![Ceph分布式存储安装部署过程-第1张-游戏相关-尔合网](/pic/ZGJiZWFKMUpIRE5EdHM5V0w0b0dOdTlhVS90QzZuWVROdmRHWHQ5Kzd5dFNJZzRyYnNORE12Zw==.jpg)
前言:
此时兄弟们对“我的世界1122怎么安装mod”大约比较讲究,看官们都想要剖析一些“我的世界1122怎么安装mod”的相关文章。那么小编在网上汇集了一些关于“我的世界1122怎么安装mod””的相关文章,希望小伙伴们能喜欢,你们快快来了解一下吧!一、环境准备
1、架构
官方文档:http://docs.ceph.org.cn/start/quick-start-preflight/
2、创建ceph.repo
[root@admin-node yum.repos.d]# cat ceph.repo
[ceph]
name=ceph
baseurl=http://mirrors.aliyun.com/ceph/rpm-jewel/el7/x86_64/
gpgcheck=0
priority=1
[ceph-noarch]
name=cephnoarch
baseurl=http://mirrors.aliyun.com/ceph/rpm-jewel/el7/noarch/
gpgcheck=0
priority=1
[root@admin-node yum.repos.d]#
3、在管理节点安装ceph部署工具
yum install ceph-deploy
4、配置NTP时间同步
yum install chrony -y && systemctl start chronyd && systemctl status chronyd && systemctl enable chronyd && egrep -v "#|^$" /etc/chrony.conf
[root@osd-node2 ~]# egrep -v "#|^$" /etc/chrony.conf
server 10.100.50.120 iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
5、创建各个节点上的ceph账户
我们建议在集群内的所有 Ceph 节点上给 ceph-deploy 创建一个特定的用户,但不要用 “ceph” 这个名字。全集群统一的用户名可简化操作(非必需),然而你应该避免使用知名用户名,因 为黑客们会用它做暴力破解(如 root 、 admin 、 {productname} )。后续步骤描述了如何创建无 sudo 密码的用户,你要用自己取的名字替换 {username} 。
useradd -d /home/{username} -m {username}
passwd {username}
6、配置SSH无密钥登录
1、生成密钥:不要使用soudu或者root用户
[sysadmin@admin-node ~]$ ssh-keygen -t rsa 使用-t 选择密钥加密类型
Generating public/private rsa key pair.
Enter file in which to save the key (/home/sysadmin/.ssh/id_rsa): 直接回车
Created directory '/home/sysadmin/.ssh'.
Enter passphrase (empty for no passphrase): 直接回车
Enter same passphrase again: 直接回车
Your identification has been saved in /home/sysadmin/.ssh/id_rsa. 私钥路径
Your public key has been saved in /home/sysadmin/.ssh/id_rsa.pub. 公钥路径
The key fingerprint is:
SHA256:2jiM64WbNMHu3n6wrHIz3dED8ezKDcFCRnZ4erps8uo sysadmin@admin-node
The key's randomart image is:
+---[RSA 2048]----+
| .o.. |
| .+.o |
| o + + |
| . o = o |
| o +S= |
| . =o+o + |
| *+**.= . |
| .o=BB.= . |
| *EX+. |
+----[SHA256]-----+
[sysadmin@admin-node ~]$
2、将密钥拷贝到其他节点
ssh-copy-id sysadmin@10.100.50.128
ssh-copy-id sysadmin@10.100.50.129
ssh-copy-id sysadmin@10.100.50.130
尝试登录测试
[sysadmin@admin-node ~]$10.100.50.130
Last login: Fri May 4 14:25:14 2018 from 10.100.50.127
[sysadmin@osdb-node2 ~]$
7、配置hosts解析
[root@admin-node ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.100.50.127 admin-node
10.100.50.128 mon-node1
10.100.50.129 osd-node1
10.100.50.130 osd-node2
8、修改管理节点的~/.ssh/config 简化SSH登录
[sysadmin@admin-node ~]$ cat ~/.ssh/config
Host mon-node1
Hostname mon-node1
User sysadmin
Host osd-node1
Hostname osd-node1
User sysadmin
Host osd-node2
Hostname osd-node2
User sysadmin
登录测试:
[sysadmin@admin-node ~]$ ssh osd-node2
Last login: Fri May 4 16:55:54 2018 from admin-node
[sysadmin@osd-node2 ~]$
二、安装部署
1、创建配置文件夹用来保存配置文件和密钥对
/home/sysadmin/my-cluster
2、禁用 requiretty
将Defaults !visiblepw 修改为Defaults visiblepw
报错信息如下:
[ceph_deploy.cli][INFO ] gpg_url : None
[ceph_deploy.install][DEBUG ] Installing stable version jewel on cluster ceph hosts mon-node1
[ceph_deploy.install][DEBUG ] Detecting platform for host mon-node1 ...
[mon-node1][DEBUG ] connection detected need for sudo
We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:
#1) Respect the privacy of others.
#2) Think before you type.
#3) With great power comes great responsibility.
sudo: no tty present and no askpass program specified
[ceph_deploy][ERROR ] RuntimeError: connecting to host: mon-node1 resulted in errors: IOError cannot send (already closed?)
[sysadmin@admin-node my-cluster]$
在某些发行版(如 CentOS )上,执行 ceph-deploy 命令时,如果你的 Ceph 节点默认设置了 requiretty 那就会遇到报错。可以这样禁用此功能:
执行 sudo visudo ,找到 Defaults requiretty 选项,把它改为 Defaults:ceph !requiretty ,这样 ceph-deploy 就能用 ceph 用户登录并使用 sudo 了。
3、如何删除ceph
如果在某些地方碰到麻烦,想从头再来,可以用下列命令清除配置:
ceph-deploy purgedata {ceph-node} [{ceph-node}]
ceph-deploy forgetkeys
用下列命令可以连 Ceph 安装包一起清除:
ceph-deploy purge {ceph-node} [{ceph-node}]
如果执行了 purge ,你必须重新安装 Ceph 。
4、创建集群
1、在所有节点安装ceph
如果报错显示:[ceph_deploy][ERROR ] RuntimeError: NoSectionError: No section: 'ceph-source'
需要修改epel源,或者yum makecache,或者删除/etc/yum.repos.d/中的repo缓存文件如: rm -rf ceph.repo.rpmnew ceph.repo.rpmsave epel.repo.rpmnew
wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
ceph-deploy install osd-node1
如果报下面错误,需要将/etc/yum.repos.d文件夹中的ceph.repo.rpmnew、epel-testing.repo、ceph.repo.rpmsave 、 epel.repo.rpmnew删除。
[osd-node1][WARNIN] ensuring that /etc/yum.repos.d/ceph.repo contains a high priority
[ceph_deploy][ERROR ] RuntimeError: NoSectionError: No section: 'ceph-source'
2、运行下面命令创建监控节点集群,目前只有一台作为monitor
ceph-deploy new mon-node1
如果遇到报错:sudo: no tty present and no askpass program specified
[ceph_deploy][ERROR ] RuntimeError: connecting to host: mon-node1 resulted in errors: IOError cannot send (already closed?)
运行 sudo visudo并添加下面一行
sysadmin ALL=(ALL) NOPASSWD: ALL
3、配置monitor并收集密钥,必须在my-cluster目录下运行
[sysadmin@admin-node my-cluster]$ ceph-deploy mon create-initial
..........................
.........................
[ceph_deploy.gatherkeys][INFO ] Storing ceph.client.admin.keyring
[ceph_deploy.gatherkeys][INFO ] Storing ceph.bootstrap-mds.keyring
[ceph_deploy.gatherkeys][INFO ] Storing ceph.bootstrap-mgr.keyring
[ceph_deploy.gatherkeys][INFO ] keyring 'ceph.mon.keyring' already exists
[ceph_deploy.gatherkeys][INFO ] Storing ceph.bootstrap-osd.keyring
[ceph_deploy.gatherkeys][INFO ] Storing ceph.bootstrap-rgw.keyring
[ceph_deploy.gatherkeys][INFO ] Destroy temp directory /tmp/tmpeANrgr
[sysadmin@admin-node my-cluster]$ ls
ceph.bootstrap-mds.keyring ceph.bootstrap-osd.keyring ceph.client.admin.keyring ceph-deploy-ceph.log
ceph.bootstrap-mgr.keyring ceph.bootstrap-rgw.keyring ceph.conf ceph.mon.keyring
[sysadmin@admin-node my-cluster]$
4、部署过程中出现问题重新部署时报错如下:
[mon-node1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.mon-node1.asok mon_status
[mon-node1][ERROR ] no valid command found; 10 closest matches:
[mon-node1][ERROR ] config set <var> <val> [<val>...]
[mon-node1][ERROR ] version
[mon-node1][ERROR ] git_version
[mon-node1][ERROR ] help
[mon-node1][ERROR ] config show
[mon-node1][ERROR ] get_command_descriptions
[mon-node1][ERROR ] config get <var>
[mon-node1][ERROR ] perfcounters_dump
[mon-node1][ERROR ] 2
[mon-node1][ERROR ] config diff
[mon-node1][ERROR ] admin_socket: invalid command
[ceph_deploy.mon][WARNIN] mon.mon-node1 monitor is not yet in quorum, tries left: 2
[ceph_deploy.mon][WARNIN] waiting 15 seconds before retrying
[mon-node1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.mon-node1.asok mon_status
[mon-node1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[ceph_deploy.mon][WARNIN] mon.mon-node1 monitor is not yet in quorum, tries left: 1
[ceph_deploy.mon][WARNIN] waiting 20 seconds before retrying
[ceph_deploy.mon][ERROR ] Some monitors have still not reached quorum:
[ceph_deploy.mon][ERROR ] mon-node1
解决方式:
1)、删除安装包
[sysadmin@admin-node my-cluster]$ ceph-deploy purge mon-node1
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/sysadmin/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.39): /bin/ceph-deploy purge mon-node1
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7f31db696ef0>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] host : ['mon-node1']
[ceph_deploy.cli][INFO ] func : <function purge at 0x7f31dbf89500>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.install][INFO ] note that some dependencies *will not* be removed because they can cause issues with qemu-kvm
[ceph_deploy.install][INFO ] like: librbd1 and librados2
[ceph_deploy.install][DEBUG ] Purging on cluster ceph hosts mon-node1
[ceph_deploy.install][DEBUG ] Detecting platform for host mon-node1 ...
[mon-node1][DEBUG ] connection detected need for sudo
[mon-node1][DEBUG ] connected to host: mon-node1
[mon-node1][DEBUG ] detect platform information from remote host
[mon-node1][DEBUG ] detect machine type
[ceph_deploy.install][INFO ] Distro info: CentOS Linux 7.5.1804 Core
[mon-node1][INFO ] Purging Ceph on mon-node1
[mon-node1][INFO ] Running command: sudo yum -y -q remove ceph ceph-release ceph-common ceph-radosgw
[mon-node1][DEBUG ] warning: /etc/yum.repos.d/ceph.repo saved as /etc/yum.repos.d/ceph.repo.rpmsave
[mon-node1][INFO ] Running command: sudo yum clean all
[mon-node1][DEBUG ] Loaded plugins: fastestmirror, langpacks, priorities
[mon-node1][DEBUG ] Cleaning repos: base epel extras updates
[mon-node1][DEBUG ] Cleaning up everything
[mon-node1][DEBUG ] Maybe you want: rm -rf /var/cache/yum, to also free up space taken by orphaned data from disabled or removed repos
[mon-node1][DEBUG ] Cleaning up list of fastest mirrors
[sysadmin@admin-node my-cluster]$
2)、 删除key
[sysadmin@admin-node my-cluster]$ ls
ceph.conf ceph-deploy-ceph.log ceph.mon.keyring
[sysadmin@admin-node my-cluster]$ ceph-deploy forgetkeys
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/sysadmin/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.39): /bin/ceph-deploy forgetkeys
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7fe4eaed35a8>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function forgetkeys at 0x7fe4eb7c2e60>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[sysadmin@admin-node my-cluster]$ ls
ceph.conf ceph-deploy-ceph.log
3)、删除部署的数据文件
[sysadmin@admin-node my-cluster]$ ceph-deploy purgedata mon-node1
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/sysadmin/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.39): /bin/ceph-deploy purgedata mon-node1
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7f51e0d0a878>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] host : ['mon-node1']
[ceph_deploy.cli][INFO ] func : <function purgedata at 0x7f51e15fd578>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.install][DEBUG ] Purging data from cluster ceph hosts mon-node1
[mon-node1][DEBUG ] connection detected need for sudo
[mon-node1][DEBUG ] connected to host: mon-node1
[mon-node1][DEBUG ] detect platform information from remote host
[mon-node1][DEBUG ] detect machine type
[mon-node1][DEBUG ] find the location of an executable
[mon-node1][DEBUG ] connection detected need for sudo
[mon-node1][DEBUG ] connected to host: mon-node1
[mon-node1][DEBUG ] detect platform information from remote host
[mon-node1][DEBUG ] detect machine type
[ceph_deploy.install][INFO ] Distro info: CentOS Linux 7.5.1804 Core
[mon-node1][INFO ] purging data on mon-node1
[mon-node1][INFO ] Running command: sudo rm -rf --one-file-system -- /var/lib/ceph
[mon-node1][INFO ] Running command: sudo rm -rf --one-file-system -- /etc/ceph/
[sysadmin@admin-node my-cluster]$
4)、重新安装部署ceph
5、修改存储副本数
[sysadmin@admin-node my-cluster]$ vim ceph.conf
添加下面信息,配置public网络,配置mon之间时间差为2s(默认为0.05s),配置副本数量为2
public network = 10.100.50.0/24
mon clock drift allowed = 2
osd pool default size = 2
5、格式化osd磁盘
查看osd节点磁盘
[sysadmin@admin-node my-cluster]$ ceph-deploy disk list osd-node1
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/sysadmin/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.39): /bin/ceph-deploy disk list osd-node1
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] subcommand : list
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x111dcb0>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function disk at 0x1108398>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] disk : [('osd-node1', None, None)]
[osd-node1][DEBUG ] connection detected need for sudo
[osd-node1][DEBUG ] connected to host: osd-node1
[osd-node1][DEBUG ] detect platform information from remote host
[osd-node1][DEBUG ] detect machine type
[osd-node1][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.4.1708 Core
[ceph_deploy.osd][DEBUG ] Listing disks on osd-node1...
[osd-node1][DEBUG ] find the location of an executable
[osd-node1][INFO ] Running command: sudo /usr/sbin/ceph-disk list
[osd-node1][DEBUG ] /dev/dm-0 other, ext4, mounted on /
[osd-node1][DEBUG ] /dev/sda :
[osd-node1][DEBUG ] /dev/sda3 other, LVM2_member
[osd-node1][DEBUG ] /dev/sda2 swap, swap
[osd-node1][DEBUG ] /dev/sda1 other, ext4, mounted on /boot
[osd-node1][DEBUG ] /dev/sdb other, unknown
[osd-node1][DEBUG ] /dev/sdc other, unknown
[osd-node1][DEBUG ] /dev/sdd other, unknown
[osd-node1][DEBUG ] /dev/sr0 other, unknown
[sysadmin@admin-node my-cluster]$
将osd节点的sdb作为日志盘
将osd节点的sdc作为数据盘并分别进行分区格式化。使用ceph推荐的xfs文件系统和gpt分区。
parted -a optimal --script /dev/sdc -- mktable gpt
parted -a optimal --script /dev/sdc -- mkpart primary xfs 0% 100%
mkfs.xfs /dev/sdc1
6、准备OSD磁盘
查看磁盘:
[sysadmin@admin-node my-cluster]$ ceph-deploy disk list osd-node1
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/sysadmin/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.39): /bin/ceph-deploy disk list osd-node1
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] subcommand : list
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x1580cb0>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function disk at 0x156b398>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] disk : [('osd-node1', None, None)]
[osd-node1][DEBUG ] connection detected need for sudo
[osd-node1][DEBUG ] connected to host: osd-node1
[osd-node1][DEBUG ] detect platform information from remote host
[osd-node1][DEBUG ] detect machine type
[osd-node1][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.4.1708 Core
[ceph_deploy.osd][DEBUG ] Listing disks on osd-node1...
[osd-node1][DEBUG ] find the location of an executable
[osd-node1][INFO ] Running command: sudo /usr/sbin/ceph-disk list
[osd-node1][DEBUG ] /dev/dm-0 other, ext4, mounted on /
[osd-node1][DEBUG ] /dev/sda :
[osd-node1][DEBUG ] /dev/sda3 other, LVM2_member
[osd-node1][DEBUG ] /dev/sda2 swap, swap
[osd-node1][DEBUG ] /dev/sda1 other, ext4, mounted on /boot
[osd-node1][DEBUG ] /dev/sdb :
[osd-node1][DEBUG ] /dev/sdb1 other, xfs
[osd-node1][DEBUG ] /dev/sdc :
[osd-node1][DEBUG ] /dev/sdc1 other, xfs
[osd-node1][DEBUG ] /dev/sdd other, unknown
[osd-node1][DEBUG ] /dev/sr0 other, unknown
[sysadmin@admin-node my-cluster]$
[sysadmin@admin-node my-cluster]$ ceph-deploy disk list osd-node2
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/sysadmin/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.39): /bin/ceph-deploy disk list osd-node2
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] subcommand : list
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x1122cb0>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function disk at 0x110d398>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] disk : [('osd-node2', None, None)]
[osd-node2][DEBUG ] connection detected need for sudo
[osd-node2][DEBUG ] connected to host: osd-node2
[osd-node2][DEBUG ] detect platform information from remote host
[osd-node2][DEBUG ] detect machine type
[osd-node2][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.4.1708 Core
[ceph_deploy.osd][DEBUG ] Listing disks on osd-node2...
[osd-node2][DEBUG ] find the location of an executable
[osd-node2][INFO ] Running command: sudo /usr/sbin/ceph-disk list
[osd-node2][DEBUG ] /dev/dm-0 other, ext4, mounted on /
[osd-node2][DEBUG ] /dev/sda :
[osd-node2][DEBUG ] /dev/sda3 other, LVM2_member
[osd-node2][DEBUG ] /dev/sda2 swap, swap
[osd-node2][DEBUG ] /dev/sda1 other, ext4, mounted on /boot
[osd-node2][DEBUG ] /dev/sdb :
[osd-node2][DEBUG ] /dev/sdb1 other, xfs
[osd-node2][DEBUG ] /dev/sdc :
[osd-node2][DEBUG ] /dev/sdc1 other, xfs
[osd-node2][DEBUG ] /dev/sdd other, unknown
[osd-node2][DEBUG ] /dev/sr0 other, unknown
[sysadmin@admin-node my-cluster]$
下面操作必须在admin的/etc/ceph下执行(请务必注意,格式为: ip地址:osd磁盘:日志盘)
ceph-deploy osd prepare osd-node1:sdc1:sdb1
ceph-deploy osd prepare osd-node2:sdc1:sdb1
如果报下面错误表示配置文件在各个节点上没有同步,需要手动推送一下:ceph-deploy --overwrite-conf config push admin-node osd-node1 osd-node2
[ceph_deploy.osd][ERROR ] RuntimeError: config file /etc/ceph/ceph.conf exists with different content; use --overwrite-conf to overwrite
[ceph_deploy][ERROR ] GenericError: Failed to create 1 OSDs
7、激活OSD磁盘
ceph-deploy osd activate osd-node1:sdc1:sdb1
ceph-deploy osd activate osd-node2:sdc1:sdb1
如果遇到下面错误,提示权限问题,需要修改磁盘的属主和属组为ceph(不是部署ceph创建的账号,而是ceph自己创建的‘ceph’账户)
[osd-node1][WARNIN] ceph_disk.main.Error: Error: ['ceph-osd', '--cluster', 'ceph', '--mkfs', '--mkkey', '-i', u'0', '--monmap', '/var/lib/ceph/tmp/mnt.wMliCA/activate.monmap', '--osd-data', '/var/lib/ceph/tmp/mnt.wMliCA', '--osd-journal', '/var/lib/ceph/tmp/mnt.wMliCA/journal', '--osd-uuid', u'14a500fd-a030-427a-b007-f16f6f4bbd4d', '--keyring', '/var/lib/ceph/tmp/mnt.wMliCA/keyring', '--setuser', 'ceph', '--setgroup', 'ceph'] failed : 2018-05-23 10:47:41.254049 7fece10c8800 -1 filestore(/var/lib/ceph/tmp/mnt.wMliCA) mkjournal error creating journal on /var/lib/ceph/tmp/mnt.wMliCA/journal: (13) Permission denied
[osd-node1][WARNIN] 2018-05-23 10:47:41.254068 7fece10c8800 -1 OSD::mkfs: ObjectStore::mkfs failed with error -13
[osd-node1][WARNIN] 2018-05-23 10:47:41.254123 7fece10c8800 -1 ** ERROR: error creating empty object store in /var/lib/ceph/tmp/mnt.wMliCA: (13) Permission denied
[osd-node1][WARNIN]
[osd-node1][ERROR ] RuntimeError: command returned non-zero exit status: 1
[ceph_deploy][ERROR ] RuntimeError: Failed to execute command: /usr/sbin/ceph-disk -v activate --mark-init systemd --mount /dev/sdc1
需要修改磁盘的属主和属组为ceph
[root@osd-node1 yum.repos.d]# cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
......
sysadmin:x:1000:1000:sysadmin:/home/sysadmin:/bin/bash
ceph:x:167:167:Ceph daemons:/var/lib/ceph:/sbin/nologin
[root@osd-node1 yum.repos.d]# chown ceph:ceph /dev/sdb1
[root@osd-node1 yum.repos.d]# chown ceph:ceph /dev/sdc1
[root@osd-node1 yum.repos.d]# ll /dev/sdb1
brw-rw---- 1 ceph ceph 8, 17 May 23 10:29 /dev/sdb1
[root@osd-node1 yum.repos.d]# ll /dev/sdc1
brw-rw---- 1 ceph ceph 8, 33 May 23 10:33 /dev/sdc1
[root@osd-node1 yum.repos.d]#
8、一次性完成磁盘的准备和激活
ceph-deploy osd create osd-node1:sdc1:sdb1
ceph-deploy osd create osd-node2:sdc1:sdb1
9、查看osd视图
[sysadmin@admin-node ceph]$ ceph osd tree
ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY
-1 0.03897 root default
-2 0.01949 host osd-node1
0 0.01949 osd.0 up 1.00000 1.00000
-3 0.01949 host osd-node2
1 0.01949 osd.1 up 1.00000 1.00000
[sysadmin@admin-node ceph]$
如果报下面错误,因为我的key都是放到/home/sysadmin/my-cluster/下面,所以直接将下面文件全部拷贝到/etc/ceph/下面,并将权限改为755。
[sysadmin@admin-node my-cluster]$ ceph osd tree
2018-05-23 11:08:38.913940 7fbd0e84d700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory
2018-05-23 11:08:38.913953 7fbd0e84d700 -1 monclient(hunting): ERROR: missing keyring, cannot use cephx for authentication
2018-05-23 11:08:38.913955 7fbd0e84d700 0 librados: client.admin initialization error (2) No such file or directory
Error connecting to cluster: ObjectNotFound
[sysadmin@admin-node my-cluster]$
[root@admin-node ceph]# cp /home/sysadmin/my-cluster/* /etc/ceph/
cp: overwrite ‘/etc/ceph/ceph.bootstrap-mds.keyring’? y
cp: overwrite ‘/etc/ceph/ceph.conf’? y
[root@admin-node ceph]# chmod 755 *
[root@admin-node ceph]# ll
total 172
-rwxr-xr-x 1 root root 113 May 23 11:08 ceph.bootstrap-mds.keyring
-rwxr-xr-x 1 root root 71 May 23 11:08 ceph.bootstrap-mgr.keyring
-rwxr-xr-x 1 root root 113 May 23 11:08 ceph.bootstrap-osd.keyring
-rwxr-xr-x 1 root root 113 May 23 11:08 ceph.bootstrap-rgw.keyring
-rwxr-xr-x 1 root root 129 May 23 11:08 ceph.client.admin.keyring
-rwxr-xr-x 1 root root 225 May 23 11:08 ceph.conf
-rwxr-xr-x 1 root root 142307 May 23 11:08 ceph-deploy-ceph.log
-rwxr-xr-x 1 root root 73 May 23 11:08 ceph.mon.keyring
-rwxr-xr-x. 1 root root 92 Oct 4 2017 rbdmap
-rwxr-xr-x 1 root root 0 May 23 10:33 tmpAlNIWB
[root@admin-node ceph]#
10、查看集群状态
[sysadmin@admin-node my-cluster]$ ceph -s
cluster 77524d79-bc18-471a-8956-f5045579cc74
health HEALTH_OK
monmap e1: 1 mons at {mon-node1=10.100.50.128:6789/0}
election epoch 3, quorum 0 mon-node1
osdmap e11: 2 osds: 2 up, 2 in
flags sortbitwise,require_jewel_osds
pgmap v19: 64 pgs, 1 pools, 0 bytes data, 0 objects
68400 kB used, 40869 MB / 40936 MB avail
64 active+clean
[sysadmin@admin-node my-cluster]$ ceph osd tree
ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY
-1 0.03897 root default
-2 0.01949 host osd-node1
0 0.01949 osd.0 up 1.00000 1.00000
-3 0.01949 host osd-node2
1 0.01949 osd.1 up 1.00000 1.00000
[sysadmin@admin-node my-cluster]$
- 随机文章