kubeadm安装高可用简介

Windows Windows 2个月前 (08-15) 4次浏览 未收录 0个评论 扫描二维码

1、系统环境

  使用kubeadm安装高可用k8s v.13.x较为简单,相比以往的版本省去了很多步骤。

  kubeadm安装高可用k8s v.11 和 v1.12点我

  主机信息

主机名 IP地址 说明 组件
k8s-master01 ~ 03 192.168.20.20 ~ 22 master节点 * 3 keepalived、nginx、etcd、kubelet、kube-apiserver
k8s-master-lb 192.168.20.10 keepalived虚拟IP
k8s-node01 ~ 08 192.168.20.30 ~ 37 worker节点 * 8 kubelet

  主机配置

[root@k8s-master01~]#uname-r4.18.9-1.el7.elrepo.x86_64
[root@k8s-master01~]#uname-a
Linuxk8s-master014.18.9-1.el7.elrepo.x86_64#1SMPThuSep2009:04:54EDT2018x86_64x86_64x86_64GNU/Linux
[root@k8s-master01~]#hostnamek8s-master01
[root@k8s-master01~]#free-g
totalusedfreesharedbuff/cacheavailable
Mem:310022Swap:000[root@k8s-master01~]#cat/proc/cpuinfo|grepprocess
processor:0processor:1processor:2processor:3[root@k8s-master01~]#cat/etc/redhat-release
CentOSLinuxrelease7.5.1804(Core)

  Docker和k8s版本

[root@k8s-master01~]#dockerversion
Client:
Version:17.09.1-ce
APIversion:1.32
Goversion:go1.8.3
Gitcommit:19e2cf6
Built:ThuDec722:23:402017
OS/Arch:linux/amd64

Server:
Version:17.09.1-ce
APIversion:1.32(minimumversion1.12)
Goversion:go1.8.3
Gitcommit:19e2cf6
Built:ThuDec722:25:032017
OS/Arch:linux/amd64
Experimental:false[root@k8s-master01~]#kubectlversion
ClientVersion:version.Info{Major:"1",Minor:"13",GitVersion:"v1.13.2",GitCommit:"cff46ab41ff0bb44d8584413b598ad8360ec1def",GitTreeState:"clean",BuildDate:"2019-01-10T23:35:51Z",GoVersion:"go1.11.4",Compiler:"gc",Platform:"linux/amd64"}
ServerVersion:version.Info{Major:"1",Minor:"13",GitVersion:"v1.13.2",GitCommit:"cff46ab41ff0bb44d8584413b598ad8360ec1def",GitTreeState:"clean",BuildDate:"2019-01-10T23:28:14Z",GoVersion:"go1.11.4",Compiler:"gc",Platform:"linux/amd64"}

 

2、配置SSH互信

  所有节点配置hosts:

[root@k8s-master01~]#cat/etc/hosts

192.168.20.20k8s-master01
192.168.20.21k8s-master02
192.168.20.22k8s-master03
192.168.20.10k8s-master-lb
192.168.20.30k8s-node01
192.168.20.31k8s-node02

  在k8s-master01上执行:

[root@k8s-master01~]#ssh-keygen-trsaGeneratingpublic/privatersakeypair.
Enterfileinwhichtosavethekey(/root/.ssh/id_rsa):
Createddirectory'/root/.ssh'.
Enterpassphrase(emptyfornopassphrase):
Entersamepassphraseagain:
Youridentificationhasbeensavedin/root/.ssh/id_rsa.
Yourpublickeyhasbeensavedin/root/.ssh/id_rsa.pub.
Thekeyfingerprintis:
SHA256:TE0eRfhGNRXL3btmmMRq+awUTkR4RnWrMf6Q5oJaTn0root@k8s-master01
Thekey'srandomartimageis:
+---[RSA2048]----+
|=*+oo+o|
|=o+.o.=|
|.=+o+o|
|o.==.|
|S+O.|
|=B=.|
|+OE=|
|=o=o|
|....o|
+----[SHA256]-----+
foriink8s-master01k8s-master02k8s-master03k8s-node01k8s-node02;dossh-copy-id-i.ssh/id_rsa.pub$i;done

  所有节点关闭防火墙和selinux

[root@k8s-master01~]#systemctldisable--nowfirewalldNetworkManager
Removedsymlink/etc/systemd/system/multi-user.target.wants/NetworkManager.service.
Removedsymlink/etc/systemd/system/multi-user.target.wants/firewalld.service.
Removedsymlink/etc/systemd/system/dbus-org.freedesktop.NetworkManager.service.
Removedsymlink/etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service.
Removedsymlink/etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@k8s-master01~]#setenforce0
[root@k8s-master01~]#sed-ri'/^[^#]*SELINUX=/s#=.+$#=disabled#'/etc/selinux/config

  所有节点关闭dnsmasq(如开启)

systemctldisable--nowdnsmasq

  所有节点关闭swap

[root@k8s-master01~]#swapoff-a&&sysctl-wvm.swappiness=0
vm.swappiness=0
[root@k8s-master01~]#sed-ri'/^[^#]*swap/s@^@#@'/etc/fstab

  所有节点升级系统

yuminstallepel-release-y
yuminstallwgetgitjqpsmiscvim-y
yumupdate-y--exclude=kernel*

  所有节点同步时间

ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
echo 'Asia/Shanghai' >/etc/timezone
ntpdate time2.aliyun.com
# 加入到crontab

  所有节点limit配置

ulimit-SHn65535

  master01下载安装文件

[root@k8s-master01~]#gitclonehttps://github.com/dotbalo/k8s-ha-install.git-bv1.13.x

  所有节点创建repo

cd/etc/yum.repos.d
mkdirbak
mv*.repobak/
cp/root/k8s-ha-install/repo/*.

  所有节点升级系统并重启

yuminstallwgetgitjqpsmisc-yyumupdate-y--exclude=kernel*&&reboot

 

  内核升级

  所有节点

[root@k8s-master01~]#rpm--importhttps://www.elrepo.org/RPM-GPG-KEY-elrepo.org[root@k8s-master01~]#rpm-Uvhhttp://www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpmRetrievinghttp://www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpmRetrievinghttp://elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpmPreparing...#################################[100%]
Updating/installing...1:elrepo-release-7.0-3.el7.elrepo#################################[100%]

  master01下载内核文件

wgethttp://mirror.rc.usf.edu/compute_lock/elrepo/kernel/el7/x86_64/RPMS/kernel-ml-devel-4.18.9-1.el7.elrepo.x86_64.rpmwgethttp://mirror.rc.usf.edu/compute_lock/elrepo/kernel/el7/x86_64/RPMS/kernel-devel-4.18.9-1.el7.elrepo.x86_64.rpm

  复制到其他节点

[root@k8s-master01~]#foriink8s-master01k8s-master02k8s-master03k8s-node01k8s-node02;doscpkernel-ml-4.18.9-1.el7.elrepo.x86_64.rpmkernel-ml-devel-4.18.9-1.el7.elrepo.x86_64.rpm$i:/root/;donekernel-ml-4.18.9-1.el7.elrepo.x86_64.rpm100%45MB147.2MB/s00:00kernel-ml-devel-4.18.9-1.el7.elrepo.x86_64.rpm100%12MB149.1MB/s00:00kernel-ml-4.18.9-1.el7.elrepo.x86_64.rpm100%45MB22.6MB/s00:02kernel-ml-devel-4.18.9-1.el7.elrepo.x86_64.rpm100%12MB20.8MB/s00:00kernel-ml-4.18.9-1.el7.elrepo.x86_64.rpm100%45MB15.1MB/s00:03kernel-ml-devel-4.18.9-1.el7.elrepo.x86_64.rpm100%12MB11.9MB/s00:01kernel-ml-4.18.9-1.el7.elrepo.x86_64.rpm100%45MB45.1MB/s00:01kernel-ml-devel-4.18.9-1.el7.elrepo.x86_64.rpm100%12MB27.4MB/s00:00kernel-ml-4.18.9-1.el7.elrepo.x86_64.rpm100%45MB45.1MB/s00:01kernel-ml-devel-4.18.9-1.el7.elrepo.x86_64.rpm100%12MB30.0MB/s00:00

  所有节点安装内核

yumlocalinstall-ykernel-ml*

  所有节点修改内核启动顺序

grub2-set-default 0 && grub2-mkconfig -o /etc/grub2.cfg

grubby--args="user_namespace.enable=1"--update-kernel="$(grubby--default-kernel)"

  所有节点重启

reboot

  确认所有节点为新内核

[root@k8s-master01~]#uname-r4.18.9-1.el7.elrepo.x86_64

  确认能否加载nf_conntrack_ipv4,ipvs依赖此模块,需要确认能正常加载

[root@k8s-master02~]#modprobenf_conntrack_ipv4
[root@k8s-master02~]#lsmod|grepnf
nf_conntrack_ipv4163840nf_defrag_ipv4163841nf_conntrack_ipv4
nf_conntrack1351681nf_conntrack_ipv4
libcrc32c163842nf_conntrack,xfs

  所有节点安装ipvsadm

yuminstallipvsadmipsetsysstatconntracklibseccomp-y

  所有节点设置开启自动加载的模块

[root@k8s-master01~]#cat/etc/modules-load.d/ipvs.conf
ip_vs
ip_vs_lc
ip_vs_wlc
ip_vs_rr
ip_vs_wrr
ip_vs_lblc
ip_vs_lblcr
ip_vs_dh
ip_vs_sh
ip_vs_fo
ip_vs_nq
ip_vs_sed
ip_vs_ftp

 

[root@k8s-master01~]#systemctldisable--nowsystemd-modules-load.service

[root@k8s-master01~]#lsmod|grepip_vs
ip_vs_ftp163840nf_nat327681ip_vs_ftp
ip_vs_sed163840ip_vs_nq163840ip_vs_fo163840ip_vs_sh163840ip_vs_dh163840ip_vs_lblcr163840ip_vs_lblc163840ip_vs_wrr163840ip_vs_rr163840ip_vs_wlc163840ip_vs_lc163840ip_vs15155224ip_vs_wlc,ip_vs_rr,ip_vs_dh,ip_vs_lblcr,ip_vs_sh,ip_vs_fo,ip_vs_nq,ip_vs_lblc,ip_vs_wrrip_vs_lc,ip_vs_sed,ip_vs_ftp
nf_conntrack1351683nf_conntrack_ipv4,nf_nat,ip_vs
libcrc32c163844nf_conntrack,nf_nat,xfs,ip_vs

  所有节点配置k8s内核

cat<
 
  /etc/sysctl.d/k8s.conf
net.ipv4.ip_forward=1net.bridge.bridge-nf-call-ip6tables=1net.bridge.bridge-nf-call-iptables=1fs.may_detach_mounts=1vm.overcommit_memory=1vm.panic_on_oom=0fs.inotify.max_user_watches=89100fs.file-max=52706963fs.nr_open=52706963net.netfilter.nf_conntrack_max=2310720EOF

sysctl--system
 

 

3、k8s服务安装

  所有节点安装docker-ce

yum-yinstalldocker-ce-17.09.1.ce-1.el7.centos

  所有节点安装集群组件

yuminstall-ykubelet-1.13.2-0.x86_64kubeadm-1.13.2-0.x86_64kubectl-1.13.2-0.x86_64

  所有节点启动docker和kubelet

systemctlenabledocker&&systemctlstartdocker
[root@k8s-master01~]#DOCKER_CGROUPS=$(dockerinfo|grep'Cgroup'|cut-d''-f3)
[root@k8s-master01~]#echo$DOCKER_CGROUPS
cgroupfs
[root@k8s-master01~]#cat>/etc/sysconfig/kubelet<
 
  KUBELET_EXTRA_ARGS="--cgroup-driver=$DOCKER_CGROUPS--pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64:3.1">EOF
[root@k8s-master01~]#
[root@k8s-master01~]#systemctldaemon-reload
[root@k8s-master01~]#systemctlenablekubelet&&systemctlstartkubelet
Createdsymlinkfrom/etc/systemd/system/multi-user.target.wants/kubelet.serviceto/etc/systemd/system/kubelet.service.
[root@k8s-master01~]#
 

  注意此时如果kubelet无法启动不用管

  在所有master节点安装并启动keepalived及docker-compose

yuminstall-ykeepalived
systemctlenablekeepalived&&systemctlrestartkeepalived

#安装docker-composeyuminstall-ydocker-compose

 

4、master01节点安装

  以下操作在master01节点

  创建配置文件

  修改对应的配置信息,注意nm-bond修改为服务器对应网卡名称

[root@k8s-master01k8s-ha-install]#./create-config.shcreatekubeadm-config.yamlfilessuccess.config/k8s-master01/kubeadm-config.yaml
createkubeadm-config.yamlfilessuccess.config/k8s-master02/kubeadm-config.yaml
createkubeadm-config.yamlfilessuccess.config/k8s-master03/kubeadm-config.yaml
createkeepalivedfilessuccess.config/k8s-master01/keepalived/createkeepalivedfilessuccess.config/k8s-master02/keepalived/createkeepalivedfilessuccess.config/k8s-master03/keepalived/createnginx-lbfilessuccess.config/k8s-master01/nginx-lb/createnginx-lbfilessuccess.config/k8s-master02/nginx-lb/createnginx-lbfilessuccess.config/k8s-master03/nginx-lb/createcalico.yamlfilesuccess.calico/calico.yaml
[root@k8s-master01k8s-ha-install]#pwd/root/k8s-ha-instal

  分发文件

[root@k8s-master01k8s-ha-install]#exportHOST1=k8s-master01
[root@k8s-master01k8s-ha-install]#exportHOST2=k8s-master02
[root@k8s-master01k8s-ha-install]#exportHOST3=k8s-master03
[root@k8s-master01k8s-ha-install]#scp-rconfig/$HOST1/kubeadm-config.yaml$HOST1:/root/kubeadm-config.yaml100%9931.9MB/s00:00[root@k8s-master01k8s-ha-install]#scp-rconfig/$HOST2/kubeadm-config.yaml$HOST2:/root/kubeadm-config.yaml100%107163.8KB/s00:00[root@k8s-master01k8s-ha-install]#scp-rconfig/$HOST3/kubeadm-config.yaml$HOST3:/root/kubeadm-config.yaml100%111227.6KB/s00:00[root@k8s-master01k8s-ha-install]#scp-rconfig/$HOST1/keepalived/*$HOST1:/etc/keepalived/
check_apiserver.sh100%47136.4KB/s00:00
keepalived.conf100%55869.9KB/s00:00
Youhavenewmailin/var/spool/mail/root
[root@k8s-master01k8s-ha-install]#scp-rconfig/$HOST2/keepalived/*$HOST2:/etc/keepalived/

check_apiserver.sh100%47110.8KB/s00:00
keepalived.conf100%558275.5KB/s00:00
[root@k8s-master01k8s-ha-install]#scp-rconfig/$HOST3/keepalived/*$HOST3:/etc/keepalived/
check_apiserver.sh100%47112.7KB/s00:00
keepalived.conf100%5581.1MB/s00:00
[root@k8s-master01k8s-ha-install]#scp-rconfig/$HOST1/nginx-lb$HOST1:/root/
docker-compose.yaml100%213478.6KB/s00:00
nginx-lb.conf100%10362.6MB/s00:00
[root@k8s-master01k8s-ha-install]#scp-rconfig/$HOST2/nginx-lb$HOST2:/root/

docker-compose.yaml100%21312.5KB/s00:00
nginx-lb.conf100%103635.5KB/s00:00
[root@k8s-master01k8s-ha-install]#scp-rconfig/$HOST3/nginx-lb$HOST3:/root/
docker-compose.yaml100%21320.5KB/s00:00
nginx-lb.conf100%103694.3KB/s00:00

  所有master节点启动nginx

启动nginx-lb,修改nginx-lb下的nginx配置文件proxy_connect_timeout60s;proxy_timeout10m;
cd
docker-compose--file=/root/nginx-lb/docker-compose.yamlup-d
docker-compose--file=/root/nginx-lb/docker-compose.yamlps

  重启keepalived

systemctlrestartkeepalived

  提前下载镜像

kubeadmconfigimagespull--config/root/kubeadm-config.yaml

  集群初始化

kubeadm init --config /root/kubeadm-config.yaml

....
kubeadm join k8s-master-lb:16443 --token cxwr3f.2knnb1gj83ztdg9l --discovery-token-ca-cert-hash sha256:41718412b5d2ccdc8b7326fd440360bf186a21dac4a0769f460ca4bdaf5d2825
....
[root@k8s-master01~]#cat<
 
  >~/.bashrc
exportKUBECONFIG=/etc/kubernetes/admin.conf
EOF
[root@k8s-master01~]#source~/.bashrc
[root@k8s-master01~]#kubectlgetnodes
NAMESTATUSROLESAGEVERSION
k8s-master01NotReadymaster2m11sv1.13.2
 

  查看pods状态

[root@k8s-master01~]#kubectlgetpods-nkube-system-owide
NAMEREADYSTATUSRESTARTSAGEIPNODENOMINATEDNODEREADINESSGATES
coredns-89cc84847-2h7r60/1ContainerCreating03m12s
 
  k8s-master01
  
   
    coredns-89cc84847-fhwbr0/1ContainerCreating03m12s
    
     k8s-master01
     
      
       etcd-k8s-master011/1Running02m31s192.168.20.20k8s-master01
       
        
         kube-apiserver-k8s-master011/1Running02m36s192.168.20.20k8s-master01
         
          
           kube-controller-manager-k8s-master011/1Running02m39s192.168.20.20k8s-master01
           
            
             kube-proxy-kb95s1/1Running03m12s192.168.20.20k8s-master01
             
              
               kube-scheduler-k8s-master011/1Running02m46s192.168.20.20k8s-master01
               
                
               
              
             
            
           
          
         
        
       
      
     
    
   
  
 

  此时CoreDNS状态为ContainerCreating,报错如下:

NormalScheduled2m51sdefault-schedulerSuccessfullyassignedkube-system/coredns-89cc84847-2h7r6tok8s-master01
WarningNetworkNotReady2m3s(x25over2m51s)kubelet,k8s-master01networkisnotready:[runtimenetworknotready:NetworkReady=falsereason:NetworkPluginNotReadymessage:docker:networkpluginisnotready:cniconfiguninitialized

  因为没有安装网络插件,暂时不用管

  安装calico

[root@k8s-master01k8s-ha-install]#kubectlcreate-fcalico/configmap/calico-configcreated
service/calico-typhacreated
deployment.apps/calico-typhacreated
poddisruptionbudget.policy/calico-typhacreated
daemonset.extensions/calico-nodecreated
serviceaccount/calico-nodecreated
customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.orgcreated
customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.orgcreated
customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.orgcreated
customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.orgcreated
customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.orgcreated
customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.orgcreated
customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.orgcreated
customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.orgcreated
customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.orgcreated
clusterrole.rbac.authorization.k8s.io/calico-nodecreated
clusterrolebinding.rbac.authorization.k8s.io/calico-nodecreated

  再次查看

[root@k8s-master01k8s-ha-install]#kubectlgetpo-nkube-system
NAMEREADYSTATUSRESTARTSAGE
calico-node-tp2dz2/2Running042s
coredns-89cc84847-2djpl1/1Running066s
coredns-89cc84847-vt6zq1/1Running066s
etcd-k8s-master011/1Running027s
kube-apiserver-k8s-master011/1Running016s
kube-controller-manager-k8s-master011/1Running034s
kube-proxy-x497d1/1Running066s
kube-scheduler-k8s-master011/1Running017s

 

5、高可用配置

  复制证书

USER=root
CONTROL_PLANE_IPS="k8s-master02k8s-master03"forhostin$CONTROL_PLANE_IPS;do
ssh"${USER}"@$host"mkdir-p/etc/kubernetes/pki/etcd"
scp/etc/kubernetes/pki/ca.crt"${USER}"@$host:/etc/kubernetes/pki/ca.crtscp/etc/kubernetes/pki/ca.key"${USER}"@$host:/etc/kubernetes/pki/ca.keyscp/etc/kubernetes/pki/sa.key"${USER}"@$host:/etc/kubernetes/pki/sa.keyscp/etc/kubernetes/pki/sa.pub"${USER}"@$host:/etc/kubernetes/pki/sa.pubscp/etc/kubernetes/pki/front-proxy-ca.crt"${USER}"@$host:/etc/kubernetes/pki/front-proxy-ca.crtscp/etc/kubernetes/pki/front-proxy-ca.key"${USER}"@$host:/etc/kubernetes/pki/front-proxy-ca.keyscp/etc/kubernetes/pki/etcd/ca.crt"${USER}"@$host:/etc/kubernetes/pki/etcd/ca.crtscp/etc/kubernetes/pki/etcd/ca.key"${USER}"@$host:/etc/kubernetes/pki/etcd/ca.keyscp/etc/kubernetes/admin.conf"${USER}"@$host:/etc/kubernetes/admin.confdone

  以下操作在master02执行

  提前下载镜像

kubeadmconfigimagespull--config/root/kubeadm-config.yaml

  master02加入集群,与node节点相差的参数就是–experimental-control-plane

kubeadmjoink8s-master-lb:16443--tokencxwr3f.2knnb1gj83ztdg9l--discovery-token-ca-cert-hashsha256:41718412b5d2ccdc8b7326fd440360bf186a21dac4a0769f460ca4bdaf5d2825--experimental-control-plane
......

Thisnodehasjoinedtheclusterandanewcontrolplaneinstancewascreated:*Certificatesigningrequestwassenttoapiserverandapprovalwasreceived.*TheKubeletwasinformedofthenewsecureconnectiondetails.*Masterlabelandtaintwereappliedtothenewnode.*TheKubernetescontrolplaneinstancesscaledup.*Anewetcdmemberwasaddedtothelocal/stackedetcdcluster.

Tostartadministeringyourclusterfromthisnode,youneedtorunthefollowingasaregularuser:mkdir-p$HOME/.kubesudocp-i/etc/kubernetes/admin.conf$HOME/.kube/configsudochown$(id-u):$(id-g)$HOME/.kube/config

Run'kubectlgetnodes'toseethisnodejointhecluster.

  master01查看状态

[root@k8s-master01k8s-ha-install]#kubectlgetno
NAMESTATUSROLESAGEVERSION
k8s-master01Readymaster15mv1.13.2k8s-master02Readymaster9m55sv1.13.2

  其他master节点类似

  查看最终master状态

[root@k8s-master01~]#ipvsadm-lnIPVirtualServerversion1.2.1(size=4096)
ProtLocalAddress:PortSchedulerFlags->RemoteAddress:PortForwardWeightActiveConnInActConn
TCP10.96.0.1:443rr->192.168.20.20:6443Masq140
->192.168.20.21:6443Masq100
->192.168.20.22:6443Masq100TCP10.96.0.10:53rr->172.168.0.10:53Masq100
->172.168.0.11:53Masq100TCP10.102.221.48:5473rr
UDP10.96.0.10:53rr->172.168.0.10:53Masq100
->172.168.0.11:53Masq100[root@k8s-master01~]#kubectlgetpo-nkube-system
NAMEREADYSTATUSRESTARTSAGE
calico-node-49dwr2/2Running026m
calico-node-kz2d42/2Running022m
calico-node-zwnmq2/2Running04m6s
coredns-89cc84847-dgxlw1/1Running027m
coredns-89cc84847-n77x61/1Running027m
etcd-k8s-master011/1Running027m
etcd-k8s-master021/1Running022m
etcd-k8s-master031/1Running04m5s
kube-apiserver-k8s-master011/1Running027m
kube-apiserver-k8s-master021/1Running022m
kube-apiserver-k8s-master031/1Running34m6s
kube-controller-manager-k8s-master011/1Running127m
kube-controller-manager-k8s-master021/1Running022m
kube-controller-manager-k8s-master031/1Running04m6s
kube-proxy-f9qc51/1Running027m
kube-proxy-k55bg1/1Running022m
kube-proxy-kbg9c1/1Running04m6s
kube-scheduler-k8s-master011/1Running127m
kube-scheduler-k8s-master021/1Running022m
kube-scheduler-k8s-master031/1Running04m6s
[root@k8s-master01~]#kubectlgetno
NAMESTATUSROLESAGEVERSION
k8s-master01Readymaster28mv1.13.2k8s-master02Readymaster22mv1.13.2k8s-master03Readymaster4m16sv1.13.2
[root@k8s-master01~]#kubectlgetcsr
NAMEAGEREQUESTORCONDITION
csr-6mqbv28msystem:node:k8s-master01Approved,Issued
node-csr-GPLcR1G4Nchf-zuB5DaTWncoluMuENUfKvWKs0j2GdQ23msystem:bootstrap:9zp70mApproved,Issued
node-csr-cxAxrkllyidkBuZ8fck6fwq-ht1_u6s0snbDErM8bIs4m51ssystem:bootstrap:9zp70mApproved,Issued

  在所有master节点上允许hpa通过接口采集数据

vi/etc/kubernetes/manifests/kube-controller-manager.yaml
---horizontal-pod-autoscaler-use-rest-clients=false

  在所有master上允许istio的自动注入,修改/etc/kubernetes/manifests/kube-apiserver.yaml

vi/etc/kubernetes/manifests/kube-apiserver.yaml
---enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota

 

6、node节点加入集群

kubeadmjoin192.168.20.10:16443--tokenll4usb.qmplnofiv7z1j0an--discovery-token-ca-cert-hashsha256:e88a29f62ab77a59bf88578abadbcd37e89455515f6ecf3ca371656dc65b1d6e

 

......
[kubelet-start]Activatingthekubeletservice
[tlsbootstrap]WaitingforthekubelettoperformtheTLSBootstrap...
[patchnode]UploadingtheCRISocketinformation"/var/run/dockershim.sock"totheNodeAPIobject"k8s-node02"asanannotation

Thisnodehasjoinedthecluster:*Certificatesigningrequestwassenttoapiserverandaresponsewasreceived.*TheKubeletwasinformedofthenewsecureconnectiondetails.

Run'kubectlgetnodes'onthemastertoseethisnodejointhecluster.

  master节点查看

[root@k8s-master01k8s-ha-install]#kubectlgetpo-nkube-system
NAMEREADYSTATUSRESTARTSAGE
calico-node-49dwr2/2Running013h
calico-node-9nmhb2/2Running011m
calico-node-k5nmt2/2Running011m
calico-node-kz2d42/2Running013h
calico-node-zwnmq2/2Running013h
coredns-89cc84847-dgxlw1/1Running013h
coredns-89cc84847-n77x61/1Running013h
etcd-k8s-master011/1Running013h
etcd-k8s-master021/1Running013h
etcd-k8s-master031/1Running013h
kube-apiserver-k8s-master011/1Running018m
kube-apiserver-k8s-master021/1Running017m
kube-apiserver-k8s-master031/1Running016m
kube-controller-manager-k8s-master011/1Running019m
kube-controller-manager-k8s-master021/1Running119m
kube-controller-manager-k8s-master031/1Running019m
kube-proxy-cl2zv1/1Running011m
kube-proxy-f9qc51/1Running013h
kube-proxy-hkcq51/1Running011m
kube-proxy-k55bg1/1Running013h
kube-proxy-kbg9c1/1Running013h
kube-scheduler-k8s-master011/1Running113h
kube-scheduler-k8s-master021/1Running013h
kube-scheduler-k8s-master031/1Running013h
Youhavenewmailin/var/spool/mail/root
[root@k8s-master01k8s-ha-install]#kubectlgetno
NAMESTATUSROLESAGEVERSION
k8s-master01Readymaster13hv1.13.2k8s-master02Readymaster13hv1.13.2k8s-master03Readymaster13hv1.13.2k8s-node01Ready
 
  11mv1.13.2k8s-node02Ready
  
   11mv1.13.2
  
 

 

7、其他组件安装

  部署metrics server 0.3.1/1.8+安装

[root@k8s-master01k8s-ha-install]#kubectlcreate-fmetrics-server/clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-readercreated
clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegatorcreated
rolebinding.rbac.authorization.k8s.io/metrics-server-auth-readercreated
apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.iocreated
serviceaccount/metrics-servercreated
deployment.extensions/metrics-servercreated
service/metrics-servercreated
clusterrole.rbac.authorization.k8s.io/system:metrics-servercreated
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-servercreated
[root@k8s-master01k8s-ha-install]#kubectlgetpo-nkube-system
NAMEREADYSTATUSRESTARTSAGE
calico-node-49dwr2/2Running014h
calico-node-9nmhb2/2Running069m
calico-node-k5nmt2/2Running069m
calico-node-kz2d42/2Running014h
calico-node-zwnmq2/2Running014h
coredns-89cc84847-dgxlw1/1Running014h
coredns-89cc84847-n77x61/1Running014h
etcd-k8s-master011/1Running014h
etcd-k8s-master021/1Running014h
etcd-k8s-master031/1Running014h
kube-apiserver-k8s-master011/1Running06m23s
kube-apiserver-k8s-master021/1Running14m41s
kube-apiserver-k8s-master031/1Running04m34s
kube-controller-manager-k8s-master011/1Running078m
kube-controller-manager-k8s-master021/1Running178m
kube-controller-manager-k8s-master031/1Running077m
kube-proxy-cl2zv1/1Running069m
kube-proxy-f9qc51/1Running014h
kube-proxy-hkcq51/1Running069m
kube-proxy-k55bg1/1Running014h
kube-proxy-kbg9c1/1Running014h
kube-scheduler-k8s-master011/1Running114h
kube-scheduler-k8s-master021/1Running014h
kube-scheduler-k8s-master031/1Running014h
metrics-server-7c5546c5c5-ms4nz1/1Running025s

  过5分钟左右查看

[root@k8s-master01k8s-ha-install]#kubectltopnodes
NAMECPU(cores)CPU%MEMORY(bytes)MEMORY%k8s-master01155m3%1716Mi44%k8s-master02337m8%1385Mi36%k8s-master03450m11%1180Mi30%k8s-node01153m3%582Mi7%k8s-node02142m3%601Mi7%[root@k8s-master01k8s-ha-install]#kubectltoppod-nkube-system
NAMECPU(cores)MEMORY(bytes)
calico-node-49dwr15m71Mi
calico-node-9nmhb47m60Mi
calico-node-k5nmt46m61Mi
calico-node-kz2d418m47Mi
calico-node-zwnmq16m46Mi
coredns-89cc84847-dgxlw2m13Mi
coredns-89cc84847-n77x62m13Mi
etcd-k8s-master0127m126Mi
etcd-k8s-master0223m117Mi
etcd-k8s-master0319m112Mi
kube-apiserver-k8s-master0129m410Mi
kube-apiserver-k8s-master0219m343Mi
kube-apiserver-k8s-master0313m343Mi
kube-controller-manager-k8s-master0123m97Mi
kube-controller-manager-k8s-master021m16Mi
kube-controller-manager-k8s-master031m16Mi
kube-proxy-cl2zv18m18Mi
kube-proxy-f9qc58m20Mi
kube-proxy-hkcq530m19Mi
kube-proxy-k55bg8m20Mi
kube-proxy-kbg9c6m20Mi
kube-scheduler-k8s-master017m20Mi
kube-scheduler-k8s-master029m19Mi
kube-scheduler-k8s-master037m19Mi
metrics-server-7c5546c5c5-ms4nz3m14Mi

  部署dashboardv1.10.0

[root@k8s-master01k8s-ha-install]#kubectlcreate-fdashboard/secret/kubernetes-dashboard-certscreated
serviceaccount/kubernetes-dashboardcreated
role.rbac.authorization.k8s.io/kubernetes-dashboard-minimalcreated
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard-minimalcreated
deployment.apps/kubernetes-dashboardcreated
service/kubernetes-dashboardcreated

  查看pod和svc

[root@k8s-master01k8s-ha-install]#kubectlgetsvc-nkube-system
NAMETYPECLUSTER-IPEXTERNAL-IPPORT(S)AGE
calico-typhaClusterIP10.102.221.48
 
  5473/TCP15h
kube-dnsClusterIP10.96.0.10
  
   53/UDP,53/TCP15h kubernetes-dashboardNodePort10.105.18.61
   
    443:30000/TCP7s metrics-serverClusterIP10.101.178.115
    
     443/TCP23m [root@k8s-master01k8s-ha-install]#kubectlgetpo-nkube-system-lk8s-app=kubernetes-dashboard NAMEREADYSTATUSRESTARTSAGE kubernetes-dashboard-845b47dbfc-j4r481/1Running07m14s
    
   
  
 

  访问:https://192.168.20.10:30000/#!/login

kubeadm安装高可用简介

  查看令牌

[root@k8s-master01k8s-ha-install]#kubectl-nkube-systemdescribesecret$(kubectl-nkube-systemgetsecret|grepadmin-user|awk'{print$1}')
Name:admin-user-token-455bd
Namespace:kube-system
Labels:
 
  Annotations:kubernetes.io/service-account.name:admin-user
kubernetes.io/service-account.uid:e6effde6-1a0a-11e9-ae1a-000c298bf023

Type:kubernetes.io/service-account-token

Data====namespace:11bytes
token:eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLTQ1NWJkIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJlNmVmZmRlNi0xYTBhLTExZTktYWUxYS0wMDBjMjk4YmYwMjMiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06YWRtaW4tdXNlciJ9.Lw8hErqRoEC3e4VrEsAkFraytQI13NWj2osm-3lhaFDfgLtj4DIadq3ef8VgxpmyViPRzPh5fhq7EejuGH6V9cPsqEVlNBjWG0Wzfn0QuPP0xkxoW2V7Lne14Pu0-bTDE4P4UcW4MGPJAHSvckO9DTfYSzYghE2YeNKzDfhhA4DuWXaWGdNqzth_QjG_zbHsAB9kT3yVNM6bMVj945wZYSzXdJixSPBB46y92PAnfO0kAWsQc_zUtG8U1bTo7FdJ8BXgvNhytUvP7-nYanSIcpUoVXZRinQDGB-_aVRuoHHpiBOKmZlEqWOOaUrDf0DQJvDzt9TL-YHjimIstzv18A
ca.crt:1025bytes
 
喜欢 (0)
[]
分享 (0)
关于作者:
发表我的评论
取消评论
表情 贴图 加粗 删除线 居中 斜体 签到

Hi,您需要填写昵称和邮箱!

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址