kubernetes

搭建本地测试平台

安装

以ArchLinux为例, yay安装:

kubectl: 用于控制k8s集群的命令行工具
kubeadm: 用于快速搭建k8s集群的工具
kubelet: k8s集群中的节点服务
containerd: k8s容器运行时 (docker 的原生支持在 1.24 被移除了. 可恶, 本就不富裕的存储空间又被压榨了)
cilium-cli: k8s网络插件

运行时需要禁用交换分区sudo swapoff -a

初始化

在containerd.service服务启动后运行:

kubeadm init

~~大概等个十天半个月~~之后就生成好了初始化配置文件, 这个指令的具体内容可以在这里找到: kubernetes.io/zh-cn/docs/reference/…/kubeadm-init/

输出:

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.0.192:6443 --token egbtk9.j997hksrcbalsghu \
        --discovery-token-ca-cert-hash sha256:b0163d603cab4a01cf9add4132c4493b81b28486f4628dbe8b6d8e87c79a4c67

复制配置文件

根据提示输入:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

安装网络插件

然后安装网络插件cilium:

cilium-cli install

添加节点

接着按照上面的提示输入:

kubeadm join 192.168.0.192:6443 --token egbtk9.j997hksrcbalsghu \
        --discovery-token-ca-cert-hash sha256:b0163d603cab4a01cf9add4132c4493b81b28486f4628dbe8b6d8e87c79a4c67

如果显示文件已经存在可以直接删除对应的文件, 如果显示端口占用可以使用sudo systemctl restart kubelet重启服务.

如果一切顺利你就会看到这样的输出:

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

就代表已经成功连接到节点了.

官方文档的警告:
不要与任何人共享admin.conf和super-admin.conf, 如果要为其他用户生成配置文件, 请使用kubeadm kubeconfig user命令.

其他

安装 Kubernetes Dashboard :

# Add kubernetes-dashboard repository
helm repo add kubernetes-dashboard https://kubernetes.github.io/dashboard/
# Deploy a Helm Release named "kubernetes-dashboard" using the kubernetes-dashboard chart
helm upgrade --install kubernetes-dashboard kubernetes-dashboard/kubernetes-dashboard --create-namespace --namespace kubernetes-dashboard

其中helm是k8s包管理器, 可以用来安装各种插件, 具体可以看他的官网: helm.sh/zh
输出:

Release "kubernetes-dashboard" does not exist. Installing it now.
NAME: kubernetes-dashboard
LAST DEPLOYED: Wed Mar 19 21:32:00 2025
NAMESPACE: kubernetes-dashboard
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
*************************************************************************************************
*** PLEASE BE PATIENT: Kubernetes Dashboard may need a few minutes to get up and become ready ***
*************************************************************************************************

Congratulations! You have just installed Kubernetes Dashboard in your cluster.

To access Dashboard run:
  kubectl -n kubernetes-dashboard port-forward svc/kubernetes-dashboard-kong-proxy 8443:443

NOTE: In case port-forward command does not work, make sure that kong service name is correct.
      Check the services in Kubernetes Dashboard namespace using:
        kubectl -n kubernetes-dashboard get svc

Dashboard will be available at:
  https://localhost:8443

可以使用kubectl -n kubernetes-dashboard port-forward svc/kubernetes-dashboard-kong-proxy 8443:443来启动dashboard.
但是, 如果不出意外的话, 就要出意外了:

kubectl -n kubernetes-dashboard port-forward svc/kubernetes-dashboard-kong-proxy 8443:443
error: unable to forward port because pod is not running. Current status=Pending

查看集群事件就会发现:

kubectl get events -n kubernetes-dashboard
LAST SEEN   TYPE      REASON              OBJECT                                                       MESSAGE
78s         Warning   FailedScheduling    pod/kubernetes-dashboard-api-699b67d7fc-g9jn5                0/1 nodes are available: 1 node(s) had untolerated taint {node-role.kubernetes.io/control-plane: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling.
6m26s       Normal    SuccessfulCreate    replicaset/kubernetes-dashboard-api-699b67d7fc               Created pod: kubernetes-dashboard-api-699b67d7fc-g9jn5
6m26s       Normal    ScalingReplicaSet   deployment/kubernetes-dashboard-api                          Scaled up replica set kubernetes-dashboard-api-699b67d7fc from 0 to 1
78s         Warning   FailedScheduling    pod/kubernetes-dashboard-auth-665595885c-vnm6d               0/1 nodes are available: 1 node(s) had untolerated taint {node-role.kubernetes.io/control-plane: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling.
6m26s       Normal    SuccessfulCreate    replicaset/kubernetes-dashboard-auth-665595885c              Created pod: kubernetes-dashboard-auth-665595885c-vnm6d
6m26s       Normal    ScalingReplicaSet   deployment/kubernetes-dashboard-auth                         Scaled up replica set kubernetes-dashboard-auth-665595885c from 0 to 1
78s         Warning   FailedScheduling    pod/kubernetes-dashboard-kong-79867c9c48-b4j2z               0/1 nodes are available: 1 node(s) had untolerated taint {node-role.kubernetes.io/control-plane: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling.
6m26s       Normal    SuccessfulCreate    replicaset/kubernetes-dashboard-kong-79867c9c48              Created pod: kubernetes-dashboard-kong-79867c9c48-b4j2z
6m26s       Normal    ScalingReplicaSet   deployment/kubernetes-dashboard-kong                         Scaled up replica set kubernetes-dashboard-kong-79867c9c48 from 0 to 1
78s         Warning   FailedScheduling    pod/kubernetes-dashboard-metrics-scraper-84655b9bd8-58nxp    0/1 nodes are available: 1 node(s) had untolerated taint {node-role.kubernetes.io/control-plane: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling.
6m26s       Normal    SuccessfulCreate    replicaset/kubernetes-dashboard-metrics-scraper-84655b9bd8   Created pod: kubernetes-dashboard-metrics-scraper-84655b9bd8-58nxp
6m26s       Normal    ScalingReplicaSet   deployment/kubernetes-dashboard-metrics-scraper              Scaled up replica set kubernetes-dashboard-metrics-scraper-84655b9bd8 from 0 to 1
78s         Warning   FailedScheduling    pod/kubernetes-dashboard-web-658946f7f9-jp8nn                0/1 nodes are available: 1 node(s) had untolerated taint {node-role.kubernetes.io/control-plane: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling.
6m26s       Normal    SuccessfulCreate    replicaset/kubernetes-dashboard-web-658946f7f9               Created pod: kubernetes-dashboard-web-658946f7f9-jp8nn
6m26s       Normal    ScalingReplicaSet   deployment/kubernetes-dashboard-web                          Scaled up replica set kubernetes-dashboard-web-658946f7f9 from 0 to 1

因为集群只有一个控制平面节点(主节点), 默认情况下k8s不会在控制平面节点上运行工作负载(出于安全考虑).
如果想要在控制平面节点上运行工作负载, 需要移除控制平面节点上的污点(taint).
注意, 在开发环境中这样做没有任何影响, 但是在生产环境中这样做可能会导致安全问题, 应该使用专门的工作节点来运行工作负载.

kubectl taint nodes --all node-role.kubernetes.io/control-plane-

然而, 这样做之后, 还是不行, 再次运行kubectl get pods -n kubernetes-dashboard, 发现所有的pod都还是Pending状态...
使用kubectl describe node echo | grep Taints查看节点的污点, 发现了一个新的污点:

Taints:             node.kubernetes.io/disk-pressure:NoSchedule

这个污点是由kubelet自动添加的, 用于标记节点的磁盘压力. 所以只是我电脑的磁盘空间不够了...(上面写的竟然还成了伏笔!)
不管他, 反正只是测试, 直接删除这个污点:

❯ kubectl taint nodes echo node.kubernetes.io/disk-pressure:NoSchedule-
node/echo untainted
❯ kubectl taint nodes echo node.kubernetes.io/disk-pressure:NoSchedule-
node/echo untainted

还删不掉...
手动清理了快 30G 的 docker 缓存之后等了几分钟才好.

使用 miniKube

装完了才发现有这么个东西...
文档里怎么有了个"入门"还有个"教程"
不过使用minikube之后, 之前创建的配置好像就被覆盖了, 只剩下一个叫minikube的 node

使用yay安装minikube
运行minikube start下载镜像并创建配置
运行minikube dashboard打开控制面板网页
运行示例镜像kubectl create deployment hello-node --image=registry.k8s.io/e2e-test-images/agnhost:2.39 -- /agnhost netexec --http-port=8080
创建暴露端口的服务kubectl expose deployment hello-node --type=LoadBalancer --port=8080
运行服务minikube service hello-node
列出插件minikube addons list
启用插件minikube addons enable metrics-server
清理kubectl delete service hello-node和kubectl delete deployment hello-node
停止集群minikube stop
删除 k8s 虚拟机minikube delete

这样体验下来 k8s 比 docker 麻烦的不是一点半点啊

概念

Kubernetes 集群

Kubernetes 集群是一组由计算机组成的集群, 通过 Kubernetes 进行协调, 使每个计算机作为独立单元相互连接工作. 允许你将容器化的应用部署到集群.

一个 Kubernetes 包含两种资源:

控制面(Control Plane): 负责调度整个集群
节点(Nodes): 负责运行应用, 同时每个节点都需要安装 Kubelet 负责节点与控制面的通信. 一般还需要安装一个容器运行时, 负责运行容器化的应用.

pod

pod是 k8s 中的最小部署单元, 可以是单个容器, 也可以是多个容器共同构成. (类似于docker compose?)