第六章：Kubernetes 容器编排
最后更新: 2024-01-01
作者: DevOps Team
页面目录
第六章：Kubernetes 容器编排
Kubernetes (K8s) 是容器编排的事实标准，本章将深入讲解 K8s 的架构、核心资源对象以及在生产环境中的部署和管理实践。
6.1 Kubernetes 架构
6.1.1 集群架构
┌─────────────────────────────────────────────────────────────────┐
│ Kubernetes 集群 │
│ │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ Control Plane │ │
│ │ │ │
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌────────┐ │ │
│ │ │ kube- │ │ kube- │ │ etcd │ │ cloud- │ │ │
│ │ │ apiserver│ │ scheduler│ │ │ │ controller│ │ │
│ │ └──────────┘ └──────────┘ └──────────┘ └────────┘ │ │
│ │ │ │
│ └───────────────────────────────────────────────────────────┘ │
│ │ │
│ │ │
│ ┌───────────────────────────┼───────────────────────────────┐ │
│ │ Worker Nodes │ │
│ │ │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
│ │ │ Node 1 │ │ Node 2 │ │ Node 3 │ │ │
│ │ │ ┌───────┐ │ │ ┌───────┐ │ │ ┌───────┐ │ │ │
│ │ │ │ Pod │ │ │ │ Pod │ │ │ │ Pod │ │ │ │
│ │ │ │ (Nginx) │ │ │ │ (App) │ │ │ │ (DB) │ │ │ │
│ │ │ └───────┘ │ │ └───────┘ │ │ └───────┘ │ │ │
│ │ │ ┌─────────┐│ │ ┌─────────┐│ │ ┌─────────┐│ │ │
│ │ │ │ kubelet ││ │ │ kubelet ││ │ │ kubelet ││ │ │
│ │ │ │ kube- ││ │ │ kube- ││ │ │ kube- ││ │ │
│ │ │ │ proxy ││ │ │ proxy ││ │ │ proxy ││ │ │
│ │ │ │ containerd││ │ │ containerd││ │ │ containerd││ │ │
│ │ │ └─────────┘│ │ └─────────┘│ │ └─────────┘│ │ │
│ │ └─────────────┘ └─────────────┘ └─────────────┘ │ │
│ └───────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
6.1.2 Control Plane 组件
| 组件 | 功能 |
|---|---|
| kube-apiserver | 集群 API 网关，处理所有 REST 操作 |
| etcd | 分布式键值存储，保存集群所有数据 |
| kube-scheduler | 调度 Pod 到合适的 Node |
| kube-controller-manager | 运行控制器进程 (Node、Replica、Endpoint 等) |
| cloud-controller-manager | 与云提供商交互的控制器 |
6.1.3 Node 组件
| 组件 | 功能 |
|---|---|
| kubelet | 代理，负责在 Node 上创建/删除 Pod |
| kube-proxy | 网络代理，维护网络规则 |
| container runtime | 容器运行时 (containerd, CRI-O) |
6.2 核心资源对象
6.2.1 Pod 详解
# basic-pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: myapp-pod
labels:
app: myapp
version: v1
namespace: default
spec:
# 容器配置
containers:
- name: myapp
image: myapp:v1.0.0
imagePullPolicy: Always # Always/IfNotPresent/Never
ports:
- name: http
containerPort: 8080
protocol: TCP
env:
- name: NODE_ENV
value: production
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: db-credentials
key: url
resources:
requests:
memory: "128Mi"
cpu: "100m"
limits:
memory: "256Mi"
cpu: "500m"
livenessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
httpGet:
path: /ready
port: 8080
initialDelaySeconds: 5
periodSeconds: 5
successThreshold: 1
failureThreshold: 3
startupProbe:
httpGet:
path: /health
port: 8080
failureThreshold: 30
periodSeconds: 10
lifecycle:
preStop:
exec:
command: ["/bin/sh", "-c", "sleep 10"]
volumeMounts:
- name: app-logs
mountPath: /app/logs
- name: config
mountPath: /app/config
readOnly: true
volumes:
- name: app-logs
emptyDir: {}
- name: config
configMap:
name: app-config
# 调度策略
nodeSelector:
disktype: ssd
region: us-east-1
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
preference:
matchExpressions:
- key: node-type
operator: In
values:
- compute-optimized
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchLabels:
app: myapp
topologyKey: kubernetes.io/hostname
tolerations:
- key: "node-type"
operator: "Equal"
value: "gpu"
effect: "NoSchedule"
# DNS 配置
dnsPolicy: ClusterFirst
hostname: myapp
subdomain: myapp.ns
restartPolicy: Always
6.2.2 Deployment 详解
# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp-deployment
labels:
app: myapp
spec:
# 副本数
replicas: 3
# 选择器
selector:
matchLabels:
app: myapp
# 更新策略
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
# Pod 模板
template:
metadata:
labels:
app: myapp
version: v1
spec:
containers:
- name: myapp
image: myapp:v1.0.0
ports:
- containerPort: 8080
name: http
resources:
requests:
memory: "128Mi"
cpu: "100m"
limits:
memory: "256Mi"
cpu: "500m"
livenessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 10
periodSeconds: 10
readinessProbe:
httpGet:
path: /ready
port: 8080
initialDelaySeconds: 5
periodSeconds: 5
# 优雅终止
terminationGracePeriodSeconds: 60
# 终止前等待
preStop:
exec:
command: ["/bin/sh", "-c", "sleep 10"]
6.2.3 Service 详解
# service.yaml
apiVersion: v1
kind: Service
metadata:
name: myapp-service
labels:
app: myapp
spec:
# Service 类型
type: ClusterIP # ClusterIP/NodePort/LoadBalancer/ExternalName
# 服务端口
ports:
- name: http
port: 80
targetPort: 8080
protocol: TCP
- name: grpc
port: 50051
targetPort: 50051
protocol: TCP
# 选择后端 Pod
selector:
app: myapp
# 流量策略
sessionAffinity: ClientIP
sessionAffinityConfig:
clientIP:
timeoutSeconds: 10800
---
# Headless Service (无头服务)
apiVersion: v1
kind: Service
metadata:
name: myapp-headless
spec:
type: ClusterIP
clusterIP: None
ports:
- port: 80
targetPort: 8080
selector:
app: myapp
---
# NodePort Service
apiVersion: v1
kind: Service
metadata:
name: myapp-nodeport
spec:
type: NodePort
ports:
- port: 80
targetPort: 8080
nodePort: 30080
selector:
app: myapp
---
# LoadBalancer Service
apiVersion: v1
kind: Service
metadata:
name: myapp-lb
spec:
type: LoadBalancer
loadBalancerIP: 1.2.3.4
loadBalancerSourceRanges:
- 10.0.0.0/8
ports:
- port: 80
targetPort: 8080
selector:
app: myapp
6.2.4 Ingress 配置
# ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myapp-ingress
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
nginx.ingress.kubernetes.io/proxy-body-size: "10m"
nginx.ingress.kubernetes.io/proxy-connect-timeout: "30"
nginx.ingress.kubernetes.io/proxy-read-timeout: "60"
nginx.ingress.kubernetes.io/proxy-send-timeout: "60"
cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
ingressClassName: nginx
tls:
- hosts:
- myapp.example.com
secretName: myapp-tls
rules:
- host: myapp.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: myapp-service
port:
number: 80
- path: /api
pathType: Prefix
backend:
service:
name: api-service
port:
number: 8080
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /api/$2
6.3 ConfigMap 和 Secret
6.3.1 ConfigMap
# configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: app-config
data:
# 键值对
DATABASE_HOST: "postgres.default.svc"
REDIS_HOST: "redis.default.svc"
LOG_LEVEL: "info"
# 配置文件
nginx.conf: |
server {
listen 80;
server_name localhost;
location / {
root /usr/share/nginx/html;
index index.html;
}
}
# JSON 配置
config.json: |
{
"database": {
"host": "postgres",
"port": 5432
}
}
---
# 使用方式
apiVersion: v1
kind: Pod
spec:
containers:
- name: app
envFrom:
- configMapRef:
name: app-config
env:
- name: SECRET_KEY
valueFrom:
configMapKeyRef:
name: app-config
key: secret.key
6.3.2 Secret
# secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: app-secrets
type: Opaque
stringData:
DATABASE_PASSWORD: "supersecretpassword"
API_KEY: "api-key-value"
TLS_CERT: |
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
TLS_KEY: |
-----BEGIN PRIVATE KEY-----
...
-----END PRIVATE KEY-----
---
# Docker Registry Secret
kubectl create secret docker-registry my-registry \
--docker-server=registry.example.com \
--docker-username=user \
--docker-password=pass \
--docker-email=user@example.com
---
# TLS Secret
kubectl create secret tls my-tls \
--cert=cert.pem \
--key=key.pem
6.4 存储管理
6.4.1 PersistentVolume 和 PersistentVolumeClaim
# persistentvolume.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-volume
labels:
type: local
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: standard
hostPath:
path: "/mnt/data"
---
# persistentvolumeclaim.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc-storage
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
storageClassName: standard
selector:
matchLabels:
type: local
---
# 使用 PVC
apiVersion: v1
kind: Pod
spec:
containers:
- name: app
volumeMounts:
- name: persistent-storage
mountPath: /app/data
volumes:
- name: persistent-storage
persistentVolumeClaim:
claimName: pvc-storage
6.4.2 StatefulSet
# statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres
spec:
serviceName: postgres
replicas: 3
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: postgres
image: postgres:15
env:
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: postgres-secret
key: password
volumeMounts:
- name: data
mountPath: /var/lib/postgresql/data
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: "standard"
resources:
requests:
storage: 10Gi
6.5 Helm 图表管理
6.5.1 Helm 基础
# 安装 Helm
curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
# 添加仓库
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
# 搜索图表
helm search repo nginx
helm search hub nginx
# 安装图表
helm install my-release bitnami/nginx
helm install my-release bitnami/nginx --namespace mynamespace --create-namespace
# 升级和回滚
helm upgrade my-release bitnami/nginx
helm rollback my-release 1
# 释放
helm uninstall my-release
6.5.2 自定义 Helm 图表
# 创建图表
helm create mychart
# 图表结构
# mychart/
# ├── Chart.yaml
# ├── values.yaml
# ├── charts/
# ├── templates/
# │ ├── NOTES.txt
# │ ├── _helpers.tpl
# │ ├── deployment.yaml
# │ ├── service.yaml
# │ └── ingress.yaml
# └── .helmignore
# Chart.yaml
apiVersion: v2
name: mychart
description: A Helm chart for My Application
type: application
version: 1.0.0
appVersion: "1.0.0"
keywords:
- myapp
- web
home: https://myapp.example.com
sources:
- https://github.com/myorg/myapp
maintainers:
- name: DevOps Team
email: devops@example.com
dependencies:
- name: postgresql
version: "12.x.x"
repository: "https://charts.bitnami.com/bitnami"
condition: postgresql.enabled
# values.yaml
replicaCount: 3
image:
repository: myapp
tag: "latest"
pullPolicy: IfNotPresent
service:
type: ClusterIP
port: 80
ingress:
enabled: true
className: nginx
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
hosts:
- host: myapp.example.com
paths:
- path: /
pathType: Prefix
tls:
- secretName: myapp-tls
hosts:
- myapp.example.com
resources:
limits:
cpu: 1000m
memory: 1Gi
requests:
cpu: 100m
memory: 256Mi
autoscaling:
enabled: true
minReplicas: 2
maxReplicas: 10
targetCPUUtilizationPercentage: 70
postgresql:
enabled: true
auth:
database: myapp
username: myapp
# templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "mychart.fullname" . }}
labels:
{{- include "mychart.labels" . | nindent 4 }}
spec:
replicas: {{ .Values.replicaCount }}
selector:
matchLabels:
{{- include "mychart.selectorLabels" . | nindent 6 }}
template:
metadata:
labels:
{{- include "mychart.selectorLabels" . | nindent 8 }}
spec:
containers:
- name: {{ .Chart.Name }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
ports:
- name: http
containerPort: {{ .Values.service.port }}
protocol: TCP
livenessProbe:
httpGet:
path: /health
port: http
readinessProbe:
httpGet:
path: /ready
port: http
resources:
{{- toYaml .Values.resources | nindent 12 }}
6.6 本章小结
| 资源对象 | 用途 |
|---|---|
| Pod | 最小部署单元，包含一个或多个容器 |
| Deployment | 管理 Pod 副本数和更新策略 |
| Service | 为 Pod 提供稳定的访问入口 |
| Ingress | HTTP/HTTPS 路由和负载均衡 |
| ConfigMap/Secret | 配置管理和敏感信息存储 |
| PersistentVolume | 持久化存储 |
| StatefulSet | 有状态应用管理 |
| Helm | 应用打包和版本管理 |
📌 下一章预告
下一章我们将学习 基础设施即代码 (IaC)，包括：
- Terraform 基础和使用
- Ansible 配置管理
- Infrastructure as Code 最佳实践
💡 提示：使用 Helm 打包应用时，应将 values.yaml 中的默认值设置为生产环境友好，并在 CI/CD 中使用自定义 values 文件覆盖。