kubernetes-scheduling

Manual Scheduling#

pod spec#

通过指定Pod的spec.nodeName字段，将Pod调度到指定Node上, 跳过scheduler直接调度

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  nodeName: my-node # 手动指定Node
  containers:
    - name: my-container
      image: nginx

Labels and Selectors#

kubectl get pods -l label-key=label-value # 通过标签选择Pod
kubectl run my-pod --image=nginx --labels="app=my-app" # 创建Pod时添加标签

apiVersion: v1
kind: ReplicaSet
metadata:
  name: my-replicaset
spec:
  replicas: 3
  selector: # 选择器
    matchLabels:
      app: my-app
  template:
    metadata:
      labels: # Pod标签
        app: my-app
    spec:
      containers:
        - name: my-container
          image: nginx

Taints and Tolerations#

Node level, 通过taints和tolerations控制Pod调度是否调度到该Node

Taints#

kubectl taint nodes node-name key=value:taint-effect

taint-effect:

NoSchedult: pod 不会被调度到该节点
PreferNoSchedlue: 尽量不调度
NoExecute: pod不调度，已存在的pod会被驱逐

Tolerations#

Pod level, 容忍node中对应的taint

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
  - name: nginx-contianer
    image: nginx
  tolerations:
    - key: "key"
      operator: "Equal"
      value: "value"
      effect: "taint-effect"

实例，通过taint，scheduler不会将pod调度到master node

$ kubectl describe node kubemaster | grep Taints
Taints:             node-role.kubernetes.io/control-plane:NoSchedule

DaemonSets#

保证每个Node上都有一个Pod副本, node创建时自动创建,适用于日志收集，监控等场景

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: my-daemonset
spec:
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
        - name: my-container
          image: nginx

Static Pods#

直接由kubelet控制的pod，不受scheduler控制，适用于kubelet启动前的pod, 对kubectl只读，无法更改或删除

static pod config目录：

/etc/kubernetes/manifests/(默认)
--pod-manifest-path指定目录
--config指定配置文件

查看static pod (假设control-plane无api-server,无法使用kubectl)

nerdctl ps # containerd

修改或删除static pod：

进入static pod所在node
找到static pod的manifest文件，修改或删除

Multiple Schedulers#

通过--scheduler-name指定scheduler名称，多个scheduler共存

# my-scheduler-1-config.yaml
apiVersion: kubescheduler.config.k8s.io/v1
kind: KubeSchedulerConfiguration
profile: 
  - schedulerName: 
      my-scheduler-1
leaderElection:  # leader选举 - 高可用选项
  leaderElect: true
  resourceNamespace: kube-system
  resourceName: lock-object-myscheduler

# 启动scheduler
ExecStart=/usr/local/bin/kubescheduler \
--config=my-scheduler-1-config.yaml

# 在Pod中指定scheduler
apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  schedulerName: my-scheduler-1 # 指定scheduler
  containers:
    - name: my-container
      image: nginx

Scheduler Profile Configuration#

Scheduling Queue -> Filtering -> Scoring -> Binding

Priority#

影响Scheduling Queue中Pod的优先级
通过Priority字段设置Pod的优先级，scheduler根据优先级调度Pod
优先级范围：0-100
默认优先级：0
PriorityClassName指定优先级类别

apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: high-priority
value: 100000
globalDefault: false
description: "This priority class should be used for XYZ service pods only."

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  priorityClassName: high-priority
  containers:
    - name: my-container
      image: nginx

Resource Requests and Limits#

影响Filtering和Scoring：
- Filtering: 根据Pod的资源需求过滤Node
- Scoring: 根据Pod的资源需求和限制评分(假设Pod部署后的剩余资源)
通过requests和limits字段设置Pod的资源需求和限制