Pod Group Disruption and Priority

PodGroup can declare a disruption mode. This mode dictates how the scheduler can disrupt a running PodGroup, for example to accommodate a higher priority PodGroup. A PodGroup also has a priority, which overrides the priority of the individual pods from the group for workload-aware preemption events.

Disruption mode types

The API supports two disruption modes: Pod and PodGroup. The default one is Pod.

Pod

The Pod mode instructs the scheduler to treat all Pods in the group as separate entities, allowing independent disruption of a single pod from a PodGroup.

PodGroup

The PodGroup mode emphasizes "all-or-nothing" semantics for disruption. It instructs the scheduler that all pods from the PodGroup have to be disrupted together.

CompositePodGroup

A CompositePodGroup can also declare a disruptionMode in its specification, which controls how the scheduler disrupts child groups within the composite group during preemption events.

The API supports two disruption modes for CompositePodGroups:

• Single: Allows individual child groups within the CompositePodGroup to be disrupted independently during preemption.
• All: Enforces all-or-nothing disruption semantics across the CompositePodGroup hierarchy. If any Pod contained in the hierarchy below this CompositePodGroup has to be preempted, all of the Pods from the entire hierarchy must be preempted.

If not specified, the mode defaults to Single.

Pod group priority

PodGroup uses the same concept of PriorityClass as single Pods. Once you have created one or more PriorityClasses, you can create a PodGroup that specifies one of those PriorityClass names in its specification. The priority admission controller uses the priorityClassName field and populates the integer value of the priority. If the priority class is not found, the PodGroup is rejected. When priorityClassName is not set for a PodGroup, Kubernetes looks for a default (a PriorityClass with globalDefault set true) If there is no PriorityClass with globalDefault set true, a PodGroup with no priorityClassName has priority zero.

The priority of the PodGroup is an authoritative priority for all pods in the group during workload-aware preemption events, even when priorities of individual pods forming this PodGroup differ.

The following YAML is an example of a PodGroup configuration that uses the high-priority PriorityClass, which maps to the integer priority value of 1000000. The priority admission controller checks the specification and resolves the priority of the PodGroup to 1000000.

apiVersion: scheduling.k8s.io/v1alpha2
kind: PodGroup
metadata:
  namespace: ns-1
  name: job-1
spec:
  priorityClassName: high-priority

CompositePodGroup priority

CompositePodGroup API has priorityClassName and priority fields as well and their resolution is performed in the same way as for the PodGroups, through the priority admission controller.

The priority of a root CompositePodGroup acts as the authoritative priority for all child groups and Pods within its hierarchy during workload-aware preemption events. All Pods within a single group hierarchy must share the exact same priority and must be equal to the priority of the root CompositePodGroup.

The value of priority is also used for the ordering of root CompositePodGroups in the scheduling active queue.

PreemptionPolicy in CompositePodGroup

The CompositePodGroup API has the preemptionPolicy field as well and its resolution is performed in the exact same way as for the PodGroup API.

The value of preemptionPolicy of the root CompositePodGroup determines whether workload-aware preemption can be invoked to fit its Pods during scheduling if needed:

• PreemptLowerPriority policy allows preempting victims with lower priority,
• Never policy disables workload-aware preemption for that root CompositePodGroup.

All Pods within a single group hierarchy must share the exact same preemption policy which must be equal to the preemption policy of the root CompositePodGroup.

If the feature flag is disabled, the root CompositePodGroup will be allowed to perform preemption unless one of the Pods that belongs to the group hierarchy has preemptionPolicy set to Never.

What's next

• Read about Workload-Aware Preemption algorithm.
• Learn about the Workload API.
• Learn about the scheduling building blocks and the workloadbuilder library, including the disruption mode building block.