ClusterClass Variable Reference

Understanding Variables in VKS

VKS uses the Cluster API Cluster objects to expose the configuration of VKS workload clusters. In the Cluster API (CAPI) ecosystem, variables are the primary mechanism for customizing the configuration of a cluster. These variables are defined within a ClusterClass, which acts as a reusable blueprint or template for clusters.

The ClusterClass defines the schema including variable names, types (e.g., string, integer, boolean), and default values. When creating a workload cluster, these values need to be provided which are then used to actualize the workload cluster. These variables can be specified at the Cluster level or can be overridden at the control plane or a per node-pool level.

Variable Hierarchy and Overrides

CAPI employs a "most-specific-wins" inheritance model. This allows you to set broad defaults at the cluster level while fine-tuning the configuration for specific components like the control plane or individual node pools. When a variable is overridden at a particular level, this override takes precedence over the value at the Cluster level.

Global Cluster Level

Variables defined at the spec.topology.variables level of the Cluster object apply to the entire cluster.

Purpose: Establishes the baseline configuration.

Scope: These values are inherited by all nodes (both control plane and workers) unless a more specific override is provided.

Control Plane Level

The control plane can have its own specific overrides located under spec.topology.controlPlane.variables.

Purpose: Tailors the infrastructure specifically for the API server, etcd, and scheduler.

Override Logic: Values set here supersede the Global Cluster values but only apply to the control plane nodes.

Node Pool (MachineDeployment) Level

Node Pools represent your worker nodes. Since a cluster can have multiple pools for different workloads (e.g., GPU-optimized vs. general purpose), variables can be overridden at the individual pool level under spec.topology.workers.machineDeployments[].variables.

Purpose: Provides granular control over specific groups of worker nodes.

Override Logic: This is the highest level of specificity. A value set here overrides both the Global defaults and any Control Plane settings for that specific pool.

Variable availability across VKS versions

The following table shows which variables and their nested properties are available in each VKS version.

Property Path3.2.03.3.03.4.03.5.03.6.03.7.0

bootstrapAddons✓✓

kubeAPIServerFQDNs✓✓✓✓✓✓

kubernetes✓✓✓✓✓✓

apiServerConfiguration✓✓✓

extraAuthentication✓

logs✓✓✓

maxMutatingRequestsInFlight✓✓✓

maxRequestsInFlight✓✓✓

oidc✓

profiling✓✓✓

requestTimeout✓✓✓

certificateRotation✓✓✓✓✓✓

endpointFQDNs✓✓✓✓✓✓

etcdConfiguration✓✓✓

maxRequestSizeKiB✓

maximumDBSizeGiB✓✓✓

kubeControllerManagerConfiguration✓✓

kubeProxyConfiguration✓✓

enabled✓

kubeletConfiguration✓✓✓

allowedUnsafeSysctls✓✓✓

containerLogMaxFiles✓✓✓

containerLogMaxSizeMiB✓✓✓

eventBurst✓✓✓

eventRecordQPS✓✓✓

healthzBindAddress✓✓✓

imageGCHighThresholdPercent✓✓✓

imageGCLowThresholdPercent✓✓✓

imageMaximumGCAge✓✓✓

imageMinimumGCAge✓✓✓

imagePullCredentialsVerificationPolicy✓✓✓

logging✓✓✓

maxParallelImagePulls✓✓✓

maxPods✓✓✓

podPidsLimit✓✓

preloadedImagesVerificationAllowlist✓✓✓

registryBurst✓✓✓

registryPullQPS✓✓✓

serializeImagePulls✓✓✓

streamingConnectionIdleTimeout✓✓✓

security✓✓✓✓✓✓

minimumTLSProtocol✓

podSecurityStandard✓✓✓✓✓✓

resourceQuotaConfiguration✓✓✓

tlsCipherSuites✓

networks✓✓

interfaces✓✓

primary✓✓

secondary✓✓

driver✓

mtu✓✓

name✓✓

network✓✓

routes✓✓

sriovResourcePool✓

node✓✓✓✓✓✓

cri✓

firewall✓✓

labels✓✓✓✓✓✓

taints✓✓✓✓✓✓

osConfiguration✓✓✓✓✓✓

directoryJoin✓✓✓✓✓

fips✓✓✓✓✓

grub✓✓✓

password✓✓✓

enabled✓✓✓

secretRef✓✓✓

user✓✓

immutability✓

ntp✓✓✓✓✓✓

securityContext✓✓

sshd✓✓✓✓✓

systemProxy✓✓✓✓✓✓

trust✓✓✓✓✓✓

tuned✓✓

ubuntuPro✓✓✓✓✓

user✓✓✓✓✓✓

password✓✓✓✓

passwordSecret✓✓✓✓✓✓

requirePasswordOnSudo✓✓✓

sshAuthorizedKey✓✓✓✓✓✓

user✓✓✓✓✓✓

resourceConfiguration✓✓✓✓✓✓

storageClass✓✓✓✓✓✓

vmClass✓✓✓✓✓✓

volumes✓✓✓✓✓✓

vsphereOptions✓✓✓✓✓✓

persistentVolumes✓✓✓✓✓✓

availableStorageClasses✓✓✓✓✓✓

availableVolumeSnapshotClasses✓✓✓✓✓✓

customizableStorageClassAnnotations✓✓✓✓

customizableStorageClassLabels✓✓✓✓

defaultStorageClass✓✓✓✓✓✓

defaultVolumeSnapshotClass✓✓✓✓✓✓

TKR_DATA✓✓✓

Variables

bootstrapAddons

BootstrapAddons defines Addons to be installed on Cluster during bootstrapping. Only supported with Kubernetes 1.35 and above.

Introduced in version: 3.6.0

Required: No

Example:

cniRef:
  name: cni-test
  namespace: cni-namespace-test

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`cniRef` object	CNIRef defines the CNI Addon to instantiate for Cluster. It is used to select CNI rather than ClusterBootstrap spec.CNI field. Compatible Addon/AddonRelease must exist.	Yes	3.6.0

cniRef

CNIRef defines the CNI Addon to instantiate for Cluster. It is used to select CNI rather than ClusterBootstrap spec.CNI field. Compatible Addon/AddonRelease must exist.

Introduced in version: 3.6.0

Required: Yes

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`name` string	Name of the Addon being referenced.	Yes	3.6.0
`namespace` string	Namespace of the addon being referenced. If not specified, will use the default public namespace defined by the addon manager.	No	3.6.0

kubeAPIServerFQDNs

Deprecated: This variable is deprecated. Use kubernetes.endpointFQDNs instead. This variable will be removed in a future release.

Introduced in version: 3.2.0

Required: No

Type: []string

kubernetes

Kubernetes configures cluster-wide settings for the Kubernetes cluster, typically applied to the control plane.

Introduced in version: 3.2.0

Required: No

Example:

apiServerConfiguration:
  extraAuthentication:
    jwt:
    - claimMappings:
        groups: {}
        uid: {}
        username:
          claim: email
          prefix: 'test-prefix:'
      issuer:
        audiences:
        - test-audience
        url: https://issuer.example.com
  logs:
    flushFrequency: 10s
    format: json
    verbosity: 5
  maxMutatingRequestsInFlight: 100
  maxRequestsInFlight: 200
  oidc:
    serviceAccountIssuerURL: https://oidc-issuer.example.com
  profiling: false
  requestTimeout: 30s
certificateRotation:
  enabled: true
  renewalDaysBeforeExpiry: 99
endpointFQDNs:
- fqdn.test1
- fqdn.test2
etcdConfiguration:
  maxRequestSizeKiB: 10240
  maximumDBSizeGiB: 5
kubeControllerManagerConfiguration:
  terminatedPodGCThreshold: 6789
kubeletConfiguration:
  allowedUnsafeSysctls:
  - kernel.msg*
  - net.ipv4.route.min_pmtu
  containerLogMaxFiles: 20
  containerLogMaxSizeMiB: 10
  eventBurst: 20
  eventRecordQPS: 20
  healthzBindAddress: 0.0.0.0
  imageGCHighThresholdPercent: 20
  imageGCLowThresholdPercent: 10
  imageMaximumGCAge: 5s
  imageMinimumGCAge: 1s
  imagePullCredentialsVerificationPolicy: AlwaysVerify
  logging:
    flushFrequency: 10s
    format: json
    verbosity: 5
  maxParallelImagePulls: 20
  maxPods: 200
  podPidsLimit: 66
  preloadedImagesVerificationAllowlist:
  - registry.example.com/myimage
  - my.registry.io/*
  registryBurst: 12
  registryPullQPS: 6
  serializeImagePulls: true
  streamingConnectionIdleTimeout: 10s
security:
  podSecurityStandard:
    audit: privileged
    deactivated: false
    enforce: restricted
    warn: baseline
  resourceQuotaConfiguration:
    enabled: true

Field	Description	Supported Scopes	Required	Since
`apiServerConfiguration` object	APIServerConfiguration contains configuration options for the Kubernetes API server. These settings control API server behavior including request limits, audit logging, admission control, authentication, and event retention. Validation: sum of `maxRequestsInFlight` and `maxMutatingRequestsInFlight` must be positive Changes require a machine rollout.	cluster, controlPlane	No	3.5.0
`certificateRotation` object	CertificateRotation configures options for the automatic rotation of control plane certificates which have a default validity of 12 months. Note that certificates are always rotated during Kubernetes upgrades. Changes require a machine rollout.	cluster, controlPlane	No	3.2.0
`endpointFQDNs` []string	EndpointFQDNs Configure FQDN aliases for the control plane endpoint for example to allow users to connect to the cluster using https://k8s.prod.example.com/	cluster, controlPlane	No	3.2.0
`etcdConfiguration` object	EtcdConfiguration contains configuration options for the etcd database used by Kubernetes. These settings control etcd behavior including database size limits and performance tuning. Changes require a machine rollout.	cluster, controlPlane	No	3.5.0
`kubeControllerManagerConfiguration` object	KubeControllerManagerConfiguration contains configuration options for the kube-controller-manager. Changes require a machine rollout.	cluster, controlPlane	No	3.6.0
`kubeProxyConfiguration` object	KubeProxyConfiguration contains configuration options for kube-proxy component. KubeProxy subscribes to services and endpoints, and programs the network data plane on all the linux nodes for load balancing service traffic. These settings control the behaviour of kube-proxy. Validation: `kubeProxyConfiguration` is immutable Changes require a machine rollout.	cluster, controlPlane	No	3.6.0
`kubeletConfiguration` object	KubeletConfiguration contains configuration options for the kubelet running on worker nodes. These settings control various aspects of kubelet behavior including resource limits, image management, logging, and security policies. Validation: `imageGCLowThresholdPercent` must be < `imageGCHighThresholdPercent` when both are specified `registryBurst` must be > 0 when `registryPullQPS` > 0 Changes require a machine rollout.	cluster, controlPlane, workers	No	3.5.0
`security` object	Security configures Kubernetes specific security settings. Changes require a machine rollout.	cluster, controlPlane	No	3.2.0

apiServerConfiguration

APIServerConfiguration contains configuration options for the Kubernetes API server. These settings control API server behavior including request limits, audit logging, admission control, authentication, and event retention.

Introduced in version: 3.5.0

Required: No

Changes require a machine rollout.

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`extraAuthentication` object	ExtraAuthentication configures external authentication (e.g. OIDC) using the Kubernetes StructuredAuthenticationConfiguration API (apiserver.config.k8s.io/v1). When set, a Secret is created from this config and the API server is started with --authentication-config pointing to the mounted file. Schema matches apiserver.config.k8s.io/v1 AuthenticationConfiguration. IMPORTANT: Anonymous authentication is automatically configured by VKS and cannot be customized. VKS hardcodes anonymous access for essential cluster operations: /healthz, /readyz, /livez, and /api/v1/namespaces/kube-public/configmaps/cluster-info (required for worker node bootstrap). Any anonymous configuration in the source AuthenticationConfiguration is ignored. For complete anonymous access lockdown, use additional network security controls. Example (Keycloak with CEL claim mappings and optional CA); claimMappings/claimValidationRules are per JWT entry (siblings of issuer): extraAuthentication: jwt: issuer: url: https://keycloak.example.com/realms/engineering audiences: [my-cluster-client-id] certificateAuthority: \| -----BEGIN CERTIFICATE----- ... claimMappings: username: { expression: '"oidc:" + claims.email' } groups: { expression: 'claims.?groups.orValue([])' } uid: { claim: sub } claimValidationRules: expression: 'claims.email_verified == true' message: "email must be verified" Validation: at least one JWT authenticator is required	No	3.7.0
`logs` object	Logging configures the logging options for the API server, including log levels, formats, and output destinations. Refer to the Kubernetes component-base logs options for more information.	No	3.5.0
`maxMutatingRequestsInFlight` integer	MaxMutatingRequestsInFlight is the maximum number of parallel mutating requests. Every further request has to wait. This field and MaxRequestsInFlight are summed to determine the server's total concurrency limit (which must be positive). Corresponds to the --max-mutating-requests-inflight flag. Default: 200	No	3.5.0
`maxRequestsInFlight` integer	MaxRequestsInFlight is the maximum number of parallel non-long-running requests. Every further request has to wait. This field and MaxMutatingRequestsInFlight are summed to determine the server's total concurrency limit (which must be positive). Corresponds to the --max-requests-inflight flag. Default: 400	No	3.5.0
`oidc` object	OIDC contains OpenID Connect configuration for service account token issuance and validation. This includes the issuer URL and other OIDC-related settings.	No	3.7.0
`profiling` boolean	Profiling enables profiling via web interface host:port/debug/pprof/ Default: false	No	3.5.0
`requestTimeout` string	RequestTimeout is the duration after which all non-long-running requests will be timed out. Corresponds to the --request-timeout flag. Default: 1m0s	No	3.5.0

extraAuthentication

ExtraAuthentication configures external authentication (e.g. OIDC) using the Kubernetes StructuredAuthenticationConfiguration API (apiserver.config.k8s.io/v1). When set, a Secret is created from this config and the API server is started with --authentication-config pointing to the mounted file. Schema matches apiserver.config.k8s.io/v1 AuthenticationConfiguration.

IMPORTANT: Anonymous authentication is automatically configured by VKS and cannot be customized. VKS hardcodes anonymous access for essential cluster operations: /healthz, /readyz, /livez, and /api/v1/namespaces/kube-public/configmaps/cluster-info (required for worker node bootstrap). Any anonymous configuration in the source AuthenticationConfiguration is ignored. For complete anonymous access lockdown, use additional network security controls.

Example (Keycloak with CEL claim mappings and optional CA); claimMappings/claimValidationRules are per JWT entry (siblings of issuer): extraAuthentication: jwt:

issuer:

url: https://keycloak.example.com/realms/engineering audiences: [my-cluster-client-id] certificateAuthority: | -----BEGIN CERTIFICATE----- ... claimMappings: username: { expression: '"oidc:" + claims.email' } groups: { expression: 'claims.?groups.orValue([])' } uid: { claim: sub } claimValidationRules:

expression: 'claims.email_verified == true'

message: "email must be verified"

Introduced in version: 3.7.0

Required: No

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`jwt` []object	JWT is a list of authenticators to authenticate Kubernetes users using JWT compliant tokens. The authenticator will attempt to parse a raw ID token, verify it's been signed by the configured issuer. The public key to verify the signature is discovered from the issuer's public endpoint using OIDC discovery. Validation: at least one JWT authenticator is required	Yes	3.7.0

jwt

JWT is a list of authenticators to authenticate Kubernetes users using JWT compliant tokens. The authenticator will attempt to parse a raw ID token, verify it's been signed by the configured issuer. The public key to verify the signature is discovered from the issuer's public endpoint using OIDC discovery.

Introduced in version: 3.7.0

Required: Yes

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`claimMappings` object	ClaimMappings points claims of a token to be treated as user attributes. Required. Validation: `username` must specify either `claim` or `expression`	Yes	3.7.0
`claimValidationRules` []object	ClaimValidationRules are rules applied to validate token claims to authenticate users.	No	3.7.0
`issuer` object	Issuer contains the basic OIDC provider connection options. Required. Validation: OIDC `issuer` URL must use HTTPS OIDC discovery URL must use HTTPS at least one audience is required	Yes	3.7.0
`userValidationRules` []object	UserValidationRules are rules applied to the final user before completing authentication.	No	3.7.0

claimMappings

ClaimMappings points claims of a token to be treated as user attributes. Required.

Introduced in version: 3.7.0

Required: Yes

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`extra` []object	Extra represents an option for the extra attribute. Expression must produce a string or string array value.	No	3.7.0
`groups` object	Groups represents an option for the groups attribute. The claim's value must be a string or string array.	No	3.7.0
`uid` object	UID represents an option for the uid attribute.	No	3.7.0
`username` object	Username represents an option for the username attribute. Required.	Yes	3.7.0

extra

Extra represents an option for the extra attribute. Expression must produce a string or string array value.

Introduced in version: 3.7.0

Required: No

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`key` string	Key is a string to use as the extra attribute key. Must be a domain-prefix path (e.g. example.org/foo). Required.	Yes	3.7.0
`valueExpression` string	ValueExpression is a CEL expression to extract the extra attribute value. Required.	Yes	3.7.0

groups

Groups represents an option for the groups attribute. The claim's value must be a string or string array.

Introduced in version: 3.7.0

Required: No

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`claim` string	Claim is the JWT claim to use. Mutually exclusive with expression.	No	3.7.0
`expression` string	Expression is a CEL expression to evaluate. Mutually exclusive with claim and prefix.	No	3.7.0
`prefix` string	Prefix is prepended to the claim's value to prevent clashes with existing names. Mutually exclusive with expression.	No	3.7.0

uid

UID represents an option for the uid attribute.

Introduced in version: 3.7.0

Required: No

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`claim` string	Claim is the JWT claim to use. Either claim or expression must be set. Mutually exclusive with expression.	No	3.7.0
`expression` string	Expression is a CEL expression to evaluate. Mutually exclusive with claim.	No	3.7.0

username

Username represents an option for the username attribute. Required.

Introduced in version: 3.7.0

Required: Yes

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`claim` string	Claim is the JWT claim to use. Mutually exclusive with expression.	No	3.7.0
`expression` string	Expression is a CEL expression to evaluate. Mutually exclusive with claim and prefix.	No	3.7.0
`prefix` string	Prefix is prepended to the claim's value to prevent clashes with existing names. Mutually exclusive with expression.	No	3.7.0

claimValidationRules

ClaimValidationRules are rules applied to validate token claims to authenticate users.

Introduced in version: 3.7.0

Required: No

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`claim` string	Claim is the name of a required claim. Same as --oidc-required-claim. Mutually exclusive with expression and message.	No	3.7.0
`expression` string	Expression is a CEL expression evaluated by the API server. Must produce a boolean. Mutually exclusive with claim and requiredValue.	No	3.7.0
`message` string	Message customizes the returned error when expression returns false. Mutually exclusive with claim and requiredValue.	No	3.7.0
`requiredValue` string	RequiredValue is the value of a required claim. Mutually exclusive with expression and message.	No	3.7.0

issuer

Issuer contains the basic OIDC provider connection options. Required.

Introduced in version: 3.7.0

Required: Yes

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`audienceMatchPolicy` string	AudienceMatchPolicy defines how audiences is used to match the "aud" claim. Use "MatchAny" when multiple audiences are specified.	No	3.7.0
`audiences` []string	Audiences is the set of acceptable audiences the JWT must be issued to. At least one must match the "aud" claim. Required.	Yes	3.7.0
`certificateAuthority` string	CertificateAuthority contains PEM-encoded CA certificates used to validate the connection when fetching discovery information. If unset, the system verifier is used.	No	3.7.0
`discoveryURL` string	DiscoveryURL, if specified, overrides the URL used to fetch discovery information instead of using "{url}/.well-known/openid-configuration".	No	3.7.0
`egressSelectorType` string	EgressSelectorType indicates which egress selection to use for this issuer (e.g. "controlplane", "cluster").	No	3.7.0
`url` string	URL points to the issuer URL in a format https://url or https://url/path. This must match the "iss" claim in the presented JWT, and the issuer returned from discovery.	Yes	3.7.0

userValidationRules

UserValidationRules are rules applied to the final user before completing authentication.

Introduced in version: 3.7.0

Required: No

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`expression` string	Expression is a CEL expression evaluated by the API server. Must return true for validation to pass. Required.	Yes	3.7.0
`message` string	Message customizes the returned error message when the rule returns false.	No	3.7.0

logs

Logging configures the logging options for the API server, including log levels, formats, and output destinations. Refer to the Kubernetes component-base logs options for more information.

Introduced in version: 3.5.0

Required: No

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`flushFrequency` string	FlushFrequency is the maximum time between log flushes. If specified as a string, it's parsed as a duration (e.g., "1s"). If specified as an integer, it's the maximum number of nanoseconds. Ignored if the selected logging backend writes log messages without buffering. Corresponds to --log-flush-frequency flag. Default: 5s	No	3.5.0
`format` string	Format specifies the structure of log messages. Supported values are "text" (default) and "json". Corresponds to --logging-format flag. Default: "text"	No	3.5.0
`verbosity` integer	Verbosity is the threshold that determines which log messages are logged. Default is zero which logs only the most important messages. Higher values enable additional messages. Error messages are always logged. Corresponds to -v or --v flag. Default: 0	No	3.5.0

oidc

OIDC contains OpenID Connect configuration for service account token issuance and validation. This includes the issuer URL and other OIDC-related settings.

Introduced in version: 3.7.0

Required: No

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`serviceAccountIssuerURL` string	ServiceAccountIssuerURL is the issuer URL used by the kube-apiserver for bound service account tokens. This value is used as the `iss` claim in issued tokens and is also used for OIDC discovery/JWKS endpoints when enabled. Must be a valid HTTPS URL as per the OpenID spec. This URL must be capable of serving the OpenID discovery documents for the Kubernetes cluster at `<service-account-issuer>/.well-known/openid-configuration`. This is not done automatically and is the responsibility of the cluster administrator to set up. Corresponds to the `--service-account-issuer` flag. WARNING - DISRUPTIVE CHANGES: Setting, changing, or clearing this value will trigger a control plane rollout (API server restart) Service account tokens issued before/after the change will have different issuer claims Applications relying on specific issuer values may need to be updated Clearing this value reverts to Kubernetes default issuer behavior Example: https://kubernetes-cluster.example.com/ Default: unset (Kubernetes uses its own default issuer)	No	3.7.0

certificateRotation

CertificateRotation configures options for the automatic rotation of control plane certificates which have a default validity of 12 months. Note that certificates are always rotated during Kubernetes upgrades.

Introduced in version: 3.2.0

Required: No

Changes require a machine rollout.

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`enabled` boolean	Enabled controls enablement of auto certificate rotation	No	3.2.0
`renewalDaysBeforeExpiry` integer	RenewalDaysBeforeExpiry states the number of days before certificate expiry to initiate the renewal of certificates.	No	3.2.0

etcdConfiguration

EtcdConfiguration contains configuration options for the etcd database used by Kubernetes. These settings control etcd behavior including database size limits and performance tuning.

Introduced in version: 3.5.0

Required: No

Changes require a machine rollout.

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`maxRequestSizeKiB` integer	MaxRequestSizeKiB specifies the maximum size of a request to etcd in KiB. This value is used to set --max-request-bytes for etcd (converted to bytes in code). When unset, etcd uses its default of 1536 KiB (1.5 MiB). Must be between 1536 KiB (1.5 MiB) and 10240 KiB (10 MiB) inclusive. Validation: `maxRequestSizeKiB` must be between 1536 (1.5 MiB) and 10240 (10 MiB)	No	3.7.0
`maximumDBSizeGiB` integer	MaximumDBSizeGiB specifies the maximum size of the etcd database in GiB. This value is used to set --quota-backend-bytes for etcd. To account for compaction, defragmentation, and temporary usage spikes, the volume is provisioned with 25% additional capacity and mounted at /var/lib/etcd path. Validation: `maximumDBSizeGiB` must be between 2 and 8 GiB `maximumDBSizeGiB` cannot be decreased (must be ≥ previous value)	Yes	3.5.0

kubeControllerManagerConfiguration

KubeControllerManagerConfiguration contains configuration options for the kube-controller-manager.

Introduced in version: 3.6.0

Required: No

Changes require a machine rollout.

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`terminatedPodGCThreshold` integer	TerminatedPodGCThreshold is the number of terminated pods that can exist before the terminated pod garbage collector starts deleting terminated pods. If ≤ 0, the terminated pod garbage collector is disabled. Default: 6000	No	3.6.0

kubeProxyConfiguration

KubeProxyConfiguration contains configuration options for kube-proxy component. KubeProxy subscribes to services and endpoints, and programs the network data plane on all the linux nodes for load balancing service traffic. These settings control the behaviour of kube-proxy.

Introduced in version: 3.6.0

Required: No

Changes require a machine rollout.

Supported Scopes: cluster

Field	Description	Required	Since
`enabled` boolean	Enabled indicates whether kube-proxy should be enabled on the cluster. When set to false, kube-proxy will not be deployed, allowing alternative service proxy implementations provided by CNIs to be used. Not supported with Antrea or KR-bundled Calico. Use with custom CNIs requires KR version 1.35 or later.	No	3.7.0

kubeletConfiguration

KubeletConfiguration contains configuration options for the kubelet running on worker nodes. These settings control various aspects of kubelet behavior including resource limits, image management, logging, and security policies.

Introduced in version: 3.5.0

Required: No

Changes require a machine rollout.

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`allowedUnsafeSysctls` []string	AllowedUnsafeSysctls is a comma separated allowlist of unsafe sysctls or sysctl patterns (ending in ``). All safe sysctls are enabled by default. A safe sysctl must be properly isolated between pods on the same node. This means that setting a safe sysctl for one pod: must not have any influence on any other pod on the node must not allow to harm the node's health must not allow to gain CPU or memory resources outside of the resource limits of a pod. Unsafe sysctls fail to meet one or more of these criteria; the use of unsafe sysctls is at-your-own-risk and can lead to severe problems like wrong behavior of containers, resource shortage, or complete breakage of a node. Only modify sysctl parameters after you understand their effects, to avoid destabilizing your operating system. With this in mind, you can allow certain unsafe sysctls for very special situations such as high-performance or real-time application tuning. It is good practice to consider node pools with special sysctl settings as tainted within a cluster, and only schedule pods onto them which need those sysctl settings. Unsafe sysctl groups are `kernel.shm`, `kernel.msg`, `kernel.sem`, `fs.mqueue.`, and `net.`. These sysctls are namespaced but not allowed by default. For example: "`kernel.msg,net.ipv4.route.min_pmtu`" Default: []	No	3.5.0
`containerLogMaxFiles` integer	ContainerLogMaxFiles is the maximum number of container log files that can be present for a container. Default: 5	No	3.5.0
`containerLogMaxSizeMiB` integer	ContainerLogMaxSize defines the maximum size of the container log file before it is rotated in MiB. The mutator converts this integer into a kubelet-compatible string with the "Mi" unit when applying the configuration. For example: "5 → 5Mi". Default: 10	No	3.5.0
`eventBurst` integer	EventBurst is the maximum size of a burst of event creations, temporarily allows event creations to burst to this number, while still not exceeding eventRecordQPS. Only used if eventRecordQPS > 0. Default: 100	No	3.5.0
`eventRecordQPS` integer	EventRecordQPS is the maximum event creations per second. If 0, there is no limit enforced. Corresponds to --event-qps kubelet flag. Default: 50	No	3.5.0
`healthzBindAddress` string	HealthzBindAddress is the IP address for the healthz server to serve on. Default: "127.0.0.1"	No	3.5.0
`imageGCHighThresholdPercent` integer	ImageGCHighThresholdPercent is the percent of disk usage after which image garbage collection is always run. The percent is calculated as this field value out of 100. Default: 85	No	3.5.0
`imageGCLowThresholdPercent` integer	ImageGCLowThresholdPercent is the percent of disk usage before which image garbage collection is never run. Lowest disk usage to garbage collect to. The percent is calculated as this field value out of 100. Default: 80	No	3.5.0
`imageMaximumGCAge` string	ImageMaximumGCAge is the maximum age an image can be unused before it is garbage collected. The default of this field is "0s", which disables this field--meaning images won't be garbage collected based on being unused for too long. Default: "0s" (disabled)	No	3.5.0
`imageMinimumGCAge` string	ImageMinimumGCAge is the minimum age for an unused image before it is garbage collected. Default: "2m"	No	3.5.0
`imagePullCredentialsVerificationPolicy` string	ImagePullCredentialsVerificationPolicy determines how credentials should be verified when pod requests an image that is already present on the node. Supported values: NeverVerify: anyone on a node can use any image present on the node NeverVerifyPreloadedImages: images that were pulled to the node by something else than the kubelet can be used without reverifying pull credentials NeverVerifyAllowlistedImages: like "NeverVerifyPreloadedImages" but only node images from `preloadedImagesVerificationAllowlist` don't require reverification AlwaysVerify: all images require credential reverification Default: "NeverVerifyPreloadedImages"	No	3.5.0
`logging` object	Logging specifies the logging configuration options for the kubelet. This controls log levels, formats, and output destinations for kubelet logs. Refer to the Kubernetes component-base logs options for more information.	No	3.5.0
`maxParallelImagePulls` integer	MaxParallelImagePulls sets the maximum number of image pulls in parallel. This field is only used when SerializeImagePulls is false. Corresponds to --max-parallel-image-pulls kubelet flag. Default: nil (no limit)	No	3.5.0
`maxPods` integer	MaxPods is the number of pods that can run on this Kubelet. Default: 110 NOTE: By default, the maximum allowed value is 250.	No	3.5.0
`podPidsLimit` integer	PodPidsLimit is the maximum number of PIDs in any pod. Use Kubelet default (-1) when omitted. Default: nil	No	3.6.0
`preloadedImagesVerificationAllowlist` []string	PreloadedImagesVerificationAllowlist specifies a list of images that are exempted from credential reverification for the "NeverVerifyAllowlistedImages" `imagePullCredentialsVerificationPolicy`. The list accepts a full path segment wildcard suffix "/*". Only use image specs without an image tag or digest.	No	3.5.0
`registryBurst` integer	RegistryBurst is the maximum size of bursty pulls, temporarily allows pulls to burst to this number, while still not exceeding registryPullQPS. Only used if registryPullQPS > 0. Default: 10	No	3.5.0
`registryPullQPS` integer	RegistryPullQPS is the limit of registry pulls per second. Set to 0 for no limit. Default: 5	No	3.5.0
`serializeImagePulls` boolean	SerializeImagePulls when enabled, tells the Kubelet to pull images one at a time. Default: true	No	3.5.0
`streamingConnectionIdleTimeout` string	StreamingConnectionIdleTimeout is the maximum time a streaming connection can be idle before the connection is automatically closed. Default: "5m"	No	3.5.0

logging

Logging specifies the logging configuration options for the kubelet. This controls log levels, formats, and output destinations for kubelet logs. Refer to the Kubernetes component-base logs options for more information.

Introduced in version: 3.5.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`flushFrequency` string	FlushFrequency is the maximum time between log flushes. If specified as a string, it's parsed as a duration (e.g., "1s"). If specified as an integer, it's the maximum number of nanoseconds. Ignored if the selected logging backend writes log messages without buffering. Corresponds to --log-flush-frequency flag. Default: 5s	No	3.5.0
`format` string	Format specifies the structure of log messages. Supported values are "text" (default) and "json". Corresponds to --logging-format flag. Default: "text"	No	3.5.0
`verbosity` integer	Verbosity is the threshold that determines which log messages are logged. Default is zero which logs only the most important messages. Higher values enable additional messages. Error messages are always logged. Corresponds to -v or --v flag. Default: 0	No	3.5.0

security

Security configures Kubernetes specific security settings.

Introduced in version: 3.2.0

Required: No

Changes require a machine rollout.

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`minimumTLSProtocol` string	MinimumTLSProtocol sets the minimum supported TLS protocol version.	No	3.7.0
`podSecurityStandard` object	PodSecurityStandard configures the PodSecurityStandard settings for the cluster.	No	3.2.0
`resourceQuotaConfiguration` object	ResourceQuotaConfiguration configures the ResourceQuota admission control settings for the cluster.	No	3.5.0
`tlsCipherSuites` []string	TLSCipherSuites sets the list of supported cipher suites. If not specified, defaults are used based on FIPS mode.	No	3.7.0

podSecurityStandard

PodSecurityStandard configures the PodSecurityStandard settings for the cluster.

Introduced in version: 3.2.0

Required: No

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`audit` string	Audit sets the level for the audit PodSecurityConfiguration mode. Policy violations trigger an audit annotation, but are otherwise allowed One of "", privileged, baseline, restricted.	No	3.2.0
`auditVersion` string	AuditVersion can be used to pin the policy to the version that shipped with a given Kubernetes minor version (e.g. v1.31) when in audit mode.	No	3.2.0
`deactivated` boolean	Deactivated disables the patches for Pod Security Standard via AdmissionConfiguration.	No	3.2.0
`enforce` string	Enforce sets the level for the enforce PodSecurityConfiguration mode. Policy violations cause the pod to be rejected. One of "", privileged, baseline, restricted.	No	3.2.0
`enforceVersion` string	EnforceVersion can be used to pin the policy to the version that shipped with a given Kubernetes minor version (e.g. v1.31) when in enforce mode.	No	3.2.0
`exemptions` object	Exemptions can be statically configured based on (requesting) user, RuntimeClass, or namespace. A request meeting exemption criteria is ignored by the admission plugin. Currently only namespace exemptions are supported.	No	3.2.0
`warn` string	Warn sets the level for the warn PodSecurityConfiguration mode. Policy violations trigger a user-facing warning, but are otherwise allowed. One of "", privileged, baseline, restricted.	No	3.2.0
`warnVersion` string	WarnVersion can be used to pin the policy to the version that shipped with a given Kubernetes minor version (e.g. v1.31) when in warn mode.	No	3.2.0

exemptions

Exemptions can be statically configured based on (requesting) user, RuntimeClass, or namespace. A request meeting exemption criteria is ignored by the admission plugin. Currently only namespace exemptions are supported.

Introduced in version: 3.2.0

Required: No

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`namespaces` []string	Namespaces where PodSecurityStandards will be ignored.	No	3.2.0

resourceQuotaConfiguration

ResourceQuotaConfiguration configures the ResourceQuota admission control settings for the cluster.

Introduced in version: 3.5.0

Required: No

Supported Scopes: cluster, controlPlane

Field	Description	Required	Since
`enabled` boolean	Enabled enables the patches for ResourceQuotaConfiguration via AdmissionConfiguration. Once enabled, the ResourceQuota admission controller will be configured to limit pods with PriorityClass In [system-node-critical, system-cluster-critical].	No	3.5.0

networks

Networks defines the network configuration for the cluster

Introduced in version: 3.6.0

Required: No

Changes require a machine rollout.

Example:

interfaces:
  primary:
    network:
      apiVersion: crd.nsx.vmware.com/v1alpha1
      kind: SubnetSet
      name: subnetset-1
  secondary:
  - name: eth1
    network:
      apiVersion: crd.nsx.vmware.com/v1alpha1
      kind: SubnetSet
      name: subnetset-2
  - name: eth2
    network:
      apiVersion: crd.nsx.vmware.com/v1alpha1
      kind: Subnet
      name: subnet-3

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`interfaces` object	Interfaces describes one primary (eth0) and zero or more secondary interfaces attached to Node virtual machine. Validation: `primary` field cannot be updated after creation	No	3.6.0

interfaces

Interfaces describes one primary (eth0) and zero or more secondary interfaces attached to Node virtual machine.

Introduced in version: 3.6.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`primary` object	Primary is the primary network interface which is used to connect the Kubernetes primary network for Load balancer, Service discovery, Pod traffic and management traffic etc. Customization is only supported with network provider NSX-VPC.	No	3.6.0
`secondary` []object	Secondary network is supported with network provider NSX-VPC and vsphere-network.	No	3.6.0

primary

Primary is the primary network interface which is used to connect the Kubernetes primary network for Load balancer, Service discovery, Pod traffic and management traffic etc. Customization is only supported with network provider NSX-VPC.

Introduced in version: 3.6.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`mtu` integer	MTU is the Maximum Transmission Unit size in bytes.	No	3.6.0
`network` object	Network is the name of the network resource to which this interface is connected.	Yes	3.6.0
`routes` []object	Routes is a list of optional, static routes. Please note this feature is available only with the following bootstrap providers: CloudInit.	No	3.6.0

network

Network is the name of the network resource to which this interface is connected.

Introduced in version: 3.6.0

Required: Yes

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`apiVersion` string	APIVersion of the remediation template. APIVersion must be fully qualified domain name followed by / and a version. NOTE: This field must be kept in sync with the APIVersion of the remediation template.	Yes	3.6.0
`kind` string	Kind of the remediation template. Kind must consist of alphanumeric characters or '-', start with an alphabetic character, and end with an alphanumeric character.	Yes	3.6.0
`name` string	Name of the remediation template. Name must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character.	Yes	3.6.0

routes

Routes is a list of optional, static routes.

Please note this feature is available only with the following bootstrap providers: CloudInit.

Introduced in version: 3.6.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`to` string	To is an IPv4 CIDR. IPv6 is not supported yet. Examples: 192.168.1.0/24, 192.168.100.100/32, 0.0.0.0/0	Yes	3.6.0
`via` string	Via is an IPv4 address. IPv6 is not supported yet.	Yes	3.6.0

secondary

Secondary network is supported with network provider NSX-VPC and vsphere-network.

Introduced in version: 3.6.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`driver` string	driver is the kernel driver to bind to the interface. Only "vfio-pci" is supported at the moment. When the node VM does not have vIOMMU enabled, the machine agent automatically enables VFIO unsafe no-IOMMU mode so that vfio-pci binding can proceed; DMA from the device is then not IOMMU-protected. Enable vIOMMU on the node VM for production deployments.	No	3.7.0
`mtu` integer	MTU is the Maximum Transmission Unit size in bytes.	No	3.6.0
`name` string	Name describes the unique name of this network interface, used to distinguish it from other network interfaces attached to node Virtual Machine. Validation: `name` cannot be 'eth0'	Yes	3.6.0
`network` object	Network is the name of the network resource to which this interface is connected.	Yes	3.6.0
`routes` []object	Routes is a list of optional, static routes. Please note this feature is available only with the following bootstrap providers: CloudInit.	No	3.6.0
`sriovResourcePool` object	sriovResourcePool contains SRIOV device plugin resource pool configuration. This field should be set when driver is specified.	No	3.7.0

network

Network is the name of the network resource to which this interface is connected.

Introduced in version: 3.6.0

Required: Yes

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`apiVersion` string	APIVersion of the remediation template. APIVersion must be fully qualified domain name followed by / and a version. NOTE: This field must be kept in sync with the APIVersion of the remediation template.	Yes	3.6.0
`kind` string	Kind of the remediation template. Kind must consist of alphanumeric characters or '-', start with an alphabetic character, and end with an alphanumeric character.	Yes	3.6.0
`name` string	Name of the remediation template. Name must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character.	Yes	3.6.0

routes

Routes is a list of optional, static routes.

Please note this feature is available only with the following bootstrap providers: CloudInit.

Introduced in version: 3.6.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`to` string	To is an IPv4 CIDR. IPv6 is not supported yet. Examples: 192.168.1.0/24, 192.168.100.100/32, 0.0.0.0/0	Yes	3.6.0
`via` string	Via is an IPv4 address. IPv6 is not supported yet.	Yes	3.6.0

sriovResourcePool

sriovResourcePool contains SRIOV device plugin resource pool configuration. This field should be set when driver is specified.

Introduced in version: 3.7.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`name` string	name is the resource name for the SRIOV device pool.	Yes	3.7.0
`prefix` string	prefix is the resource prefix for the SRIOV device pool.	Yes	3.7.0

node

Node configures Kubernetes node specific settings.

Introduced in version: 3.2.0

Required: No

Example:

cri:
  runtimeClasses:
  - name: high-pid-5000
    podSpec:
      pidsLimit: 5000
  - name: high-ulimit-50000
    podSpec:
      uLimits:
      - hard: 50000
        soft: 50000
        type: nofile
firewall:
  inboundRules:
  - fromPort: 100
    protocol: 6
    source: 10.10.10.0/24
    toPort: 100
  - fromPort: 200
    protocol: 22
    source: ""
    toPort: 200
labels:
  label-test1: label-value-test1
  label-test2: label-value-test2
taints:
- effect: NoExecute
  key: taint-test1
  value: taint-value-test1
- effect: NoSchedule
  key: taint-test2
  value: taint-value-test2

Field	Description	Supported Scopes	Required	Since
`cri` object	CRI defines the Container Runtime Interface settings for the node. This allows for specialized execution isolation layers (e.g., gVisor). When set at cluster scope, configuration applies only to workers. Runtime class additions are applied in-place; deletes and modifications roll the node. Changes require a machine rollout.	cluster, workers	No	3.7.0
`firewall` object	Firewall specifies the firewall configuration that should be created on the node to allow specific kinds of traffic. Any rules defined will be merged with the default set of rules required for the cluster to function normally.	cluster, controlPlane, workers	No	3.6.0
`labels` map	Labels is a list of user defined name-value pairs	cluster, controlPlane, workers	No	3.2.0
`taints` []object	Taints specifies the taints the Node API object should be registered with. If this field is unset, i.e. nil, it will be defaulted with a control-plane taint for control-plane nodes. If you don't want to taint your control-plane node, set this field to an empty slice, i.e. `taints: []` in the YAML file. This field is solely used for Node registration.	cluster, controlPlane, workers	No	3.2.0

cri

CRI defines the Container Runtime Interface settings for the node. This allows for specialized execution isolation layers (e.g., gVisor). When set at cluster scope, configuration applies only to workers. Runtime class additions are applied in-place; deletes and modifications roll the node.

Introduced in version: 3.7.0

Required: No

Changes require a machine rollout.

Supported Scopes: cluster, workers

Field	Description	Required	Since
`runtimeClasses` []object	RuntimeClasses defines custom execution handlers available to the CRI. This allows for specialized isolation (e.g., gVisor) by mapping names to specific kernel-level configurations.	Yes	3.7.0

runtimeClasses

RuntimeClasses defines custom execution handlers available to the CRI. This allows for specialized isolation (e.g., gVisor) by mapping names to specific kernel-level configurations.

Introduced in version: 3.7.0

Required: Yes

Supported Scopes: cluster, workers

Field	Description	Required	Since
`name` string	Name corresponds to the PodSpec.runtimeClassName used in workloads.	Yes	3.7.0
`podSpec` object	PodSpec contains the kernel-level parameters for the runtime class.	Yes	3.7.0

podSpec

PodSpec contains the kernel-level parameters for the runtime class.

Introduced in version: 3.7.0

Required: Yes

Supported Scopes: cluster, workers

Field	Description	Required	Since
`pidsLimit` integer	PidsLimit is the maximum number of processes allowed per container. Updating or removing this value will trigger a node rollout.	No	3.7.0
`uLimits` []object	ULimits specifies POSIX resource limits (rlimits) for the process, enforced at process start. Updating or removing limits will trigger a node rollout. Supported types: core, memlock, nice, nofile, rtprio, stack. Resource exhaustion risks (KEP-5758): nofile—high values can exhaust the kernel FD table (node-wide "too many open files"). memlock—high values can exhaust physical RAM (OOM kills). nproc—applied per-UID on host; containers sharing a UID share the pool. Mitigations: nofile capped at 1,048,576; avoid unnecessarily high values; use node-level monitoring.	No	3.7.0

uLimits

ULimits specifies POSIX resource limits (rlimits) for the process, enforced at process start. Updating or removing limits will trigger a node rollout. Supported types: core, memlock, nice, nofile, rtprio, stack. Resource exhaustion risks (KEP-5758): nofile—high values can exhaust the kernel FD table (node-wide "too many open files"). memlock—high values can exhaust physical RAM (OOM kills). nproc—applied per-UID on host; containers sharing a UID share the pool. Mitigations: nofile capped at 1,048,576; avoid unnecessarily high values; use node-level monitoring.

Introduced in version: 3.7.0

Required: No

Supported Scopes: cluster, workers

Field	Description	Required	Since
`hard` integer	Hard is the ceiling value for the soft limit. Use -1 for unlimited. Must be greater than or equal to Soft.	Yes	3.7.0
`soft` integer	Soft is the current value enforced by the kernel. Use -1 for unlimited (maps to RLIM_INFINITY). Must be less than or equal to Hard.	Yes	3.7.0
`type` string	Type is the POSIX resource name, accepted in short form (e.g., "nofile") or with the RLIMIT_ prefix (e.g., "RLIMIT_NOFILE"). Allowlist follows the upstream KEP: Per Container ulimits configuration (https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/5758-per-container-ulimits-configuration#design-details).	Yes	3.7.0

firewall

Firewall specifies the firewall configuration that should be created on the node to allow specific kinds of traffic. Any rules defined will be merged with the default set of rules required for the cluster to function normally.

Introduced in version: 3.6.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`inboundRules` []object	InboundRules is a list of firewall rules that will be configured on each node to allow or deny specific kinds of traffic.	Yes	3.6.0

inboundRules

InboundRules is a list of firewall rules that will be configured on each node to allow or deny specific kinds of traffic.

Introduced in version: 3.6.0

Required: Yes

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`fromPort` integer	FromPort is the low end (inclusive) of the port range that this rule applies to.	No	3.6.0
`protocol`	Protocol is the type of traffic that this rule applies to. Allowed protocols include "tcp", "udp", "icmp", or an any valid IANA protocol number.	Yes	3.6.0
`source` string	Source is the CIDR range of the originating traffic that this rule applies to. If unset, the rule will apply to any source network. IP addresses specified to this option can be provided using CIDR notation: an appended slash and number specifies the number of "network bits" out of the address to use in the comparison. For example "192.168.0.0/16" would match all addresses starting with "192.168".	No	3.6.0
`toPort` integer	ToPort is the high end (inclusive) of the port range that this rule applies to.	No	3.6.0

labels

Labels is a list of user defined name-value pairs

Introduced in version: 3.2.0

Required: No

Type: map

taints

Taints specifies the taints the Node API object should be registered with. If this field is unset, i.e. nil, it will be defaulted with a control-plane taint for control-plane nodes. If you don't want to taint your control-plane node, set this field to an empty slice, i.e. taints: [] in the YAML file. This field is solely used for Node registration.

Introduced in version: 3.2.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`effect` string	Effect of the taint on pods that do not tolerate the taint. Valid effects are NoSchedule, PreferNoSchedule and NoExecute.	Yes	3.2.0
`key` string	Key is the taint key to be applied to a node.	Yes	3.2.0
`value` string	Value is the taint value corresponding to the taint key.	Yes	3.2.0

osConfiguration

OSConfiguration configures the system settings of nodes that are independent of Kubernetes.

Introduced in version: 3.2.0

Required: No

Example:

directoryJoin:
  credentialSecretRef: credential-secret-name-test
  domain: contoso.test
  gmsaControlSecurityGroupDN: CN=VKSClusterMembers,OU=VKSGMSAAccounts,DC=contoso,DC=test
  organizationalUnitDN: OU=VKSComputers,DC=contoso,DC=test
fips:
  enabled: true
grub:
  password:
    enabled: true
    secretRef:
      key: some-grub-secret-key
      name: my-grub-secret
    user: my-grub-user
ntp:
  servers:
  - ntp.test.1
  - ntp.test.2
securityContext:
  appArmor:
    profiles:
    - name: apparmor-profile-1
sshd:
  banner: Test banner
systemProxy:
  http: http://1.2.3.4:2139
  https: http://4.3.2.1:2139
  noProxy:
  - no.proxy.test1
  - no.proxy.test2
trust:
  additionalTrustedCAs:
  - caCert:
      secretRef:
        key: trust-ca-test1
        name: ""
  - caCert:
      secretRef:
        key: trust-ca-test2
        name: ""
  - caCert:
      secretRef:
        key: trust-ca-test3
        name: custom-trust-secret-1
tuned:
  active:
  - builtin-3.6.0
  profiles:
    builtin-3.6.0:
      profileRef:
        name: builtin-vks-v3.6.0
user:
  password:
    renewalDaysBeforeExpiry: 25
  passwordSecret:
    key: user-secret-key-test
    name: user-secret-name-test
  sshAuthorizedKey: sshAuthorizedKeyTest...
  user: customuser

Field	Description	Supported Scopes	Required	Since
`directoryJoin` object	DirectoryJoin configures the node to join a Windows Active Directory. Only supported on Windows at present. Changes require a machine rollout.	cluster, controlPlane, workers	No	3.3.0
`fips` object	FIPS configures FIPS related settings for the Kubernetes cluster to run in FIPS mode. Changes require a machine rollout.	cluster	No	3.3.0
`grub` object	GRUB configures GRUB Boot Loader. Changes require a machine rollout.	cluster, controlPlane, workers	No	3.5.0
`immutability` object	Immutability configures OS immutability settings for Photon OS nodes. Changes require a machine rollout.	cluster, controlPlane, workers	No	3.7.0
`ntp` object	NTP sets the time servers that will be used by nodes in the cluster. By default, NTP servers are inherited from vCenter. Changes require a machine rollout.	cluster, controlPlane, workers	No	3.2.0
`securityContext` object	SecurityContext holds security configurations that will be applied to node.	cluster, controlPlane, workers	No	3.6.0
`sshd` object	SSHD configures the sshd config of the node.	cluster, controlPlane, workers	No	3.3.0
`systemProxy` object	SystemProxy configures parameters that reference a proxy server for outbound cluster connections.	cluster, controlPlane, workers	No	3.2.0
`trust` object	Trust configures system-wide certificate trust for nodes	cluster, controlPlane, workers	No	3.2.0
`tuned` object	TuneD injects TuneD profiles and activate specified profile on Linux nodes. Only supported on Linux. Changes require a machine rollout.	cluster, controlPlane, workers	No	3.6.0
`ubuntuPro` object	UbuntuPro configures the Ubuntu Pro subscription of the node. Only supported on Ubuntu. Changes require a machine rollout.	cluster, controlPlane, workers	No	3.3.0
`user` object	User is an administrative user that will be created on all nodes. If not set, this is defaulted to "vmware-system-user". The administrative user will be added to "wheel" group on Linux and "Administrators" on Windows Validation: When `user` is set, atleast one of `passwordSecret` or `sshAuthorizedKey` must be set Changes require a machine rollout.	cluster, controlPlane, workers	No	3.2.0

directoryJoin

DirectoryJoin configures the node to join a Windows Active Directory. Only supported on Windows at present.

Introduced in version: 3.3.0

Required: No

Changes require a machine rollout.

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`credentialSecretRef` string	CredentialSecretRef is the name of the secret containing Active Directory join credentials. These will be passed to all new Windows new nodes to allow them to join the Active Directory domain. The secret must have the following keys: username - The username for the Active Directory account. This must be in one of the following formats: <username> <domain>\<username> <username>@<fqdn of domain> password - The password for the account	Yes	3.3.0
`domain` string	Domain is the FQDN of the Active Directory Kerberos domain to join.	Yes	3.3.0
`gmsaControlSecurityGroupDN` string	GMSAControlSecurityGroupDN is an optional Windows Active Directory security group that has permissions to access the password of the Group Managed Service Accounts. The value will be validated according to https://tools.ietf.org/html/rfc4514	No	3.3.0
`organizationalUnitDN` string	OrganizationalUnitDN is an optional organizational unit where the node will be added to in Active Directory. The value will be validated according to https://tools.ietf.org/html/rfc4514	No	3.3.0

fips

FIPS configures FIPS related settings for the Kubernetes cluster to run in FIPS mode.

Introduced in version: 3.3.0

Required: No

Changes require a machine rollout.

Supported Scopes: cluster

Field	Description	Required	Since
`enabled` boolean	Enable specifies whether FIPS settings are enabled and enforced on the node	No	3.3.0

grub

GRUB configures GRUB Boot Loader.

Introduced in version: 3.5.0

Required: No

Changes require a machine rollout.

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`password` object	Password configures the password protection for GRUB Boot Loader (Only applicable on Linux). Validation: `secretRef` cannot be set when `enabled` is false	No	3.5.0

password

Password configures the password protection for GRUB Boot Loader (Only applicable on Linux).

Introduced in version: 3.5.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`enabled` boolean	Enabled defines if the GRUB Boot Loader must be protected with a password	No	3.5.0
`secretRef` object	SecretRef is the name of the secret containing the password to protect GRUB Key is the data.key field within the secret containing the password value. If not specified, the secret will be automatically generated as <cluster name>-grub-password when `Enabled` is set to true. The password value must be a PBKDF2 SHA-512 hashed password using grub2-mkpasswd-pbkdf2. The value will set to GRUB password as superuser password. User must type in the plain-text password to boot from non-default entries or modify boot parameters for any boot entry. See https://www.gnu.org/software/grub/manual/grub/html_node/Invoking-grub_002dmkpasswd_002dpbkdf2.html for more information.	No	3.5.0
`user` string	User specifies the username to use for GRUB password protection.	No	3.6.0

secretRef

SecretRef is the name of the secret containing the password to protect GRUB Key is the data.key field within the secret containing the password value. If not specified, the secret will be automatically generated as <cluster name>-grub-password when Enabled is set to true.

The password value must be a PBKDF2 SHA-512 hashed password using grub2-mkpasswd-pbkdf2. The value will set to GRUB password as superuser password. User must type in the plain-text password to boot from non-default entries or modify boot parameters for any boot entry.

See https://www.gnu.org/software/grub/manual/grub/html_node/Invoking-grub_002dmkpasswd_002dpbkdf2.html for more information.

Introduced in version: 3.5.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`key` string	Key is the entry in the secret's data map that is to be used to retrieve secret data.	Yes	3.5.0
`name` string	Name is the name of the secret and must be in the same namespace as the cluster.	Yes	3.5.0

immutability

Immutability configures OS immutability settings for Photon OS nodes.

Introduced in version: 3.7.0

Required: No

Changes require a machine rollout.

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`usrPartitionMode` string	UsrPartitionMode configures the /usr partition should be read-only or read-write on Photon OS nodes. When set to "ReadOnly", the /usr partition stays read-only, providing defence-in-depth against accidental modification and unprivileged code; it does not prevent a privileged Pod (root with CAP_SYS_ADMIN) from remounting /usr as read-write at runtime. When set to "ReadWrite" or omitted, /usr partition stays as read-write, and nodes behave identically to mutable nodes.	No	3.7.0

ntp

NTP sets the time servers that will be used by nodes in the cluster. By default, NTP servers are inherited from vCenter.

Introduced in version: 3.2.0

Required: No

Changes require a machine rollout.

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`servers` []string	NTP sets the time servers that will be used by nodes in this cluster. By default, NTP servers are inherited from vCenter.	Yes	3.2.0

securityContext

SecurityContext holds security configurations that will be applied to node.

Introduced in version: 3.6.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`appArmor` object	AppArmor configures the appArmor profiles of the node. Only supported on Ubuntu and Photon nodes.	No	3.6.0

appArmor

AppArmor configures the appArmor profiles of the node. Only supported on Ubuntu and Photon nodes.

Introduced in version: 3.6.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`profiles` []object	Profiles is a list of appArmor profiles to be added to the node.	Yes	3.6.0

profiles

Profiles is a list of appArmor profiles to be added to the node.

Introduced in version: 3.6.0

Required: Yes

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`name` string	Name is the AppArmorProfile object name, and the object should be in the same namespace of this cluster. The .spec.content value would be automatically loaded from this object.	Yes	3.6.0

sshd

SSHD configures the sshd config of the node.

Introduced in version: 3.3.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`banner` string	Banner specifies the login message used for sending a legal warning message before authentication	No	3.3.0

systemProxy

SystemProxy configures parameters that reference a proxy server for outbound cluster connections.

Introduced in version: 3.2.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`http` string	HTTP is the proxy server to be used for all http connections. This should be a hostname or dotted numerical IP address. To specify port number in this string, append :[port] to the end of the host name. The proxy string may be prefixed with [scheme]:// to specify which kind of proxy is used. This configures the http_proxy environment variable for profile login and systemd on Linux, and is inherited by kubelet and containerd. On Windows, the http_proxy environment variable is set for the kubelet and containerd service. In addition, WinHTTP proxy server is also set for the http protocol. Note, that for WinHTTP, a https scheme for proxy servers is not supported, and will be ignored for WinHTTP configuration. WinHTTP is used by services such as Windows Update Services. Validation: `http` must be a valid URL	Yes	3.2.0
`https` string	HTTPS configures the proxy server to be used for all https connections. This should be a hostname or dotted numerical IP address. To specify port number in this string, append :[port] to the end of the host name. The proxy string may be prefixed with [scheme]:// to specify which kind of proxy is used. This configures the http_proxy environment variable for profile login and systemd on Linux, and is inherited by kubelet and containerd. On Windows, the http_proxy environment variable is set for the kubelet and containerd service. In addition, WinHTTP proxy server is also set for the http protocol. Note, that for WinHTTP, a https scheme for proxy servers is not supported, and will be ignored for WinHTTP configuration. WinHTTP is used by services such as Windows Update Services. All proxy servers MUST support the CONNECT method to allow pass-through TLS. Intercepting proxies will cause connectivity issues for websockets, HTTP/2 & HTTP/3. Validation: `https` must be a valid URL	Yes	3.2.0
`noProxy` []string	NoProxy configures the list of hostnames and CIDR ranges that should be reached without the configured proxy servers. The only wildcard available is a single * character, which matches all hosts, and effectively disables the proxy. Each name in this list is matched as either a domain which contains the hostname, or the hostname itself. For example, "ample.com" would match ample.com, ample.com:80, and www.ample.com, but not www.example.com or ample.com.org. IP addresses specified to this option can be provided using CIDR notation: an appended slash and number specifies the number of "network bits" out of the address to use in the comparison. For example "192.168.0.0/16" would match all addresses starting with "192.168". Enter IPv6 numerical addresses in the list of hostnames without enclosing brackets. On Linux, this configures no_proxy environment variables for systemd and the login profile, and is inherited by containerd and kubelet. On Windows, this configures the no_proxy environment variable for the containerd and kubelet services. The NoProxy list is also translated into a system winhttp bypass list. However, note that Windows does not support CIDR notation for the bypass list, and instead, CIDR ranges are converted 'best-effort' to wildcards, e.g. "192.168.0.0/16" becomes "192.168.."	Yes	3.2.0

trust

Trust configures system-wide certificate trust for nodes

Introduced in version: 3.2.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`additionalTrustedCAs` []object	AdditionalTrustedCAs is a list of additional CAs to be added to the system trust store of nodes.	Yes	3.2.0

additionalTrustedCAs

AdditionalTrustedCAs is a list of additional CAs to be added to the system trust store of nodes.

Introduced in version: 3.2.0

Required: Yes

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`caCert` object	SecretContent configures a reference to or content of secret data.	Yes	3.2.0

caCert

SecretContent configures a reference to or content of secret data.

Introduced in version: 3.2.0

Required: Yes

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`content` string	Content should only be used where it is ok if the secret data is viewable on the Cluster CR, and allows direct injection of a secret.	No	3.2.0
`secretRef` object	SecretRef is the name of a secret within the namespace that contains secret data to be used.	No	3.2.0

secretRef

SecretRef is the name of a secret within the namespace that contains secret data to be used.

Introduced in version: 3.2.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`key` string	Key is the entry in the secret's data map that is to be used to retrieve secret data.	Yes	3.2.0
`name` string	Name is the name of the secret and must be in the same namespace as the cluster.	Yes	3.2.0

tuned

TuneD injects TuneD profiles and activate specified profile on Linux nodes. Only supported on Linux.

Introduced in version: 3.6.0

Required: No

Changes require a machine rollout.

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`active` []string	Active is a list of tuned profile name will be activated on node.	Yes	3.6.0
`profiles` map	Profiles is a map of tuned profiles will be injected on node. Key is the desired tuned profile name, value is the TunedProfile CR reference which contains the profile content.	No	3.6.0

profiles

Profiles is a map of tuned profiles will be injected on node. Key is the desired tuned profile name, value is the TunedProfile CR reference which contains the profile content.

Introduced in version: 3.6.0

Required: No

Type: map

ubuntuPro

UbuntuPro configures the Ubuntu Pro subscription of the node. Only supported on Ubuntu.

Introduced in version: 3.3.0

Required: No

Changes require a machine rollout.

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`services` []string	Services specifies the Ubuntu Pro services to be enabled.	No	3.3.0
`settings` []object	Settings specifies the Ubuntu Pro client (ubuntu-advantage-tools) settings to be configured.	No	3.3.0
`tokenSecretRef` string	TokenSecretRef is the name of the secret containing a valid Ubuntu Pro Subscription token. The secret must have a key token with the content of a valid token.	Yes	3.3.0

settings

Settings specifies the Ubuntu Pro client (ubuntu-advantage-tools) settings to be configured.

Introduced in version: 3.3.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`key` string	Key is the setting key for Ubuntu Pro client.	Yes	3.3.0
`value` string	Value is the setting value for Ubuntu Pro client.	Yes	3.3.0

user

User is an administrative user that will be created on all nodes. If not set, this is defaulted to "vmware-system-user". The administrative user will be added to "wheel" group on Linux and "Administrators" on Windows

Introduced in version: 3.2.0

Required: No

Changes require a machine rollout.

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`password` object	Password configures the password policy such as password max age and renewal settings.	No	3.4.0
`passwordSecret` object	Key is the data.key field within the secret containing the password value. If not specified, the secret will be automatically generated as <cluster name>-ssh-password.	No	3.2.0
`requirePasswordOnSudo` boolean	RequirePasswordOnSudo configures whether password re-authentication is required on sudo.	No	3.5.0
`sshAuthorizedKey` string	The string of the SSH public key that is to be used for the administrative account. The public key must be of any FIPS-140 approved algorithm. If not specified, a key will be automatically generated and the private key stored in the secret <cluster-name>-ssh	No	3.2.0
`user` string	Name is the name of the user to be created. By default, this is vmware-system-user.	Yes	3.2.0

password

Password configures the password policy such as password max age and renewal settings.

Introduced in version: 3.4.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`renewalDaysBeforeExpiry` integer	RenewalDaysBeforeExpiry configures the days to renew the password before it gets expired. The expired password will re-newed when it reaches the validity MaxAgeInDays - RenewalDaysBeforeExpiry. On Linux, it also manages PASS_WARN_AGE in /etc/login.defs on Linux to give a warning when the password reaches the validity. Defaults to 7 days if omitted. Only supported on Linux at present.	No	3.4.0

passwordSecret

Key is the data.key field within the secret containing the password value. If not specified, the secret will be automatically generated as <cluster name>-ssh-password.

Introduced in version: 3.2.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`key` string	Key is the data.key field within the secret containing the password value. For Linux, this must be the hashed value that should be inserted into /etc/shadow. For Windows, only plain text is supported, and is therefore not recommended and is not configured by default.	Yes	3.2.0
`name` string	Name is the name of the secret containing the password for the administrative account.	Yes	3.2.0

resourceConfiguration

ResourceConfiguration configures kubelet resource options. Currently, only CPU and memory reservations are supported.

Introduced in version: 3.2.0

Required: No

Changes require a machine rollout.

Example:

systemReserved:
  cpu: 500m
  memory: 100Mi

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`systemReserved` object	SystemReserved defines the system reserved CPU and memory reservations.	No	3.2.0

systemReserved

SystemReserved defines the system reserved CPU and memory reservations.

Introduced in version: 3.2.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`automatic` boolean	Automatic controls the automatic calculation of system reserved resources.	No	3.2.0
`cpu`	CPU describes the number of CPU cores reserved for system processes. Inputs should be of type https://k8s.io/docs/reference/kubernetes-api/common-definitions/quantity/ For a detailed explanation of CPU resource units in Kubernetes, refer to https://k8s.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-cpu Example: The input value of 1 describes 1 virtual CPU core. Validation: `cpu` must be a valid Kubernetes quantity	No	3.2.0
`memory`	Memory describes the memory resources reserved for system processes. Inputs should be of type https://k8s.io/docs/reference/kubernetes-api/common-definitions/quantity/ For a detailed explanation of memory resource units in Kubernetes, refer to https://k8s.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-memory Example: 4096Mi, 8G Validation: `memory` must be a valid Kubernetes quantity	No	3.2.0

storageClass

StorageClass sets the StorageClass that will be used to create node root volumes.

Introduced in version: 3.2.0

Required: Yes

Type: string

Changes require a machine rollout.

Example:

storage-class-test

vmClass

VMClass sets the VMClass that will be used to create nodes.

Introduced in version: 3.2.0

Required: Yes

Type: string

Changes require a machine rollout.

Example:

vm-class.test

volumes

Volumes configures additional disks to be attached to node virtual machines.

Introduced in version: 3.2.0

Required: No

Changes require a machine rollout.

Example:

- capacity: 1T
  mountPath: volume/test1
  name: volume-test1
  storageClass: volume-class-test1

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`capacity` string	Capacity defines the storage capacity of the volume.	Yes	3.2.0
`mountPath` string	MountPath defines the mount path for the volume.	Yes	3.2.0
`name` string	Name defines the name of the volume.	Yes	3.2.0
`storageClass` string	StorageClass defines the Storage class to use for the volume.	No	3.2.0

vsphereOptions

VSphereOptions configures vSphere specific options related to nodes

Introduced in version: 3.2.0

Required: No

Changes require a machine rollout.

Example:

persistentVolumes:
  customizableStorageClassAnnotations:
  - annotation-key-1
  - annotation-key-2
  customizableStorageClassLabels:
  - label-key-1
  - label-key-2

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`persistentVolumes` object	PersistentVolumes configures what is available for PVCs to be used in the cluster.	No	3.2.0

persistentVolumes

PersistentVolumes configures what is available for PVCs to be used in the cluster.

Introduced in version: 3.2.0

Required: No

Supported Scopes: cluster, controlPlane, workers

Field	Description	Required	Since
`availableStorageClasses` []string	AvailableStorageClasses lists the storage classes that can be used in the cluster.	No	3.2.0
`availableVolumeSnapshotClasses` []string	AvailableVolumeSnapshotClasses lists the volume snapshot classes that can be used in the cluster.	No	3.2.0
`customizableStorageClassAnnotations` []string	CustomizableStorageClassAnnotations is a list of annotation keys set on the storage classes within the cluster which can be customized by the user. Any edits made to the values of these annotations will not be overridden by VKS.	No	3.4.0
`customizableStorageClassLabels` []string	CustomizableStorageClassLabels is a list of label keys set on the storage classes within the cluster which can be customized by the user. Any edits made to the values of these labels will not be overridden by VKS.	No	3.4.0
`defaultStorageClass` string	DefaultStorageClass sets the default storage class inside the cluster. The StorageClass with this name gets a "storageclass.kubernetes.io/is-default-class: true" label added during storage class sync into the guest cluster.	No	3.2.0
`defaultVolumeSnapshotClass` string	DefaultVolumeSnapshotClass sets the default volume snapshot class inside the cluster. The VolumeSnapshotClass with this name gets a "snapshot.storage.kubernetes.io/is-default-class: true" label added during volume snapshot class sync into the guest cluster.	No	3.2.0

TKR_DATA

TKRData is an internal variable used to populate the Kubernetes configuration with the correct OCI images to bootstrap Kubernetes. Do not modify or set this manually without the assistance of VMware Support.

Introduced in version: 3.2.0

Removed since: 3.5.0

Required: No

Type: map