kubevirt¶

KubeVirt - Virtual Machine Management on Kubernetes - Deploy and manage VMs as native Kubernetes resources

Homepage: https://github.com/encircle360-oss/helm-charts/tree/main/charts/kubevirt

⚠️ UNDER CONSTRUCTION This chart is currently under active development and has not been battle-tested in production environments. NOT PRODUCTION READY - Use at your own risk and thoroughly test in non-production environments first. KubeVirt is a complex system requiring deep Kubernetes and virtualization knowledge. Only use this chart if you are an experienced Kubernetes operator.

Description¶

KubeVirt is a Kubernetes add-on that enables you to run and manage virtual machines alongside container workloads. This Helm chart provides a deployment of KubeVirt v1.7.0 with comprehensive configuration options.

Key Features: - Full KubeVirt v1.7.0 support with all feature gates - Operator-based lifecycle management - Comprehensive RBAC configuration - Monitoring integration (ServiceMonitor & PrometheusRule) - High availability configuration - Extensive configuration options

Prerequisites¶

• Kubernetes 1.32+ (vanilla Kubernetes, K3s, K0s, or OpenShift)
• Helm 3.8+
• Nodes with KVM support (hardware virtualization) OR software emulation enabled
• Sufficient cluster resources for VM workloads

Platform Compatibility: This chart works on both vanilla Kubernetes (including K3s, K0s) and OpenShift. OpenShift-specific RBAC rules are included but will be safely ignored on non-OpenShift clusters.

Check Node Virtualization Support:

# Check if nodes have KVM support
kubectl get nodes -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.allocatable.devices\.kubevirt\.io/kvm}{"\n"}{end}'

# Verify KVM is available on nodes
kubectl debug node/<node-name> -it --image=ubuntu -- bash
apt update && apt install -y cpu-checker
kvm-ok

Installation¶

Quick Start¶

# Add the Helm repository
helm repo add encircle360-oss https://encircle360-oss.github.io/helm-charts
helm repo update

# Install KubeVirt
helm install kubevirt encircle360-oss/kubevirt \
  --namespace kubevirt \
  --create-namespace

# Wait for KubeVirt to be ready
kubectl wait kubevirt kubevirt -n kubevirt \
  --for=condition=Available \
  --timeout=10m

Installation with Custom Values¶

helm install kubevirt encircle360-oss/kubevirt \
  --namespace kubevirt \
  --create-namespace \
  -f custom-values.yaml

Install with Monitoring¶

# values-monitoring.yaml
monitoring:
  enabled: true
  namespace: monitoring
  serviceAccount: prometheus-k8s
  prometheusRule:
    enabled: true
    labels:
      prometheus: kube-prometheus

helm install kubevirt encircle360-oss/kubevirt \
  -f values-monitoring.yaml \
  --namespace kubevirt \
  --create-namespace

Configuration¶

Basic Configuration¶

Feature Gates¶

KubeVirt uses feature gates to control optional functionality. Feature gates are categorized as: - Alpha: Experimental features that must be explicitly enabled - Beta: Features that are stable but may have limitations - GA: Stable features (enabled by default, no gate needed)

Alpha/Beta Feature Gates (v1.7.0)¶

kubevirt:
  configuration:
    developerConfiguration:
      featureGates:
        # Storage
        - ExpandDisks              # Dynamic disk expansion
        - HostDisk                 # Host disk access
        - ImageVolume              # Image volume support (Beta in v1.7.0)

        # Performance
        - CPUManager               # CPU pinning and NUMA topology
        - AlignCPUs                # Align guest and host CPU topology

        # Hardware
        - HostDevices              # PCI device passthrough
        - GPUsWithDRA              # GPU with Dynamic Resource Allocation
        - HostDevicesWithDRA       # Host devices with DRA

        # Security
        - WorkloadEncryptionSEV    # AMD SEV memory encryption
        - KubevirtSeccompProfile   # Custom seccomp profiles
        - SecureExecution          # IBM Secure Execution

        # Networking
        - VSOCK                    # AF_VSOCK for host-guest communication
        - PasstIPStackMigration    # PASST IP stack migration

        # Other
        - ExperimentalIgnitionSupport  # Ignition config support
        - HypervStrictCheck        # Strict Hyper-V feature checking
        - Sidecar                  # Sidecar container injection
        - DownwardMetrics          # Expose metrics to VMs
        - Root                     # Run virt-launcher as root
        - DisableMDEVConfiguration # Disable automatic MDEV configuration
        - PersistentReservation    # SCSI persistent reservations
        - MultiArchitecture        # Multi-architecture support
        - VirtIOFSConfigVolumesGate    # VirtioFS for config volumes
        - VirtIOFSStorageVolumeGate    # VirtioFS for storage volumes
        - DecentralizedLiveMigration   # Decentralized live migration
        - ObjectGraph              # Object graph feature
        - DeclarativeHotplugVolumes    # Declarative hotplug volumes
        - VideoConfig              # Video device configuration
        - PanicDevices             # Panic device support

GA Feature Gates (No Configuration Needed)¶

The following features are stable and enabled by default in v1.7.0. You do NOT need to specify these in feature gates:

• LiveMigration - Live migration of VMs
• SRIOVLiveMigration - Live migration with SR-IOV
• NonRoot - Run as non-root user
• PSA - Pod Security Admission
• CPUNodeDiscovery - CPU node discovery
• NUMA - NUMA topology support
• GPU - GPU passthrough
• VMLiveUpdateFeatures - Live update VM features (GA in v1.5.0)
• CommonInstancetypesDeploymentGate - Common instance types (GA in v1.4.0)
• HotplugNICs - Hot-plug network interfaces (GA in v1.4.0)
• BochsDisplayForEFIGuests - Bochs display for EFI (GA in v1.4.0)
• AutoResourceLimitsGate - Automatic resource limits (GA in v1.5.0)
• NetworkBindingPlugins - Network binding plugins (GA in v1.5.0)
• DynamicPodInterfaceNaming - Dynamic pod interface naming (GA in v1.5.0)
• VolumesUpdateStrategy - Volumes update strategy (GA in v1.5.0)
• VolumeMigration - Volume migration (GA in v1.5.0)
• InstancetypeReferencePolicy - Instance type reference policy (GA in v1.6.0)
• Snapshot - VM snapshot support (GA in v1.7.0)
• VMExport - VM export functionality (GA in v1.7.0)
• HotplugVolumes - Hot-plug/unplug volumes (GA in v1.7.0)
• NodeRestriction - Node restriction (GA in v1.7.0)

CPU Configuration¶

kubevirt:
  configuration:
    # Default CPU model for VMs
    cpuModel: "host-passthrough"

    # Default CPU request
    cpuRequest: "100m"

    # Mark obsolete CPUs as unusable
    obsoleteCPUModels:
      pentium: true
      pentium2: true
      pentium3: true
      Conroe: true
      Penryn: true

Network Configuration¶

kubevirt:
  configuration:
    network:
      # Default network interface type
      defaultNetworkInterface: "masquerade"

      # Allow bridge on pod network
      permitBridgeInterfaceOnPodNetwork: false

      # Allow SLIRP interface
      permitSlirpInterface: false

Live Migration Configuration¶

kubevirt:
  configuration:
    migration:
      # Disable TLS (not recommended for production)
      disableTLS: false

      # Allow auto-converge for slow migrations
      allowAutoConverge: false

      # Bandwidth limit per migration
      bandwidthPerGiB: "64Mi"

      # Timeouts
      completionTimeoutPerGiB: 800
      progressTimeout: 150

      # Parallelism
      parallelMigrationsPerCluster: 5
      parallelOutboundMigrationsPerNode: 2

      # Post-copy migration (use with caution)
      allowPostCopy: false

      # Dedicated migration network
      network: ""

Storage Configuration¶

kubevirt:
  configuration:
    # Storage class for VM snapshots and state
    vmStateStorageClass: "standard"

Host Device Passthrough¶

kubevirt:
  configuration:
    permittedHostDevices:
      # PCI devices
      pciHostDevices:
        - pciVendorSelector: "10DE:1EB8"  # NVIDIA Tesla T4
          resourceName: "nvidia.com/T4"
        - pciVendorSelector: "8086:1572"  # Intel X710
          resourceName: "intel.com/X710"

      # Mediated devices (vGPU)
      mediatedDevices:
        - mdevNameSelector: "GRID T4-1Q"
          resourceName: "nvidia.com/GRID_T4-1Q"

Node Placement¶

Infrastructure Components (virt-api, virt-controller)¶

kubevirt:
  infra:
    nodePlacement:
      nodeSelector:
        node-role.kubernetes.io/control-plane: ""
      tolerations:
        - key: node-role.kubernetes.io/control-plane
          effect: NoSchedule

Workload Components (virt-handler, virt-launcher)¶

kubevirt:
  workloads:
    nodePlacement:
      nodeSelector:
        kubevirt.io/vm-workload: "true"
      tolerations:
        - key: kubevirt.io/vm-workload
          effect: NoSchedule

Monitoring Configuration¶

monitoring:
  enabled: true
  namespace: monitoring
  serviceAccount: prometheus-k8s

  serviceMonitorLabels:
    prometheus: kube-prometheus

  scrapeInterval: "30s"

  prometheusRule:
    enabled: true
    labels:
      prometheus: kube-prometheus

    # Add custom alerting rules
    additionalRules:
      - alert: MyCustomAlert
        expr: up == 0
        for: 5m
        labels:
          severity: critical

Usage Examples¶

Creating a Virtual Machine¶

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: ubuntu-vm
spec:
  running: true
  template:
    metadata:
      labels:
        kubevirt.io/vm: ubuntu-vm
    spec:
      domain:
        devices:
          disks:
            - name: containerdisk
              disk:
                bus: virtio
            - name: cloudinitdisk
              disk:
                bus: virtio
        resources:
          requests:
            memory: 2Gi
            cpu: 2
      volumes:
        - name: containerdisk
          containerDisk:
            image: quay.io/containerdisks/ubuntu:22.04
        - name: cloudinitdisk
          cloudInitNoCloud:
            userData: |
              #cloud-config
              password: ubuntu
              chpasswd: { expire: False }
              ssh_pwauth: True

Using virtctl¶

# Install virtctl
kubectl krew install virt

# Start/Stop VMs
kubectl virt start ubuntu-vm
kubectl virt stop ubuntu-vm
kubectl virt restart ubuntu-vm

# Console access
kubectl virt console ubuntu-vm

# VNC access
kubectl virt vnc ubuntu-vm

# SSH into VM (requires SSH service in VM)
kubectl virt ssh ubuntu@ubuntu-vm

VM Snapshots (requires Snapshot feature gate)¶

# Enable snapshot feature
kubevirt:
  configuration:
    developerConfiguration:
      featureGates:
        - Snapshot

# Create snapshot
apiVersion: snapshot.kubevirt.io/v1alpha1
kind: VirtualMachineSnapshot
metadata:
  name: ubuntu-vm-snapshot
spec:
  source:
    apiGroup: kubevirt.io
    kind: VirtualMachine
    name: ubuntu-vm

Upgrading¶

Upgrade the Chart¶

helm repo update
helm upgrade kubevirt encircle360-oss/kubevirt \
  --namespace kubevirt \
  -f values.yaml

Upgrade Strategy¶

KubeVirt supports rolling updates of VMs during upgrades:

kubevirt:
  workloadUpdateStrategy:
    workloadUpdateMethods:
      - LiveMigrate  # Try live migration first
      - Evict        # Fall back to eviction
    batchEvictionSize: 10
    batchEvictionInterval: "1m"

Frequently Asked Questions (FAQ)¶

Does this work on K3s/K0s/vanilla Kubernetes?¶

Yes! This chart is designed to work on: - Vanilla Kubernetes (upstream) - K3s (lightweight Kubernetes) - K0s (zero friction Kubernetes) - MicroK8s - OpenShift/OKD

The chart includes OpenShift-specific RBAC rules, but these are safely ignored on non-OpenShift clusters.

Why are there OpenShift resources in the RBAC?¶

KubeVirt officially supports both vanilla Kubernetes and OpenShift. The OpenShift-specific resources (security.openshift.io/securitycontextconstraints, route.openshift.io/routes) are: - Optional and only used on OpenShift - Ignored on vanilla Kubernetes/K3s (API groups don't exist) - Standard practice in the official KubeVirt manifests

Your K3s cluster will simply skip these rules - no issues!

Why do some labels have empty values like operator.kubevirt.io: ""?¶

This is valid and intentional in Kubernetes! Empty-string label values are used as: - Marker labels: Indicate that something is tagged without needing a specific value - Selector labels: Can be matched with matchLabels: { "operator.kubevirt.io": "" } - KubeVirt convention: How KubeVirt identifies its own resources internally

Do I need hardware virtualization (KVM)?¶

Recommended but not required: - With KVM (hardware virtualization): Full performance VMs - Without KVM (software emulation): Slower VMs using QEMU emulation

Enable software emulation if nodes lack KVM:

kubevirt:
  configuration:
    developerConfiguration:
      useEmulation: true

Can I run this on ARM64 nodes?¶

Yes, with limitations. Enable the MultiArchitecture feature gate:

kubevirt:
  configuration:
    developerConfiguration:
      featureGates:
        - MultiArchitecture

Note: Not all VM images support ARM64.

Troubleshooting¶

Check Installation Status¶

# Check KubeVirt CR
kubectl get kubevirt -n kubevirt

# Check all components
kubectl get pods -n kubevirt

# Check operator logs
kubectl logs -n kubevirt deployment/virt-operator

# Check virt-handler logs
kubectl logs -n kubevirt daemonset/virt-handler

Common Issues¶

1. Nodes without KVM support¶

Error: VMs fail to start with "KVM not available"

Solution: Enable software emulation (not recommended for production):

kubevirt:
  configuration:
    developerConfiguration:
      useEmulation: true

2. VM fails to start due to CPU model¶

Error: "Requested CPU model is not supported"

Solution: Use a more compatible CPU model:

kubevirt:
  configuration:
    cpuModel: "host-passthrough"

3. Migration failures¶

Error: "Migration failed due to incompatible nodes"

Solution: Ensure nodes have compatible CPU features or use live migration policies.

Debug Commands¶

# Describe VM
kubectl describe vm <vm-name>

# Describe VMI (running instance)
kubectl describe vmi <vm-name>

# Check virt-launcher logs
kubectl logs <virt-launcher-pod>

# Check events
kubectl get events -n <namespace> --sort-by='.lastTimestamp'

Uninstallation¶

# Delete all VMs first
kubectl delete vms --all -A

# Uninstall the chart
helm uninstall kubevirt -n kubevirt

# Delete namespace (if desired)
kubectl delete namespace kubevirt

Note: By default, CRDs are kept on uninstallation to prevent data loss (helm.sh/resource-policy: keep). To also remove CRDs: kubectl delete crd kubevirts.kubevirt.io

Upgrading from v0.1.x to v0.2.0¶

In v0.2.0, CRD management moved from crds/ directory (install-only) to regular Helm templates (updated on every upgrade). Existing installations need a one-time CRD adoption before upgrading:

# Annotate the existing CRD for Helm adoption (one-time, safe, idempotent)
kubectl annotate crd kubevirts.kubevirt.io \
  meta.helm.sh/release-name=kubevirt \
  meta.helm.sh/release-namespace=kubevirt \
  --overwrite
kubectl label crd kubevirts.kubevirt.io \
  app.kubernetes.io/managed-by=Helm \
  --overwrite

# Then upgrade normally
helm upgrade kubevirt encircle360-oss/kubevirt -n kubevirt -f values.yaml

Helmfile users can automate this with a presync hook (see examples in the repository).

Migration from Raw Manifests¶

If you're currently using raw KubeVirt manifests:

1. Export current configuration:

   kubectl get kubevirt kubevirt -n kubevirt -o yaml > current-config.yaml
   

2. Create values.yaml from current config: Review your current KubeVirt CR and translate settings to Helm values.

3. Uninstall existing KubeVirt:

   kubectl delete kubevirt kubevirt -n kubevirt
   kubectl delete deployment virt-operator -n kubevirt
   

4. Install using Helm:

   helm install kubevirt encircle360-oss/kubevirt \
     -f values.yaml \
     --namespace kubevirt
   

Breaking Changes¶

v1.6.x → v1.7.0¶

• Minimum Kubernetes version: Bumped from 1.30 to 1.32
• GA Feature Gates: Snapshot, VMExport, HotplugVolumes, and NodeRestriction are now GA. Listing them in featureGates is harmless but unnecessary.
• Operator scheduling: Default affinity now prefers control-plane nodes. Override with operator.affinity if needed.
• Instancetype API versions: v1alpha1 and v1alpha2 instancetype APIs are removed. Migrate to v1beta1 or v1 before upgrading.

v1.5.0 → v1.6.x¶

• VirtualMachineInstanceMigration RBAC: Namespace admins no longer have default permissions to create/edit/delete migrations. Grant explicitly if needed.

Resources¶

• KubeVirt Documentation
• KubeVirt API Reference
• Feature Gates Documentation
• Chart Repository
• Issue Tracker

Contributing & Maintainership¶

We Welcome Contributors! 🎉¶

We're actively seeking contributors and co-maintainers for this KubeVirt chart! Whether you want to: - Become a co-maintainer for this chart - Submit pull requests for bug fixes, features, or documentation improvements - Help with testing virtualization features and VM workloads - Improve documentation with real-world VM deployment examples - Share your virtualization and KubeVirt expertise with the community

Your expertise is valuable! Whether you're a virtualization expert, a KubeVirt user, or someone passionate about running VMs on Kubernetes - we'd love your contribution.

Become a Chart Co-Maintainer¶

Interested in becoming a co-maintainer for this KubeVirt chart? We'd be excited to collaborate!

What we're looking for: - Experience with virtualization (KVM, QEMU, libvirt) or KubeVirt - Strong Kubernetes and Helm knowledge - Understanding of VM networking, storage, and live migration - Willingness to review PRs and help with issues - Passion for bringing virtualization to Kubernetes

How to get involved: - Start by contributing PRs or helping in issues/discussions - Reach out to us at oss@encircle360.com expressing your interest - We'll work together to onboard you as a co-maintainer

We especially welcome virtualization experts who can help users successfully run VMs on Kubernetes!

Support & Professional Services¶

Community Support¶

For issues and questions about this Helm chart: - Matrix Community: Join our Matrix Chat to chat with maintainers and other users - Chart Issues: Create an Issue for Helm chart bugs and feature requests - General Questions: Start a Discussion for questions and general support

For KubeVirt specific issues: - Visit the KubeVirt GitHub repository - Check the KubeVirt documentation - Join the KubeVirt Slack channel

Professional Support¶

For professional support, consulting, custom development, or enterprise solutions, contact hello@encircle360.com

Disclaimer¶

⚠️ This chart is under active development and NOT production-ready.

This Helm chart is provided "AS IS" without warranty of any kind. encircle360 GmbH and the contributors: - Make no warranties about the completeness, reliability, or accuracy of this chart - Are not liable for any damages arising from the use of this chart - Strongly recommend thorough testing in non-production environments only - Do not recommend this chart for production use at this time - This chart requires expert-level Kubernetes and virtualization knowledge

Use this chart at your own risk. For production-ready virtualization solutions with SLA requirements, contact our professional support services at hello@encircle360.com

Maintainers¶

Source Code¶

• https://github.com/encircle360-oss/helm-charts
• https://github.com/kubevirt/kubevirt
• https://kubevirt.io

Requirements¶

Kubernetes: >=1.32.0-0

Values¶