kubevirt¶

KubeVirt - Virtual Machine Management on Kubernetes - Deploy and manage VMs as native Kubernetes resources

Homepage: https://github.com/encircle360-oss/helm-charts/tree/main/charts/kubevirt

⚠️ UNDER CONSTRUCTION This chart is currently under active development and has not been battle-tested in production environments. NOT PRODUCTION READY - Use at your own risk and thoroughly test in non-production environments first. KubeVirt is a complex system requiring deep Kubernetes and virtualization knowledge. Only use this chart if you are an experienced Kubernetes operator.

Description¶

KubeVirt is a Kubernetes add-on that enables you to run and manage virtual machines alongside container workloads. This Helm chart provides a deployment of KubeVirt v1.6.2 with comprehensive configuration options.

Key Features: - Full KubeVirt v1.6.2 support with all feature gates - Operator-based lifecycle management - Comprehensive RBAC configuration - Monitoring integration (ServiceMonitor & PrometheusRule) - High availability configuration - Extensive configuration options

Prerequisites¶

• Kubernetes 1.30+ (vanilla Kubernetes, K3s, K0s, or OpenShift)
• Helm 3.8+
• Nodes with KVM support (hardware virtualization) OR software emulation enabled
• Sufficient cluster resources for VM workloads

Platform Compatibility: This chart works on both vanilla Kubernetes (including K3s, K0s) and OpenShift. OpenShift-specific RBAC rules are included but will be safely ignored on non-OpenShift clusters.

Check Node Virtualization Support:

# Check if nodes have KVM support
kubectl get nodes -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.allocatable.devices\.kubevirt\.io/kvm}{"\n"}{end}'

# Verify KVM is available on nodes
kubectl debug node/<node-name> -it --image=ubuntu -- bash
apt update && apt install -y cpu-checker
kvm-ok

Installation¶

Quick Start¶

# Add the Helm repository
helm repo add encircle360-oss https://encircle360-oss.github.io/helm-charts
helm repo update

# Install KubeVirt
helm install kubevirt encircle360-oss/kubevirt \
  --namespace kubevirt \
  --create-namespace

# Wait for KubeVirt to be ready
kubectl wait kubevirt kubevirt -n kubevirt \
  --for=condition=Available \
  --timeout=10m

Installation with Custom Values¶

helm install kubevirt encircle360-oss/kubevirt \
  --namespace kubevirt \
  --create-namespace \
  -f custom-values.yaml

Install with Monitoring¶

# values-monitoring.yaml
monitoring:
  enabled: true
  namespace: monitoring
  serviceAccount: prometheus-k8s
  prometheusRule:
    enabled: true
    labels:
      prometheus: kube-prometheus

helm install kubevirt encircle360-oss/kubevirt \
  -f values-monitoring.yaml \
  --namespace kubevirt \
  --create-namespace

Configuration¶

Basic Configuration¶

Feature Gates¶

KubeVirt uses feature gates to control optional functionality. Feature gates are categorized as: - Alpha: Experimental features that must be explicitly enabled - Beta: Features that are stable but may have limitations - GA: Stable features (enabled by default, no gate needed)

Alpha Feature Gates (v1.6.2)¶

kubevirt:
  configuration:
    developerConfiguration:
      featureGates:
        # Storage
        - ExpandDisks              # Dynamic disk expansion
        - HotplugVolumes           # Hot-plug/unplug volumes
        - HostDisk                 # Host disk access
        - ImageVolume              # Image volume support (New in v1.6.0)

        # Performance
        - CPUManager               # CPU pinning and NUMA topology
        - AlignCPUs                # Align guest and host CPU topology

        # Hardware
        - HostDevices              # PCI device passthrough
        - GPUsWithDRA              # GPU with Dynamic Resource Allocation (New in v1.6.0)
        - HostDevicesWithDRA       # Host devices with DRA (New in v1.6.0)

        # Security
        - WorkloadEncryptionSEV    # AMD SEV memory encryption
        - KubevirtSeccompProfile   # Custom seccomp profiles
        - SecureExecution          # IBM Secure Execution (New in v1.6.0)

        # Snapshot & Export
        - Snapshot                 # VM snapshot support (Beta since v1.3.0)
        - VMExport                 # VM export functionality (Beta since v1.3.0)

        # Networking
        - VSOCK                    # AF_VSOCK for host-guest communication
        - PasstIPStackMigration    # PASST IP stack migration (New in v1.6.0)

        # Other
        - ExperimentalIgnitionSupport  # Ignition config support
        - HypervStrictCheck        # Strict Hyper-V feature checking
        - Sidecar                  # Sidecar container injection
        - DownwardMetrics          # Expose metrics to VMs
        - Root                     # Run virt-launcher as root
        - DisableMDEVConfiguration # Disable automatic MDEV configuration
        - PersistentReservation    # SCSI persistent reservations
        - MultiArchitecture        # Multi-architecture support
        - NodeRestriction          # Node restriction (Beta since v1.6.0)
        - VirtIOFSConfigVolumesGate    # VirtioFS for config volumes
        - VirtIOFSStorageVolumeGate    # VirtioFS for storage volumes
        - DecentralizedLiveMigration   # Decentralized live migration
        - ObjectGraph              # Object graph feature (New in v1.6.0)
        - DeclarativeHotplugVolumes    # Declarative hotplug volumes
        - VideoConfig              # Video device configuration (New in v1.6.0)
        - PanicDevices             # Panic device support (New in v1.6.0)

GA Feature Gates (No Configuration Needed)¶

The following features are stable and enabled by default in v1.6.2. You do NOT need to specify these in feature gates:

• LiveMigration - Live migration of VMs
• SRIOVLiveMigration - Live migration with SR-IOV
• NonRoot - Run as non-root user
• PSA - Pod Security Admission
• CPUNodeDiscovery - CPU node discovery
• NUMA - NUMA topology support
• GPU - GPU passthrough
• VMLiveUpdateFeatures - Live update VM features (GA in v1.5.0)
• CommonInstancetypesDeploymentGate - Common instance types (GA in v1.4.0)
• HotplugNICs - Hot-plug network interfaces (GA in v1.4.0)
• BochsDisplayForEFIGuests - Bochs display for EFI (GA in v1.4.0)
• AutoResourceLimitsGate - Automatic resource limits (GA in v1.5.0)
• NetworkBindingPlugins - Network binding plugins (GA in v1.5.0)
• DynamicPodInterfaceNaming - Dynamic pod interface naming (GA in v1.5.0)
• VolumesUpdateStrategy - Volumes update strategy (GA in v1.5.0)
• VolumeMigration - Volume migration (GA in v1.5.0)
• InstancetypeReferencePolicy - Instance type reference policy (GA in v1.6.0)

CPU Configuration¶

kubevirt:
  configuration:
    # Default CPU model for VMs
    cpuModel: "host-passthrough"

    # Default CPU request
    cpuRequest: "100m"

    # Mark obsolete CPUs as unusable
    obsoleteCPUModels:
      pentium: true
      pentium2: true
      pentium3: true
      Conroe: true
      Penryn: true

Network Configuration¶

kubevirt:
  configuration:
    network:
      # Default network interface type
      defaultNetworkInterface: "masquerade"

      # Allow bridge on pod network
      permitBridgeInterfaceOnPodNetwork: false

      # Allow SLIRP interface
      permitSlirpInterface: false

Live Migration Configuration¶

kubevirt:
  configuration:
    migration:
      # Disable TLS (not recommended for production)
      disableTLS: false

      # Allow auto-converge for slow migrations
      allowAutoConverge: false

      # Bandwidth limit per migration
      bandwidthPerGiB: "64Mi"

      # Timeouts
      completionTimeoutPerGiB: 800
      progressTimeout: 150

      # Parallelism
      parallelMigrationsPerCluster: 5
      parallelOutboundMigrationsPerNode: 2

      # Post-copy migration (use with caution)
      allowPostCopy: false

      # Dedicated migration network
      network: ""

Storage Configuration¶

kubevirt:
  configuration:
    # Storage class for VM snapshots and state
    vmStateStorageClass: "standard"

Host Device Passthrough¶

kubevirt:
  configuration:
    permittedHostDevices:
      # PCI devices
      pciHostDevices:
        - pciVendorSelector: "10DE:1EB8"  # NVIDIA Tesla T4
          resourceName: "nvidia.com/T4"
        - pciVendorSelector: "8086:1572"  # Intel X710
          resourceName: "intel.com/X710"

      # Mediated devices (vGPU)
      mediatedDevices:
        - mdevNameSelector: "GRID T4-1Q"
          resourceName: "nvidia.com/GRID_T4-1Q"

Node Placement¶

Infrastructure Components (virt-api, virt-controller)¶

kubevirt:
  infra:
    nodePlacement:
      nodeSelector:
        node-role.kubernetes.io/control-plane: ""
      tolerations:
        - key: node-role.kubernetes.io/control-plane
          effect: NoSchedule

Workload Components (virt-handler, virt-launcher)¶

kubevirt:
  workloads:
    nodePlacement:
      nodeSelector:
        kubevirt.io/vm-workload: "true"
      tolerations:
        - key: kubevirt.io/vm-workload
          effect: NoSchedule

Monitoring Configuration¶

monitoring:
  enabled: true
  namespace: monitoring
  serviceAccount: prometheus-k8s

  serviceMonitorLabels:
    prometheus: kube-prometheus

  scrapeInterval: "30s"

  prometheusRule:
    enabled: true
    labels:
      prometheus: kube-prometheus

    # Add custom alerting rules
    additionalRules:
      - alert: MyCustomAlert
        expr: up == 0
        for: 5m
        labels:
          severity: critical

Usage Examples¶

Creating a Virtual Machine¶

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: ubuntu-vm
spec:
  running: true
  template:
    metadata:
      labels:
        kubevirt.io/vm: ubuntu-vm
    spec:
      domain:
        devices:
          disks:
            - name: containerdisk
              disk:
                bus: virtio
            - name: cloudinitdisk
              disk:
                bus: virtio
        resources:
          requests:
            memory: 2Gi
            cpu: 2
      volumes:
        - name: containerdisk
          containerDisk:
            image: quay.io/containerdisks/ubuntu:22.04
        - name: cloudinitdisk
          cloudInitNoCloud:
            userData: |
              #cloud-config
              password: ubuntu
              chpasswd: { expire: False }
              ssh_pwauth: True

Using virtctl¶

# Install virtctl
kubectl krew install virt

# Start/Stop VMs
kubectl virt start ubuntu-vm
kubectl virt stop ubuntu-vm
kubectl virt restart ubuntu-vm

# Console access
kubectl virt console ubuntu-vm

# VNC access
kubectl virt vnc ubuntu-vm

# SSH into VM (requires SSH service in VM)
kubectl virt ssh ubuntu@ubuntu-vm

VM Snapshots (requires Snapshot feature gate)¶

# Enable snapshot feature
kubevirt:
  configuration:
    developerConfiguration:
      featureGates:
        - Snapshot

# Create snapshot
apiVersion: snapshot.kubevirt.io/v1alpha1
kind: VirtualMachineSnapshot
metadata:
  name: ubuntu-vm-snapshot
spec:
  source:
    apiGroup: kubevirt.io
    kind: VirtualMachine
    name: ubuntu-vm

Upgrading¶

Upgrade the Chart¶

helm repo update
helm upgrade kubevirt encircle360-oss/kubevirt \
  --namespace kubevirt \
  -f values.yaml

Upgrade Strategy¶

KubeVirt supports rolling updates of VMs during upgrades:

kubevirt:
  workloadUpdateStrategy:
    workloadUpdateMethods:
      - LiveMigrate  # Try live migration first
      - Evict        # Fall back to eviction
    batchEvictionSize: 10
    batchEvictionInterval: "1m"

Frequently Asked Questions (FAQ)¶

Does this work on K3s/K0s/vanilla Kubernetes?¶

Yes! This chart is designed to work on: - Vanilla Kubernetes (upstream) - K3s (lightweight Kubernetes) - K0s (zero friction Kubernetes) - MicroK8s - OpenShift/OKD

The chart includes OpenShift-specific RBAC rules, but these are safely ignored on non-OpenShift clusters.

Why are there OpenShift resources in the RBAC?¶

KubeVirt officially supports both vanilla Kubernetes and OpenShift. The OpenShift-specific resources (security.openshift.io/securitycontextconstraints, route.openshift.io/routes) are: - Optional and only used on OpenShift - Ignored on vanilla Kubernetes/K3s (API groups don't exist) - Standard practice in the official KubeVirt manifests

Your K3s cluster will simply skip these rules - no issues!

Why do some labels have empty values like operator.kubevirt.io: ""?¶

This is valid and intentional in Kubernetes! Empty-string label values are used as: - Marker labels: Indicate that something is tagged without needing a specific value - Selector labels: Can be matched with matchLabels: { "operator.kubevirt.io": "" } - KubeVirt convention: How KubeVirt identifies its own resources internally

Do I need hardware virtualization (KVM)?¶

Recommended but not required: - With KVM (hardware virtualization): Full performance VMs - Without KVM (software emulation): Slower VMs using QEMU emulation

Enable software emulation if nodes lack KVM:

kubevirt:
  configuration:
    developerConfiguration:
      useEmulation: true

Can I run this on ARM64 nodes?¶

Yes, with limitations. Enable the MultiArchitecture feature gate:

kubevirt:
  configuration:
    developerConfiguration:
      featureGates:
        - MultiArchitecture

Note: Not all VM images support ARM64.

Troubleshooting¶

Check Installation Status¶

# Check KubeVirt CR
kubectl get kubevirt -n kubevirt

# Check all components
kubectl get pods -n kubevirt

# Check operator logs
kubectl logs -n kubevirt deployment/virt-operator

# Check virt-handler logs
kubectl logs -n kubevirt daemonset/virt-handler

Common Issues¶

1. Nodes without KVM support¶

Error: VMs fail to start with "KVM not available"

Solution: Enable software emulation (not recommended for production):

kubevirt:
  configuration:
    developerConfiguration:
      useEmulation: true

2. VM fails to start due to CPU model¶

Error: "Requested CPU model is not supported"

Solution: Use a more compatible CPU model:

kubevirt:
  configuration:
    cpuModel: "host-passthrough"

3. Migration failures¶

Error: "Migration failed due to incompatible nodes"

Solution: Ensure nodes have compatible CPU features or use live migration policies.

Debug Commands¶

# Describe VM
kubectl describe vm <vm-name>

# Describe VMI (running instance)
kubectl describe vmi <vm-name>

# Check virt-launcher logs
kubectl logs <virt-launcher-pod>

# Check events
kubectl get events -n <namespace> --sort-by='.lastTimestamp'

Uninstallation¶

# Delete all VMs first
kubectl delete vms --all -A

# Uninstall the chart
helm uninstall kubevirt -n kubevirt

# Delete namespace (if desired)
kubectl delete namespace kubevirt

Note: By default, CRDs are kept on uninstallation to prevent data loss.

Migration from Raw Manifests¶

If you're currently using raw KubeVirt manifests:

1. Export current configuration:

   kubectl get kubevirt kubevirt -n kubevirt -o yaml > current-config.yaml
   

2. Create values.yaml from current config: Review your current KubeVirt CR and translate settings to Helm values.

3. Uninstall existing KubeVirt:

   kubectl delete kubevirt kubevirt -n kubevirt
   kubectl delete deployment virt-operator -n kubevirt
   

4. Install using Helm:

   helm install kubevirt encircle360-oss/kubevirt \
     -f values.yaml \
     --namespace kubevirt
   

Breaking Changes¶

v1.5.0 → v1.6.x¶

• VirtualMachineInstanceMigration RBAC: Namespace admins no longer have default permissions to create/edit/delete migrations. Grant explicitly if needed.

Resources¶

• KubeVirt Documentation
• KubeVirt API Reference
• Feature Gates Documentation
• Chart Repository
• Issue Tracker

Contributing & Maintainership¶

We Welcome Contributors! 🎉¶

We're actively seeking contributors and co-maintainers for this KubeVirt chart! Whether you want to: - Become a co-maintainer for this chart - Submit pull requests for bug fixes, features, or documentation improvements - Help with testing virtualization features and VM workloads - Improve documentation with real-world VM deployment examples - Share your virtualization and KubeVirt expertise with the community

Your expertise is valuable! Whether you're a virtualization expert, a KubeVirt user, or someone passionate about running VMs on Kubernetes - we'd love your contribution.

Become a Chart Co-Maintainer¶

Interested in becoming a co-maintainer for this KubeVirt chart? We'd be excited to collaborate!

What we're looking for: - Experience with virtualization (KVM, QEMU, libvirt) or KubeVirt - Strong Kubernetes and Helm knowledge - Understanding of VM networking, storage, and live migration - Willingness to review PRs and help with issues - Passion for bringing virtualization to Kubernetes

How to get involved: - Start by contributing PRs or helping in issues/discussions - Reach out to us at oss@encircle360.com expressing your interest - We'll work together to onboard you as a co-maintainer

We especially welcome virtualization experts who can help users successfully run VMs on Kubernetes!

Support & Professional Services¶

Community Support¶

For issues and questions about this Helm chart: - Open an issue in GitHub Issues - Start a discussion in GitHub Discussions

For KubeVirt specific issues: - Visit the KubeVirt GitHub repository - Check the KubeVirt documentation - Join the KubeVirt Slack channel

Professional Support¶

For professional support, consulting, custom development, or enterprise solutions, contact hello@encircle360.com

Disclaimer¶

⚠️ This chart is under active development and NOT production-ready.

This Helm chart is provided "AS IS" without warranty of any kind. encircle360 GmbH and the contributors: - Make no warranties about the completeness, reliability, or accuracy of this chart - Are not liable for any damages arising from the use of this chart - Strongly recommend thorough testing in non-production environments only - Do not recommend this chart for production use at this time - This chart requires expert-level Kubernetes and virtualization knowledge

Use this chart at your own risk. For production-ready virtualization solutions with SLA requirements, contact our professional support services at hello@encircle360.com

Maintainers¶

Source Code¶

• https://github.com/encircle360-oss/helm-charts
• https://github.com/kubevirt/kubevirt
• https://kubevirt.io

Requirements¶

Kubernetes: >=1.30.0-0

Values¶