Pet VMs in Kubernetes? WTH
Fabian Deutsch, Red Hat
@dummdida - dummida.tumblr.com
- Pet VMs
- Kubernetes & Containers
- What they share
- How they differ
- A first try
- A good representation
- And how to add the functionality
- Many knobs and switches
- Hand crafted - VM and it's contents
"Oh! I deleted the VM. $EXPLETIVE"
- It's all about running containers
- Images become ready to use containers
"Oh - I deleted the container. And here it is again."
- The assumptions of Kubernetes do not apply to pet VMs
- Pet VMs can
Strongest point: The assumptions about them differ
Scheduling ~ Storage ~ Network ~ Host life-cycle
… they have much in common: Infrastructure.
Cluster, Schedule, Storage, Network, …
Even if the workload is different, can both workloads share the same infrastructure?
Let's see how we can run both workloads on the same infra. Would allow sharing.
Try: Don't run a pod but a pet VM instead
Keep everything Kubernetes, just replace the launcher
Let's see pods as VM representations and let's get the kubelet to launch VMs.
Representation + Handling
- Tried to use Pod Spec to launch a VM
- Before CRI ...
- Representation difficulties
- Boot order
- Technical difficulties
- Storage - file-system vs block storage
- Assumption: Stateless
- Virtualization add-on to Kubernetes
- Resources, controllers & daemons
$ kubectl create -f - <<EOF kind: VM
metadata: name: testvm
spec: nodeSelector: kubernetes.io/hostname: master domain: devices: interfaces: - source: network: default type: network video: - model: type: qxl heads: 1 graphics: - autoPort: 'yes' defaultMode: any type: spice memory: unit: KiB value: 8192 os: type: os: hvm type: qemu
RE: VM Representation
VM: 3rd Party Resource
OTHER EXAMPLES? LEGIT TO
- Pet VMs have many more knobs than Pods.
- Custom Type to represent VMs
- Defined and hosted by Kubernetes.
- Can be manipulated through the native Kubernetes REST API
RE: VM Handling
Controllers & Daemons
Watch for VMs, create associated pod: Ability to reuse pod features.
Handle virtualization specific cluster tasks: live-migration
Handle virtualization functionality on the node side.
Responsible for launching VMs.
Pet VMs can not be handled by Kubernetes.
Kubernetes has a core assumption of stateless workloads - this shines through everywhere.
Some functionality is missing - live migration.
Needs to be extended.
Up Next →
K8s TPR Improvements
K8s Snapshot Improvements
K8s Volume Cloning
Ouch. Issues. Ouch. Gaps.
Design allows us to solve in Kubernetes if it can be solved there.
Otherwise workaround/bypass in Controllers and Daemons
Pet VMs and Kubernetes
By Fabian Deutsch