The Nested Lab

The full VMware Cloud Foundation stack, engineered live. Written from a nested VCF 9.x lab that the team at Comms-care builds and looks after — a platform where every consultant gets a complete VCF instance of their own to demo, prove and break. The standing rule: every command, every error and every fix you read here was captured from a running environment.

What gets covered — because all of it runs in the lab:

  • Foundation — VCF 9 bringup from the Installer API, SDDC Manager, vCenter and nested ESXi, vSAN ESA, fleet components, identity.
  • Networking — NSX and NSX VPCs: isolation, overlapping addressing, trunk subnets, transit gateways, load balancers.
  • Automation — VCF Automation both ways: classic VM Apps (Aria templates, vRO, event broker) and supervisor-native All Apps (CCI blueprints, VM Service, the catalog).
  • Kubernetes — VKS clusters, supervisor services, Harbor, packages, air-gapped upgrades.
  • Operations — VCF Operations, Operations for Logs, fluent-bit, Telegraf, and metrics pipelines for things that don’t have adapters (like vLLM).
  • AI infrastructure — GPU Operator, vGPU licensing and LLM serving on VKS, including in dark sites.

New here? Start here: reading paths by what you are trying to do. Or browse by product, series or month below.

Browse the lab notes

By month

November 20265
October 20269
September 202623

Full archive · Search

Org Builder's Student progress page: a bar per part of the runbook showing how many students have reached it, and a row per student giving where they are, their milestones as a row of squares, and flags such as Ahead or No change for 52 min

A VCF lab each

What a student actually receives: a private nested VCF stack, separated by NSX VPCs and the site’s own directory, handed out phase by phase so a lesson starts where the trainer wants it to. How many run at once is up to the hardware.

11 November 2026 · Adam Stretton
VCF Ready's Step 4, Customer Deliverables and Export Package, for the example estate: the Download customer package (.zip) button, the ticks to include the answers file and to anonymise names and addresses, the engagement file, the customer name Example Freight Group and the first three cards, for the Word report, the findings workbook and the evidence request; the step reads 12 Deliverables.

The deliverables, and sharing them without the customer

Part 3 of VCF Ready: what Step 4 hands over, who each deliverable is for, why they all tell the same story, and how an assessment leaves the customer’s site without the customer in it.

11 November 2026 · Adam Stretton
The VCF Ready home page with the example estate loaded: the header reads 4 Sources, 13 Clusters, 65 Hosts (2 standalone) and 596 VMs; four steps run from Estate Discovery and Scope through Readiness and Findings (6 Blockers) and Upgrade Plan and Sizing to Deliverables and Reports; and Step 1 offers live vCenter discovery or imported files.

Introducing VCF Ready

Are you VCF ready? Are you sure? The answer is spread across hundreds of Broadcom articles and pages. Introducing VCF Ready, our upgrade readiness tool: what it reads, what it finds, how it turns open questions into a workshop, and why every check cites Broadcom.

9 November 2026 · Adam Stretton
VCF Ready's Step 3, Upgrade Plan and Sizing, with its four sections closed. Route and target reads VCF 9.1.1.0, Converge into New VCF Fleet, from vSphere 8.x / 7.x. Sizing reads 26 appliances, 274 vCPU, 896 GB RAM, 33.4 TB on vSAN ESA, and 2 of 13 clusters fit with a host failed. Upgrade runbook reads 8 phases to 9.1.1.0. Architecture recommends VCF Fleet with Multiple Sites Across Multiple Regions.

The upgrade plan and sizing, from Broadcom's own sources

Part 1 read the estate and found what stands in the way. Part 2 is the plan: how VCF Ready brings Broadcom’s VCF Upgrade Planner, Planning and Preparation Workbook and design blueprints onto one page, filled in from the estate, so the route, the phases, the size of the management domain and the hosts to buy are all in view.

9 November 2026 · Adam Stretton
Org Builder's Home page: four tiles reading Make a site file, Create a package, Install or upgrade and Move an organization, a Run a class panel below them, and a sidebar listing the install steps Connect, Discover, Packages, Choose, Validate and Plan

A VCF classroom from one site file

A classroom needs a complete VCF stack per student, and hand-building them does not scale. One site file, a catalog of nested labs handed out phase by phase, and a kit that installs with no internet at all.

4 November 2026 · Adam Stretton
site-f06.yaml feeds gen_lab_blueprint.py, which writes eight blueprints and two property groups; each request builds a lab in its own VPC with a jump host, dc01, four nested hosts, vCenter, VCF Operations and AD, routed by the VPC gateway with no router VM

Nested labs in VCF Automation 9.1, built from code instead of captured

A VCF Automation 9.1 catalog that builds a complete nested VCF lab per request, each in its own VPC. How it differs from capturing a hand-built lab, and what building from code buys you.

28 October 2026 · Adam Stretton
Everything that has to cross the airlock: images, signatures, node OVAs, kernel headers, licences

VKS and the GPU Operator in a dark site: the whole airlock

Upgrading VKS, Kubernetes and the NVIDIA GPU Operator with no internet means five supply chains arriving intact. Each has a trap that passes every check until activation. This is the map.

21 October 2026 · Adam Stretton
Org Builder's Home page, four ways in, with the Install or upgrade tile highlighted and its note: Install, upgrade or uninstall an organization from packages

Install, upgrade, uninstall: a whole lab catalog with Org Builder

The nested-lab catalog as three Org Builder packages that install, upgrade and uninstall a VCF Automation organization. How made and found keep an uninstall honest, and what two live runs proved.

18 October 2026 · Adam Stretton
Two tarballs that look the same: manifest.json vs oci-layout

imgpkg tar vs OCI archive: why your air-gap tarballs won't load

Two tarballs full of container images, both bound for the same Harbor. imgpkg loads one and rejects the other. The layout difference nobody explains, what survives a copy, and three ways across.

14 October 2026 · Updated 18 October 2026 · Adam Stretton
Telegraf runs the script every 60 s; the script's stdout is one integer; the integer is the health of the integration

vLLM metrics into VCF Operations, part 2: monitor the monitor

Part 1’s design only helps if it runs every minute, for ever, and says when it stops. Self-monitoring through Telegraf, batched pushes, the four alerts for day one, and an offline test mode.

7 October 2026 · Adam Stretton