Managed node pools

Early access. Managed node pools are rolling out behind an organization flag. If the commands below report that the surface is unavailable, ask us to enable it for your organization.

A managed node pool is a group of worker nodes the platform provisions, heals, and autoscales in your own cloud account. You store a cloud credential once, set a floor and a ceiling, and the cluster adds nodes when pods are pending and removes them when they go idle. The machines run in your account and count against your cloud bill; we run the elasticity, the joins, and the repair.

This is the complement of joining your own nodes, not a replacement — a cluster can mix both. Bring your own machines for the baseline, and let a pool absorb the spikes.

What a pool needs: a cloud account we can provision in (Hetzner Cloud today) and a cloud credential stored on your organization. That’s it — we build the machines, join them over the same outbound fabric as any node, and install the CNI. You never run an autoscaler or write per-cloud glue.

Store a cloud credential

A cloud credential is a cloud API token, held at the organization level and shared by every pool that names it. It is write-only: you set it on create or rotate, and it is never shown again — the platform stores it, probes it, and reports only its health and how many pools use it.

lbr cloud-credential create hetzner-prod --token <hetzner-api-token>

Use a Hetzner Cloud API token with read/write on the project you want the nodes in. List your credentials (health and usage, never the secret), rotate the token in place, or remove one you no longer need:

lbr cloud-credential list
lbr cloud-credential rotate hetzner-prod --token <new-token>
lbr cloud-credential delete hetzner-prod

A delete is refused while any pool still references the credential — remove the pools first. Rotating never disturbs running pools; they pick up the new token on the next reconcile.

Prefer the browser? The Console stores and rotates credentials under Organization → Cloud credentials, listing each by presence and health.

Create a pool

Point a pool at a cloud credential, choose where its machines live, and set the autoscaler bounds:

lbr cp pool create prod workers \
  --credential hetzner-prod \
  --location hel1 --server-type cx23 \
  --min 1 --max 5

The platform provisions the first machines, and each one self-registers over the outbound fabric — the same NAT-friendly, no-inbound join any node uses — so pods schedule onto them within a couple of minutes. --min/--max are the autoscaler’s floor and ceiling; --initial seeds the starting size once (it defaults to --min and is never reconciled after). Watch the pool come up:

lbr cp pool list prod        # or: kubectl get nodes

How scaling works

The cluster runs a cluster-autoscaler for you. When pods can’t schedule for want of capacity, the pool adds nodes — up to --max. When nodes sit idle, it drains and removes them — down to --min. Set --min 0 and a pool scales to zero when there’s nothing to run, and back up from cold on the next pending pod. You don’t deploy, configure, or upgrade the autoscaler; it’s part of the managed promise.

Labels, taints, and placement

Give a pool’s nodes labels and taints so workloads land where you intend — a GPU pool, a batch pool, a spot-priced pool:

lbr cp pool create prod batch \
  --credential hetzner-prod --location hel1 --server-type cx23 \
  --min 0 --max 10 \
  --label workload=batch \
  --taint dedicated=batch:NoSchedule

The autoscaler understands the labels and taints even when the pool is at zero, so it only scales a pool up for pods that actually belong on it.

The managed lifecycle

Once a pool exists, the platform keeps it healthy without you:

• Auto-repair. A node that goes NotReady and stays there is replaced — the machine is deleted, a fresh one is provisioned and rejoins, and the stale node object is reaped.
• Upgrade-follow. When your control plane’s Kubernetes version moves, the pool rolls its nodes to match — replace, not in-place patch — respecting pod disruption budgets as it drains.
• Pause. Freeze a pool — no scaling, no repair, no upgrade — when you want it held exactly where it is:

lbr cp pool update prod workers --paused    # freeze
lbr cp pool update prod workers --paused=false

Update a pool

min, max, labels, taints, and paused are mutable; the cloud credential and the machine placement are fixed for a pool’s life (make a new pool to change them). Raise the ceiling as your workload grows:

lbr cp pool update prod workers --max 10

Delete a pool

lbr cp pool delete prod workers

This drains and terminates the pool’s machines in your cloud and tears down the elasticity around them. The nodes are yours — nothing lingers in your account afterward.

What you pay

Management is free: we don’t mark up or tax your nodes, and there’s no per-node fee. You pay your cloud provider directly for the machines a pool runs, at their price. We meter two dimensions — pool management and managed-node hours — so usage is visible in the Console, but nothing is billed during early access.

Troubleshooting

• Pool stuck in Provisioning. Check the cloud credential is Valid (lbr cloud-credential list) and that your cloud project has quota for the server type. A brand-new cloud project is often rate-limited on rapid creates — give it a moment.
• InstanceTypeUnavailable. The --server-type isn’t offered in that --location. Pick a type the location sells (for Hetzner, e.g. cx23 in hel1).
• A credential won’t delete. It’s still referenced by a pool — delete the pools that use it first.
• Nodes join but stay NotReady. The CNI is installing; give it a minute. If it persists, check the nodes’ clocks are in sync.