Nuon's First Deprecation: Ripping Out Container Builds

The Dream, The Vision, and Our First Deprecation

The magic of Heroku in the 2010s was simple: you ran git push , and the platform handled everything else. When we built Nuon, we wanted that exact developer experience for enterprise SaaS. We wanted software vendors to push code and watch it deploy seamlessly inside their end-customer’s AWS, Azure, or GCP account. We wanted to make Bring Your Own Cloud simple.

To make that magic work out of the box, our early architecture included a complete container build-and-deploy pipeline alongside our delivery engine. Fetch source, build containers, package artifacts, and deploy: all handled as part of provisioning software into a customer's cloud environment.

Recently, we made a major architectural decision. For the first time in Nuon’s history, we deliberately deprecated a core feature: container builds.

Deleting features is always harder than adding them. It takes discipline to admit when an abstraction has outlived its usefulness. But real platform velocity requires trimming system components that blur fundamental boundaries. The reality is simple: BYOC platforms belong squarely in Continuous Delivery (CD). Modern software teams already build container images in dedicated CI pipelines like GitHub Actions or Buildkite. Forcing container builds inside our managed delivery infrastructure added massive drag without adding real customer value.

Why Operating Build Infrastructure Is Hard

Running long-lived build runners in our managed SaaS infrastructure seemed logical at first, but container build infrastructure is notoriously difficult to operate at scale. There is a reason a whole category of dedicated build platforms (like Depot and Namespace) exists[^1]. Building containers quickly and securely requires isolated compute, custom layer caching, rootless execution environments, and micro-VM isolation. Trying to bolt ad-hoc container builds onto a specialized BYOC delivery engine forces you to reinvent an entire build platform from scratch.

First, container workloads are notoriously unpredictable. Compiling code and assembling layers creates massive, erratic CPU spikes, Buildkit memory leaks, and OOM kills. Heavy operations like repository compression would pin runner CPUs, preventing health check heartbeats from reaching our control plane. This triggered false health check failures, marked healthy runners as dead, and stalled customer deployments.

Second, our reliance on Kaniko inside those runners actively blocked critical BYOC platform capabilities. Kaniko executes builds by unpacking container layers directly into the running pod's filesystem, mutating and wiping the roots in the process.

This behavior directly broke artifact vendoring: a core feature for enterprise BYOC that pre-packages all Terraform modules, Helm charts, and binary dependencies upfront, so customer environments never have to fetch third-party resources over the open internet at runtime. Because Kaniko wiped the execution context during image generation, we couldn't bundle vendor artifacts cleanly alongside build tasks.

Finally, there was a heavy onboarding penalty. Because every new organization required dedicated build infrastructure to be provisioned before anything else could happen, users were forced to sit through a five-minute bootstrap process just to try the product.

The Realization: It Was Just a Packaging Problem

Looking across the chronic operational fires, Kaniko’s unmaintained state, and the hard blocker on critical features like artifact vendoring, a key realization kicked in: if we got rid of container builds entirely, we were left with just a packaging problem. And file packaging isn't heavy compute: it is mostly network I/O.

Fetching files, compressing bundles, and pushing artifacts to target registries doesn't require permanent, always-on compute or dedicated per-organization[^2] runner nodes.

With that realization, we decoupled packaging completely. We moved file packaging and artifact-pushing handlers out of individual organization runners and shifted them into centralized, shared Temporal workers in our control plane. These workers operate with globally managed concurrency queues and compute profiles tuned specifically for high-throughput network I/O.

Security was preserved strictly at the boundary. Control-plane Temporal workers use temporary IAM role assumption to push packaged artifacts directly to target container registries like ECR without requiring permanent stored credentials or always-on per-organization build nodes.

This cleanly separated CI from CD. Customers bring their container artifacts straight from their existing pipelines, and Nuon takes over to orchestrate environments, infrastructure updates, and delivery across customer clouds.

The Telemetry of Subtraction

Deleting the build engine delivered immediate, measurable improvements across provisioning speed, control plane efficiency, and operating costs:

• Instant Product Usability: Setup time dropped from 5 minutes down to ~1 second. Because there is no longer any build infrastructure to bootstrap upfront, users can hop directly into the product and start deploying immediately.
• Infra Cost Saved: Decommissioning dedicated runner compute clusters and eliminating per-organization pre-provisioned VMs shaved roughly $6,000 in infrastructure costs this month alone. When factoring in current usage and projected growth, this will potentially save well over $100,000 a year going forward.
• Control-Plane and Telemetry Relief: Previously, hundreds of active runners continuously sent background heartbeats, health checks, and status logs to monitor system state. Eliminating those ping loops instantly quieted system load across our API runners and ClickHouse database:
  • 64.6% CPU load decrease on control-plane API runners alongside a 68–71% reduction in ClickHouse telemetry database inserts.
  • 74.6% reduction in database operations, a 67.5% drop in total API requests, and a 64–67% drop in total ClickHouse query traffic

What Platform Teams Can Learn

Deleting a major feature is uncomfortable, but it is often the single best decision you can make for your architecture. If you are building platform infrastructure, two key principles stand out from this shift:

First, be picky about the engineering challenges you choose to tackle. Operating container build infrastructure at scale is genuinely tough work. Unless running container builds is your core value proposition, you do not need to reinvent that platform from scratch. Meet your users where they actually are. In our case, we realized that our customers were already producing pre-built artifacts in their existing CI pipelines. Aligning with those existing workflows removed friction for our users and instantly simplified our operational overhead.

Finally, velocity comes from subtraction. The cleanest code and most reliable infrastructure is what you don't have to run. True platform maturity is knowing when to delete abstractions that no longer serve your core product vision.