How Nuon Works - Part 1, Configuring Your App

Our mission at Nuon is to enable the Bring Your Own Cloud (BYOC) deployment model for everyone, but what does that actually mean?

This is a three-part series on How Nuon Works. We discuss our platform in detail and explain how to create a Nuon application, how installs work and what tools you have for day-2 operations.

If you are considering offering BYOC for your product, or just curious what the hell we even do, then this series is for you.

What is BYOC?

BYOC is an emerging deployment model where software products are deployed directly inside a customer owned cloud account, and managed by the vendor. Done right, BYOC should have the ease of use of SaaS, but the isolation, security, and peace of mind of self-hosted. In fact, it's a bit of a hybrid of both.

BYOC is seeing the quickest adoption for AI products, data, infrastructure, and security products. Since all software and infrastructure end up being deeply connected, we at Nuon have long held conviction that eventually these early BYOC products will force connected apps to also be run in customer environments.

Building BYOC yourself

If you are a company in one of the categories I mentioned above, your company most likely already has dev-ops and operational DNA. These products are not easy to run in the cloud, and if you have a self-hosted offering you probably have had to figure out things like packaging, documenting how to run the software and more.

Let's be honest, your first instinct is probably to build BYOC yourself. Why trust a third party vendor for something so critical, you already have a leg up on?

After all, you just need to package your app, deploy it into a customer's account and push updates, right?

The Devil is in the Details

Packaging your application into something you can deploy into a customer account with an installer is just the beginning. Most of the time this means scripting together your Terraform or Infrastructure as Code (IaC), Helm charts, container images and other automations to set up your app. The customer can either run an automated installer, or manually install each piece, one by one.

While this sounds easy in practice, in the wild there are two places this breaks down:

Last Mile Automation and Testing - Automating your entire application stack requires an enormous amount of last-mile automation and testing. Testing different permutations and automating the entire thing together requires a lot more effort than just having the automation code in the first place.

Supporting customer configurations - Each customer will want to control different aspects of the application. From bringing their own network, connecting data sources, or other infrastructure.

In the cloud, you typically have one footprint - one architecture, cloud, and configuration to support. While taking your cloud automation and handing it to customers is the easiest way to get started, you quickly outgrow this with customer configuration and deployment requirements and figuring out how to reach the coveted 100% automation of your product (after all, the dream is that it's just a single click to install right?).

Day-2 and Security

Once you have deployed your application into a customer's cloud account, you (the vendor) are tasked with operating your software and making it ... just work. Your customer is paying you to make it feel like SaaS after all.

However, at the same time this software is in your customer's account, your customer most likely will not grant you direct external access. Relying on run-books and having the customer self-triage takes your offering from BYOC, back towards self-hosted.

This is where building your own BYOC offering goes from handing your customer your cloud automation and a docs page (yeah, really), to building your own internal BYOC platform. As a vendor, to offer BYOC you need to build your own tooling to figure out when things are going wrong, fix them on behalf of your customer and securely push updates and patches. As a vendor, you need to do this without having direct access to your customer's cloud environment, otherwise you are just offering a more dangerous form of Cloud SaaS.

Why we Built Nuon

Most infrastructure vendors struggle to deploy software securely in customer environments. The existing solutions are brittle: either you're giving customers a complex self-hosted nightmare, or you're compromising their security with what amounts to "god mode" access in their account.

We built Nuon to solve this fundamental infrastructure problem: how do you run software in a customer's environment without breaking their security model?

Our first attempts were naive. We thought packaging and configuration were the hard problems. In reality, the real challenge is continuous operation: how do you run, update, and maintain software in an environment where you have zero direct access?

BYOC lacks the infrastructure primitives we take for granted in SaaS. There's no standard toolkit for running software across customer environments with zero-trust access. We had to rebuild the entire operational model from scratch, designing tools that work natively in environments where you can't simply SSH or have direct system access.

What is Nuon?

Nuon gives you a workflow to package your application, deploy it into customer accounts and make it feel like SaaS. We give you the tools to not only unlock new customers, but you can continuously push updates, monitor and debug your application. We have designed Nuon with security first, and a zero-access model so your customer never has to grant you direct access to their account.

We deploy a runner alongside your application which powers the functionality of our platform. The runner is responsible for:

• Setting up and provisioning your application.
• Managing updates.
• Monitoring your application and letting you run debug scripts when things go wrong.

Nuon then gives you a control plane where you can push updates, debug when things go wrong, and proactively monitor for issues. You can offer a secure, automated experience without direct access to the customer account and toggleable permissions for your customer.

Nuon gives you secure BYOC, by default.

Package your App

Nuon lets you package your existing product into a Nuon app, which can be installed into any customer account. You can create apps that run on different clouds (AWS, Azure, GCP) and with different architectures (Kubernetes, Serverless, Containers). Basically, anything you can package with IAC and automate with scripts you can turn into a Nuon app.

App Config

You manage your Nuon app with a directory of config files that you use to tell us where to find your app, what permissions you need, the policies you want to govern with and more.

A typical app config looks something like this:

├── README.md
├── actions
│   ├── kubectl_list_pods.toml
│   ├── kubectl_logs.toml
│   ├── nuon_rds_creds.toml
│   ├── temporal_init_db.toml
├── components
│   ├── rds_cluster_terraform.toml
│   ├── image-api.toml
│   ├── helm-api.toml
├── input_groups
│   ├── auth.toml
├── inputs
│   ├── auth
├── metadata.toml
├── permissions
│   ├── deprovision.toml
│   ├── maintenance.toml
│   ├── provision.toml
├── policies
│   └── prevent-secrets-access.yaml
├── runner.toml
├── sandbox.toml
├── secrets
│   ├── auth
└── stack.toml

We will talk through each part of the config in the following sections. You can also reference our docs for more information.

Pick your Sandbox

A sandbox is the compute environment inside your customer's cloud where your application runs. Most of the time, the sandbox will be a Kubernetes cluster or a container runtime such as AWS ECS.

We provide templates for common architectures you can use, such as Kubernetes with Karpenter on AWS.

App sandboxes can work with a customer managed VPC, or a new VPC depending on which the customer picks when installing. When your app is installed in a customer account, Nuon automatically treats your sandbox with additional controls, to prevent downtime and ensure quality of service. You can do things like roll out cluster updates, support customer cluster configuration options and more with your sandbox.

Connect your Components

Components represent individual parts of your app such as a container image, Helm chart, or Terraform module. Components are stateful, and have state management and change approval built into Nuon. Your components are your app, and Nuon deploys them into the sandbox in each install.

Nuon takes each of your components and defines them into a graph, based on which components reference each other.

Let's talk through a hypothetical application: a full-stack web app that is deployed on Kubernetes using Helm. It needs a database, a s3 bucket, and has a container image. Each part of the stack will be defined with a Nuon component.

First, you define a container_image which tells Nuon where to import the container image from.

name   = "img_nuon_dashboard_ui"
type   = "container_image"

[aws_ecr]
image_url    = "431927561584.dkr.ecr.us-west-2.amazonaws.com/mono/api"
tag          = "0.19.619"
region       = "us-west-2"
iam_role_arn = "arn:aws:iam::431927561584:role/nuon-ecr-access"

When your app is installed, Nuon will sync the container image into the customer install. For air-gapped installs, apps can define all of their required images as components, so the customer install never relies directly on a third-party container image.

Next, we define a terraform component which tells Nuon where to find the IAC for our S3 bucket and database.

name              = "rds_cluster_nuon"
type              = "terraform_module"
terraform_version = "1.11.3"

[public_repo]
repo      = "nuonco/byoc"
directory = "byoc-nuon/src/components/rds_cluster_nuon"
branch    = "main"

[vars]
identifier      = "nuon-{{ .nuon.install.id }}"
port            = "5432"
db_name         = "nuonadmin"
db_user         = "nuon"
instance_class  = "{{ .nuon.install.inputs.nuon_db_instance_type }}"
subnet_group_id = "{{ .nuon.components.rds_subnet.outputs.id }}"

region     = "{{ .nuon.install_stack.outputs.region }}"
vpc_id     = "{{ .nuon.install_stack.outputs.vpc_id }}"
subnet_ids = "{{ index .nuon.sandbox.outputs.vpc.private_subnet_ids 0}},{{ index .nuon.sandbox.outputs.vpc.private_subnet_ids 1 }},{{ index .nuon.sandbox.outputs.vpc.private_subnet_ids 2 }}"

nuon_id = "{{ .nuon.install.id }}"

iam_database_authentication_enabled = "true"
deletion_protection                 = "false"  # until we're out of dev

allocated_storage = "100"
apply_immediately = "true"

When your app is installed, Nuon will automatically provision the Terraform module inside each install and manage the state, outputs, and variables.

Finally, we define a helm_chart component which tells Nuon where to find the Helm Chart, and how to configure it. You can reference outputs from other components, so you can use infrastructure resources (such as the database) in dependent components.

#:schema https://api.nuon.co/v1/general/config-schema?source=helm
name           = "ctl_api"
type           = "helm_chart"
chart_name     = "ctl-api"
namespace      = "ctl-api"
storage_driver = "configmap"
dependencies   = ["rds_cluster_nuon", "karpenter_nodepools", "clickhouse_cluster", "temporal", "management", "ctl_api_init_db"]

[public_repo]
repo      = "nuonco/byoc"
directory = "byoc-nuon/src/components/ctl_api"
branch    = "main"

[[values_file]]
contents = "./values/ctl-api.yaml"

When your app is installed in a customer install, Nuon will automatically deploy things in the correct order, so all outputs are present for each step (we call this the App Graph).

Actions for lifecycle hooks

While components are stateful and represent infrastructure, images and resources, actions can be used for stateless parts of your application. Actions are useful for things like database migrations, init scripts and various one-time steps that only need to be run as part of provisioning the install, or updating the app.

Let's define a lifecycle action that initializes our database, you can define a lifecycle trigger, and run an action immediately after a component was deployed:

#:schema https://api.nuon.co/v1/general/config-schema?source=action
name    = "ch_init"
timeout = "1m30s"

# It can take about 20 minutes for clickhouse to be ready,
# so we don't use the post-deploy hook on clickhouse itself,
# because that would add 20 minutes to an already lengthy provision.
#
# Instead, while clickhouse is provisioning, we move forward with deploying all the other components.
#
# We attach this to ctl-api's post deploy, because ctl-api will need clickhouse to be ready.
[[triggers]]
type           = "pre-deploy-component"
component_name = "ctl_api"

[[triggers]]
type = "manual"

[[steps]]
name    = "Initialize the DB"
command = "./clickhouse/init.sh"

[steps.public_repo]
repo      = "nuonco/byoc"
directory = "byoc-nuon/src/actions"
branch    = "main"

[steps.env_vars]
FOO = "BAR"

Actions can also be manually triggered for one-off customer setup operations like importing a database migration or configuring a certificate.

#:schema https://api.nuon.co/v1/general/config-schema?source=action
name    = "ch_init"
timeout = "1m30s"

[[triggers]]
type = "manual"

[[steps]]
name    = "Initialize the DB"
command = "./clickhouse/init.sh"

[steps.public_repo]
repo      = "nuonco/byoc"
directory = "byoc-nuon/src/actions"
branch    = "main"

[steps.env_vars]
FOO = "BAR"

Actions are packaged by Nuon, and executed inside the runner in each install. The audit log for each install contains the source, input and, output of what was executed for customer review.

Customer Configurations and Secrets

Once you have configured the components, sandbox and, actions for deploying your app, you can define customer configuration inputs, secrets and, permissions. This gives you the ability to support custom customer configurations and allow for architectures such as bring-your-own-key or integrating with customer databases, and more.

Define Customer Secrets

You can define customer secrets to allow your customer to bring their own secret, and use it as part of your application configuration. Secrets are stored directly in the customer account and added via the CloudFormation Stack (or cloud equivalent). The Nuon runner can automatically sync secrets into your sandbox cluster, such as copying them into place in a Kubernetes secret for your components to use.

You can define secrets in your application config, and mark them as required or optional:

name          = "clickhouse_cluster_pw"
display_name  = "Clickhouse Cluster Password"
description   = "Password for the Clickhouse Cluster for the Nuon CTL API User."
auto_generate = true

kubernetes_sync             = true
kubernetes_secret_namespace = "clickhouse"
kubernetes_secret_name      = "clickhouse-cluster-pw"

Customer secrets can be a big source of risk, and you can use permission boundaries and policies to guarantee that your application never can read a customer secret directly via an Action, or within a deployment.

Create Customer Configuration Options

You can expose customer configuration options that enable customers to control different parts of your app in their account.

Inputs can be defined in your config and have controls to mark them as required, group them, and set defaults. For example, if you wanted to let a customer control the size of the database in their install, you could define an input:

#:schema http://localhost:8081/v1/general/config-schema?type=inputs
name         = "nuon_db_instance_type"
description  = "What RDS DB instance type would you like to deploy for nuon?"
default      = "db.r6gd.large"
display_name = "RDS Instance Type"
group        = "rds-nuon"
internal     = true

Then, we could reference the input from our database component:

name           = "ctl_api"
type           = "helm_chart"
chart_name     = "ctl-api"
namespace      = "ctl-api"
storage_driver = "configmap"

[public_repo]
repo      = "nuonco/byoc"
directory = "byoc-nuon/src/components/ctl_api"
branch    = "main"

[[values_file]]
contents = "./values/ctl-api.yaml"

[values]
DB_HOST = "{{ .nuon.inputs.db_address }}"
DB_USER = "{{ .nuon.inputs.db_user }}"

Customer inputs can be managed in git, using a config file, or via the Nuon UI and API. Whenever inputs are changed, Nuon will automatically redeploy each part of your application that references the input.

Define Permissions and Policies

Permissions are configured and bundled with your Nuon application. Permissions are templated into the install stack for each install, and your customer directly grants permissions to your Nuon runner via the stack, and can toggle them on or off at any time.

When you set up your application, you will define three permission sets:

• Provision - the permissions your application needs during initial setup.
• Deprovision - the permissions needed to remove your application (automatically disabled).
• Maintenance - the ongoing permissions to update, monitor and debug your app during day-2 operations.

Next we define the permissions for our app, enumerating every permission needed by defining policies individually:

type = "maintenance"
name                 = "{{ .nuon.install.id }}-maintenance"
description          = "maintenance"
display_name         = "byoc-nuon maintenance role"


[[policies]]
name = "bucket-put-policy-scoped"
contents = """
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:PutBucketPolicy",
            "Resource": "*"
        }
    ]
}
"""

Or, you can use a built-in role, and define policies which remove permissions from the role:

type = "maintenance"
name                 = "{{ .nuon.install.id }}-maintenance"
description          = "maintenance"
display_name         = "byoc-nuon maintenance role"

[[policies]]
managed_policy_name = "AdministratorAccess"

[[policies]]
name = "{{ .nuon.install.id }}-limited-secrets-manage-rds"
contents = """
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowSecretsManagerReadScoped",
            "Effect": "Allow",
            "Action": [
                "secretsmanager:GetSecretValue"
            ],
            "Resource": "arn:aws:secretsmanager:{{ .nuon.cloud_account.aws.region }}::secret:rds!*",
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/install.nuon.co/id": "{{ .nuon.install.id }}"
                }
            }
        }
    ]
}
"""

Each role supports defining a permission's boundary to limit escalation of permissions:

```toml
type = "maintenance"
name                 = "{{ .nuon.install.id }}-maintenance"
description          = "maintenance"
display_name         = "byoc-nuon maintenance role"

permissions_boundary = """
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "*",
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/install.nuon.co/id": "{{ .nuon.install.id }}"
        }
      }
    }
  ]
}
"""

[[policies]]
managed_policy_name = "AdministratorAccess"

Each role can only be assumed by the runner, and your customer can toggle the permissions on/off directly from the CloudFormation stack, to grant/remove access as needed.

We will talk more in the next post about how permissions work inside each install.

Manage your App

Once you have your application configured and synced with Nuon, you then get our control plane to deep-dive and manage each part of it. You can have installs running on different versions of your app, add new actions, and push updates to installs.

Each install runs a full-stack version of your application and the control-plane gives you a single pane of glass to work.

Part 2 - Installs

In the next part of this series, we will dive into what happens after you have your app setup and how installs work. Stay tuned for more!