Nivis

Infrastructure as Nix Code. All your base belongs to Nix. (Nivis, Latin, "of snow"; it belongs to Nix. Formerly nixform, then Terrae Nivis.)

A Nix-native infrastructure tool where Terraform/OpenTofu provider resources are first-class Nix values. A thin Go executor speaks the Terraform plugin protocol directly to unmodified provider binaries: Nix is the configuration frontend, Go is pure orchestration.

The headline capability (the reason this project exists) is the round trip: a provider-created resource returns computed values (an IP, an ID, a generated secret) back into Nix, which re-evaluates to produce dependent configuration, repeating to a fixpoint. This is proven end to end across two providers with unknown values originating on both sides.

How it works

Nix evaluates your configuration to a JSON IR (docs/IR-CONTRACT.md). Values that aren't known until apply-time are emitted as typed placeholders: a __ref (a direct reference to another resource's output) or a __derived (a value Nix computed from an output, e.g. a string built from an IP). The Go executor ingests the IR, spawns the relevant provider binaries, drives GetProviderSchema/PlanResourceChange/ApplyResourceChange, and collects the real outputs into an outputs ledger. It then re-evaluates Nix with the ledger injected, so placeholders resolve to concrete values; the new IR may unlock more resources. This loop repeats to a fixpoint (no new value resolves). Because each Nix-mediated (__derived) hop needs its own re-evaluation, deep chains take more than two phases; the loop generalizes to N phases. See DESIGN.md for why this (not an Output<T> promise model) is the honest, Nix-shaped approach.

Where to start

• Getting started: a hands-on walkthrough against the in-repo fake providers (offline, no credentials).
• Real providers (AWS): drive a real provider end to end.
• Architecture & decisions: why it is the way it is (spawn-not-link, batch-not-live, phased re-eval to a fixpoint).
• The IR contract: the stable interface between the Nix frontend and the Go executor.

Installing Nivis

The Nivis CLI is nivis (schema codegen is the nivis gen subcommand). It is distributed as a Nix flake, so you can run it without cloning anything. Pick whichever fits how you work.

Runtime needs. nivis shells out to nix to evaluate your configuration, so Nix must be on your PATH (with flakes enabled). The first time you use a real provider, nivis downloads it from the OpenTofu registry (e.g. the AWS provider is ~900 MB) and caches it, so that first run needs network and a little patience.

Run it ad hoc (no install)

The quickest way: run straight from the flake; Nix builds it on first use and caches the result:

nix run github:nivis-project/nivis#nivis -- --version
nix run github:nivis-project/nivis#nivis -- plan      # in your infra dir

Everything after -- is passed to nivis; codegen is nivis -- gen ….

A throwaway shell

Drop into a shell with nivis on PATH for the session, handy while iterating:

nix shell github:nivis-project/nivis#nivis
nivis --version

Install it persistently

Add nivis to your user profile so it's always available:

nix profile install github:nivis-project/nivis#nivis
nivis --version

Update later with nix profile upgrade, remove with nix profile remove.

From a clone (contributors)

If you've checked out the repository:

nix run .#nivis -- --version          # from the repo root
# or build a binary:
go build -o bin/nivis ./cmd/nivis
nix build .#nivis                     # -> ./result/bin/nivis

Shell completion

nivis completion <shell> prints a completion script for bash, zsh, fish, or powershell. It completes commands and flags, and dynamically completes resource ids (for state show, state rm, and --target) from your state file.

# bash: load it for the current shell, or drop it in your completions dir
source <(nivis completion bash)

# zsh: write it where your $fpath looks (then restart the shell)
nivis completion zsh > "${fpath[1]}/_nivis"

# fish
nivis completion fish > ~/.config/fish/completions/nivis.fish

Run nivis completion --help for per-shell details.

Working with state

Nivis keeps state in a local JSON file (--state, default nivis.state.json). Day-to-day:

nivis state list                 # ids in state (or "No resources in state.")
nivis state show <id>            # one resource's stored attributes
nivis state rm <id>              # drop a resource from state

Move the whole state document around with pull/push (the same shape a future remote backend uses):

nivis state pull > backup.json           # whole state to stdout (or --out)
nivis state pull --out backup.json

nivis state push --in backup.json        # replace state from a file
cat backup.json | nivis state push       # or from stdin

push replaces all of state, so it confirms first and reports the resource counts. Pass --force (or --yes) to skip the prompt; --force is required when the input is piped (non-interactive), so a scripted push is always explicit.

If a command reports that the state is locked by another nivis process, another run holds the advisory lock. Wait for it to finish; if it crashed and left a stale *.lock file, remove that file by hand. (Nivis no longer hangs on a held lock; it times out with that message.)

Pinning

The github: reference floats on the default branch. For reproducible infra, pin it in your own flake's flake.lock (the AWS S3 tutorial does this: Nivis becomes an input, and nix flake lock records the exact revision). Re-pin deliberately with nix flake update nivis.

Getting started with Nivis

A hands-on walkthrough using the in-repo fake providers. Everything here runs offline: no provider registry, no cloud account, no credentials. You need Go 1.22+ and Nix.

1. Get the binaries on your PATH

Enter a shell with nivis and the in-repo fake providers, in one command:

nix shell .#nivis .#fake-providers

provider-alpha and provider-beta are minimal tfprotov6 providers used as a hermetic test substrate. Their outputs are a deterministic function of inputs (a per-process counter seeded by TERRAE_NIVIS_FAKE_COUNTER, default 0), so every run is reproducible. The example configs reference them by bare name, so once they are on your PATH there is nothing to build or copy.

Contributors who'd rather use the Go toolchain directly can build instead: go build -o bin/provider-alpha ./cmd/provider-alpha (and provider-beta, nivis). If you do, prepend ./bin to your PATH (export PATH=$PWD/bin:$PATH) so the nivis and bare-name provider sources resolve just as in the Nix shell.

Scaffold a tutorial with nivistutor

If you do not have a repo checkout and just want to try Nivis in a sandbox, nivistutor writes a tutorial's starter files (a ready flake.nix, the config, and a README) into a directory of your choosing, so you run it with plain nivis, with no --flake/--attr flags. It scaffolds the files for you to read and run; it does not run nivis for you (you learn by doing).

nix shell github:nivis-project/nivis#nivis github:nivis-project/nivis#tutor
nivistutor

It greets you, lists the available tutorials (this getting-started one and the current release's features tutorial), asks whether to write into a new subdirectory or the current one, writes the files, and prints the exact nivis commands to run next. The #tutor shell carries the fake providers too, so the scaffolded project runs immediately. Non-interactively:

nivistutor --list                                   # the available tutorials
nivistutor --tutorial getting-started --dir my-nivis # write without prompts

The starter's flake.nix is pinned to the nivis release that scaffolded it, so the library and your nivis binary agree. Existing files are never overwritten without --force.

2. The example configuration

The flake's nivis.plan (in nix/example/) describes three resources and a consumer, wired so each hop crosses the Nix boundary:

alpha_token.A            (alpha)            -- no inputs; A.value computed at apply
   └─ name = "rec-" + A.value               (a __derived value)
beta_record.B  (beta)    from = name        -- B.endpoint computed at apply
   └─ final = B.endpoint + "::" + A.value    (a __derived on BOTH providers)
alpha_token.C  (alpha)   label = final

systemConfig (a Nix consumer) reads:
  recordEndpoint = B.endpoint   # from beta
  tokenValue     = A.value      # from alpha
  combined       = final        # from both

Because name and final are values Nix computes from provider outputs, they can't be known until those outputs exist and Nix is re-evaluated, which is what forces multiple phases.

3. Plan and apply

nivis plan

  + alpha.alpha_token.A (alpha_token)
  + beta.beta_record.B (beta_record)
  + alpha.alpha_token.C (alpha_token)

3 change(s) across 3 resource(s) (+ create, ~ update, -/+ replace, = no change). Run `nivis apply`.

nivis apply

Applied 3 resource(s) across 3 phase(s):

Phase 1
  + alpha.alpha_token.A
Phase 2
  + beta.beta_record.B
Phase 3
  + alpha.alpha_token.C

Three phases, not one: phase 1 applies A (nothing else is ready); re-evaluating with A.value known unlocks B; re-evaluating with B.endpoint known unlocks C. The loop halts at a fixpoint once nothing new resolves.

Provider notes

Providers log about their own internals while they work. Nivis surfaces what they say as provider notes — one line each, next to (never inside) the change list:

provider note aws_amplify_app.description: unable to require attribute replacement (detail: ForceNew: No changes for description)

Read that as: while planning an aws_amplify_app, the provider mentioned something about the description attribute. A note is not a failure. The detail: part is the provider's own explanation of why it said something — in this example, the AWS SDK considered forcing a replacement of description and decided against it, because nothing changed. Providers report that as an "error" field internally; Nivis renders it as a detail precisely so it does not read as one. If a run actually fails you get an error: line and a non-zero exit status, not a note.

A note that recurs — many providers log the same internal remark once per resource — is printed once, and the end of the run reports the rest:

provider note aws_amplify_app.description: unable to require attribute replacement (11 further occurrence(s) not shown)

Notes go to stderr, while the change list goes to stdout, so nivis plan > plan.txt keeps the change list unmixed.

To see more or less of this, use --provider-log-level:

trace is the setting for debugging a provider or filing a provider bug report: it restores the request ids, caller sites and RPC names that a note deliberately drops. Expect a lot of output — providers are verbose at that level.

The flag governs what you see. It is separate from TF_LOG, which the provider process inherits from your environment and which governs how much the provider emits: setting TF_LOG=trace cannot flood a run whose --provider-log-level is warn.

Note that a plan of a stack that is not yet in state reports creates without contacting the provider, so it produces no provider notes. Notes appear from apply onward, and on a re-plan of a stack that already has state.

--provider-log-level governs the providers. For Nivis's own progress reporting — how much it tells you while a long run is working, which channel each kind of output goes to, and the colour and terminal rules — see Reading Nivis output.

4. Inspect the round trip

nivis state list
nivis state show alpha.alpha_token.C

alpha.alpha_token.C (alpha_token)
  id = alpha-1
  label = beta://rec-alpha::0::alpha::0
  value = alpha:beta://rec-alpha::0::alpha::0:1

C.label is final: a string Nix built from both B.endpoint (beta) and A.value (alpha). That value only became concrete after both providers applied and Nix re-evaluated. That is the round trip.

Stack outputs

To surface named values out of a run (the Terraform output "x" {} equivalent), declare them with the outputs argument to toIR:

toIR {
  providers = { ... };
  resources = [ A B C ];
  outputs = {
    token = A.refAttr "value";          # from one resource
    combined = final;                   # composed across both providers
  };
  inherit ledger;
}

Read them after apply with nivis output (resolved from current state):

nivis output

combined = beta://rec-alpha::0::alpha::0
token = alpha::0

nivis output <name> prints a single value, and nivis output --json prints a JSON object ({ "<name>": <value> }) for a CI step or another stack to consume. Outputs reuse the round trip's resolution, so a value composed across providers and phases comes back concrete.

5. Refresh and destroy

nivis refresh    # reconciles state via ReadResource; no changes here
nivis destroy    # tears down in reverse dependency order

Destroyed 3 resource(s):
  - alpha.alpha_token.C
  - beta.beta_record.B
  - alpha.alpha_token.A

6. Generate constructors from a provider schema

nivis gen turns any provider's schema into typed Nix constructors:

nivis gen --provider provider-alpha --out ./generated
cat ./generated/alpha/alpha_token.nix

The generated constructor requires the provider's required inputs (throwing a named error if missing), passes optional inputs through, omits computed-only attributes (they're outputs), and accepts an overrides argument so you can adjust the generated output.

It also includes the resource's nested blocks, each with the shape that matches its nesting, so you never guess list-vs-single: a list/set-nested block (e.g. ingress, disk_container) is an argument defaulting to [] and written [ { ... } ]; a single-nested block is a plain attrset; a map-nested block is a map of attrsets. Every block is documented in the generated file with its nesting and inner attribute names.

So the generated constructor is your per-provider argument reference: instead of translating a provider's Terraform docs into Nivis terms by hand, run nivis gen and read the .nix. It lists every argument (with type and required/optional), every nested block (with its nesting), and the computed outputs, all in Nivis form.

7. A real provider (AWS)

Everything above is offline against the fakes. The same nivis commands drive real providers: nivis resolves a provider by address from the OpenTofu registry, downloads and checksum-verifies the binary, negotiates the plugin protocol (AWS speaks v5), configures it, and runs plan/apply/destroy. The example nix/example/aws.nix (flake attr nivis.aws) declares the hashicorp/aws provider with mkProvider and one aws_s3_bucket.

⚠️ This creates a real resource in your AWS account: one (free-tier) S3 bucket, then destroys it. The provider's region lives in the Nix config; only credentials come from the environment (the AWS SDK default chain), so set AWS_PROFILE (or AWS_ACCESS_KEY_ID/…). The first run downloads the ~900 MB AWS provider (cached afterwards).

For the full, hand-held walkthrough (prerequisites, writing the config line by line, plan/apply/inspecting state/destroy, and troubleshooting) follow the AWS S3 tutorial.

Where to go next

• IR-CONTRACT.md + ir-schema.json: the IR, the stable contract between the Nix frontend and the Go executor.
• TESTING.md: the test layers and the headline two-provider e2e.
• DESIGN.md: why the architecture is the way it is (spawn-not-link, batch-not-live, phased re-eval to a fixpoint).

The core test suite is hermetic (fakes, no network/credentials); real-provider support (registry download + checksum verification, tfprotov5/6) is proven against AWS as shown above.

Real providers (AWS)

Everything above is offline against the fakes. The same nivis commands drive real providers: nivis resolves a provider by address from the OpenTofu registry, downloads and checksum-verifies the binary, negotiates the plugin protocol (AWS speaks v5), configures it, and runs plan/apply/destroy. The example nix/example/aws.nix (flake attr nivis.aws) declares the hashicorp/aws provider with mkProvider and one aws_s3_bucket.

⚠️ This creates a real resource in your AWS account: one (free-tier) S3 bucket, then destroys it. The provider's region lives in the Nix config; only credentials come from the environment (the AWS SDK default chain), so set AWS_PROFILE (or AWS_ACCESS_KEY_ID/…). The first run downloads the ~900 MB AWS provider (cached afterwards).

For the full, hand-held walkthrough (prerequisites, writing the config line by line, plan/apply/inspecting state/destroy, and troubleshooting) follow the AWS S3 tutorial.

Tutorial: an S3 bucket on AWS

A genuinely from-scratch walkthrough. You start in an empty directory on your own machine (not a checkout of Nivis), install the nivis CLI, scaffold a fresh flake that uses Nivis as a dependency, declare one S3 bucket, and drive it through plan → apply → inspect → destroy. By the end you'll have a small infra flake you own and a real bucket created and torn down.

⚠️ This creates a real resource in your AWS account: a single S3 bucket (no objects, negligible cost) that you destroy at the end. The commands and outputs below come from real runs.

Prerequisites: Nix (with flakes enabled) on your PATH, and AWS credentials you can use locally.

Part 1: Install nivis

The CLI is nivis. You don't need to clone anything; the quickest path is to run it straight from the flake:

nix run github:nivis-project/nivis#nivis -- --version

If you'd rather have nivis on your PATH for the rest of this tutorial, install it persistently or open a shell with it: see Installing Nivis for all the options (nix run, nix shell, nix profile install, building from a clone). The rest of this tutorial writes nivis …; if you chose the ad-hoc form, read that as nix run github:nivis-project/nivis#nivis -- ….

Part 2: A fresh infra flake

2.1 Scaffold the flake

mkdir my-infra && cd my-infra
nix flake init

nix flake init drops a placeholder flake.nix (a hello package). Replace its contents with the infra flake below.

2.2 The boilerplate

{
  description = "My infrastructure, as Nix code (Nivis).";

  # Pull Nivis in as a dependency. `nix flake lock` (run automatically by
  # the first nivis command) records the exact revision in flake.lock.
  inputs.nivis.url = "github:nivis-project/nivis";

  outputs =
    { self, nivis }:
    let
      # The Nivis Nix library: mkResource, mkProvider, toIR, str, …
      lib = nivis.lib;
    in
    {
      # `nivis` (the CLI) evaluates the attribute `nivis.plan` by default. It's a
      # function of the outputs ledger (the apply-time values fed back in each
      # phase).
      nivis.plan =
        ledger:
        let
          # A bucket. `bucket` (the name) is omitted, so AWS generates one.
          bucket = lib.mkResource {
            provider = "aws";
            type = "aws_s3_bucket";
            name = "demo"; # id becomes aws.aws_s3_bucket.demo
            config = {
              force_destroy = true; # let `nivis destroy` delete it even if non-empty
            };
          };

          # A text file whose CONTENT is generated by Nix, from the bucket's own
          # output. `bucket = bucket.refAttr "id"` makes the object depend on the
          # bucket; `content` is a Nix string embedding the bucket's generated
          # name, which doesn't exist until the bucket is applied. This is the
          # round trip (see below).
          note = lib.mkResource {
            provider = "aws";
            type = "aws_s3_object";
            name = "note";
            config = {
              bucket = bucket.refAttr "id";
              key = "hello-from-nix.txt";
              content = lib.str [
                "This file's content was generated by Nix.\n"
                "It is stored in the bucket named: "
                (bucket.refAttr "id")
                "\n"
              ];
              content_type = "text/plain";
            };
          };
        in
        lib.toIR {
          # --- providers --------------------------------------------------
          providers.aws = lib.mkProvider {
            source = "registry.opentofu.org/hashicorp/aws";
            config = {
              region = "eu-central-1"; # set in Nix, not via AWS_REGION
              # default_tags is a *list-nested* block in the AWS provider, so it
              # takes a list (a bare attrset is rejected at configure time).
              default_tags = [ { tags = { managed-by = "nivis"; }; } ];
            };
          };

          # --- resources --------------------------------------------------
          resources = [ bucket note ];

          inherit ledger;
        };
    };
}

Reading it:

• inputs.nivis.url makes Nivis a dependency; lib = nivis.lib binds its Nix library.
• nivis.plan is the attribute nivis evaluates by default: a function ledger → IR. (Name it something else and pass nivis plan --attr <name>.)
• mkProvider declares the AWS provider: source is its registry address, region lives in Nix, and default_tags is a one-element list because that block is list-nested in the AWS provider.
• mkResource declares the aws_s3_bucket (force_destroy for easy teardown; bucket omitted so AWS picks a unique name) and an aws_s3_object.
• The object is the interesting part. bucket = bucket.refAttr "id" is a reference to the bucket's output, so Nivis knows the object depends on the bucket. And content = lib.str [ … (bucket.refAttr "id") … ] builds the file's body in Nix, embedding the bucket's name, a value that does not exist until the bucket is applied. See A file whose content comes from Nix below.

2.3 Credentials

nivis uses the AWS SDK default credential chain (the same one the AWS CLI uses). Point it at your account, typically a named profile:

export AWS_PROFILE=your-profile
aws sts get-caller-identity   # sanity check

Only credentials come from the environment; the region is in the flake above.

2.4 Parameterise with a variable (optional)

Hard-coding eu-central-1 is fine for one environment. To set it per run, declare a variable with nivis.mkVars and read it in the plan. mkVars takes a declaration (type and optional default) and the values Nivis injects, and returns the resolved, typed result:

nivis.plan =
  ledger:
  let
    vars = lib.mkVars {
      region = { type = "str"; default = "eu-central-1"; };
    } (ledger.vars or { });
  in
  lib.toIR {
    providers.aws = lib.mkProvider {
      source = "registry.opentofu.org/hashicorp/aws";
      config.region = vars.region;   # was the hard-coded string
    };
    # ... resources ...
    inherit ledger;
  };

Then override it at the CLI:

nivis plan --var region=us-east-1

A variable with no default is required. For the full story (types, defaults, --var-file, NIVIS_VAR_*, and precedence) see Variables.

Part 3: Plan, apply, inspect, destroy

Run these from your my-infra directory.

Plan

nivis plan

+ aws.aws_s3_bucket.demo (aws_s3_bucket)
+ aws.aws_s3_object.note (aws_s3_object)

2 resource(s) to resolve across phases (+ create, ~ change). Run `nivis apply`.

The first nivis command resolves the nivis input (writing flake.lock) and, on first use of a real provider, downloads it, so the first run is slower.

Apply

nivis apply

Applied 2 resource(s) across 2 phase(s):

Phase 1
  + aws.aws_s3_bucket.demo
Phase 2
  + aws.aws_s3_object.note

(On a terminal the markers are colored by change type: + create green, ~ update yellow, -/+ replace, - destroy red, = no-op dim, and a datasource read shows a dim r. Piped or with NO_COLOR set, it is plain text.)

Two phases, not one. Phase 1 creates the bucket: nothing else can run, because the object's content needs the bucket's name. Nivis then re-evaluates the Nix config with the bucket's generated name injected, which resolves the object's content; phase 2 uploads it. nivis writes the resulting state to nivis.state.json in my-infra.

What appears in your working directory

The run writes two files next to your config:

nivis.state.json          the state document
nivis.state.json.ledger   the outputs ledger

Neither may be committed. Both can hold secrets that came back from a provider: an access key, a generated password, a token. The ledger in particular is where Nivis keeps sensitive provider outputs in plaintext, deliberately, to keep them out of the world-readable Nix store. Both are written 0600, but file permissions do not stop git add.

Add this to your .gitignore before your first commit:

nivis.state.json*
*.ledger

*.ledger is a suffix rule, so it still covers the ledger if you point --state somewhere else. Nivis also names these paths once, the first time a run creates them.

If one of them is already committed, adding it to .gitignore is not enough: the file stays in your history. Remove it from the repository and rotate whatever it contained.

Inspect the round trip

nivis state show aws.aws_s3_bucket.demo

  arn = arn:aws:s3:::terraform-20260615181937557000000001
  tags_all = map[managed-by:nivis]
  force_destroy = true
  region = eu-central-1
  id = terraform-20260615181937557000000001
  …

The bucket name (terraform-2026…) was generated by AWS and read back into state: a value that didn't exist until apply is now concrete. tags_all shows the provider's default_tags were applied.

A file whose content comes from Nix

This is the point of the project. Look at the object:

nivis state show aws.aws_s3_object.note

  bucket = terraform-20260615181937557000000001
  key = hello-from-nix.txt
  content_type = text/plain
  content = This file's content was generated by Nix.
  id = terraform-20260615181937557000000001/hello-from-nix.txt

The content was built in Nix, and it embeds the bucket's AWS-generated name, which did not exist until phase 1 created the bucket. Fetch the actual object from S3 to see it landed in the real world:

aws s3 cp "s3://$(nivis state show aws.aws_s3_bucket.demo | awk '/^  id = /{print $3}')/hello-from-nix.txt" -

This file's content was generated by Nix.
It is stored in the bucket named: terraform-20260615181937557000000001

A value computed in the Nix domain became the body of a real resource in the Terraform/AWS domain, resolved across phases. That round trip, not just "Terraform from Nix," but Nix and provider state feeding each other, is why Nivis exists.

Destroy

nivis destroy

Destroyed 2 resource(s):
  - aws.aws_s3_object.note
  - aws.aws_s3_bucket.demo

Resources tear down in reverse dependency order: the object first, then the bucket. Confirm nothing's left:

aws s3api list-buckets --query 'Buckets[?contains(Name, `terraform-`)].Name'

Make it your own

• A specific bucket name: add bucket = "globally-unique-name"; to the resource config.
• A different region: change region in the provider config.
• More resources: add more mkResource entries to the resources list; wire one resource's output into another with the reference helpers (refAttr) and Nivis resolves them across phases.
• Pin Nivis: the input floats on the default branch; flake.lock pins the exact revision. Re-pin deliberately with nix flake update nivis.

Troubleshooting

• NoCredentialProviders / could not find credentials: the SDK chain found nothing. Set AWS_PROFILE (or the access-key vars) and confirm with aws sts get-caller-identity.
• this is a list-nested block; wrap the value in a one-element list: you wrote a provider block as a bare attrset { … }, but it is list-nested, so it takes a one-element list, e.g. default_tags = [ { tags = { … }; } ]. The error names the attribute (e.g. ["default_tags"]: …). The symmetric case, a single-nested block given a list, says to pass one attrset instead.
• First run is slow / seems to hang: it's resolving the flake input and downloading the ~900 MB AWS provider once; later runs use the cache.
• BucketAlreadyExists: you set an explicit bucket name someone already owns (S3 names are global). Omit bucket, or pick another.
• nivis can't find your flake: run nivis from the directory containing flake.nix, or pass nivis plan --flake /path/to/my-infra.

Tutorial: a NixOS machine on EC2

This goes further than the S3 tutorial: you build a NixOS image in Nix, register it as an AMI, and launch it as an EC2 instance, and the entire AWS pipeline (upload, import, register, launch) is driven by Nivis. The machine runs a tiny web server, and you verify the running instance answers HTTP 200 on port 80, then tear it all down.

⚠️ This creates real, billable AWS resources (an EBS snapshot, an AMI, and a t3.micro instance) and uploads a ~2 GB image to S3. The walkthrough destroys everything at the end. Credentials come from the environment (AWS_PROFILE); the region is in the Nix config.

The shape:

nivis apply ──►  NixOS amazon image (a .vhd, nginx baked in)   ← built by the run
                      │
   Nivis ── aws_iam_role + policy (vmimport)      the VM-import service role
         ── aws_s3_bucket + aws_s3_object         the .vhd uploaded to S3
         ── aws_ebs_snapshot_import               S3 .vhd → EBS snapshot
         ── aws_ami                               register the snapshot as an AMI
         ── aws_security_group                    ingress :80
         ── aws_instance                          launch it  (public_ip → Nix)
                      │
   curl http://<public_ip>/  ──►  200

This mirrors elastinix (the wearetechnative NixOS-on-AWS flake) and its Terraform module, but driven by Nivis instead of a Terraform module.

Part 1: The OS and the infra in one file

The key idea: the image and the infrastructure live in the same Nix file. A NixOS "amazon image" is itself a Nix derivation, config.system.build.images.amazon, a .vhd disk image of a machine configuration, so you reference its build output directly as aws_s3_object.source. When nivis apply evaluates the flake, Nix realises the image as part of evaluation and its store path flows straight into the upload. One expression defines the OS and the cloud resources that ship it: that two-domain mix is the whole point of this tutorial.

{
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-25.05";
  inputs.nivis.url = "github:nivis-project/nivis";

  outputs = { self, nixpkgs, nivis }:
    let
      # --- domain 1: the OS, built in Nix (nginx, returns 200 on :80) -----
      image = (nixpkgs.lib.nixosSystem {
        system = "x86_64-linux";
        modules = [ ({ modulesPath, ... }: {
          imports = [ (modulesPath + "/virtualisation/amazon-image.nix") ];
          services.nginx.enable = true;
          services.nginx.virtualHosts."_".locations."/".return =
            ''200 "hello from NixOS on EC2, built and launched by Nivis\n"'';
          networking.firewall.allowedTCPPorts = [ 80 ];
          system.stateVersion = "25.05";
        }) ];
      }).config.system.build.images.amazon;

      # --- domain 2: the infra, as Nivis resources, fed by that image -----
      pipeline = import (nivis + "/nix/example/ec2.nix") {
        nivis = nivis.lib;
        nixosImage = image;   # `drv image` -> aws_s3_object.source (realised at apply)
      };
    in {
      nivis.plan = ledger: pipeline (ledger // { vars.suffix = "demo"; });
    };
}

nivis apply builds image first (the heavy step, the one that uses the Nix binary cache, ≈2 GB), then drives the AWS pipeline; everything after the build is pure AWS. (This repo's nix/example/ec2.nix + the nivis.ec2 flake attr are exactly this, ready to run.)

Part 2: The Nivis pipeline

Here is the whole AWS side: a function of the built image and the outputs ledger, returning the IR. It's the contents of this repo's nix/example/ec2.nix (exposed as the flake attr nivis.ec2); drop it in your own repo and import it as shown in Part 1, or inline it.

{
  nivis,        # the Nivis library (mkResource / mkProvider / toIR / drv)
  nixosImage,   # the NixOS amazon image derivation from Part 1
}:
ledger:
let
  inherit (nivis) mkResource mkProvider toIR drv;

  suffix = (ledger.vars or { }).suffix or "demo";
  bucketName = "nivis-ec2nix-${suffix}";

  # The OS crossing into the infra: `drv` marks the image as a build output,
  # a __build leaf that `nivis apply` realises (builds) before uploading, then
  # substitutes the concrete .vhd path. No manual store-path interpolation; no
  # separate `nix build`. (drv uses the image's passthru.filePath, the .vhd.)
  imgSource = drv nixosImage;

  # --- the vmimport service role AWS requires to import a disk image ---------
  role = mkResource {
    provider = "aws"; type = "aws_iam_role"; name = "vmimport";
    config = {
      name = "nivis-vmimport-${suffix}";
      assume_role_policy = builtins.toJSON {
        Version = "2012-10-17";
        Statement = [{
          Effect = "Allow";
          Principal.Service = "vmie.amazonaws.com";
          Action = "sts:AssumeRole";
          Condition.StringEquals."sts:Externalid" = "vmimport";
        }];
      };
    };
  };

  policy = mkResource {
    provider = "aws"; type = "aws_iam_policy"; name = "vmimport";
    config = {
      name = "nivis-vmimport-${suffix}";
      policy = builtins.toJSON {
        Version = "2012-10-17";
        Statement = [
          {
            Effect = "Allow";
            Action = [ "s3:GetBucketLocation" "s3:GetObject" "s3:ListBucket" "s3:PutObject" "s3:GetBucketAcl" ];
            Resource = [ "arn:aws:s3:::${bucketName}" "arn:aws:s3:::${bucketName}/*" ];
          }
          {
            Effect = "Allow";
            Action = [ "ec2:ModifySnapshotAttribute" "ec2:CopySnapshot" "ec2:RegisterImage" "ec2:Describe*" ];
            Resource = "*";
          }
        ];
      };
    };
  };

  attach = mkResource {
    provider = "aws"; type = "aws_iam_role_policy_attachment"; name = "vmimport";
    config = { role = role.refAttr "name"; policy_arn = policy.refAttr "arn"; };
  };

  # --- upload the built .vhd to S3 ------------------------------------------
  bucket = mkResource {
    provider = "aws"; type = "aws_s3_bucket"; name = "image";
    config = { bucket = bucketName; force_destroy = true; };
  };

  image = mkResource {
    provider = "aws"; type = "aws_s3_object"; name = "image";
    config = { bucket = bucket.refAttr "id"; key = "nixos.vhd"; source = imgSource; };
  };

  # --- S3 .vhd -> EBS snapshot ----------------------------------------------
  # disk_container and user_bucket are LIST-nested blocks, so each is a
  # one-element list (a bare attrset is rejected at apply).
  snapshot = mkResource {
    provider = "aws"; type = "aws_ebs_snapshot_import"; name = "nixos";
    config = {
      role_name = role.refAttr "name";
      disk_container = [{
        format = "VHD";
        user_bucket = [{ s3_bucket = bucket.refAttr "id"; s3_key = "nixos.vhd"; }];
      }];
    };
  };

  # --- register the snapshot as a bootable AMI ------------------------------
  ami = mkResource {
    provider = "aws"; type = "aws_ami"; name = "nixos";
    config = {
      name = "nivis-ec2nix-${suffix}";
      virtualization_type = "hvm";
      root_device_name = "/dev/xvda";
      ena_support = true;
      ebs_block_device = [{ device_name = "/dev/xvda"; snapshot_id = snapshot.refAttr "id"; }];
    };
  };

  # --- a security group opening port 80 -------------------------------------
  sg = mkResource {
    provider = "aws"; type = "aws_security_group"; name = "web";
    config = {
      name = "nivis-ec2nix-web-${suffix}";
      description = "Nivis EC2+NixOS demo: allow HTTP";
      ingress = [{ from_port = 80; to_port = 80; protocol = "tcp"; cidr_blocks = [ "0.0.0.0/0" ]; description = "http"; }];
      egress  = [{ from_port = 0;  to_port = 0;  protocol = "-1";  cidr_blocks = [ "0.0.0.0/0" ]; description = "all";  }];
    };
  };

  # --- launch the AMI -------------------------------------------------------
  instance = mkResource {
    provider = "aws"; type = "aws_instance"; name = "web";
    config = {
      ami = ami.refAttr "id";
      instance_type = "t3.micro";
      vpc_security_group_ids = [ (sg.refAttr "id") ];
      tags = { Name = "nivis-ec2nix-${suffix}"; managed-by = "nivis"; };
    };
  };
in
toIR {
  providers.aws = mkProvider {
    source = "registry.opentofu.org/hashicorp/aws";
    config = { region = "eu-central-1"; };
  };
  resources = [ role policy attach bucket image snapshot ami sg instance ];
  inherit ledger;
}

Reading the chain: image's source is the built .vhd path (Part 1): the OS crossing into the infra. Every later resource references the previous one's output with refAttr (a __ref), so Nivis resolves the chain across phases: the snapshot import waits on the upload, the AMI on the snapshot, the instance on the AMI. The IAM role + policy create the vmimport service role AWS requires for disk-image import. The only per-deployment knob is suffix (unique resource names).

Part 3: Apply

export AWS_PROFILE=your-profile
nivis plan      # 9 resources to create across phases
nivis apply     # build the image, upload (~2 GB), import, register, launch

Because source is a drv (__build) leaf, nivis apply builds the image itself before uploading it: no separate nix build step. The leaf carries the image's derivation, which is what lets the run build it rather than only fetch a prebuilt copy — so this works on a machine where the image has never been built and is in no binary cache.

The image build is the heavy part (≈2 GB, mostly from the Nix binary cache) and it happens inside the apply: nivis reports Building nixos-image-… when it starts and Built … when it finishes, so a long silence is not a hang. Note that the state lock is held for the whole run, build included, so on a shared remote backend a first build blocks other runs against that state. (Pass --build=false to skip realising if you have pre-built, e.g. with nix build .#ec2-image.)

A real run of this pipeline (verified against AWS) resolves across four phases: the AWS chain can't all happen at once:

Applied 9 resource(s) across 4 phase(s):

Phase 1
  + aws.aws_iam_role.vmimport
  + aws.aws_iam_policy.vmimport
  + aws.aws_s3_bucket.image
  + aws.aws_security_group.web
Phase 2
  + aws.aws_iam_role_policy_attachment.vmimport
  + aws.aws_s3_object.image          # the ~2 GB NixOS .vhd
  + aws.aws_ebs_snapshot_import.nixos
Phase 3
  + aws.aws_ami.nixos
Phase 4
  + aws.aws_instance.web

(On a terminal each + is colored by change type; piped or with NO_COLOR it is plain text. A datasource read, if the config had one, would show a dim r.)

Read the instance's public address back out of state and check it serves:

nivis state show aws.aws_instance.web    # public_ip / public_dns / instance_state
curl -sS -o /dev/null -w '%{http_code}\n' "http://<public_ip>/"
# 200

The instance boots, nginx comes up on port 80, and returns 200: a machine whose OS you built in Nix, registered as an AMI through Nivis, and launched, all from one flake. (Give it a minute after apply: the instance has to boot before nginx answers.)

That public_ip did not exist until AWS launched the instance; it was read back into state (and is available to Nix for dependent config). The instance is running an OS you built in Nix, from an image you registered through Nivis.

Tear it all down (reverse dependency order: instance, AMI, snapshot, bucket, role):

nivis destroy

Notes

• Cost & safety: a t3.micro is cheap, but don't leave it running; nivis destroy removes everything this created. The EBS snapshot import takes a few minutes; that's AWS, not Nivis.
• The vmimport role: AWS requires this specific service role for disk-image import; the example creates it (and a least-privilege policy) so the pipeline is self-contained. If your account already has a vmimport role, point aws_ebs_snapshot_import.role_name at it instead.
• Production: for a real fleet, use elastinix: it owns the image-build + upload pipeline and a maintained module. This tutorial shows the mechanism, end to end, driven entirely by Nivis.

Reading Nivis output

A nivis run spends most of its wall-clock time inside three slow things: a Nix evaluation of your configuration, a Nix build of whatever artifacts it references, and provider calls that wait on real infrastructure. This page describes what Nivis tells you while that is happening, where each kind of output goes, and how to turn the detail up or down.

The three channels

Nivis divides its output into three kinds of content, each with a fixed destination. The split is what lets a script and a human share one command.

The result is what the command concluded. It is byte-identical whether or not a terminal is attached, so this keeps working exactly as before:

$ nivis apply > result.txt      # the same bytes a pipeline would see
$ nivis plan | grep '^\s*+'     # stable enough to parse

The narrative is what happened along the way. It is durable — redirect it and you get a full transcript, differing from the on-screen form only in carrying no colour:

$ nivis apply 2> run.log

The live region is the small block of status lines at the bottom of the screen during a run, rewritten in place and erased before the run ends. It never appears in redirected output, so a log file never contains half-drawn spinner frames.

The rule dividing the first two: content that reports what happened is narrative, and content that reports what resulted is the result. Acquiring the state lock, for instance, is narrative.

Note A node you watch complete in the narrative also appears in the result's phase-grouped recap at the end. That is deliberate — they are different views. The narrative is chronological and carries durations; the recap is grouped by phase and carries change markers.

Verbosity

--log-level controls how much of Nivis's own narrative you see. It can also be set once with the NIVIS_LOG environment variable; an explicit flag wins.

$ nivis apply --log-level verbose
$ NIVIS_LOG=quiet nivis plan

The default reports what changed, not what happened

At info, a bulk operation reports its outcome rather than narrating every step. A plan refreshes every resource in state through its provider; naming all forty buries the three that matter, so the default names only the drifted ones and counts the rest. --log-level verbose names every read, when you want it.

--log-level is not --provider-log-level

They govern different things and are independently settable:

• --log-level — how much Nivis says about its own execution.
• --provider-log-level — how much of a spawned provider's logging is surfaced as notes. See Getting started.

Silencing one does not silence the other. --log-level quiet still shows a provider's deprecation warning.

Nix's own output

Nix has a perfectly good progress display of its own, and a build of an operating-system image can take many minutes. But Nix's display and Nivis's live region both drive the cursor, so they can never both be active. The verbosity picks which one you get:

At the default level Nivis reads Nix's own event stream and reports it itself, so the live region can stay up. A running build shows what is building, how many derivations are done of how many expected, how long it has been going, and the latest line of the build's own output:

  ⠹  + aws_s3_object.image                   2m14s
     building nixos-image              [2/4 drv]  1m14s
     creating disk image (2048 MiB)

The [2/4 drv] total can go up while you watch. Nix discovers work as it proceeds, so the number it expects is a running figure, not a promise.

At verbose Nivis hands the terminal to Nix instead, so you get Nix's own display verbatim:

$ nivis apply --log-level verbose

That is also the fallback. Nix's event stream is not a documented interface, so if Nivis meets a version whose stream it cannot read, it stops interpreting and passes the output through rather than guessing — you get plainer output, never a broken build. --log-level verbose also reports which Nix is being driven.

Whichever level you choose, a failing evaluation or build always reports its actionable error text.

Colour

--color accepts auto (the default), always and never.

• auto colourises a colour-capable terminal, unless colour is disabled by the environment.
• always colourises even when output is redirected — useful for a CI system that renders ANSI in its log viewer.
• never never colourises.

NO_COLOR disables colour when set to anything; CLICOLOR_FORCE enables it, as its counterpart. An explicit --color wins over both. A terminal reporting itself as dumb is never treated as colour-capable.

Colour changes presentation only. The markers, the text and the counts are identical either way, so a test or a script sees stable output.

Colour and the live region are separate questions

Turning colour off does not turn the live region off, and forcing colour on does not turn it on. A terminal can render colour correctly and still mishandle cursor movement, so Nivis asks the two questions separately — conflating them corrupts output on exactly the terminals least able to cope.

The live region is used only when all of these hold: output is a terminal, TERM is not dumb, no CI environment is detected, and the level is info. Otherwise you get one line per transition carrying the same information.

Reading a phased apply

Nivis resolves your configuration over phases. A phase boundary means the nodes in the next phase could not have run any earlier — their inputs only became known once the previous phase's outputs were fed back into Nix and it was re-evaluated. That round trip is the thing Nivis exists to do, and the phase headings are where you watch it happen.

Phase 1  3 nodes
  + alpha.alpha_token.app        1s   alpha-0
  r data.alpha.alpha_lookup.app  0ms
Phase 2  1 node
  + beta.beta_record.dns        42s   rec-17

A phase costs one Nix evaluation, so a phase boundary is a real round trip and nothing else. An ordinary reference from one resource to another — B's input is A's output — does not start a new phase, however long the chain: Nivis resolves those itself as each resource is applied, with no need to ask Nix again. Only a value Nix must compute from an apply-time result forces another phase. That is why a stack of nine AWS resources wired to each other can apply in a single phase, while a two-resource stack that builds a hostname in Nix takes two.

Each phase announces how many nodes are ready when it starts. That figure can grow as the phase runs, because applying one resource can make another ready; it is an opening count, not a total. There is deliberately no total for the whole run either: Nivis cannot know it in advance, because a later phase can reveal resources that phase 0 could not see. A count that looked authoritative would be a guess.

The markers are the same ones plan uses: + create, ~ update, -/+ replace, = no change, r a datasource read.

Variables

Variables let you parameterise a Nivis configuration instead of hard-coding it: declare them in Nix with nivis.mkVars, read them in your plan, and set them per run from the command line, a file, or the environment. This is how you keep one configuration and vary the region, a name suffix, an instance size, or any other input between environments.

Declare variables with mkVars

mkVars takes a declaration attrset (each variable with an optional type and default) and the values Nivis injects (ledger.vars), and returns the resolved, validated values your plan reads:

nivis.plan =
  ledger:
  let
    vars = lib.mkVars {
      region = { type = "str"; default = "eu-central-1"; };
      suffix = { type = "str"; };          # no default -> required
      replicas = { type = "int"; default = 2; };
    } (ledger.vars or { });
  in
  lib.toIR {
    providers.aws = lib.mkProvider {
      source = "registry.opentofu.org/hashicorp/aws";
      config.region = vars.region;
    };
    resources = [
      (lib.mkResource {
        provider = "aws"; type = "aws_s3_bucket"; name = "demo";
        config = { bucket = "myapp-${vars.suffix}"; force_destroy = true; };
      })
    ];
    inherit ledger;
  };

Read each variable as vars.<name>. Pass (ledger.vars or { }) so a run with no variables set still evaluates (the declared defaults fill in).

Types

type is one of:

A value whose type does not match its declaration is an error that names the variable and the expected type.

Defaults and required variables

• A variable with a default uses that default when unset.
• A variable without a default is required: if it is unset when the config reads it, evaluation fails with an error naming the variable. This is how you make a configuration refuse to run until a needed value is supplied.

Set variables (and precedence)

A variable's value is resolved from these sources, listed lowest to highest priority (a higher source overrides a lower one for the same name):

1. the default declared in Nix;
2. the environment variable NIVIS_VAR_<name>;
3. a --var-file <file> (a JSON object; when given more than once, a later file overrides an earlier one);
4. a --var name=value flag (when given more than once, a later flag overrides an earlier one).

So an explicit --var on the command line always wins. This is Terraform's precedence: it avoids a stale environment variable silently overriding what you typed, and it matches the mental model if you are coming from Terraform or OpenTofu.

# default in Nix (eu-central-1), overridden per run:
nivis plan --var region=us-east-1

# from a file (later --var-file wins; an explicit --var still beats the file):
nivis apply --var-file prod.json --var suffix=prod

# from the environment (lowest override; a file or flag beats it):
NIVIS_VAR_region=eu-west-1 nivis plan

A --var-file is a JSON object, so it can carry non-string values for int / bool / any variables:

{ "region": "us-east-1", "replicas": 4, "enabled": true }

Errors

• A --var without =, or with an empty name, is rejected with a message naming the offending flag.
• A --var-file that is missing or is not a JSON object is rejected naming the file.
• A required variable that is never set is an evaluation error naming it.

Purity and secrets

Variable values travel only inside the executor's 0600 ledger file (the same file that carries resolved outputs), never on the Nix command line and never into the Nix store. The Nix evaluation reads them as plain data with builtins.fromJSON; it does not read your environment. So variables introduce no impurity and no new path for a secret to leak into a world-readable store.

Variables are not yet a secrets mechanism: there is no "sensitive variable" marking in this version. Do not treat a plain variable as a secure secret store; a dedicated secrets integration is planned separately.

How variables fit the phased-eval loop

Variables are known inputs, resolved once before the first phase and injected unchanged on every phase (unlike resource outputs, which accumulate across phases to a fixpoint). A variable value is therefore always concrete: it is never a cross-resource reference or an unknown placeholder. In the injected ledger they appear as a vars object alongside outputs (see docs/IR-CONTRACT.md).

What is not here yet

• Module-system declaration (NixOS-style options.vars via lib.mkOption). mkVars is the current API; resolved values land in ledger.vars, the same place a future module-options layer would write to, so it can be added later without changing how you set variables.
• Rich types beyond str / int / bool / any (lists, attrsets, enums).
• Automatic file loading by name (a .auto-style convention); use an explicit --var-file.
• Sensitive variables; see the note above.

Datasources

A datasource reads existing infrastructure (an AMI by filter, a VPC, an availability zone, an existing bucket) and feeds it into your resources. Where a resource is created, a datasource is read: Nivis never plans, applies, writes to state, or destroys it. Declare one with nivis.mkData and wire its outputs into resources exactly as you wire one resource into another.

Declare a datasource with mkData

mkData mirrors mkResource but for a read:

nivis.plan =
  ledger:
  let
    ami = lib.mkData {
      provider = "aws";
      type = "aws_ami";
      name = "ubuntu";
      config = {
        most_recent = true;
        owners = [ "099720109477" ];        # canonical
        filter = [ { name = "name"; values = [ "ubuntu/images/*-24.04-*" ]; } ];
      };
    };
  in
  lib.toIR {
    providers.aws = lib.mkProvider {
      source = "registry.opentofu.org/hashicorp/aws";
      config.region = "eu-central-1";
    };
    dataSources = [ ami ];
    resources = [
      (lib.mkResource {
        provider = "aws"; type = "aws_instance"; name = "web";
        config = {
          ami = ami.refAttr "id";           # the datasource output feeds the resource
          instance_type = "t3.micro";
        };
      })
    ];
    inherit ledger;
  };

Pass datasources to toIR in a dataSources = [ ... ] list (distinct from resources). A datasource exposes refAttr/refPath just like a resource, so ami.refAttr "id" is an ordinary cross-node reference that creates a dependency edge.

Its id is namespaced data.<provider>.<type>.<name> (so it never collides with a resource id), and it carries no lifecycle: a datasource is read, never created.

When a datasource is read (the phased model)

Nivis reads a datasource when its config inputs are fully known, using the same per-phase readiness as resources. Two cases:

• Fully-known config (the common case): the datasource reads in the first phase, before the resources that depend on it, and its outputs are available immediately.
• Config that depends on a resource's apply-time output: the resource applies first, then the datasource reads in a later phase once that output is known, then resources that consume the datasource apply after that.

So a datasource participates in the round trip: you can apply a resource, read a datasource computed from its output, and feed that back into more resources, all in one nivis apply. A datasource whose config never becomes fully known is reported as a stuck node, the same as an unresolvable resource.

How it differs from a resource

What is not here yet

• Typed mkData constructors from a provider schema (nivis gen). mkData is hand-usable now; codegen for datasources comes later.
• Caching / staleness policy. A datasource is read once per run when ready; there is no cross-run cache.
• depends_on on a datasource. Implicit refs are enough for now.

Remote state (the S3 backend)

By default Nivis keeps state in a local JSON file (--state, default ./nivis.state.json). For a team or CI, declare a remote backend so state lives in a shared store instead. The first backend is S3.

State stays Nivis's own format. There is no tfstate compatibility (a deliberate design choice): Nivis state is not a Terraform/OpenTofu state file and the two are not interchangeable.

Configure it in the flake

The backend is part of your configuration, not a flag or an env var. Declare it with backend on your top-level config (it flows through the IR to the executor):

{ nivis }:
ledger:
nivis.toIR {
  backend = {
    type = "s3";
    bucket = "my-company-nivis-state";
    key = "prod/app.json";          # the object key for THIS stack's state
    region = "eu-west-1";
  };
  providers = { /* ... */ };
  resources = [ /* ... */ ];
  inherit ledger;
}

Then nivis plan / nivis apply read and write state in s3://my-company-nivis-state/prod/app.json instead of a local file. The whole state document is stored as that one object.

Keys

• type (required): "s3".
• bucket, key, region (required for s3): where the state object lives.
• endpoint (optional): override the S3 endpoint (for an S3-compatible store or a test server). Unset in production, where the AWS SDK resolves the real endpoint.
• sseAlgorithm, kmsKeyId (optional): how the objects are server-side encrypted. See Encryption.

This list is the complete set of keys the s3 backend accepts. Any other key is an error rather than being ignored, so a misspelling is reported instead of silently taking effect as its default. A local backend accepts only type.

The backend is static: its values must be plain (no references to resource outputs), because the executor has to know where state lives before it evaluates anything.

The local files a run writes

Even with a remote backend, a run writes the outputs ledger into your working directory, next to where --state points (<state path>.ledger). With the local store it writes the state document there too.

Both can hold secrets from provider outputs, and the ledger is where Nivis keeps sensitive ones in plaintext by design, to keep them out of the world-readable Nix store (see docs/IR-CONTRACT.md, "Sensitive values across the boundary"). They are written 0600; that stops another user on the machine, not git add. Ignore them:

nivis.state.json*
*.ledger

Nivis names these paths once, the first time a run creates them. A file already committed needs removing from the repository and its secrets rotating; ignoring it from now on leaves it in your history.

Credentials

Secrets are never in the config. The S3 backend uses the AWS default credential chain (the same chain the AWS CLI/SDK use): environment variables (AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY / AWS_PROFILE), the shared credentials/config files, or an instance/role profile. Set AWS_PROFILE (or the keys) in your shell or CI.

The config may name a role to assume (see below). That is an identity, not a credential: the credentials that authorize assuming it still come from the chain.

Assuming a role for state access

In a landing zone an operator usually authenticates once, as a management-account user, and reaches each workload account by assuming a role there. Declare that in the backend and the mapping lives in the repository instead of in every operator's ~/.aws/config:

backend = {
  type = "s3";
  bucket = "terraform-state-104144963194-production";
  key = "nivis/app.json";
  region = "eu-central-1";

  assumeRole = {
    roleArn = "arn:aws:iam::104144963194:role/landing_zone_devops_user";
    sessionName = "nivis";        # optional, defaults to "nivis"
    externalId = "...";           # optional, when the trust policy requires one
  };
};

• roleArn is required whenever the block is present. A block without it would assume nothing while reading as configured, so it is refused.
• sessionName is what appears in CloudTrail against every state write. The default names the tool; set it to something identifying the operator or the CI job when state changes need to be attributable to a person.
• externalId is sent only when set. Third-party and cross-account trust policies commonly require it.

The credentials from the default chain are the source identity for the assumption, so an operator still needs their base profile; they no longer need a per-account assume-role profile. The assumed credentials are cached and refreshed, so a long apply does not make an STS call per request and does not fail partway through when the first session ages out.

Assuming a role for state needs sts:AssumeRole on the role, and the role itself needs access to the bucket (and to its KMS key, if the bucket is CMK-encrypted).

This is not the provider's assume_role

They look alike and are different mechanisms:

Provider config follows the provider's schema, which is why it is snake_case and passed through untouched. The backend block is nivis's own schema. A config that switches accounts for both will carry both, and they are set independently.

Why there is no profile key

profile is refused deliberately. A role ARN means the same thing from any machine and any CI runner; a profile name points into one operator's ~/.aws/config, resolves differently elsewhere, and is usually absent in CI entirely. The backend block is committed configuration, so it carries the globally meaningful identifier and leaves the machine-local one to the environment. Use AWS_PROFILE for the base identity.

Encryption

Both objects the backend writes (the state object and the lock object) are server-side encrypted. How is set by sseAlgorithm:

kmsKeyId is required when sseAlgorithm is "aws:kms", and refused otherwise.

None of these modes writes state unencrypted. Every S3 bucket has had default encryption applied since January 2023, so "bucket-default" still produces an encrypted object: SSE-S3 at minimum, and the bucket's KMS key when one is configured.

Sharing a bucket that enforces SSE-KMS

Hardened state buckets commonly carry a policy that denies any write whose encryption header is present and is not the bucket's own default, for example:

{ "Effect": "Deny", "Principal": "*", "Action": "s3:*",
  "Condition": {
    "Null": { "s3:x-amz-server-side-encryption": "false" },
    "StringNotEquals": { "s3:x-amz-server-side-encryption": "aws:kms" } } }

"Null": "false" means "the header is present", so the statement fires only on a header that is there and wrong. Against such a bucket the default AES256 is denied and "bucket-default" is the mode to use:

backend = {
  type = "s3";
  bucket = "terraform-state-123456789012-production";
  key = "nivis/app.json";
  region = "eu-central-1";
  sseAlgorithm = "bucket-default";
};

Sending no header satisfies the policy and lets the bucket encrypt the object with its own CMK, which is also what Terraform does against these buckets when its backend sets no encrypt.

Use "aws:kms" instead when the bucket policy requires the key to be stated explicitly (the variant that denies a write whose key-id header is missing). Then kmsKeyId must be the exact key ARN the policy compares against: such policies test it as a string, so an alias or a bare key id names the same key and is still denied.

The symptom

A bucket that refuses your encryption mode fails the lock object first, since that is the first write of a plan or apply, and S3 answers an explicit policy Deny with a bare AccessDenied that never mentions encryption:

error: state: s3: acquire lock my-bucket/nivis/app.json.lock: ... AccessDenied
  This may be a server-side encryption mismatch rather than a credentials problem:
  nivis requested "AES256". A bucket whose policy enforces its own default
  encryption (commonly SSE-KMS with a fixed key) denies that write. If the bucket
  already defaults to the right key, set backend.sseAlgorithm = "bucket-default".

KMS permissions

Reading and writing a KMS-encrypted object needs permission on the key, not only on the bucket: kms:GenerateDataKey to write and kms:Decrypt to read. A run that can list the bucket but lacks kms:Decrypt fails on the read with another bare AccessDenied.

Enable bucket policies/versioning on your side as you would for any state bucket.

Locking

nivis apply and nivis destroy take an advisory lock on the state before they run, so two people (or two CI jobs) cannot mutate the same stack at the same time and corrupt it. The lock is a small sibling object next to your state object (<key>.lock), created atomically with an S3 conditional write (no DynamoDB or other service is needed). It is released automatically when the run finishes, including when it fails.

Read-only commands (plan, refresh, output, state pull) do not lock.

If a run is already holding the lock, the next apply/destroy stops before doing anything and tells you who holds it and since when:

error: state is locked by alice@ci-runner since 2026-06-22T10:31:04Z for "apply"; run `nivis force-unlock` to override

force-unlock

If a run crashes (or is killed) while holding the lock, the lock object is left behind and the next run is blocked. Clear it with:

nivis force-unlock

It confirms first in an interactive shell; pass --force (or --yes) to skip the prompt in CI. Only force-unlock when you are sure no other run is active, or you risk two concurrent applies. The local file store does not use this lock (it is single-machine and has its own per-operation file lock), so force-unlock there reports there is nothing to clear.

Moving state between backends

nivis state migrate moves the whole state document between your local state file and the backend your configuration declares, in the direction you choose:

nivis state migrate --to-remote     # local state file  ->  the declared backend
nivis state migrate --from-remote   # the declared backend  ->  local state file

It reports both sides before it does anything, and on success it says how many resources moved:

Migrating the state document
  from the local state file ./nivis.state.json
  to   s3://my-company-nivis-state/prod/app.json
Moved 7 resource(s) to s3://my-company-nivis-state/prod/app.json; removed the source document at ./nivis.state.json.

You must pass exactly one direction — Nivis will not guess which way your state of record should move. If your configuration declares no backend, there is nothing to migrate to or from and the command says so.

What it does, in order

1. Takes the state lock on both sides (for backends that support locking), so a migration cannot race an apply.
2. Copies the document to the destination.
3. Verifies the destination by reading it back.
4. Only then removes the source document — the local state file (along with its derived .ledger and .lock siblings), or the remote state object.

If anything fails before step 4, the source document is untouched and the error says so, so nothing is lost and you can re-run the command. If a run is interrupted between steps 3 and 4, the destination already holds the document: re-running finishes the job (it reports "an interrupted migration") instead of complaining that the destination is occupied.

Because the source is removed, you never end up with a stale local state file sitting next to a live remote backend — the single most confusing state a project can be in. If you want a copy first, take one with nivis state pull --out state.json (below).

When it refuses

A migration will not silently destroy state you did not mean to overwrite:

A refusal reports the resource count on both sides and changes nothing:

error: the destination already holds 3 resource(s) and the source holds 7;
refusing to overwrite state that is not a copy of the source (pass --force to overwrite it anyway)

Pass --force (or --yes) only when you are sure the destination's state is disposable. Enable bucket versioning on your state bucket: it is the backstop both for a forced overwrite and for the source-removal step.

The lower-level pair: pull and push

nivis state pull / state push still move the document as bytes, and remain the tool for anything migrate does not cover — taking a backup, editing a document, or seeding a backend from a file:

nivis state pull --out state.json          # export through the selected backend
nivis state push --in state.json --force   # replace through the selected backend

Both operate on whichever backend the configuration currently selects, so migrate is the one-step version of the old export/edit-config/import dance.

Bootstrapping a self-managed state bucket

A common setup has a configuration create the very bucket it declares as its own backend. That is a chicken-and-egg problem: on the first run the bucket does not exist, so state cannot live there yet. Bootstrap it in three commands:

# 1. Apply with local state, creating the bucket (the declared backend is ignored
#    for this run only; the configuration is not modified).
nivis apply --backend=local

# 2. Move the state document into the bucket that now exists.
nivis state migrate --to-remote

# 3. From here on, ordinary runs use the declared backend.
nivis apply       # reports no changes

--backend=local is the escape hatch: it makes a single run use the local state file at --state even though the configuration declares a remote backend. When it overrides a declared backend, the run says so, so it is never ambiguous which state a run operated on:

Using local state at ./nivis.state.json (--backend=local overrides the s3 backend this configuration declares).

It applies to every state-using command (plan, apply, destroy, refresh, and the state subcommands) and affects only the run it is given on. local is its only accepted value.

A missing bucket is an error, not empty state

Nivis distinguishes two things that look similar and are not:

• A missing state object — the bucket exists, but no state has been written yet. This is a fresh stack: reads return an empty state document and the first write creates the object.
• A missing state bucket — the state location itself is unreachable. This is an error on every operation:

error: state bucket "acme-nivis-state" does not exist in eu-west-1: the state LOCATION is
missing, not just the state document, so this is not treated as an empty state (that would
re-create resources you already own).
  If this configuration creates that bucket, bootstrap it with:
      nivis apply --backend=local
      nivis state migrate --to-remote
  Otherwise correct backend.bucket / backend.region in your configuration.

The reason for the distinction is blunt: "empty state" and "I cannot see your state" are opposite instructions to apply. The first means create; the second, if mistaken for the first, means re-create infrastructure you already own. So a missing bucket never reads as an empty stack — it stops the run and tells you which of the two situations you are in. A failure that is neither (a permission denial, say) keeps reporting its own cause.

Tutorial: remote state on S3, with locking (hands-on)

This walks you through the M2 team-ready features added in this release: keeping your state in a shared S3 object instead of a local file, and the lock that stops two applies from corrupting it.

To keep it focused on state (not cloud resources), the resources in this tutorial are the offline fake providers — but the state is stored in a real S3 bucket. So you exercise the real remote-state path (and real locking, a real lock object, a real force-unlock) without creating any billable infrastructure.

Prerequisites: Nix on your PATH, an S3 bucket you own, and AWS credentials in your environment (the AWS default chain). Set AWS_PROFILE (or AWS_ACCESS_KEY_ID/…). Credentials are never in the config: only the bucket/key/ region are.

Setup

Enter a shell with nivis and the fake providers, and point AWS at your profile:

nix shell github:nivis-project/nivis#nivis github:nivis-project/nivis#fake-providers
export AWS_PROFILE=your-profile

The config for this tutorial ships as the flake attribute nivis.remoteState (nix/example/remote-state.nix). Open it and set the bucket and region to one you own:

backend = {
  type = "s3";
  bucket = "your-state-bucket";              # <- yours
  key = "nivis-tutorial/remote-state/app.json";
  region = "eu-west-1";                      # <- your bucket's region
};

That backend block is the whole feature: it tells Nivis to store state in s3://your-state-bucket/nivis-tutorial/remote-state/app.json instead of a local file.

State lives where your config says, not where a flag says. Declare a backend on toIR and nivis reads and writes state there:

toIR {
  backend = { type = "s3"; bucket = "your-state-bucket"; key = "prod/app.json"; region = "eu-west-1"; };
  providers = { /* ... */ };
  resources = [ /* ... */ ];
  inherit ledger;
}

Credentials come from the AWS chain (AWS_PROFILE/keys); only the location is in config. Absent backend keeps the local file store (unchanged).

1. Apply: state goes to S3

AWS_PROFILE=$AWS_PROFILE nivis apply --attr nivis.remoteState

Acquired state lock.
Applied 2 resource(s) across 2 phase(s):

Phase 1
  + alpha.alpha_token.app
Phase 2
  + beta.beta_record.app
Released state lock.

Notice the Acquired/Released state lock lines: apply took the lock for the duration of the run. Now look in your bucket — the state object is there:

aws s3 ls s3://your-state-bucket/nivis-tutorial/remote-state/
# app.json

There is no local nivis.state.json: the state of record is the S3 object.

2. Plan reads state back from S3

AWS_PROFILE=$AWS_PROFILE nivis plan --attr nivis.remoteState

  = alpha.alpha_token.app (alpha_token)
  = beta.beta_record.app (beta_record)

No changes. 2 resource(s) up to date.

plan read the state straight from S3 and saw nothing to do. (Read-only commands like plan do not take the lock.)

3. Locking: concurrent applies are kept apart

apply and destroy take an advisory lock (a small <key>.lock object created atomically in S3) so two people or CI jobs cannot mutate the same state at once. If a run is already holding it, the next one stops before doing anything:

error: state is locked by alice@ci-runner since 2026-06-22T10:31:04Z for "apply"; run `nivis force-unlock` to override

You can see this for yourself: in one terminal, hold the lock by pausing an apply (or simulate a crashed run), and in another run nivis apply — it refuses with the holder's name and time.

If a run crashes while holding the lock, the lock object is left behind and the next run is blocked. Clear it:

AWS_PROFILE=$AWS_PROFILE nivis force-unlock --attr nivis.remoteState

It confirms first (pass --force in CI). Only do this when you are sure no other run is active.

4. Read outputs and clean up

AWS_PROFILE=$AWS_PROFILE nivis output --attr nivis.remoteState
AWS_PROFILE=$AWS_PROFILE nivis destroy --attr nivis.remoteState

destroy also takes the lock. When you are done, remove the state object (and any leftover .lock) from your bucket:

aws s3 rm s3://your-state-bucket/nivis-tutorial/remote-state/app.json
aws s3 rm s3://your-state-bucket/nivis-tutorial/remote-state/app.json.lock 2>/dev/null || true

Notes

• State is Nivis's own format: it is not a Terraform/OpenTofu tfstate file and the two are not interchangeable.
• Every write to the state object requests server-side encryption (AES256).
• To move an existing local state into S3, declare the backend and run nivis state migrate --to-remote (and --from-remote to bring it back). It copies the document, verifies it at the destination, and only then removes the source.
• If the bucket is one this configuration creates itself, bootstrap it with nivis apply --backend=local, then nivis state migrate --to-remote.

See Remote state (the S3 backend) for the full reference.

DESIGN.md: Nivis architecture & decisions

This is the decision ledger. It exists so that a future session does not re-derive (or undo) conclusions that were expensive to reach. Each decision records the choice, the reasoning, and the alternative that was rejected.

D1. Don't fork OpenTofu; drive the provider plugin protocol

Decision. Build a minimal Go engine that speaks the Terraform plugin protocol (tfprotov6, over HashiCorp go-plugin/gRPC) to provider binaries. Use terraform-plugin-go as the dependency and read OpenTofu's internal/plugin as the reference for how to use it.

Why. Forking to "strip what we don't need" inverts the cost. The parser and HCL loader are the small, easily-replaced parts (Nix replaces them). The provider plugin client, state engine, dependency graph, and DAG scheduler are the large parts, and we need those, so we'd inherit exactly the maintenance burden we wanted to avoid. The protocol is stable; the config frontend is the part we're actually changing.

Rejected. Forking OpenTofu and removing HCL. Higher ongoing cost, no upside.

D2. Spawn unmodified providers; do not link (contrast with Pulumi)

Decision. Launch the upstream provider binary as a subprocess and talk tfprotov6 to it.

Why / prior art. The Pulumi Terraform Bridge is the closest prior art and is worth mining, but it makes the opposite choice here, and the contrast is instructive. Pulumi does not use provider binaries; it compiles the provider's Go modules (against a forked plugin SDK) into its own provider binary, per-provider, with a shim. That buys Pulumi tighter integration at the cost of a per-provider build and a maintained SDK fork. Our headline goal is universal support for all existing providers with zero per-provider work, which spawn-not-link delivers directly. So we deliberately diverge from Pulumi here. Do not refactor toward the link model.

Mine from Pulumi instead: its schema type-mapping (required/optional/ computed/sensitive, sets vs lists, nested blocks), its ProviderInfo/overlay pattern (raw schema→code is usable but not idiomatic, so plan an override seam), and how it encodes unknown values to the provider during plan/diff (relevant to D4 below).

D3. Nix as a batch frontend; resolution by phased re-evaluation to a fixpoint

Decision. Nix evaluates configuration to a JSON IR. Cross-resource and cross-domain references that aren't yet known are emitted as typed placeholders. The Go executor applies what it can, collects real outputs, and the system re-evaluates Nix with those outputs injected, repeating until a fixpoint (no phase produces a new resolved value). Two phases is the shallow case; deeper Nix-mediated dependency chains need more. We explicitly support N phases.

Why. This is the central constraint of the whole project. Nix evaluation is a single forward batch pass that completes or errors: there is eval-time, then build/apply-time, and they are separate. A value a provider computes at apply (an IP, an ID, a generated secret) does not exist at eval time. Anything Nix must compute from that value (a hostname string, a NixOS option, another resource's input) therefore cannot be produced in the same evaluation. The only faithful way to feed apply-time values back into Nix is to evaluate again with them in scope.

The two flavors of reference (the executor must distinguish):

• TF→TF: resource A's output feeds resource B's input. Resolved inside the executor during apply; no re-eval needed.
• *→Nix: a Nix expression computes something from an apply-time value (and that result may feed further resources). Requires re-eval with the value injected. This is what drives phase count.

Why not Pulumi's elegant model. Pulumi represents not-yet-known values as Output<T> (a promise) and resolves them in-process as the program runs, because a Pulumi program is a live running process the engine can feed values back into. Nix has no promise, no suspend/resume, no live runtime to re-enter. Pulumi's model is unavailable to us not because it's cleverer but because its substrate is a different kind of thing. Our phased re-eval is the honest Nix-shaped equivalent, not a workaround to feel bad about.

Rejected (for now). "Option B": a live evaluator the engine drives via suspend/resume (libexpr internals). Elegant in theory, fragile and effectively unsupported in practice. Our phased loop converges toward B's expressiveness as iterations grow, without B's dependence on Nix internals. Revisit only if re-eval cost becomes a measured problem.

D4. The IR is the single frozen contract

Decision. IR-CONTRACT.md defines the JSON IR. It is the API between the Nix library (Epic 1), the codegen (Epic 2), and the executor (Epic 3/3.5). Breaking changes require an OpenSpec change to the contract first.

Why. Three workstreams depend on it; once stable they can progress in parallel. An underspecified linchpin is how this kind of project fragments. The hard parts the contract must pin down: reference encoding (nested attrs, list/set indices, refs inside for_each/count), for_each/count expansion timing (Nix expands, executor receives concrete resources), unknown-value representation toward the provider, and how sensitive values cross the JSON boundary without landing in world-readable nix eval output / the Nix store, are decided in the contract, not improvised per-epic.

D5. Prove the round trip before building breadth

Decision. Critical path is: Nix lib core → IR contract → executor that drives one (fake) provider through plan/apply → the phased-eval loop → the two-provider e2e. General schema codegen for arbitrary providers and registry integration come after the thesis is proven.

Why. The conceptual risk lives entirely in the round trip and the phased loop. Codegen is breadth (how we reach "all providers"), not risk. Hand-written constructors for the fake providers are enough to validate everything. Building the generation machinery first means a lot of code before a single resource round-trips.

D6. Hermetic testing via in-repo fake providers

Decision. Write minimal Go binaries that speak tfprotov6 and return canned/computed values (no real APIs, no credentials, no network). The executor drives them exactly as it would a real provider.

Why. Proves the protocol client and the whole pipeline deterministically and offline, essential given the restricted network, and the right substrate for the headline e2e. Real-provider runs are low conceptual risk and network-gated; they are out of scope for the PoC and tracked as a separate bean.

D7. Flake apps use nixpkgs; the library stays input-free

Decision. The flake exposes packages/apps for the nivis and nivis gen CLIs, built with nixpkgs buildGoModule (Go toolchain from a pinned nixpkgs input, module deps pinned by a committed vendorHash). The library outputs (lib, nivis.*) remain pure builtins and do not depend on the nixpkgs input: evaluating them imports nothing from nixpkgs.

Why. Originally the flake took no inputs at all, so the library evaluated without the binary cache (the configuration frontend must be cheap to evaluate every phase, and the cache was unreachable). A runnable CLI needs a real Go toolchain, which means nixpkgs. The refinement keeps the property that actually matters (the configuration-frontend outputs never force nixpkgs) while letting nix run .#nivis build the executor from source. The two concerns are kept separate in flake.nix: only packages/apps touch nixpkgs.

Rejected. flake-utils (replaced by a few lines of Nix that enumerate systems); a committed vendor/ directory (a one-line vendorHash keeps the repo lean). Keeping the CLIs go-build-only was the prior state; nix run is strictly additive: go build/go run still work unchanged.

Nivis vs the usual suspects

How does Nivis relate to the other tools people reach for when they want infrastructure-as-code, especially in (or near) the Nix world? This page is an honest comparison: where Nivis is genuinely different, and where it is young and the others are mature.

Maturity, stated plainly. Nivis is early (0.x, alpha). The round trip works across two providers and real providers (AWS today) apply / update / replace / destroy, but it has not been run at scale, the surface is small, and the contracts (the IR, the flake interface) are the stable parts while the rest moves. Everything below should be read with that in mind: a tool is not "better" than CloudFormation because a feature table has more checkmarks in its column. Maturity, ecosystem, and operational track record are features too, and there the established tools lead.

The one-line positioning

What actually makes Nivis different

Three choices, none of which the others combine:

1. Provider resources are first-class Nix values. Not generated HCL (Terranix), not a separate program (Pulumi/CDK), not a bespoke resource DSL. You write mkResource { … } and wire outputs with refAttr in plain Nix.

2. Spawn unmodified providers; do not link. Nivis talks the Terraform plugin protocol (tfprotov5/v6) over gRPC to the same provider binaries OpenTofu uses. Contrast Pulumi, which compiles each provider's Go into its own plugin via a bridge and a maintained SDK fork, per provider. Spawn-not-link is what buys universal, zero-per-provider compatibility with the OpenTofu ecosystem, at the cost of the tighter integration Pulumi's bridge gives.

3. The round trip via phased re-evaluation. A provider-created value (an IP, an ID, a generated secret) flows back into Nix, which re-evaluates to produce dependent config, repeating to a fixpoint. Pulumi gets a live Output<T> promise model for free because a Pulumi program is a running process; Nix is a batch evaluator with no live runtime, so Nivis does the honest Nix-shaped thing (re-eval to a fixpoint) rather than pretending to have promises. Terranix has no round trip at all: it generates HCL once and stops.

   This one is not exclusive to Nivis. NixOps 4 does it too: its manual documents output→input dependencies and "structural dependencies", where the graph topology itself is computed from a resource's outputs (lib.optionalAttrs (members.selector.outputs.value == "enabled") { … }), and its integration tests on main exercise exactly that. Treat the round trip as the reason Nivis exists, not as a moat.

The closest neighbor by intent — and, for the tfproto branch, by mechanism too — is NixOps 4: the same "one model of the world instead of IaC islands" premise, Nix-native, with provider binaries driven directly. Pulumi rides the same providers but compiles them; Terranix shares the language but not the ambition (it generates HCL once and stops).

What is left, honestly, is scope and licensing. NixOps 4 is a platform whose Terraform support is one provider kind among many, still in progress, under LGPL-2.1. In Nivis the coupling is the entire product — every OpenTofu provider works on day one because we spawn the unmodified binary — and the code is Apache-2.0, which matters if you intend to build on it.

Feature comparison

Legend: ✅ yes · ⚠️ partial / with caveats · ❌ no · n/a not applicable.

Essential features

Enterprise / operational features

This is where Nivis is youngest. Honest status:

Licensing (a real differentiator)

When to pick what

• You live in Nix and want real, multi-cloud infra with provider outputs feeding back into your Nix config: Nivis is the only tool aimed squarely at that, but accept the alpha maturity.
• You want Nix to author config but run it through battle-tested tooling: Terranix (Nix generates HCL, OpenTofu/Terraform runs it). No round trip, but mature and boring in the good way.
• You want a mature engine and the biggest provider ecosystem, HCL is fine: OpenTofu (open) or Terraform (BUSL).
• You want general-purpose languages and a hosted control plane: Pulumi.
• You are AWS-only and want native, deeply-integrated IaC: CloudFormation, or CDK if you want a real language synthesizing it.
• You deploy NixOS machines and want a Nix-native orchestrator: NixOps 4.

Sources (re-verify against these)

External facts above (versions, licenses, features of other tools) drift. When re-checking, confirm against the upstream docs and update the last-verified date at the top of this file:

• OpenTofu: https://opentofu.org · license & registry
• Terraform: https://developer.hashicorp.com/terraform · BUSL relicense notes
• Terranix: https://terranix.org
• NixOps 4: https://github.com/nixops4/nixops4 — doc/manual/src/index.md ("In progress: Terraform resource provider compatibility"), doc/manual/src/concept/resource.md (output→input and structural dependencies), rust/nixops4-resources-terraform/README.md and the tfproto branch, test/integration-test-nixops4-with-local/flake/flake.nix (the structural-dependency cases), and its LICENSE (LGPL-2.1).
• Pulumi & the Terraform bridge: https://www.pulumi.com/docs/ · https://github.com/pulumi/pulumi-terraform-bridge
• AWS CDK / CDKTF: https://docs.aws.amazon.com/cdk/ · https://developer.hashicorp.com/terraform/cdktf
• AWS CloudFormation: https://docs.aws.amazon.com/cloudformation/

Nivis's own claims are grounded in this repo: docs/DESIGN.md (the spawn-not-link and phased-eval decisions) and docs/OVERVIEW.md (the round trip).

docs/IR-CONTRACT.md: the frozen IR

This is the contract between the Nix library (Epic 1), codegen (Epic 2), and the executor (Epic 3 / 3.5). It is an API. Changing its shape requires an OpenSpec change to this document first, then downstream updates. Version it.

schemaVersion is bumped on any breaking change.

Top-level shape

{
  "schemaVersion": 1,
  "backend": {                                   // OPTIONAL; where state is stored
    "type": "s3",                                // required; the backend kind
    "bucket": "my-state", "key": "prod/app", "region": "eu-west-1"
    // backend-specific keys; STATIC only (no refs/unknowns); NO credentials here
    // (those come from the provider/AWS credential chain). Absent => local file store.
  },
  "providers": {
    "<provider-id>": {                 // e.g. "alpha", "registry.opentofu.org/x/alpha"
      "source": "<source-or-path>",    // for PoC: filesystem path to the binary
      "config": { /* provider config, may contain refs */ }
    }
  },
  "resources": [
    {
      "id": "<provider>.<type>.<name>",       // stable identity, unique
      "provider": "<provider-id>",
      "type": "<resource-type>",               // e.g. "alpha_token"
      "name": "<name>",
      "config": { /* attribute tree; leaves may be values, refs, or unknowns */ },
      "meta": {
        "dependsOn": ["<id>", ...],            // explicit edges (additive to implicit)
        "lifecycle": { "preventDestroy": false, "ignoreChanges": ["<path>"] }
        // count/for_each are NOT here: expansion already happened in Nix (see below)
      }
    }
  ],
  "dataSources": [                              // OPTIONAL; read, not created (see below)
    {
      "id": "data.<provider>.<type>.<name>",   // stable identity, unique, "data." prefix
      "provider": "<provider-id>",
      "type": "<datasource-type>",             // e.g. "alpha_lookup"
      "name": "<name>",
      "config": { /* attribute tree; leaves may be values, refs, or unknowns */ }
      // no meta/lifecycle: a datasource is read, never applied or destroyed
    }
  ],
  "edges": [
    { "from": "<id>", "to": "<id>", "via": "<config-path-in-to>" }  // dependency graph
  ],
  "nixConsumers": [
    // values Nix computed FROM resource outputs, surfaced for the round trip.
    // On a given phase these may still be unknown; they become concrete once
    // their inputs are resolved and Nix is re-evaluated.
    { "id": "<consumer-id>", "value": { /* tree; leaves may be values/refs/unknowns */ } }
    // A consumer whose id is "output.<name>" is a declared STACK OUTPUT (the
    // `outputs` arg to toIR), with value { "value": <expr> }. It is an ordinary
    // consumer (same resolution); the "output." prefix is a reserved convention
    // the executor reads for `nivis output`. No separate node type.
  ]
}

Backend (backend)

The OPTIONAL top-level backend object declares where state is stored. It is static configuration: it must be known before any evaluation (the executor has to know where state lives before it evaluates anything), so its leaves are plain JSON scalars/objects and may not contain a __ref, __derived, or other unknown leaf. It has a required non-empty string type naming the backend kind (e.g. "s3"); other keys are backend-specific and are interpreted by that backend, not the IR layer. Each backend defines and enforces the COMPLETE set of keys it accepts, so a key outside that set is an error rather than being ignored: for "s3" that set is type, bucket, key, region, endpoint, sseAlgorithm, kmsKeyId and the assumeRole block (roleArn, sessionName, externalId; see docs/REMOTE-STATE.md); for "local" it is type alone. Validation reaches inside a declared block, so a key misspelled one level down is reported rather than dropped.

Secrets are never in backend. It MAY name an identity to assume, such as a role ARN, which is a globally resolvable identifier rather than a credential; the credentials that authorize assuming it always come from the provider/AWS credential chain. It does NOT carry machine-local credential selectors such as a shared-config profile name, because backend is committed configuration and such a name resolves differently on each machine. Beyond an identity to assume, backend carries the location of state and the settings its backend defines for reaching it. When backend is absent, the executor uses the local file store (the default). Adding the optional field is additive: schemaVersion stays 1.

Reference encoding (the core of the contract)

A not-yet-known cross-resource or cross-domain value is a typed ref leaf:

{ "__ref": { "resource": "<id>", "path": ["attr"] } }                 // scalar attr
{ "__ref": { "resource": "<id>", "path": ["net", 0, "ip"] } }         // nested + list index
{ "__ref": { "resource": "<id>", "path": ["tags", "env"] } }          // map key

• path is an ordered list of string keys / integer indices into the source resource's output object. This covers nested attributes, list/set indices, and map keys uniformly. (For sets, index is the post-apply stable ordering the provider returns.)
• A ref inside an expanded for_each/count instance is just a normal ref whose resource is the concrete expanded id (e.g. alpha.alpha_token.web["a"] → id alpha.alpha_token.web__a). There is no special "expansion ref" because expansion is already done (below).
• A ref whose target resource does not yet exist in state is unresolved, not an error, until fixpoint (Epic 3.5.3).

Ref classification (drives phase behavior, DESIGN D3)

The executor classifies each ref:

• TF→TF: the ref appears in a resources[].config. Resolved in-executor when the target's output is known; does not require Nix re-eval.
• *→Nix: the ref appears in a nixConsumers[].value, or a resources[].config leaf that Nix itself derived from another resource's output (Nix marks these, see "derived" below). Resolving these requires re-eval with the outputs ledger injected.

Unknown values (toward the provider)

When the executor calls PlanResourceChange with inputs that are still refs, it must present them to the provider as the protocol's unknown value sentinel, not as the __ref JSON (providers don't understand our refs). The mapping { "__ref": ... } → tfprotov6 unknown is the executor's responsibility (Epic 3a.5). Mine Pulumi's bridge for how it encodes unknowns at plan/diff time.

for_each / count expansion timing

Expansion happens in Nix. The IR contains concrete, already-multiplied resources with deterministic ids (<base>__<key>). The executor never sees count/for_each; it only sees resolved instances and edges between them. This keeps the Go ResourceNode simple and the graph explicit.

Datasources (dataSources)

A datasource reads existing infrastructure (an AMI by filter, a VPC, an availability zone) rather than creating it. The optional top-level dataSources array carries them, distinct from resources. A datasource node is { id, provider, type, name, config } with id data.<provider>.<type>.<name> (the data. prefix keeps it from colliding with a resource id). It has no meta/lifecycle: a datasource is read via the provider's ReadDataSource, never planned, applied, written to state, or destroyed.

A datasource is a first-class node in the dependency graph: a __ref/__derived in a resource (or another datasource) config MAY target a datasource id, and a datasource config MAY reference a resource or datasource, producing edges like any other node. So datasources participate in the phased fixpoint: the executor reads a datasource when its config inputs are fully known. A datasource with a fully-known config reads in the first phase; one whose config depends on a resource's apply-time output reads in a later phase, after that output lands in the ledger. Its read attributes enter the outputs ledger keyed by its id, so downstream nodes resolve against them exactly as they resolve resource outputs.

"Derived" Nix values

A config leaf that Nix computed from a resource output (e.g. "web-" + alpha.id) cannot be a plain __ref (it's a transformation). Nix emits such a leaf as unknown-pending until the inputs are available:

{ "__derived": { "inputs": ["<id>.attr", ...] } }   // value computed by Nix once inputs known

The executor treats __derived leaves as *→Nix: it cannot compute them; it records that the listed inputs are required, and once those outputs are in the ledger, the next Nix re-eval produces the concrete value. This is the mechanism that forces N>2 phases for chained Nix-mediated dependencies.

Build outputs (__build)

A config leaf that is the output of a Nix build (e.g. a resource source that is a built disk image) is a __build leaf carrying two store paths — the build output the provider must be given, and the derivation that produces it:

{ "__build": {
    "path": "/nix/store/<hash>-<name>/<file>",   // the output the provider reads
    "drv":  "/nix/store/<hash>-<name>.drv"       // the derivation that produces it
} }

Both are needed, and neither substitutes for the other, and the executor does two distinct things with them:

• Substituting path into the config happens for every config handed to a provider — a plan, an apply, and a datasource read alike. It is a pure rewrite, performed once where configs are resolved, so no operation can hand a provider a raw leaf (a provider's encoder expects the value the leaf stands for).
• Realising the leaf's derivation — building the output, or substituting it if the store prefers — happens only when a resource is applied, per resource, as it becomes ready. nivis evaluates; it does not build until it applies.

So a plan on a configuration that builds an artifact reports its diff without building anything: a plan compares a path, and whether the artifact exists yet does not change the comparison. A datasource carrying a build leaf is likewise substituted and never built — it reads existing infrastructure.

The derivation is what makes the output producible. An output path names a result, not a recipe: realising an output path can reuse a path that is already valid or fetch one from a substituter, but it cannot build one, and the derivation is not recoverable from it (the hashes are unrelated, and a store cannot report the deriver of a path that is not yet valid). A leaf carrying only path is therefore substitute-only; it remains valid IR for compatibility with a Nix library predating the field, and the executor says so when such a leaf cannot be realised.

Realising happens per resource as it becomes ready, so a build whose derivation depends on an earlier resource's apply-time output is realised in a later phase: the build participates in the phased fixpoint. That case is also why building from the derivation is required rather than convenient — there, the derivation does not exist until the earlier resource is applied, so the author cannot pre-build the path, and substitution can never satisfy it.

Unlike __ref/__derived, a __build leaf is a known value in one specific sense: it does not depend on the outputs ledger, so it passes through resolution unchanged and is neither an edge nor unknown-pending. That is not the same as its path existing. Evaluation fixes the path string; for a derivation that has never been built, the artifact it names does not exist at all. Conflating the two is what makes an output-path-only leaf look sufficient when it is not.

Authors emit it with the drv helper (source = drv image), or drvFile for a file inside the output.

Sensitive values across the boundary

Provider schema marks attributes sensitive. Sensitive outputs:

• Must not be written into the IR JSON emitted by nix eval (that output and the Nix store are world-readable). The Nix side emits a ref/placeholder only.
• Live only in the executor's outputs ledger, which is written with restricted permissions (0600) and is not a Nix store path.
• When a sensitive output must feed a later Nix re-eval, it is injected via a private channel (file path passed as --argstr, file mode 0600), never baked into a derivation. The re-eval may use it but must not re-emit it into a world-readable output.

This is a hard requirement; getting it wrong leaks secrets into the store.

Outputs ledger (the phased-eval injection format)

The file the executor accumulates and injects on each re-eval:

{
  "phase": 2,
  "outputs": {
    "<resource-id>": { "<attr>": <value-or-{__sensitiveRef}>, ... }
  },
  "vars": {
    "<name>": <value>
  }
}

Nix reads this (path passed in via the flake plan argument) to resolve refs and compute __derived values on the next phase. __sensitiveRef points at the restricted-mode channel rather than embedding the secret.

outputs accumulates across phases. vars is the resolved configuration variables (see nivis.mkVars): a map of declared-variable name to its known value, resolved once by the executor before phase 0 and re-injected unchanged on every phase (variables are constant inputs, not resolved-across-phases outputs, so a vars value is never a ref or unknown). vars is optional: a ledger with no variables may omit it, and a plan that declares none may ignore it. The executor resolves vars from --var / --var-file / NIVIS_VAR_* with Terraform precedence (an explicit --var flag wins); the values travel only in this 0600 file, never on the Nix command line.

Validation

The contract is machine-checkable, not just prose. Two artifacts make it so:

• docs/ir-schema.json: the normative JSON Schema (Draft 2020-12) encoding the structural rules of everything above: top-level shape, the __ref/__derived/__sensitiveRef leaf encodings, and the no count/ for_each in the IR rule. A leaf-marker object (__ref etc.) is dispatched to its exact subschema, so a malformed marker reports a precise, addressed error (e.g. at resources/1/config/label/__ref: 'path' is a required property) rather than a generic failure.
• tests/ir-conformance/: the executable conformance suite. check.py layers (1) JSON-Schema structural validation over (2) the referential rules JSON Schema cannot express: unique ids, every provider declared, every edge endpoint present, every __ref/__sensitiveRef target existing. Fixtures under fixtures/valid and fixtures/invalid lock both directions; each invalid fixture asserts the error names the offending element.

Both producer/consumer sides MUST conform to these artifacts:

• Nix (Epic 1.5 toIR): a property test that, for arbitrary valid resource graphs, toIR output passes check.py validate: every leaf is a value, a well-formed __ref, a __derived, or a __sensitiveRef; ids unique; every edge endpoint exists.
• Go (Epic 3a.1 IngestIR): rejects malformed IR with an actionable error naming the offending resource/path (Epic 4c), matching the failure classes in tests/ir-conformance/fixtures/invalid. check.py is the reference behavior the Go validator is tested against.

Run the suite: python3 tests/ir-conformance/check.py test.

docs/TESTING.md: testing strategy & the headline e2e

Testing is part of "done." No OpenSpec change is complete without its tests.

Layers

1. Pure Nix functions, property tests. mkResource, the reference system, for_each/count expansion, and toIR conformance to docs/IR-CONTRACT.md. Property: for arbitrary valid resource graphs, every IR leaf is a value, a well-formed __ref, or a __derived; ids are unique; every edge endpoint exists.
2. Go, table-driven unit tests. IR ingestion/validation, DAG construction, ref classification (TF→TF vs *→Nix), TF→TF in-executor resolution, the __ref→tfprotov6-unknown mapping, state read/write/lock, fixpoint detection.
3. Integration, against fake providers, no network. Executor spawns a fake tfprotov6 provider, completes the go-plugin handshake, drives GetProviderSchema/PlanResourceChange/ApplyResourceChange, and persists state. Proves the protocol client end-to-end offline.
4. E2E, the full pipeline. .nix → IR → phased plan/apply → state → refresh → destroy, culminating in the headline test below.

All provider-touching tests use in-repo fake providers so the suite is hermetic and runs in CI without credentials or registry/network access.

The fake providers (Epic 4a)

Two minimal Go binaries that speak tfprotov6. Each returns a static schema and produces computed (unknown-at-plan) outputs at apply.

provider-alpha, resource alpha_token:

• inputs: label (string, optional)
• computed outputs (known only after apply): id (string), value (string, derived deterministically from label + a counter so tests are reproducible)

provider-beta, resource beta_record:

• inputs: from (string, required)
• computed output (known only after apply): endpoint (string, derived from from)

Determinism: outputs are a pure function of inputs (+ a per-process counter that the test harness seeds), so assertions are exact. No clocks, no randomness, no external calls.

Headline e2e: milestone exit criterion (Epic 4b)

What it must prove, in one test: two providers, unknown values originating on both sides, resolution requiring ≥3 phases, and a Nix-side consumer reading outputs from both providers (the round trip). The dependency graph is acyclic; the phase count comes from each hop being Nix-mediated (__derived), which is exactly what forces N>2.

Topology (tests/e2e/two-providers.nix)

alpha_token.A           (alpha)  : no inputs
   └─ A.value  ─┐
                ▼  Nix: name = "rec-" + A.value          (__derived on A.value)
beta_record.B  (beta)   from = name
   └─ B.endpoint ─┐
                  ▼ Nix: final = B.endpoint + "::" + A.value   (__derived on B.endpoint, A.value)
alpha_token.C  (alpha)  label = final

# Nix-side consumer reading from BOTH providers (simulated NixOS option):
systemConfig = {
  recordEndpoint = B.endpoint;   # from beta
  tokenValue     = A.value;      # from alpha
  combined       = final;        # from both
}

• Unknowns originate on both sides: A.value/C.* from provider alpha and B.endpoint from provider beta.
• The chain A → (Nix name) → B → (Nix final) → C is acyclic but each arrow crosses the Nix boundary via __derived, so it cannot collapse into one pass.

Required phase progression

• Phase 0 eval: A.config fully known; name is __derived on A.value (unknown); B, final, C, systemConfig all pending.
• Phase 1 apply: only A is ready → apply A → ledger gains A.id, A.value.
• Phase 1→2 eval: re-eval injects A.value → name resolves → B.config.from now known; final still pending on B.endpoint.
• Phase 2 apply: B ready → apply B → ledger gains B.endpoint.
• Phase 2→3 eval: re-eval injects B.endpoint → final resolves → C.config.label known; systemConfig fully resolves (both providers present).
• Phase 3 apply: C ready → apply C. No pending refs remain.
• Phase 4 eval: produces no new resolved value → fixpoint → halt.

Assertions

• Total phases that performed an apply == 3 (and the loop halts at fixpoint, not by a hardcoded count).
• Attempting to resolve with a 2-phase cap leaves final/C/systemConfig unresolved → the engine reports them as pending (proves >2 phases is required, not incidental).
• Final outputs ledger contains A.id,A.value,B.endpoint,C.*.
• systemConfig evaluates to concrete values for recordEndpoint, tokenValue, and combined, each matching the deterministic provider outputs, proving TF→Nix feedback from both providers.
• A cycle variant (make A.label depend on C.*) is rejected at fixpoint with an actionable "unresolvable / cycle" error naming A and C (Epic 3.5.3).
• destroy removes C, B, A in reverse dependency order; refresh reconciles state via ReadResource without changing the plan.

Why this is the right exit test

It exercises every load-bearing decision at once: the protocol client (real tfprotov6 handshake to two providers), TF→TF and *→Nix ref handling, the __derived mechanism, N-phase fixpoint resolution with N>2, and the round trip that is the project's entire reason for existing, with unknowns genuinely originating on both provider sides.

ROADMAP.md: Nivis

Nivis cleared its proof-of-concept milestone. The thesis is proven: real Terraform/OpenTofu provider resources are first-class Nix values, driven by a thin Go executor that spawns unmodified provider binaries, and provider outputs round-trip back into Nix across phases to a fixpoint. On top of that we have real AWS apply/update/replace/destroy, schema codegen, and an end-to-end "build a NixOS AMI and launch it" example.

That makes Nivis experimental / alpha (0.3.x): real, but small. This roadmap is about the next thing, taking Nivis from "the demo works" to "I can run my real infrastructure on this," and eventually to something an enterprise can adopt. The PoC roadmap that got us here is preserved at the bottom as history.

How this maps to beans. Each phase below is a beans milestone; each theme under it is a beans epic; each task inside an epic is an OpenSpec change (spec before code). See CLAUDE.md §3. The doc is the why and what; beans is the audit trail.

Where we are honestly weak

docs/COMPARISON.md states this plainly. Versus Terraform/OpenTofu, Pulumi and CDK, the gaps that actually block adoption today are:

• State is local-only. There is a Store interface seam, but no shared backend and no locking. Two people (or CI) cannot safely touch the same infra.
• No variables / overrides. Config is whatever the flake hard-codes plus an ad-hoc ledger.vars. There is no first-class way to parameterise per environment or pass values at the CLI.
• No datasources. The provider protocol's ReadDataSource is unused; you cannot look up an existing AMI, VPC, or zone the way every other tool can.
• Thin DX. Plan/apply/destroy output is not colorised by change type, there is no shell completion, and there is no per-provider reference documentation.
• No enterprise controls. No policy-as-code, no RBAC/audit, no hosted control plane, and provider download from the registry is network-gated and not the default path.

The phases close these in the order that unlocks the widest audience soonest.

Architecture invariants (do not regress)

Every phase below is bound by docs/DESIGN.md. In particular: spawn unmodified providers, do not link them; Nix is a batch evaluator resolved by phased re-evaluation to a fixpoint, not a live Output<T> runtime; the IR is the frozen contract (docs/IR-CONTRACT.md), so any feature that changes the IR shape needs an OpenSpec change to the contract first; and tests run against in-repo fake providers (hermetic, no network, no credentials).

Phase A: a daily-driver for Nix developers ⟵ the next milestone

Beans milestone: nixform2-zdj0 ("Road to v1"). Epics: A1 nixform2-kym5, A2 nixform2-6e6i, A3 nixform2-yqd3, A4 nixform2-oycy, A5 nixform2-n2rg, A6 nixform2-z8e1.

Definition of done: a Nix developer can manage a real, multi-resource project end to end, day to day, without dropping back to Terraform, with shared state, parameterised config, datasource lookups, and a plan they can actually read. This is the headline goal for the next milestone, the same role the round-trip e2e played for the PoC.

• A1. Variables and overrides. First-class inputs to a plan: typed variables with defaults, a CLI way to set them (--var, --var-file), and a clear precedence (defaults < file < flag < environment). Must thread cleanly through the phased-eval loop (the ledger already carries vars; formalise it) and stay pure: no impurity sneaks into the Nix evaluation. Probably an IR-contract touch for how vars enter the plan function.
• A2. Datasources. Drive the provider protocol's ReadDataSource so a config can read existing infrastructure (an AMI by filter, a VPC, an availability zone) and feed it into resources. Needs a Nix-lib constructor (mkData or similar), executor support, and an IR-contract addition for the datasource node and its outputs. Datasource reads happen per phase like any other node.
• A3. Legible plan/apply/destroy output. Colorise by change type (+ create, ~ update, -/+ replace, - destroy, = no-op), summarise counts, and make the phased nature visible (which resources resolved in which phase). Respect NO_COLOR and non-TTY output. No behaviour change, pure DX.
• A4. Shell completion. Cobra can generate bash/zsh/fish completion; wire it up (nivis completion <shell>) and complete resource ids for state show / --target from the state file.
• A5. Per-provider reference docs. Today a user reads the provider's Terraform registry docs and mentally translates HCL to Nivis. Generate or curate a "Terraform docs to Nivis" mapping so aws_instance's arguments are discoverable in Nivis terms. Couples naturally to schema codegen (Epic 2, already built).
• A6. State ergonomics. Configurable state path is done; add the small things a real project needs: state list/show/rm polish, a state pull/push shape that the remote backend (Phase B) will reuse, and clear errors on a stale or locked state file.

Phase B: team-ready ⟵ after Phase A

Beans milestone: nixform2-kovh. Epics: B1 nixform2-izhk, B2 nixform2-0oqk, B3 nixform2-tyzs, B4 nixform2-cdfj.

Definition of done: multiple people and CI can safely operate the same infrastructure concurrently.

• B1. Remote state backend (S3 first). Implement the Store seam against S3 (object per state, server-side encryption, the credential chain Nivis already uses). Keep the format Nivis's own; no tfstate compatibility guarantee (DESIGN). Configured in the flake, not via env soup.
• B2. State locking. A lock so two concurrent applies cannot corrupt state (DynamoDB-style advisory lock for the S3 backend, with a force-unlock escape hatch and clear "who holds the lock" errors).
• B3. Drift detection. refresh exists; build a real "plan shows drift" experience that reconciles remote reality against stored state and surfaces out-of-band changes.
• B4. Multiple environments. A clean pattern for dev/staging/prod from one config: workspaces or per-environment var-files + state keys, decided in a spec, not improvised.

Phase C: enterprise-credible ⟵ the longer horizon

Beans milestone: nixform2-1okn. Epics: C1 nixform2-alr9, C2 nixform2-84fs, C3 nixform2-m83a, C4 nixform2-q7fx, C5 nixform2-7evo.

NixOS is gaining enterprise traction; this is where Nivis earns a seat there. These are deliberately later, after the basics are solid, and several are large enough to be their own milestones.

• C1. Policy as code / guardrails. A pre-apply policy hook (deny by rule, required tags, allowed regions). Evaluate doing this in Nix (assertions in the module system) versus an external engine; Nix-native is the differentiator.
• C2. RBAC, teams, audit. The story for who can apply what, and an audit trail. Likely pairs with a remote backend and possibly a hosted control plane; scope carefully, this is where tools grow a SaaS.
• C3. Provider registry integration. Real provider download/verify/cache from the OpenTofu registry. Network-gated (CLAUDE.md §6); today providers are fetched on first use but this needs hardening, offline/air-gapped mirrors, and supply-chain verification for enterprise.

  Companion project: a separate nivis-registry project has been started to own (some or all of) this. The exact split between nivis-registry and the in-repo C3 epic (nixform2-m83a) is not decided yet; once it is, this entry and the bean will be reconciled (C3 may move wholesale to the companion project or keep only the client-side integration here).

• C4. Secrets at scale. The IR already keeps sensitive values out of the world-readable store; extend to integration with real secret stores (Vault, SSM, sops-nix) so secrets never transit the Nix store at all.
• C5. Scale and performance. Phased re-eval cost on large graphs is currently unmeasured. Measure it; optimise only if it is a measured problem (DESIGN rejects premature cleverness like a live evaluator).

Cross-cutting, every phase

• Stay hermetic. Every feature lands with tests against the in-repo fakes; the fakes grow new capabilities (a datasource-serving fake, a drift-injecting fake) as the features that need them arrive.
• Keep the lib pure. nivis.lib stays builtins-only (no nixpkgs); only packages/apps may force nixpkgs.
• Spec before code. IR-affecting work (vars entry, datasource node) updates IR-CONTRACT.md via an OpenSpec change first.

History: the PoC milestone (delivered)

Kept for the record. This is the roadmap that proved the thesis; every epic below is complete (see the beans milestone nixform PoC / alpha base and its epics).

Milestone exit criterion (met): the headline e2e in TESTING.md: two providers, unknown values originating on both sides, resolved across ≥3 phases, with a Nix-side consumer reading outputs from both providers.

Critical path that was followed:

E1 (Nix lib core: mkResource + refs + IR serializer)
        │
E1.5 ── IR CONTRACT  (linchpin; written & frozen first)
        │
E4a ── fake tfprotov6 providers (alpha, beta)  (test substrate)
        │
E3a ── executor: ingest IR, spawn ONE fake provider, plan+apply, write state
        │
E3.5 ── PHASED EVALUATION TO FIXPOINT  (the thesis)
        │
E4b ── headline two-provider / unknowns-both-sides e2e  (milestone exit)
        │
(then breadth:) E2 schema codegen · E3b refresh/destroy/CLI · E4c/4d error UX & docs

Epics delivered (PoC and the alpha follow-ons):

• E1 Nix library core: mkResource, mkProvider, the reference system, meta-arguments (depends_on, lifecycle, count/for_each expanded in Nix), the module system, toIR, and the flake interface (nivis.plan).
• E1.5 The IR contract: IR-CONTRACT.md + ir-schema.json, the frozen JSON contract pinning ref encoding, expansion timing, unknown representation, and sensitive-value handling.
• E2 Provider schema codegen (nivis gen): typed Nix constructors from a provider's GetProviderSchema.
• E3 Go executor: IR ingestion, the lockable local Store, the plugin manager (spawn + gRPC handshake, v5 and v6), the DAG, plan and apply engines.
• E3.5 Phased evaluation to fixpoint: the outputs ledger, the phase driver, fixpoint and cycle detection, and verified *→Nix feedback (the round trip).
• E3b Refresh and destroy engines and CLI (plan/apply/destroy/refresh/state, --target, --refresh, --build).
• E4a Fake tfprotov6/tfprotov5 providers (the hermetic test substrate).
• E4b The headline two-provider, unknowns-both-sides, ≥3-phase e2e.
• E4c/4d Error UX and docs (actionable errors; README, getting-started, the stable-contract docs).
• Real-provider support (M2): real tfprotov5 + on-first-use registry fetch, proven against AWS.
• Resource lifecycle: update and replace beyond create-only, with prevent_destroy.
• EC2 + NixOS: build a NixOS AMI in Nix and launch it through Nivis, with nivis apply realising the image itself (__build / nivis.drv).
• Branding, rename to Nivis, release management (versioning, changelog, releases, the docs site).

Nivis: brand reference

Nivis (Latin, "of snow") · tagline "Infrastructure as Nix Code." Formerly nixform. This file records the brand tokens so future work (docs site, UI, slides) has the palette and type in-repo, and is the in-repo source of truth for the logo geometry and treatments.

Logo

• assets/nivis-emblem.svg: full emblem (snow-capped twin summit on a navy disc with silver ring + ember star). Use at ≥40px.
• assets/nivis-glyph.svg: simplified single-peak mark for 16-64px (favicons, tabs, avatars).
• Never recolour (beyond the ember star), stretch, skew, rotate, or shadow the emblem. Clear space ≥ the ring thickness. Below 40px use the glyph.

Colour tokens

Typography (all Google Fonts, OFL)

• Cinzel (600): wordmark / display, Roman caps, letter-spacing ≈ .07em, used UPPERCASE ("NIVIS").
• Schibsted Grotesk (400/500/600): UI, body, tagline.
• IBM Plex Mono (400/500): code, CLI, labels.

CLI colours (nivis)

Truecolor ANSI, applied only on a TTY (honours NO_COLOR): ember \e[38;2;242;99;46m for the ❯ prompt and "fixpoint reached"; ice blue \e[38;2;174;207;230m for resource names/values; dim grey for secondary text.

Regenerating the raster assets

The logo SVGs are the source; the icons and banner PNG are generated from them. Requires rsvg-convert and ImageMagick (magick), plus the Cinzel and IBM Plex Mono fonts available to fontconfig.

# favicon (32px) + apple-touch-icon (180px glyph on a #0E3157 tile, ~14% padding)
cp assets/nivis-glyph.svg assets/favicon.svg
rsvg-convert -w 32 -h 32 assets/nivis-glyph.svg -o /tmp/f32.png
magick /tmp/f32.png assets/favicon.ico
rsvg-convert -w 180 -h 180 docs/assets/apple-touch-icon.svg -o assets/apple-touch-icon.png

# README hero banner (1280×640): needs Cinzel + IBM Plex Mono on fontconfig
rsvg-convert -w 1280 -h 640 docs/assets/banner.svg -o docs/assets/banner.png

The banner/apple-touch source SVGs live in docs/assets/ / generated inline; the committed PNGs are reproducible from them.

Releases

Per-version release notes: what each release delivered, with the runnable examples from the tutorials. (For the per-version changelog, see CHANGELOG.md; for the process, see Releasing.)

Nivis 0.4 (Road to v1)

The daily-driver milestone: variables, datasources, stack outputs, colored phased plan/apply, completion, generated provider references, state ergonomics.

• Release Notes 0.4 - Road to v1.
• Tutorial: Nivis 0.4 features explained: a hands-on, hermetic tour, runnable against the in-repo fake providers (no cloud).

Release notes: M1: Road to v1 (a daily-driver for Nix developers)

Goal

A Nix developer can manage a real, multi-resource project end to end, day to day, WITHOUT dropping back to Terraform: typed variables/overrides, datasource lookups, a legible (colorised, phase-aware) plan/apply/destroy, shell completion, and per-provider reference docs. Shared remote state and locking are Phase B; this milestone is the single-operator daily-driver.

Highlights

What you can do, with verified examples from the tutorials:

Apply shows the round trip, grouped by phase

A single apply reads a datasource, then resolves resources across phases (the fixpoint made visible), colorised by change type:

nivis apply --attr nivis.tutorial --var env=prod

Applied 3 resource(s) across 3 phase(s):

Phase 1
  r data.alpha.alpha_lookup.existing
Phase 2
  + alpha.alpha_token.app
Phase 3
  + beta.beta_record.app

From TUTORIAL-FEATURES.md.

Read named outputs out of a run

Surface named values out of a run with nivis output (text, a single value, or --json for a CI step):

nivis output --attr nivis.tutorial --var env=prod

endpoint = beta://env-prod-alpha:found:prod:0
env = prod
lookupResult = found:prod
replicas = 2

From TUTORIAL-FEATURES.md.

What shipped

• A1: Variables and overrides
• A2: Datasources
• A3: Legible plan/apply/destroy output
• A4: Shell completion
• A5: Per-provider reference docs
• A6: State ergonomics
• A7: Stack outputs (declare nivis.outputs + a nivis output command)
• Docs-coverage agent-gate + Variables document
• Nested-block ergonomics: list-vs-single is a cryptic apply-time trap

Changelog

The Road to v1 milestone (M1): the daily-driver features, so a Nix developer can manage a real, multi-resource project end to end without dropping back to Terraform. See docs/TUTORIAL-FEATURES.md for a hands-on, no-cloud tour and docs/releases/release-0.4/release-notes-0.4.md for the milestone notes.

Added

• Variables (nivis.mkVars): declare typed config variables (str/int/ bool/any) with defaults; required when no default. Set them with --var name=value, --var-file <json>, or NIVIS_VAR_<name>, with Terraform precedence (an explicit --var wins). String values are coerced to the declared scalar type, so --var replicas=5 satisfies an int var.
• Datasources (nivis.mkData): read existing infrastructure (an AMI, a VPC, a lookup) and feed it into resources. Read per phase, so a datasource may depend on a resource's apply-time output (it rides the round trip). Never planned, applied, or written to state.
• Stack outputs: declare named values with the outputs argument to toIR and read them with nivis output [name] (human-readable, a single value, or --json for a CI step / another stack).
• Shell completion: nivis completion <bash|zsh|fish|powershell>, with dynamic completion of resource ids (from state) for state show, state rm, and --target.
• State pull/push: nivis state pull / nivis state push move the whole state document (the seam a remote backend will reuse); push confirms before overwriting and requires --force when non-interactive.
• Codegen now emits nested blocks: nivis gen constructors include a resource's nested blocks with the correct list-vs-single shape, so the generated constructor doubles as the per-provider argument reference.
• Docs: a hands-on feature tutorial (against the in-repo fakes, no cloud), Variables and Datasources reference pages, a comparison page vs other IaC tools, and a forward-looking roadmap.

Changed

• Plan/apply/destroy output is colorised by change type and grouped by phase (+ create, ~ update, -/+ replace, - destroy, = no-op, r datasource read), so the phased fixpoint is visible. Respects NO_COLOR and non-TTY output.
• State commands report clearly: state list notes when empty, state rm of a missing id says so, and a held state lock now times out with an actionable message instead of hanging.

Fixed

• Nested-block shape mistakes (a list-nested block written as a bare attrset) now produce an actionable error naming the attribute and the fix, instead of a cryptic codec error.

Tutorial: the daily-driver features (hands-on, no cloud)

This walks you through everything added on the road to v1, in one config you can run right now against the in-repo fake providers: no AWS account, no credentials, no network, no cost. You will use:

• variables (mkVars, --var),
• a datasource (read existing infra and feed it in),
• the round trip across phases (a value a provider computes, fed back through Nix into the next resource),
• stack outputs (nivis output),
• the colored, phase-grouped plan/apply output,
• shell completion and state pull/push.

The fake providers are deterministic, so your output will match what is shown below exactly.

Setup

From a checkout of the Nivis repo, enter a shell with nivis and the in-repo fake providers on your PATH, in one command:

nix shell .#nivis .#fake-providers

That is all the setup. nivis and the fake providers (provider-alpha, provider-beta) are now on your PATH, and the example config references them by bare name, so there is nothing to build or copy.

The config for this tutorial ships with the repo as the starter nix/example/tutorial-features-0.4/, exposed as the flake attribute nivis.tutorial. Every command below passes --attr nivis.tutorial. (In your own project you would write your own flake with nivis.plan; here we point at the bundled one so there is nothing to scaffold.)

No repo checkout? Scaffold it into a sandbox. nivistutor writes this tutorial's files (a ready flake.nix, the config, and a README) into your own directory, so you run it with plain nivis (no --attr). In a throwaway shell:

nix shell github:nivis-project/nivis#nivis github:nivis-project/nivis#tutor
nivistutor --tutorial features-0.4 --dir nivis-features
cd nivis-features
nivis plan --var env=prod

See the nivistutor section in Getting started.

Here is the whole config, annotated. It is one small graph that touches every feature:

{ nivis }:
ledger:
let
  inherit (nivis) mkResource mkData str toIR mkVars;

  # VARIABLES: typed inputs. `env` is required (no default); `replicas` defaults.
  vars = mkVars {
    env = { type = "str"; };                 # required
    replicas = { type = "int"; default = 2; };
  } (ledger.vars or { });

  # DATASOURCE: read "existing" infra. The fake returns result = "found:<query>".
  lookup = mkData {
    provider = "alpha"; type = "alpha_lookup"; name = "existing";
    config = { query = vars.env; };
  };

  # A resource whose label embeds the datasource result: data flows IN.
  token = mkResource {
    provider = "alpha"; type = "alpha_token"; name = "app";
    config = { label = lookup.refAttr "result"; };
  };

  # ROUND TRIP: a beta record whose `from` is a Nix string over the token's
  # apply-time value -> resolves in a later phase.
  record = mkResource {
    provider = "beta"; type = "beta_record"; name = "app";
    config = { from = str [ "env-${vars.env}-" (token.refAttr "value") ]; };
  };
in
toIR {
  providers = {
    alpha = { source = "provider-alpha"; config = { }; };  # on $PATH (nix shell)
    beta  = { source = "provider-beta";  config = { }; };
  };
  dataSources = [ lookup ];
  resources = [ token record ];
  # OUTPUTS: named values surfaced out of the run.
  outputs = {
    env = vars.env;
    replicas = vars.replicas;
    lookupResult = lookup.refAttr "result";
    endpoint = record.refAttr "endpoint";
  };
  inherit ledger;
}

1. Variables: a required variable

env has no default, so it is required. Run a plan without it:

nivis plan --attr nivis.tutorial

error: nivis.mkVars: required variable 'env' is not set (declare a default or pass --var env=...)

That is mkVars refusing to proceed until you supply env. Set it with --var:

nivis plan --attr nivis.tutorial --var env=prod

  + alpha.alpha_token.app (alpha_token)
  + beta.beta_record.app (beta_record)

2 change(s) across 2 resource(s) (+ create, ~ update, -/+ replace, = no change). Run `nivis apply`.

On a terminal the + markers are green. Note the datasource is not in the plan: a datasource is read, not created.

Variable precedence (lowest to highest, like Terraform): a default in Nix, then NIVIS_VAR_<name>, then --var-file, then --var. So an explicit --var always wins. An int variable accepts the string form from the CLI, so --var replicas=5 works.

2. Apply: the datasource read and the round trip, by phase

A single apply reads a datasource, then resolves resources across phases (the fixpoint made visible), colorised by change type:

nivis apply --attr nivis.tutorial --var env=prod

Applied 3 resource(s) across 3 phase(s):

Phase 1
  r data.alpha.alpha_lookup.existing
Phase 2
  + alpha.alpha_token.app
Phase 3
  + beta.beta_record.app

Read the phases top to bottom, the fixpoint made visible:

• Phase 1 reads the datasource (r = read, distinct from + create). Its result is found:prod.
• Phase 2 creates the token, whose label is the datasource result.
• Phase 3 creates the beta record, whose from is a Nix string built from the token's apply-time value. That value did not exist until phase 2, so Nix re-evaluated with it injected and phase 3 resolved. That is the round trip.

On a terminal, r is dim and + is green; piped or with NO_COLOR set the same markers are plain text.

3. Inspect the round trip in state

nivis state show alpha.alpha_token.app

alpha.alpha_token.app (alpha_token)
  label = found:prod
  value = alpha:found:prod:0
  id = alpha-0

label = found:prod is the datasource result that flowed into the resource. value then embeds it, and that value is what the beta record's from is built from in the next phase.

4. Stack outputs

Surface named values out of a run with nivis output (text, a single value, or --json for a CI step):

nivis output --attr nivis.tutorial --var env=prod

endpoint = beta://env-prod-alpha:found:prod:0
env = prod
lookupResult = found:prod
replicas = 2

• lookupResult is from the datasource,
• endpoint is the round-trip value (built across both providers and phases),
• env is your variable echoed out, replicas is the int default.

Print one output, or get JSON for a CI step / another stack:

nivis output endpoint --attr nivis.tutorial --var env=prod
# beta://env-prod-alpha:found:prod:0

nivis output --attr nivis.tutorial --var env=prod --json

{
  "endpoint": "beta://env-prod-alpha:found:prod:0",
  "env": "prod",
  "lookupResult": "found:prod",
  "replicas": 2
}

Change the variable and watch the outputs change: --var env=dev gives lookupResult = found:dev, and --var replicas=5 gives replicas = 5.

5. Move state around (pull / push)

The whole state document is portable:

nivis state pull > backup.json     # whole state to a file
nivis state list                   # the resource ids in state

state push replaces state from a file or stdin; it confirms first (and requires --force when piped), so you cannot clobber your state of record by accident:

nivis state push --in backup.json --force

If another nivis is running, a state command reports state appears locked by another nivis process (...) and times out, instead of hanging.

6. Shell completion

Install tab-completion for your shell (it completes commands, flags, and resource ids in state for state show / --target):

source <(nivis completion bash)        # bash, current shell
# zsh:  nivis completion zsh  > "${fpath[1]}/_nivis"
# fish: nivis completion fish > ~/.config/fish/completions/nivis.fish

7. Generate a provider reference

nivis gen turns a provider's schema into typed Nix constructors. The generated file lists every argument (with type and required/optional), every nested block (with the correct list-vs-single shape), and the computed outputs, so it doubles as the per-provider argument reference:

nivis gen --provider provider-alpha --out ./generated   # provider-alpha is on $PATH
cat ./generated/alpha/alpha_token.nix

8. Tear down

nivis destroy --attr nivis.tutorial --var env=prod

Destroyed 2 resource(s):
  - beta.beta_record.app
  - alpha.alpha_token.app

Resources are destroyed in reverse dependency order (the record before the token). The datasource is not destroyed, because it was only ever read.

What you just exercised

In one config: typed variables with CLI overrides, a datasource read, the round trip across three phases, stack outputs (text and JSON), phase-grouped colored plan/apply, state pull/push with locking, completion, and codegen. That is the whole "daily-driver" surface, with no cloud account in sight. To do the same against real infrastructure, see the AWS S3 tutorial and the EC2 + NixOS tutorial.

Release notes: v0.4.5 (remote state + locking)

The first slice of M2 (team-ready): keep your state in a shared S3 object instead of a local file, and a lock so two applies cannot corrupt it. This is a patch release on the 0.4 line; the M2 milestone continues (drift detection and multiple environments are still to come).

Hands-on, runnable walkthrough: Tutorial: remote state on S3, with locking. It uses a real S3 bucket for state while keeping the resources offline (the fake providers), so you exercise the real remote-state path without creating billable infrastructure.

Highlights

Configure remote state in the flake

State lives where your config says, not where a flag says. Declare a backend on toIR and nivis reads and writes state there:

toIR {
  backend = { type = "s3"; bucket = "your-state-bucket"; key = "prod/app.json"; region = "eu-west-1"; };
  providers = { /* ... */ };
  resources = [ /* ... */ ];
  inherit ledger;
}

Credentials come from the AWS chain (AWS_PROFILE/keys); only the location is in config. Absent backend keeps the local file store (unchanged). State is Nivis's own format (no tfstate compatibility), and every write is server-side encrypted.

State locking keeps concurrent applies safe

apply and destroy take an advisory lock (a small <key>.lock object created atomically in S3) so two people or CI jobs cannot mutate the same state at once. If a run is already holding it, the next one stops before doing anything:

error: state is locked by alice@ci-runner since 2026-06-22T10:31:04Z for "apply"; run `nivis force-unlock` to override

Read-only commands (plan, refresh, output, state pull) do not lock.

force-unlock clears a stuck lock

If a run crashes while holding the lock, clear it with:

nivis force-unlock --attr nivis.remoteState

It confirms first (pass --force in CI). Only do this when you are sure no other run is active.

What shipped

• B1 Remote state backend (S3 first) — the IR backend block, an S3-backed Store (one object per state, server-side encrypted, AWS credential chain), and backend selection from the config. A hermetic in-repo fake S3 makes it testable.
• B2 State locking — an advisory lock on the S3 backend via an atomic conditional-put lock object, held across apply/destroy, with a nivis force-unlock escape hatch and "who holds it / since when" errors.

Changelog

See the [0.4.5] section of CHANGELOG.md.

Try it

nix shell github:nivis-project/nivis#nivis github:nivis-project/nivis#fake-providers
export AWS_PROFILE=your-profile
# edit nix/example/remote-state.nix: set your bucket + region
nivis apply --attr nivis.remoteState     # state -> s3://<bucket>/nivis-tutorial/remote-state/app.json
nivis plan  --attr nivis.remoteState     # reads state back from S3 -> no changes