# Network

**Spinbox is experimental: APIs change and data may be lost.** What is built and what proves it is on [Status](https://spin-stack.github.io/website/reference/status.md); what Spinbox does not do is on [Limitations](https://spin-stack.github.io/website/overview/limitations.md).

> What the internet can reach in an installation, which machine talks to which on what port, and how to close the installation to your own networks.

## What the internet reaches

Only the **proxy**, on two ports:

| Port | From | For |
|---|---|---|
| 80 | Anywhere | Let's Encrypt's challenge, and a redirect to 443 |
| 443 | `proxy_allowed_cidrs` (anywhere by default) | The dashboard, the API, workspaces' ports, SSH and terminals |

Nothing else in the installation has an inbound rule for the internet. The control plane's
public address is only a way out. A runner has no inbound rule at all. No machine runs SSH.

## Inside the VPC

| From | To | Port | Who the other end is |
|---|---|---|---|
| Proxy | Control plane | 8080 | TLS under the installation's CA |
| Runners | Control plane | 8080 | TLS under the installation's CA, and the runner's token |
| Runners | Proxy (relay) | 443 | The relay's certificate, and a one-time token per session |
| Proxy, runners | Collector on the control plane | 4317 | Only the security group |
| Control plane, runners | S3 | 443 | Instance roles; the buckets refuse object requests from anywhere but the VPC's gateway endpoint |

Machines find each other by name in a private zone, `spin.internal`: `cp.spin.internal` and
`proxy.spin.internal`, with a ten-second TTL. Each machine writes its own record when it starts
serving, so a replaced machine is a record changed, not an address in every configuration.

Runners reach the relay inside the VPC, at `proxy.spin.internal`, so the proxy's port 443 does
not need to be open to wherever runners are.

*Diagram: Every connection in an installation, opened in the direction of its arrow; nothing else is let in.* Its labels: Traffic: every arrow is a connection opened in its direction, and nothing else is let in · the installation's VPC · proxy · sg proxy: in 443 (proxy_allowed_cidrs) and 80 from anywhere, 443 from the VPC · control plane · sg controlplane: in 8080 from proxy and runners, 4317 from proxy and runners · 127.0.0.1 only - each a systemd unit of a user of its own · runners · sg runner: nothing in, everything out · users and administrators · browser · CLI · SSH · Caddy · TLS for app.<domain>, *.ws.<domain> · spin_relay: pairs the two ends of a session · 443 · session cookie or API key · Let's Encrypt · (ACME challenge) · 80 · challenge, redirect · controlplane :8080 · session / API key / runner token · scopes, per-resource authz, admin gate · 8080 TLS · (installation CA) · Alloy :4317 (OTLP) · no credential: the security group is · what keeps it to the fleet · 4317 · Quickwit :7280/:7281 · logs and traces · search, retention · Grafana :3000 · believes a token the · control plane signed, only · /monitoring/grafana · admins only, signed token · PostgreSQL :5432 · scram as spin, password derived · from the installation's key · 5432 · S3 - volumes, logs and database buckets · through the VPC's gateway endpoint alone · (bucket policy: aws:SourceVpce) · 443 · instance role · AWS APIs, over 443 · SSM (shells, documents) · STS · Auto Scaling · GitHub releases (the signed release) · 443 · OTLP · VictoriaMetrics :8428 · administration takes its key · remote write · indexes · backups, metrics/ · its only data source · WAL and base backups, sealed · runner · installation CA + runner token · 443 from the VPC · dials out to the relay · 8080 TLS · runner token · 4317 · workspaces (one VM each) · no route in; egress default-deny · (nftables, docs/network) · vsock · (no network) · 443 · credentials minted for its volumes · 443 · vsock 1030 · supervisor, relayed · vsock 1031 · identity, signed by the host. ([source](https://github.com/spin-stack/spin/blob/8eee32b9ddc8d535b3300d0da3927a30b33f22c4/docs/security/traffic))

## Closing the installation to your networks

Set `proxy_allowed_cidrs` to the networks your users are on:

```hcl
proxy_allowed_cidrs = ["203.0.113.0/24", "198.51.100.10/32"]
```

Port 80 stays open to everyone, for Let's Encrypt.

## Why there is no NAT gateway

Every machine has a public address in a public subnet, and reaches S3 through the free gateway
endpoint. A NAT gateway costs more per month than the machines that run all the time, plus a
charge for every gigabyte, before any workspace runs. A public address is not an open door: the
security groups decide what reaches each machine, and for all but the proxy that is nothing from
outside the VPC.

## What crossed the network is kept

The VPC's flow log goes to CloudWatch for 30 days (`log_retention_days`); `flow_logs = false`
turns it off. `dns_query_logs = true` turns on Route 53 Resolver's query log.

A workspace's own traffic is controlled separately, by its network policies, on the runner. See
[Network and egress](https://spin-stack.github.io/website/concepts/network.md).