Runtimes¶

The runtime is what runs the environment's image. It's either a VM with its own kernel, or a container that shares the host's kernel. Pick one with --runtime on chalk run and chalk build:

The default¶

If you don't pass --runtime, Chalk uses your platform's VM, so the student gets its own kernel:

chalk run logs which runtime it picked. If your machine can run the VM but it isn't set up yet, Chalk stops before the run and tells you what's missing and how to fix it. Pass --runtime modal to run in a VM on Modal (see its setup). Pass --runtime docker (or podman) if you want to run in a container instead of a VM.

Tasks that need devices passed through, such as a GPU, run under docker. A plugin marks that hardware passthrough in chalk.hardware_limits and adds the device flags through chalk.container_run_args.

Setup¶

Apple Container¶

Install Apple's container (1.4.1 or later) and start its service:

container system start

Don't put the environment under /tmp, /private/tmp or any other path that goes through a symlink, such as /var. Apple's builder copies the directories in it as empty directories, so Chalk refuses to build from there.

Images that only have an amd64 variant run under Rosetta.

--mount paths aren't shared live. They're copied into the VM when the run starts, and read-write ones are copied back when it ends.

Firecracker¶

You need:

• read and write access to /dev/kvm
• docker with buildx to build the image
• pasta (package passt), ip and iptables for the guest's network
• mkfs.ext4 and debugfs (package e2fsprogs) to build the guest's drives

To get access to /dev/kvm, add yourself to the kvm group and log in again:

sudo usermod -aG kvm $USER

On the first run, Chalk downloads Firecracker and a guest kernel from a Kata Containers release into ~/.cache/chalk/firecracker (under $XDG_CACHE_HOME if it's set). Each Chalk release fixes which versions it downloads and their SHA-256 checksums, and stops with an error if a download doesn't match.

pasta gives the guest a network without needing root. On Ubuntu 24.04, AppArmor stops pasta from creating the user namespace it needs. To lift that restriction until the next reboot, run:

sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0

Or set CHALK_FIRECRACKER_JAILER=1 and run chalk run with sudo. Chalk then starts the VM through Firecracker's jailer, which doesn't need that change.

The guest can't reach a model proxy running on the same machine, only one running somewhere else. Use --runtime docker if your proxy runs locally.

The VM can't share a writable directory with the host, so firecracker only accepts read-only mounts (--mount source:target:ro).

Modal¶

Add the Modal client to the environment and log in:

uv add 'chalk-rl-env[modal]'
modal token new

Chalk uses your Modal profile's environment, or the one MODAL_ENVIRONMENT names.

chalk build --runtime modal builds the image on Modal from the Containerfile, and Modal caches its layers. Modal's builder requires a few changes to the Containerfile before the build:

• dnf installs packages without file capabilities, which Modal's builder can't set.
• A COPY of a folder that holds only files the .dockerignore leaves out (such as a .gitkeep) still creates the folder, empty.

Modal ignores USER instructions and RUN --mount=type=secret, and Chalk doesn't take --build-secret for Modal yet.

Each run gets its own sandbox under Modal's VM runtime, in a Modal app named chalk. Chalk terminates the sandbox when the run ends, including on Ctrl-C. If Chalk itself dies, the sandbox shuts down 5 minutes after the run inside it ends, and after 24 hours at most.

The run reads its config from a file in the sandbox, so the API key isn't on any command line. The terminal UI reaches the run's websocket through a Modal connect token, so nobody without the token can read the run's events. When the run ends, Chalk copies /out back next to the transcript, keeping only regular files and directories.

Model calls go out from the sandbox, so a model proxy has to be on a public address. Chalk refuses a proxy on localhost or a private address.

Modal doesn't filter the network around the sandbox the way firecracker does on the host, so only the student firewall inside the sandbox applies.

The modal runtime doesn't support --mount, --dev, --keep-containers or --prepare-only yet. It can't run hardware a plugin marks passthrough, such as a GPU, because Modal's VM runtime has no GPUs.

docker, podman, nerdctl¶

docker needs buildx to build the image. On Ubuntu, install docker.io and docker-buildx; Docker's own packages from its apt repository include buildx already. Then add yourself to the docker group (sudo usermod -aG docker $USER) and log in again.

podman builds on its own. On macOS, start its VM first with podman machine init and podman machine start.

nerdctl needs BuildKit: buildctl on your PATH and buildkitd running. nerdctl's "full" release bundles both. Chalk checks for buildctl but not for buildkitd.

On a machine with /dev/kvm, the default runtime is firecracker, so pass --runtime to use one of these.

docker:gvisor¶

Install gVisor and register runsc as a docker runtime in /etc/docker/daemon.json with these runtimeArgs:

-net-raw
--systrap-disable-syscall-patching
-overlay2=none
-file-access=shared
-network=sandbox
--net-disconnect-ok

Chalk checks this before each run. If something is missing, it prints the exact daemon.json to write. Then reload docker with sudo systemctl reload docker.

VM size¶

A VM gets the CPUs, memory and disk that the chalk.hardware_limits plugin assigns to the task's hardware. Without such a plugin, the sandbox gets 2 CPUs and 4 GiB of memory.

The student's memory limit is the sandbox's memory minus 1 GiB, which is kept for the harness. The VM itself gets 1 GiB more than the sandbox's memory, for the guest kernel and page cache. So without a plugin, the student has a 3 GiB limit inside a 5 GiB VM. apple-container won't start a VM that's larger than your machine's RAM. On modal, the sandbox's root filesystem is 512 GiB.

The student's disk budget is 80% of the host's free disk space, capped at the plugin's disk budget if it sets one. With chalk run -n N, that budget is split evenly between the N runs. Separate chalk run commands started at the same time don't know about each other, so each one counts all of the free space. See Student resources for how these limits are enforced.

What each runtime gives the student¶

"Watchdog" means nothing stops the student from going over the limit, but Chalk notices and kills all of the student's processes.

The docker/podman/nerdctl column is what you get in the usual case. Chalk tries a writable cgroup first, so if the container has one, the kernel enforces the memory and process limits as it does in a VM.

On firecracker, the host-side filter blocks the whole guest, root included, from reaching link-local addresses, private address ranges and the host itself. The model proxy is the only exception. This holds even with CHALK_STUDENT_NETWORK=internal.

To see what your sandbox actually enforces, run chalk check confinement as root inside it. It exits with 1 if the student gets less protection than it should:

chalk check confinement
chalk check confinement --json

It tests limits and firewall rules on an unused uid, mounts and unmounts a small file quota, and starts one student session under the default limits. It undoes each of these afterwards, with one exception: it can leave the harness in the chalk_harness cgroup, which every run does at start anyway. The report lists anything left behind. --hardware picks which hardware's default memory limit to apply.

Environment variables¶

Variables a launcher or task can set inside the sandbox:

Variables for the host:

CHALK_STUDENT_NETWORK, CHALK_INFERENCE_SERVICE_TIER and LOGURU_LEVEL are forwarded from your shell into the sandbox.