Penelope

The promise made in Phase 5

The Design choices section ended with: "the natural successor to the TS implementation is a Penelope-implemented runtime — not a Rust rewrite that would be obsolete the moment the self-hosted compiler can target WASM or native."

std/wasm.pen is a WASM backend written in Penelope. It takes an AST bundle (the kind std/parser.pen produces) and emits a complete WebAssembly module as a list of bytes — valid binary, loadable by any WASM runtime, no external assembler required.

$ pen wasm examples/09-fib.pen --out fib.wasm
wrote fib.wasm

$ node -e "
  const m = await WebAssembly.compile(require('fs').readFileSync('fib.wasm'));
  const i = await WebAssembly.instantiate(m, { js: {
    print: (p, l) => { /* ... */ }, now: () => Date.now(),
    random_int: (lo, hi) => lo + Math.floor(Math.random() * (hi - lo)),
    net_fetch: () => 0, read_file: () => 0, write_file: () => {}
  }});
  console.log('fib(20) =', i.exports.main());
"
fib(20) = 6765

What works

96 end-to-end tests in test/wasm-backend.test.ts cover each piece. Every test runs the same source through both the pen-built WASM backend and the TS interpreter and asserts the results match.

How closures work (Phase 6.E)

An anonymous fn(x) { x + d } appearing inside fn(d) { fn(x) { x + d } } is the canonical closure: the inner fn captures d from the outer's scope. WASM has no native closures — every fn lives in a flat fn table with a fixed signature. So the backend does closure conversion:

1. Free-variable analysis: find_free_vars walks each anonymous fn's body, computing which names it references that aren't parameters, locals, top-level fns, or builtins.
2. Lifting: collect_lifted_fns assigns each anonymous Fn a top-level WASM function index plus a table index. Its signature becomes (env_ptr, ...original_params) -> i32.
3. Env block layout: when a Fn-literal expression is evaluated, the backend allocates 4 + 4*N bytes where N is the number of captures. The first 4 bytes hold the table index; the next slots hold each captured Value pointer.
4. Closure Value: the env block pointer is wrapped in a CLOSURE-tagged 8-byte cell (tag=7, bytes 4–7 = env_ptr). That's a first-class value: storeable in locals, passable as arguments, returnable.
5. Indirect call: calling f(x) where f is a local holding a closure Value: load env_ptr from f+4, push it as the first arg, push user args, load table_idx from env_ptr+0, call_indirect.
6. Capture prelude: the lifted fn's body starts with a generated prelude that loads each captured value from env_ptr+(4+i*4) into a local slot, so the body's x + d compiles to a plain local.get for d.

let make_adder = fn(d) {
  fn(x) { x + d }
};
let add5 = make_adder(5);
add5(37);  // 42

Two closures from the same make_adder capture distinct values:

let add1 = make_adder(1);
let add10 = make_adder(10);
add1(5) + add10(5);  // 6 + 15 = 21

What's out of scope (documented honestly)

For pause/resume today, run programs through the regular interpreter or the JIT. The WASM backend is for the hot-path subset where you actually benefit from native execution.

How it's built

One file: std/wasm.pen (~5400 lines of Penelope). Depends only on std/iter.pen. Structure:

1. LEB128 encoders — leb128_u for unsigned (sizes, indexes), leb128_s_nonneg for signed-but-non-negative literals.
2. Byte-buffer helpers — buffers are plain Penelope lists of ints. vec_of_bytes, name_bytes, str_to_bytes.
3. Three-pass compilation — pass 1 collects top-level fn names and assigns indexes; pass 2 finds all anonymous Fn nodes and assigns lifted fn_idx + table_idx + captures; pass 3 emits each fn body's WASM instructions.
4. AST → instructions — compile_expr walks the AST and emits opcode bytes. Builtins (print, str_concat, list_get, ...) are intercepted and routed to compiled-in helpers.
5. 45 helper fns emitted into the code section: allocator, tag wrappers (make_int, make_bool, ...), tag unwrappers, string/list/dict helpers, char predicates, host-import shims, and make_closure.
6. Module assembler — type → import → function → table → memory → global → export → element → code. Wraps each section in (id, length, payload), prepends the \0asm\01\00\00\00 header.

Why this matters

A Penelope program, written by a user, parsed by a Penelope-implemented parser, compiled by a Penelope-implemented WASM backend, runs in any environment that supports WebAssembly — Node, browsers, Cloudflare Workers, Wasmtime, the WASM runtime baked into your phone. The TS implementation is the bootstrap stage; for the WASM-targetable subset of the language, the TS code is now optional, not load-bearing.

Each phase from 6.A to 6.G expanded that subset:

• 6.A: ints + control flow + top-level fns. "Recursive math runs natively."
• 6.B: tagged Values + heap. "Multiple types coexist."
• 6.C: strings + 13 builtins. "Text processing runs natively."
• 6.D: lists + dicts. "Data structures work."
• 6.E: closures + first-class fns. "Higher-order code works."
• 6.F: match. "Pattern matching works."
• 6.G: 6 host effects. "I/O works — print, time, randomness, fetch, file read/write."

What's left in the language vs the WASM target is essentially: floats, snapshot/resume, and a handful of edge cases. For everything else, the WASM backend is the path forward.