Memory model

Ownership in zpack is deliberate rather than hidden. Every allocation has a named owner and a stated lifetime.

Ownership at a glance

Data	Allocator	Freed by
Directory metadata during pack	An arena inside pack	Discarded when pack returns
File contents while packing	Reused 64 KiB buffers	Reused, then freed by pack
Deflate compressor (~230 KB)	Heap, one instance for the whole run	pack
Archive index	Owned by the Archive	Archive.deinit
Bytes from read	The caller’s buffer	Nothing is allocated
Bytes from entryReader	The caller’s buffer	Nothing is allocated
Bytes from readAlloc	The caller’s allocator	The caller
verify and unpack scratch	The Archive’s allocator	Freed before returning
Ignore rules and pattern text	The allocator passed to parse/load	Ignore.deinit

Nothing is ever resident whole

File contents stream through fixed buffers in both directions. Packing a 40 GB tree uses the same working set as packing a 40 MB one.

The peak scales with the number of paths, not their total size - the index has to be held in memory while it is built, and again while an archive is open. At 35 bytes plus the path per entry, an archive of a million files holds a few tens of megabytes of index.

Packing

arena           directory metadata, scanned paths, the entry array
64 KiB × 2      one read buffer, one write buffer
64 KiB          the deflate window (flate.max_window_len)
~230 KB         one flate.Compress, reused for every file

The compressor is heap-allocated because it is far too large for the stack, and created once rather than per file. The arena is discarded wholesale when pack returns, so scan metadata has no individual frees.

Reading

Archive.open allocates two things and keeps them for the archive’s lifetime:

• The entry array, and one heap copy of each entry’s path
• The asset-id hash map, sized up front from the entry count

The data region is never read by open. It stays on disk and is read on demand, which is why opening a 4 GB archive costs the same as opening a 4 KB one.

deinit frees every path, the entry array, and the map, then closes the file.

Entry borrows its path

An Entry returned by find or findId is a value copy, but its path field borrows from the Archive. It is invalidated by deinit:

const entry = archive.find("a.png").?;
archive.deinit();
// entry.path is now dangling. entry.size, .offset, .hash are still fine.

Copy the path if it needs to outlive the archive.

Reading contents

Three routes, in increasing order of who does the allocating:

read allocates nothing. You supply a buffer of at least entry.size bytes, which find told you before any I/O. This is the hot path - size one buffer to your largest asset at startup and reuse it for everything.

entryReader allocates nothing. You supply a buffer of at least stream_buffer_len for a deflated entry, any size for a stored one, and the entry streams through it however large it is.

readAlloc allocates the result with the allocator you pass, and hands you ownership.

Scratch during verify and unpack

verify allocates one stream_buffer_len buffer for the whole pass.

unpack allocates four: a stream buffer, a 64 KiB write buffer, and two max_path_len path buffers - one for converting to the native separator, one to remember the last parent directory created. All four are freed before returning, and none scale with the archive.

The parent-directory buffer is why sorting matters at pack time: entries sharing a directory are contiguous, so the same parent is not created twice.

Buffer size constants

/// Reused while streaming file contents into and out of archives.
pub const copy_buffer_len = 64 * 1024;

/// Smallest buffer `Archive.entryReader` accepts for a deflated entry:
/// a full history window plus room to read the compressed bytes.
pub const stream_buffer_len = flate.max_window_len + 4096;

stream_buffer_len is split internally: the first flate.max_window_len bytes become the decompressor’s history window, and the rest holds compressed bytes read from disk. A stored entry uses the whole buffer as one plain read buffer and accepts any size.

Allocation-free asset loading

The intended shape for a game:

// Startup: one archive, one buffer.
var archive = try zpack.Archive.open(gpa, io, .cwd(), "game.zpak");
defer archive.deinit();

const buffer = try gpa.alloc(u8, largest_asset_bytes);
defer gpa.free(buffer);

// Per frame, per level, per asset: no allocation at all.
const entry = archive.find(path) orelse return error.MissingAsset;
const bytes = try archive.read(entry, buffer);

Nothing past open touches the allocator. If an asset is too large for the budget, stream it with entryReader instead - see streaming.

To size the buffer without hardcoding, walk archive.entries at startup:

var largest: u64 = 0;
for (archive.entries) |e| largest = @max(largest, e.size);

Thread safety

An Archive is safe to read from multiple threads at once, provided nobody mutates it:

• find and findId read a hash map that is fixed after open
• read uses positional reads, so concurrent reads do not race on a shared file cursor
• Each entryReader owns its buffer and its position

Give each thread its own buffer. open, deinit, and unpack are not concurrent operations.