Lab A - Write a bounds-checked binary parser¶
This is a lab, not a lecture.
You will write code, not just read it. Follow the guided steps in order. Each step has an expected outcome — run the command and check it before moving on.
Objectives: Learn to read structured binary data with
encoding/binary, defend every length field with a bounds check before slicing, and understand how dr-wav-go applies exactly these techniques to a real-world RIFF/WAV file. Estimated time: 40 minutes.
What this actually means (plain English)¶
No jargon — here's what the ideas in this lesson actually mean, and why they matter.
- Binary format = "a book with no chapter titles — you must already know that page 4 is the index and page 6 is the preface." Every field sits at a fixed byte offset; there are no
{or"separators, so the parser must know the schema ahead of time and read each field at the exact right position. - Length field = "a stranger's claim about how heavy their suitcase is before you try to lift it." A file can declare its data chunk is 4 GB long while the actual file is 100 bytes; trust it blindly and
make([]byte, n)crashes the process with an out-of-memory panic. - Bounds check = "measuring the shelf before buying the bookcase." Cap any allocation to
reader.Len()(bytes actually remaining in the buffer) before callingmake, so a lying length field cannot exhaust RAM. encoding/binary.Read= "a calibrated tape measure instead of eyeballing it." It reads fixed-size values in the specified byte order and returns an error on short reads, rather than silently misaligning every subsequent field.- Chunk-based format = "a train where each car has a label and a length, so you can skip cargo cars you don't recognise." RIFF, IFF, PNG, and LP-Chunk all use a repeating
[4-byte tag][4-byte length][data…]pattern; unknown tags areSeek-ed past, not crashed on.
Why it matters: a single missing bounds check in a binary parser is a denial-of-service waiting to happen — pass in a crafted file, exhaust the server's memory, game over. The fix is three lines; skipping it costs you the production incident.
See it — how the bounds check intercepts an oversized chunk claim.
The format you will parse: LP-Chunk¶
You will build a parser for LP-Chunk, a tiny invented binary format (binary means the file is stored as raw numbers — a stream of bytes, where a byte is a chunk of 8 bits, each bit being a single 0-or-1 switch — rather than as readable text):
File layout
───────────────────────────────────────
Offset Size Field
0 4 Magic: "LPCK" (ASCII)
4 2 Version (uint16, little-endian)
6 2 ChunkCount (uint16, little-endian)
── repeated ChunkCount times ──────────
0 4 ChunkID (4 ASCII bytes, e.g. "NAME", "DATA", "CSUM")
4 4 ChunkLen (uint32, little-endian) — length of Payload
8 N Payload (ChunkLen bytes)
───────────────────────────────────────
Every number is little-endian (a convention for which order the bytes of a multi-byte number are stored in — you don't need to compute this by hand, encoding/binary handles it, but the file has to agree on one order or the number reads back wrong). This is the same layout RIFF/WAV uses: a file-level
header, a count of sub-chunks, then repeating [tag][length][data] records. dr-wav-go
(dr-wav-go/dr_wav.go) does exactly this for real WAV files.
Step 0 — create the module¶
A "module" here is just Go's name for a self-contained folder of code with its own name and dependency list — think of it as declaring "this folder is one buildable project called example.com/lpchunk."
In plain terms: make a new folder called lpchunk, move into it, tell Go this folder is a project named example.com/lpchunk, then create two empty files — one for the parser code, one for its tests (a test is a small piece of code that runs your other code and checks the result is what you expect, instead of you checking by hand every time).
Step 1 — define the types¶
Open parser.go and paste the skeleton:
// Package lpchunk parses the LP-Chunk binary format.
// This is a lab companion to the SafeHeaders-Go course.
package lpchunk
import (
"bytes"
"encoding/binary"
"errors"
"fmt"
"io"
)
// Header is the 6-byte file header.
type Header struct {
Version uint16
ChunkCount uint16
}
// Chunk is one [tag][len][payload] record.
type Chunk struct {
ID [4]byte
Payload []byte
}
// IDString returns the tag as a printable string.
func (c Chunk) IDString() string { return string(c.ID[:]) }
// File is the fully parsed result.
type File struct {
Header Header
Chunks []Chunk
}
In plain terms: a "package" is just a named folder of Go code that other code can borrow from — package lpchunk declares that this file belongs to the lpchunk package. import (...) pulls in four ready-made packages from Go's own toolbox (its standard library — code Go ships with so you don't have to write it yourself) so this file can use their pieces. type Header struct { ... } defines a "struct" — a labeled bundle of values glued together under one name, the way a form has named boxes for "Name" and "Date"; here Version and ChunkCount are the two boxes, each called a field. type Chunk struct { ... } defines another such bundle: ID is a fixed-size list of exactly 4 bytes (a list of same-typed values sitting one after another in memory like this is called an array when its length is fixed), and Payload []byte is a byte list whose length can grow or shrink — that flexible kind of list is called a slice. func (c Chunk) IDString() string { return string(c.ID[:]) } defines a method — a function that is attached to a particular type, here Chunk — named IDString; running it (a function starts running only when something "calls" it, i.e. explicitly asks it to run) converts the raw 4-byte tag into an ordinary readable string and returns it, meaning the method stops and hands that string back to whatever code asked for it.
Step 2 — read the magic bytes¶
Still in parser.go, add the Parse function:
// Parse reads and validates an LP-Chunk file from data.
func Parse(data []byte) (*File, error) {
if len(data) < 8 {
return nil, errors.New("lpchunk: data too short for header")
}
r := bytes.NewReader(data)
// ── Magic ──────────────────────────────────────────────────────────────
var magic [4]byte
if err := binary.Read(r, binary.LittleEndian, &magic); err != nil {
return nil, fmt.Errorf("lpchunk: read magic: %w", err)
}
if string(magic[:]) != "LPCK" {
return nil, fmt.Errorf("lpchunk: bad magic %q", magic)
}
// ── Header ─────────────────────────────────────────────────────────────
var hdr Header
if err := binary.Read(r, binary.LittleEndian, &hdr.Version); err != nil {
return nil, fmt.Errorf("lpchunk: read version: %w", err)
}
if err := binary.Read(r, binary.LittleEndian, &hdr.ChunkCount); err != nil {
return nil, fmt.Errorf("lpchunk: read chunk count: %w", err)
}
// TODO: read chunks (Step 3)
_ = hdr
return nil, errors.New("lpchunk: chunk reading not yet implemented")
}
In plain terms: func Parse(data []byte) (*File, error) { ... } defines the function that does the real work — you "call" (run) it later by writing Parse(someBytes), and it hands back either a filled-in File or an error explaining what went wrong. bytes.NewReader(data) wraps the raw byte slice in a small helper (r) that remembers "how far have we read so far," so each subsequent read continues from where the last one stopped instead of you tracking that position (an offset) by hand. binary.Read(r, binary.LittleEndian, &magic) asks that helper for the next few bytes, interprets them using the little-endian byte order, and writes the result into magic; the & means "give me a pointer to this variable" — a pointer is just the location in the computer's memory where a value lives, handed over so the function can fill in that exact spot rather than a copy. Each if err := ...; err != nil { return nil, ... } line means: try the step, and if it produced an error, stop immediately and hand that error back to whoever called Parse (the caller) instead of continuing with bad data.
Try it
Expected outcome: no compiler errors. The_ = hdr line silences the "declared
and not used" error while you stub the next step. You should see nothing printed —
silence is success. (Building here means Go compiles the code: it translates the human-readable source text you just wrote into a program the machine can actually execute, and checks along the way that everything is well-formed.)
Step 3 — read the chunks (with bounds checking)¶
Replace the // TODO block and the stub return with the real loop.
This is where bounds-checking matters most.
// ── Chunks ─────────────────────────────────────────────────────────────
chunks := make([]Chunk, 0, hdr.ChunkCount)
for i := 0; i < int(hdr.ChunkCount); i++ {
var ch Chunk
// 4-byte tag
if err := binary.Read(r, binary.LittleEndian, &ch.ID); err != nil {
return nil, fmt.Errorf("lpchunk: chunk %d: read id: %w", i, err)
}
// declared payload length
var declaredLen uint32
if err := binary.Read(r, binary.LittleEndian, &declaredLen); err != nil {
return nil, fmt.Errorf("lpchunk: chunk %d: read length: %w", i, err)
}
// ── THE CRITICAL CHECK ──────────────────────────────────────────────
// Trust the bytes that are actually present, not the number in the file.
// A crafted file can claim declaredLen = 4 GB; r.Len() tells us the truth.
allocLen := int(declaredLen)
if allocLen > r.Len() {
allocLen = r.Len() // cap to bytes present — never OOM on a lie
}
// ───────────────────────────────────────────────────────────────────
ch.Payload = make([]byte, allocLen)
if _, err := io.ReadFull(r, ch.Payload); err != nil && !errors.Is(err, io.EOF) {
return nil, fmt.Errorf("lpchunk: chunk %d: read payload: %w", i, err)
}
chunks = append(chunks, ch)
}
return &File{Header: hdr, Chunks: chunks}, nil
In plain terms: make([]Chunk, 0, hdr.ChunkCount) reserves a chunk of memory (this reserving step is called allocating) big enough to hold hdr.ChunkCount items, starting empty — a starting size hint so the computer doesn't have to keep re-growing the list as you add to it. The for i := 0; ...; i++ { ... } line is a loop: it repeats everything inside the { } once per chunk the header says to expect, with i counting 0, 1, 2, … each time round. Inside, it reads the 4-byte tag, then reads the 4-byte declaredLen — the file's own claim about how long the payload is. "THE CRITICAL CHECK" is the bounds check the earlier analogy described: allocLen := int(declaredLen) starts by believing the file's claim, then if allocLen > r.Len() { allocLen = r.Len() } asks the reader helper how many bytes are actually still left (r.Len()) and, if the claim is bigger than that, shrinks allocLen down to the truth — so the next line, make([]byte, allocLen), never tries to reserve more memory than could possibly exist in the file. io.ReadFull(r, ch.Payload) then reads exactly that many bytes into the payload, and chunks = append(chunks, ch) adds the finished chunk onto the end of the growing list. When the loop finishes, return &File{...}, nil hands back the fully assembled result and a nil (meaning "no error").
The three lines around "THE CRITICAL CHECK" are directly modeled on readDataChunk in
dr-wav-go/dr_wav.go:
// from dr-wav-go/dr_wav.go — readDataChunk
allocSize := int(subchunkSize)
if allocSize > r.Len() {
allocSize = r.Len() // never trust the declared size past EOF
}
pcmData := make([]byte, allocSize)
Same shape, same reason: subchunkSize is a uint32 from an untrusted file; r.Len()
is the ground truth.
What happens without the check
If you delete those three lines and a malicious file claims declaredLen = 0xFFFFFFFF
(about 4 GB), make([]byte, 4294967295) panics with runtime: out of memory — a
"panic" is Go's term for the program hitting a fatal error it cannot recover from and
crashing right there, in this case because it tried to reserve more memory than the
computer has available (an "out of memory," or OOM, condition) — or,
on a machine with enough swap, it actually allocates 4 GB and stalls the process.
This is a real denial-of-service class (an attack or bug that doesn't steal data but
knocks a service offline, here by exhausting its memory): the fuzzer found exactly
this shape in dr-wav-go before the fix landed.
Step 4 — write a helper to build test files¶
Add builder.go in the same package so your tests can construct valid (and malicious)
LP-Chunk files without hand-encoding bytes:
package lpchunk
import (
"bytes"
"encoding/binary"
)
// BuildFile serializes an LP-Chunk file. Use in tests only.
func BuildFile(version uint16, chunks []Chunk) []byte {
var buf bytes.Buffer
buf.WriteString("LPCK")
binary.Write(&buf, binary.LittleEndian, version)
binary.Write(&buf, binary.LittleEndian, uint16(len(chunks)))
for _, ch := range chunks {
buf.Write(ch.ID[:])
binary.Write(&buf, binary.LittleEndian, uint32(len(ch.Payload)))
buf.Write(ch.Payload)
}
return buf.Bytes()
}
In plain terms: this is the reverse of Parse — instead of reading bytes into typed values, BuildFile takes typed values and writes them out as bytes into an in-memory buffer (bytes.Buffer, a growable holding area for bytes you're accumulating before using them all at once), in the same magic-then-header-then-chunks order the parser expects. It exists purely so the tests below can manufacture both well-formed and deliberately broken LP-Chunk files without typing out raw byte values by hand.
binary.Write to a bytes.Buffer cannot fail (fixed-size values, in-memory buffer), so
ignoring the error here is safe — but only in a test helper. In production parsers,
always check it. dr-wav-go/dr_wav.go's Serialize function uses a tiny put closure
to capture and propagate write errors:
// from dr-wav-go/dr_wav.go — Serialize
var werr error
put := func(v any) {
if werr == nil {
werr = binary.Write(&buf, binary.LittleEndian, v)
}
}
In plain terms: a "closure" is a small, unnamed function created on the spot and stored in a variable (here put) — it "closes over," i.e. remembers and can modify, the werr variable from the surrounding code even after that surrounding code has moved on. Each time put runs, it only attempts the write if no earlier write already failed (if werr == nil), so the very first error to occur is captured in werr and every later call becomes a harmless no-op — a compact way to check one error at the end instead of after every single write.
Step 5 — write the tests¶
Paste this into parser_test.go (a Go test is an ordinary function, named starting
with Test, that Go's test tool runs automatically; it calls your real code with known
inputs and uses t.Errorf/t.Fatalf to report when the output doesn't match what's
expected — replacing the tedium of manually re-checking behavior after every change):
package lpchunk_test
import (
"testing"
"example.com/lpchunk"
)
func TestRoundTrip(t *testing.T) {
original := []lpchunk.Chunk{
{ID: [4]byte{'N', 'A', 'M', 'E'}, Payload: []byte("hello")},
{ID: [4]byte{'D', 'A', 'T', 'A'}, Payload: []byte{0x01, 0x02, 0x03}},
}
data := lpchunk.BuildFile(1, original)
f, err := lpchunk.Parse(data)
if err != nil {
t.Fatalf("Parse: %v", err)
}
if len(f.Chunks) != len(original) {
t.Fatalf("got %d chunks, want %d", len(f.Chunks), len(original))
}
for i, ch := range f.Chunks {
if ch.IDString() != original[i].IDString() {
t.Errorf("chunk %d: id %q, want %q", i, ch.IDString(), original[i].IDString())
}
if string(ch.Payload) != string(original[i].Payload) {
t.Errorf("chunk %d: payload mismatch", i)
}
}
}
func TestBadMagic(t *testing.T) {
data := lpchunk.BuildFile(1, nil)
data[0] = 'X' // corrupt the magic
if _, err := lpchunk.Parse(data); err == nil {
t.Fatal("expected error for bad magic, got nil")
}
}
func TestTruncatedData(t *testing.T) {
// Data too short for the header at all
if _, err := lpchunk.Parse([]byte{1, 2, 3}); err == nil {
t.Fatal("expected error for truncated input")
}
}
func TestBoundsCheck_OversizedChunk(t *testing.T) {
// Build a valid file with one chunk, then overwrite its declared length
// with a giant number. The parser must NOT panic or OOM.
chunk := lpchunk.Chunk{
ID: [4]byte{'D', 'A', 'T', 'A'},
Payload: []byte{0xAA, 0xBB},
}
data := lpchunk.BuildFile(1, []lpchunk.Chunk{chunk})
// Offset of the chunk length field:
// 4 (magic) + 2 (version) + 2 (count) + 4 (chunk id) = 12
const lenOffset = 12
// Write 0xFFFFFFFF (4 GB) as little-endian uint32
data[lenOffset+0] = 0xFF
data[lenOffset+1] = 0xFF
data[lenOffset+2] = 0xFF
data[lenOffset+3] = 0xFF
// Must return without panicking. The payload will be truncated to
// whatever bytes are actually remaining — that's the correct behavior.
f, err := lpchunk.Parse(data)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(f.Chunks[0].Payload) > 2 {
t.Errorf("payload longer than bytes present: %d bytes", len(f.Chunks[0].Payload))
}
t.Logf("claimed 4 GB, got %d bytes (correct)", len(f.Chunks[0].Payload))
}
Try it
Expected outcome:--- PASS: TestRoundTrip (0.00s)
--- PASS: TestBadMagic (0.00s)
--- PASS: TestTruncatedData (0.00s)
--- PASS: TestBoundsCheck_OversizedChunk (0.00s)
parser_test.go:XX: claimed 4 GB, got 2 bytes (correct)
PASS
ok example.com/lpchunk
TestBoundsCheck_OversizedChunk panics instead, your bounds check is missing
from Step 3.
Step 6 — prove there is no data race¶
Try it
Expected outcome: all four tests pass, no DATA RACE output.
If you see a race warning, look for any shared variable written from two goroutines
without a mutex. (This lab is single-threaded, so you should be clean.)
Step 7 — add a fuzz target¶
The dr-wav OOM bug was found by go test -fuzz. Add the same tool to your parser:
// In parser_test.go, add:
func FuzzParse(f *testing.F) {
// Seed corpus: a valid file and an empty input
f.Add(lpchunk.BuildFile(1, []lpchunk.Chunk{
{ID: [4]byte{'D', 'A', 'T', 'A'}, Payload: []byte("seed")},
}))
f.Add([]byte{})
f.Fuzz(func(t *testing.T, data []byte) {
// Must not panic, must not OOM, must not hang.
lpchunk.Parse(data) //nolint:errcheck
})
}
Try it
Expected outcome: the fuzzer runs for 30 seconds and reports no crashes. Any failure written totestdata/fuzz/FuzzParse/ is a reproducible regression seed — commit it
alongside a fix, exactly as dr-wav-go does under testdata/fuzz/.
How dr-wav-go applies these same patterns¶
| LP-Chunk lab pattern | dr-wav-go equivalent (dr-wav-go/dr_wav.go) |
|---|---|
Check len(data) < 8 before reading the header |
if len(data) < 44 { return nil, errors.New("data too short…") } |
binary.Read(r, binary.LittleEndian, &magic) |
Same for RIFF/WAVE/fmt tags |
Cap allocLen to r.Len() |
if allocSize > r.Len() { allocSize = r.Len() } in readDataChunk |
io.ReadFull(r, ch.Payload) |
io.ReadFull(r, pcmData) |
Skip unknown chunks with r.Seek |
r.Seek(int64(subchunkSize), io.SeekCurrent) for non-"data" subchunks |
Serialize with a binary.Write error wrapper |
put closure capturing werr |
Fuzz with go test -fuzz |
Fuzz regression seeds in dr-wav-go/testdata/fuzz/ |
The guard in readDataChunk was added after the OOM bug was found by fuzzing — a crafted
WAV declared a 4 GB data chunk in a 50-byte file. The same three-line pattern now lives in
your parser too.
Read the real source
Open dr-wav-go/dr_wav.go and read readDataChunk (around line 121) and Serialize
(around line 329). Every technique in this lab appears there verbatim. Seeing a pattern
in a toy parser and then recognising it in a production file is how you build the instinct.
Key takeaways¶
- Every length field from a file is a claim. Validate it against the bytes actually
present (
reader.Len()) before allocating. Three lines prevent a class of OOM panics. encoding/binary.Readgives you typed, endian-aware reads with proper error propagation. Prefer it over manual slice indexing for fixed-size fields.- Chunk-based formats let you skip unknown sections safely. Use
Seekrather than allocating a throw-away buffer — especially for untrusted size fields. go test -fuzzfinds the bugs you didn't think to write tests for. The OOM in dr-wav-go was found this way. A 30-second fuzz run is cheap insurance.- Wrap every
binary.Writeerror in production serializers. Writing to abytes.Bufferlooks infallible but isn't; dr-wav-go'sputclosure is the right pattern.