Skip to content

07 · Binary parsing: dr-wav RIFF/PCM

Objectives: Understand how a WAV file is laid out as a sequence of binary chunks, learn how encoding/binary reads little-endian integers directly into Go structs, and see why every size field from an untrusted file is a potential OOM vector that must be capped before allocation. Estimated time: 20 minutes.

What this actually means (plain English)

No jargon — here's what the ideas in this lesson actually mean, and why they matter.

  • RIFF/WAV chunk layout = "a set of shipping boxes stacked inside a bigger shipping box — each box has a label tag on the outside and a size stamp, so you know exactly how far to reach before the next box starts." The outermost RIFF box holds a WAVE stamp, a fmt metadata box, and a data box; the parser walks them left to right by reading the 4-byte tag and the 4-byte size before touching the payload.
  • Little-endian byte order = "reading a price tag that lists cents before dollars — the least-significant digit comes first." WAV files store every multi-byte integer with the least-significant byte at the lowest address; encoding/binary.LittleEndian reverses the bytes into the normal Go value so you never do that arithmetic by hand.
  • binary.Read = "a tape measure that advances itself — each time you call it, it reads exactly as many bytes as the target type needs and moves the cursor forward by that amount." Pass it a bytes.Reader, binary.LittleEndian, and a pointer to a Go integer or struct field; it fills the value and leaves the reader positioned right after those bytes, with no manual offset tracking.
  • Untrusted size field as OOM vector = "a forged hotel booking that claims 4 000 rooms are reserved — blindly honouring it collapses the building." A crafted WAV can set the data chunk size to 4 GB in four bytes; calling make([]byte, subchunkSize) on that number crashes the process, so readDataChunk caps allocSize to r.Len(), the bytes actually present.
  • Seek to skip = "fast-forwarding a tape instead of listening to the part you don't need." r.Seek(int64(subchunk1Size-16), io.SeekCurrent) jumps over extra fmt bytes or unknown subchunks without allocating a buffer, so an oversized size field wastes no memory even before the cap check fires.
  • Separate parse from validate (ValidateWAV) = "a customs officer who first checks whether a package is sealed correctly, then hands it to an inspector who checks whether the contents are legal." Parse accepts any byte layout that is structurally sound; ValidateWAV enforces semantic rules like NumChannels != 0, AudioFormat == 1, and a supported BitsPerSample, keeping each concern in one place.

Why it matters: audio processing pipelines ingest user-supplied WAV files. A single malformed header field can turn "decode this file" into "allocate all available RAM and crash."

See it — the byte layout. A WAV file is a flat strip of labelled chunks. Each chunk is a 4-byte ASCII tag, a 4-byte little-endian size, then that many bytes. The parser walks it left to right with binary.Read. The one field you can never trust is the highlighted data size — cap it before any make([]byte, n).

RIFF/WAV byte layout A WAV file is a strip of chunks: RIFF tag, size, WAVE, fmt chunk, then a data chunk whose size field is the untrusted OOM vector. "RIFF" filesize "WAVE" "fmt " 16 fmt: channels,rate, bits… "data" datasize samples… 048121620364044 each chunk = 4-byte tag · 4-byte little-endian size · payload — read left to right untrusted — a crafted file can claim 4 GB here cap it, then make([]byte, n) — never trust the number

The RIFF/WAV byte layout

A canonical 44-byte WAV header looks like this. (A byte is the basic unit computers store data in — think of it as one small slot that holds one number from 0 to 255; a text character, or a small piece of a bigger number, typically fits in one byte.)

Offset  Size  Field
------  ----  -----
0       4     "RIFF"            (ASCII magic)
4       4     fileSize - 8      (uint32 LE)
8       4     "WAVE"            (format tag)
12      4     "fmt "            (subchunk ID)
16      4     16                (subchunk1 size, uint32 LE)
20      2     audioFormat       (1 = PCM, uint16 LE)
22      2     numChannels       (uint16 LE)
24      4     sampleRate        (uint32 LE)
28      4     byteRate          (uint32 LE)
32      2     blockAlign        (uint16 LE)
34      2     bitsPerSample     (uint16 LE)
36      4     "data"            (subchunk ID)
40      4     dataSize          (uint32 LE)
44      …     PCM samples

After the mandatory header, any number of extra subchunks may appear before data. The parser must scan forward until it finds the "data" tag. ("The parser" is the piece of code whose job is to read the raw bytes and make sense of their layout — the thing that walks through this table field by field.)

Reading fixed fields with encoding/binary

encoding/binary is a package — a bundle of ready-made code that someone else wrote, which you can pull into your own program instead of writing that logic yourself. (You bring a package in with an import line at the top of a file; more on that below.)

From dr-wav-go/dr_wav.go, the WAVHeader struct maps directly onto the fixed fields in the fmt subchunk. (A struct is a labeled bundle of values that belong together — like a form with named blanks. Each named blank inside it, such as AudioFormat or NumChannels below, is called a field.)

// from dr-wav-go/dr_wav.go
type WAVHeader struct {
    AudioFormat   uint16 // 1 = PCM
    NumChannels   uint16
    SampleRate    uint32
    ByteRate      uint32
    BlockAlign    uint16
    BitsPerSample uint16
}

In plain terms: this defines a form named WAVHeader with six labeled blanks. uint16 and uint32 just say how big a number can fit in that blank (16 bits or 32 bits — a bit is a single yes/no, 0-or-1 switch, and 8 bits make one byte, so uint16 is 2 bytes and uint32 is 4 bytes). Nothing runs here; this only describes the shape of the form so the rest of the code can fill it in.

Parse wraps the input slice in a bytes.Reader and reads each field in order.

In plain terms: Parse is a function — a named, reusable piece of code that does one job when you run ("call" or "invoke") it. A slice is Go's flexible view onto a sequence of bytes: a window over part of a longer list that you can resize without copying the underlying data. bytes.Reader is a small helper that remembers where you've read up to, so each read continues from where the last one left off.

// from dr-wav-go/dr_wav.go
r := bytes.NewReader(data)

var riff [4]byte
binary.Read(r, binary.LittleEndian, &riff)   // advances 4 bytes
// … validate "RIFF" …

var chunkSize uint32
binary.Read(r, binary.LittleEndian, &chunkSize) // advances 4 bytes

// … read "WAVE", "fmt ", subchunk1Size …

var header WAVHeader
binary.Read(r, binary.LittleEndian, &header.AudioFormat)
binary.Read(r, binary.LittleEndian, &header.NumChannels)
binary.Read(r, binary.LittleEndian, &header.SampleRate)
// … and so on for ByteRate, BlockAlign, BitsPerSample

In plain terms: each line asks the reader for the next few bytes off the file, converts them from little-endian order into an ordinary number, and stores that number into one named blank of the header form (&header.AudioFormat means "put the result into this specific field" — the & says "here is where in memory that field lives, write directly there"). The reader automatically remembers how far it has gotten, so the next call picks up right after the last one stopped.

Each binary.Read call consumes exactly the number of bytes that the target type requires. There is no manual offset arithmetic — bytes.Reader tracks the position internally. (An offset is just a distance, in bytes, from the start of the data — "byte number 40" rather than "the 40th item in a list." "Manual offset arithmetic" would mean the programmer has to calculate and track that distance by hand; here the reader does it for you.)

Why not read the whole struct at once?

binary.Read can decode a whole struct in one call if every field is a fixed-size type. The code reads field-by-field to make each error message name the failing field explicitly, which is more useful when debugging corrupted files. Both styles are correct.

Skipping unknown fmt bytes safely

The fmt subchunk is 16 bytes for PCM but can be larger for compressed formats. The size is declared by subchunk1Size, a uint32 from the file.

// from dr-wav-go/dr_wav.go
// Skip any extra format bytes. Seek rather than allocate: subchunk1Size is an
// untrusted uint32, so make([]byte, subchunk1Size-16) is an OOM vector. If the
// declared size runs past EOF, the next chunk read fails cleanly.
if subchunk1Size > 16 {
    if _, err := r.Seek(int64(subchunk1Size-16), io.SeekCurrent); err != nil {
        return nil, fmt.Errorf("failed to skip extra format bytes: %w", err)
    }
}

In plain terms: if the file says its fmt section is bigger than the normal 16 bytes, jump the reading position forward past the extra bytes without ever copying them into memory. If that jump fails (for example, it would land past the end of the file), the function returns an error instead of the parsed data — meaning it immediately stops and hands that error back to whatever code called it, rather than continuing.

Seek with io.SeekCurrent moves the internal cursor by n bytes without reading or allocating. (Allocating means asking the computer to reserve a chunk of memory — its working space — for something; the risk in this lesson is reserving far more than the machine actually has.) If the seek lands past EOF, the next read operation returns an error — a clean, safe failure rather than an OOM crash. (EOF = "end of file," the point where there is no more data left to read. OOM = "out of memory" — the computer tried to reserve more working space than it has available, which crashes the program.)

Finding the data chunk

WAV files can have subchunks between fmt and data (e.g. LIST, cue). readDataChunk loops over them, skipping unknown ones and stopping when it sees "data":

// from dr-wav-go/dr_wav.go
func readDataChunk(r *bytes.Reader) ([]byte, error) {
    for {
        var subchunkID [4]byte
        binary.Read(r, binary.LittleEndian, &subchunkID)

        var subchunkSize uint32
        binary.Read(r, binary.LittleEndian, &subchunkSize)

        if string(subchunkID[:]) == "data" {
            allocSize := int(subchunkSize)
            if allocSize > r.Len() {
                allocSize = r.Len() // never trust the declared size past EOF
            }
            pcmData := make([]byte, allocSize)
            io.ReadFull(r, pcmData)
            return pcmData, nil
        }

        // Skip this subchunk.
        r.Seek(int64(subchunkSize), io.SeekCurrent)
    }
}

In plain terms: this function repeats ("loops") the same steps over and over: read a 4-byte tag, read a 4-byte size, and check whether the tag is "data". If it's not, jump past that subchunk and go around again. If it is "data", first shrink the claimed size down to what's actually left in the file (the cap discussed below), then reserve exactly that much memory (make([]byte, allocSize) — this is the allocation step), fill it by reading that many bytes, and hand the result back to whoever called this function.

The critical line is the cap:

if allocSize > r.Len() {
    allocSize = r.Len()
}

r.Len() returns the number of bytes actually remaining in the reader. A file claiming a 4 GB data chunk but containing only 100 bytes will allocate 100 bytes, not 4 GB. This is the fix for the fuzz-discovered OOM bug (see Lesson 17 for how go test -fuzz found it).

In plain terms: fuzzing is an automated testing technique that bombards code with huge numbers of randomly mutated, often malformed inputs to try to trigger crashes. A test here is a small piece of code, separate from the main program, whose only job is to check that other code behaves correctly.

Size fields are always untrusted

Any integer read from a file, network packet, or user input that controls an allocation is an OOM vector. The rule is: cap to bytes present, not bytes declared. The same principle applies in miniz-go for ZIP entry sizes and in tinyxml2-go for nesting depth.

Division-by-zero guards in derived calculations

After parse, GetSampleCount computes how many samples are in the data. (The (w *WAV) before the function name makes this a method — a function that is attached to a particular kind of struct, here WAV, and is called as someWav.GetSampleCount() rather than standing alone.)

// from dr-wav-go/dr_wav.go
func (w *WAV) GetSampleCount() int {
    bytesPerSample := int(w.Header.BitsPerSample) / 8
    if bytesPerSample == 0 || w.Header.NumChannels == 0 {
        return 0
    }
    return len(w.Data) / bytesPerSample / int(w.Header.NumChannels)
}

In plain terms: work out how many bytes make up one audio sample; if that comes out to zero, or if there are zero channels, just report a count of 0 instead of doing a division that would blow up the program. Otherwise, divide the total data length by the sample size and by the channel count to get the sample count.

Parse does not reject a zero-channel header — that is left to ValidateWAV. So GetSampleCount must guard the division independently. The same guard applies to bytesPerSample: if BitsPerSample is 0, dividing by 8 gives 0, and dividing by that would panic. (A panic is Go's term for the program hitting an error so severe it stops running immediately, right there — dividing any number by zero is one of the classic ways to trigger one.)

Serialization: the 4 GB guard

Serialize writes a WAV file back to bytes. RIFF size fields are uint32, so a PCM payload larger than 2^32 - 1 - 44 bytes cannot be represented:

// from dr-wav-go/dr_wav.go
const maxWAVDataSize = math.MaxUint32 - 44

func Serialize(wav *WAV) ([]byte, error) {
    if len(wav.Data) > maxWAVDataSize {
        return nil, fmt.Errorf("WAV data too large to serialize: %d bytes", len(wav.Data))
    }
    // … write fields …
}

In plain terms: before doing any real work, check whether the audio data is too big to fit in a WAV file's size fields; if it is, immediately hand back an error and nothing else (the nil means "no bytes to give back"), rather than pressing on and writing a broken file.

The check happens before any allocation. Failing fast with a clear error is preferable to writing a truncated file that silently corrupts audio data.

Validation as a second pass

ValidateWAV enforces semantic constraints that Parse intentionally skips:

// from dr-wav-go/dr_wav.go
func ValidateWAV(wav *WAV) error {
    if wav.Header.AudioFormat != 1 {
        return fmt.Errorf("unsupported audio format: %d (only PCM supported)", wav.Header.AudioFormat)
    }
    if wav.Header.NumChannels == 0 {
        return errors.New("invalid number of channels: 0")
    }
    if wav.Header.SampleRate == 0 {
        return errors.New("invalid sample rate: 0")
    }
    if wav.Header.BitsPerSample != 8 && wav.Header.BitsPerSample != 16 &&
        wav.Header.BitsPerSample != 24 && wav.Header.BitsPerSample != 32 {
        return fmt.Errorf("unsupported bits per sample: %d", wav.Header.BitsPerSample)
    }
    return nil
}

This separation keeps Parse focused on structure (does the byte layout make sense?) and ValidateWAV focused on semantics (do the values make sense?). A library that silently rejects odd-but-parseable headers surprises callers; one that parses everything and surfaces validation as a separate step is more composable.

In plain terms: a library (also called a package) is a bundle of code someone wrote and shared so other programs can reuse it. A caller is whatever other code runs ("calls") a given function — here, whoever uses this WAV library.

Try it

Run the dr-wav tests, including the fuzz regression corpus:

cd dr-wav-go
go test ./... -v -count=1

Expected outcome: all tests pass in a few milliseconds. You should see test names like TestParse, TestValidateWAV, TestWAV_GetSampleCount, and TestSerialize. None should report a FAIL line.

To replay just the fuzz regression seeds (no new mutation, fast):

go test ./... -run=FuzzParse -v

If fuzz seeds exist under testdata/fuzz/FuzzParse/, each one is run as a named sub-test and should pass cleanly — these are the exact byte sequences that previously caused OOM crashes.

Reading deeper

ParseBatch in dr-wav-go/dr_wav.go uses a worker pool (one goroutine per CPU — a goroutine is a lightweight, independently-running slice of a program that Go can run at the same time as others, which is how a program does several things concurrently, i.e. with multiple parts genuinely making progress simultaneously instead of one strictly after another) to parse multiple WAV files concurrently. The channel buffer sizes (a channel is a pipe that goroutines use to hand values to each other safely) match len(dataList) exactly so no goroutine blocks on a send (to "block" means the line of code simply waits right there, doing nothing else, until some condition is met — here, until there is room in the channel to accept the value) — compare this with the deadlock bug in jsmn-go described in Lesson 14, where a buffer that was one slot too small caused wg.Wait to hang forever. (A deadlock is when one or more of these waiting points can never be satisfied, so the program freezes permanently instead of just pausing briefly.)

Key takeaways

  • encoding/binary.LittleEndian + bytes.Reader gives you a streaming, stateful view of raw bytes; each binary.Read advances the cursor by exactly the size of its target type — no offset math needed.
  • Every size field from an untrusted source is an OOM vector. Cap allocations to r.Len() (bytes actually present), never to the declared size.
  • Seek to skip, never allocate-and-discard. r.Seek(n, io.SeekCurrent) is free; make([]byte, n) followed by a read is proportional to n.
  • Separate parse from validate. Parse checks byte-level structure; ValidateWAV checks semantic invariants. Callers that only need to transcode a file can skip validation; callers that need clean data run both.
  • Guard every division by a field that could be zero. BitsPerSample and NumChannels come from the file and can both be zero; GetSampleCount returns 0 rather than panicking when either is absent.