Play recorded notes
Recorded notes, played at any pitch. audiomixer.Mixer plays sources at
their own rate and synthio plays single-cycle waveforms; neither can take a
recorded note, play it a fourth up, loop a region of it while the key is held
and let the rest of it out when the key lifts. Sampler does that, which is
what a SoundFont instrument and every drum machine need.
sampler = audiosampler.Sampler(voice_count=8, sample_rate=48000)
audio_out.play(sampler)
sampler.voice[0].play(piano_c4, rate=44100, pitch=2 ** (5 / 12),
loop_start=11000, loop_end=19000,
loop_mode=audiosampler.LOOP_UNTIL_RELEASE,
attack=0.005, release=0.3)
...
sampler.voice[0].release()
A Sampler is an audio source like a Mixer: play it into an output, a
mixer voice or an effect. Notes start on its voices. A buffer is mono signed
16-bit — an array('h'), a memoryview, anything with a buffer — and rate
is the buffer’s own sample rate (the sampler’s when left out), so the step per
output frame is pitch * rate / sample_rate and a 44.1 kHz recording plays at
the right speed through a 48 kHz output without being told anything else.
The buffer is held for as long as the voice sounds and let go when it stops.
The loop modes are SoundFont’s sampleModes under their own numbers, so a
reader writes the file’s value straight in: LOOP_NONE (0) plays to the end
and stops, LOOP_FOREVER (1) loops [loop_start, loop_end) until
release() or stop(), and LOOP_UNTIL_RELEASE (3) loops while the key is
held, then plays on past the loop to the end of the buffer.
Each voice has an amplitude DAHDSR — delay, attack, hold, decay and
release in seconds, sustain a linear level — run per sample in C, so a
two-millisecond attack is a slope and not a step. delay postpones the start
of the sound itself, which is what a flam wants. level, pan and pitch
are live properties: a change lands at the next block, the first two as a
ramp across it so a fader move is not a click, and the third is how a pitch
bend is played. release() enters the release stage from wherever the
envelope is; stop() is silence from the next block; playing says whether
the voice still sounds.
Filter, modulation envelope and LFOs
A voice can also do what a SoundFont region asks for besides its amplitude envelope. Give it a resonant low-pass, a second envelope that sweeps the cutoff and the pitch, a vibrato, and a slower LFO on pitch, cutoff and volume:
sampler.voice[0].play(brass_c4, rate=44100, loop_start=2000, loop_end=6000,
loop_mode=audiosampler.LOOP_UNTIL_RELEASE,
attack=0.02, release=0.2,
cutoff=800, resonance=6, # Hz, dB
mod_attack=0.05, mod_decay=0.4, mod_sustain=0.3,
mod_to_cutoff=2400, # cents at full scale
vibrato_delay=0.3, vibrato_rate=5.5, vibrato_depth=15)
...
sampler.voice[0].vibrato_depth = 50 # a mod wheel
sampler.voice[0].cutoff = 1200
cutoff is in Hz and resonance in dB: the filter’s response at the cutoff
sits that far above its response at DC, and DC drops by half of it, as the
SoundFont specification puts it. Zero resonance is a Butterworth. A cutoff of
19.9 kHz or more, with no resonance and nothing moving it, is no filter at
all, and so is leaving it out.
The modulation envelope has the amplitude envelope’s six stages, prefixed
mod_, and moves pitch by mod_to_pitch and the cutoff by mod_to_cutoff
cents at full scale. The vibrato LFO (vibrato_delay, vibrato_rate in Hz,
vibrato_depth in cents) moves pitch. The modulation LFO (lfo_delay,
lfo_rate) moves pitch by lfo_to_pitch cents, the cutoff by
lfo_to_cutoff cents and the level by lfo_to_volume dB at its peak. Both
LFOs are triangles that start at zero, rising, once their delay is over.
cutoff and vibrato_depth are live properties.
These are worked out every 32 frames, which at 48 kHz is about fourteen times a cycle of the fastest LFO a SoundFont can ask for (110 Hz). A voice that uses none of them renders exactly as before and costs the same. One with a filter costs about 1.7 times as much on a desktop and twice as much on an ESP32-P4, where 48 filtered voices fit a 256-frame block and 64 only just do: see what the filter and the LFOs cost.
They cost no memory until a voice uses them. The first play() that asks for
any of them, or the first cutoff or nonzero vibrato_depth set on a voice,
gives that voice about 270 bytes for their state, which it then keeps. A
Sampler(voice_count=32) on MicroPython takes about 19 KB whatever its notes
ask for, and one whose 32 voices all use them takes about 7 KB more, in 32
small pieces rather than one large one.
How it renders
Every voice is mixed in one C call per block, summed in float and quantised once, through a 4-point Hermite interpolator, so a loop at pitch 1.0 is a wire. For why the position is fixed point and how the pitch is verified by measurement, see the upstream record.
SampleBuffer(size) is memory for the samples, outside the collected heap:
size zeroed bytes, with a buffer, so you can read into it and play views of
it. A bank of samples in bytearrays makes every gc.collect() read all of
them looking for pointers, which on a board with megabytes of samples is
seconds; a SampleBuffer’s bytes are never read. A view of one doesn’t keep
it alive, so hold the buffer itself for as long as a voice plays from it.
deinit(), or the buffer being collected, frees the bytes. On a board whose
audio driver says where (PSRAM on the ESP32 ports) they come from there; on
CPython it is a bytearray.