Glossary and numbers worth knowing

Embedded and machine-learning basics

Terms introduced in the primer sections, for participants new to one side of the field.

Embedded systems

Embedded system

A computer built into a larger product to perform a dedicated function, usually under real-time, memory and energy constraints. §1.1

Microcontroller (MCU)

A single chip containing a processor core, flash, SRAM and peripherals; the typical target of TinyML. Fig 1.1

Flash / MRAM

Non-volatile memory that keeps its contents without power; holds the program and the model weights.

SRAM

Fast volatile working memory; holds activations, buffers and the stack. Usually the scarcest resource.

Bare-metal / RTOS

Firmware running directly on the hardware, or under a small real-time operating system that schedules tasks with guaranteed timing.

DMA

Direct memory access: a hardware engine that moves data (e.g. sensor samples) into memory without the CPU, so the core can sleep.

SIMD

Single instruction, multiple data: one instruction operating on several packed values, e.g. two 16-bit multiply–accumulates at once on a Cortex-M4.

MEMS

Micro-electro-mechanical system: a microscopic mechanical sensor structure on silicon, as in most microphones and IMUs. §2.2

IMU

Inertial measurement unit: accelerometer and gyroscope (sometimes magnetometer) in one package.

TinyML

Machine learning on microcontroller-class devices, typically at milliwatt power and below.

Machine learning

Model, parameters

A function f(x; θ) of fixed form whose parameters θ are learned from data. §1.3

Training / inference

Fitting the parameters to examples (expensive, usually off-device) / evaluating the fitted model on a new input (cheap, on-device).

Loss function

A measure of how wrong a prediction is; training minimises its average over the training set. §3.1

Cross-entropy, softmax

The standard classification loss, −log of the probability assigned to the true class; the softmax turns scores into probabilities.

Gradient descent, learning rate

Iteratively moving the parameters against the gradient of the loss; the step size is the learning rate. §3.3

SGD, mini-batch, epoch

Gradient descent with the gradient estimated on a small random batch; one epoch is one pass over the training set.

Overfitting / underfitting

Fitting the noise of the training data and failing on new data / being too simple to capture the real pattern. §3.2

Train / validation / test split

Data for fitting / for every design choice / for the final, once-only estimate. Fig 3.3

Hyper-parameter

A setting chosen by the engineer, not learned: learning rate, model size, regularisation strength, window length.

Precision, recall

Of the alarms, the fraction that are right / of the real events, the fraction caught. §3.4

Neuron, dense layer

A weighted sum plus bias followed by a non-linearity; a layer is many neurons sharing the same inputs, i.e. a matrix–vector product. §3.10

Activation (function)

The non-linearity φ applied element-wise (ReLU, sigmoid…). "Activations" also means the intermediate tensors a network produces — the quantity that sizes SRAM.

Backpropagation

Computing all parameter gradients by the chain rule, layer by layer from output to input; requires the stored activations of every layer.

Convolutional layer

A layer that slides small shared kernels over the input; few parameters, translation-equivariant. §3.11

Course terms

Cost and performance

MAC / MAdd

One multiply–accumulate: a += b·c. Depending on the paper, counted as one or two FLOPs — always check which. §3.12

Arithmetic intensity

MACs performed per byte of memory traffic. Determines whether a kernel can reach peak compute. §4.3

Roofline

The bound P ≤ min(π, β·I): performance is limited by peak compute or by bandwidth × intensity, whichever is smaller. §4.2

Ridge point

I* = π/β, the arithmetic intensity above which a kernel can be compute-bound. §4.2

Compute-bound

Time is dominated by arithmetic; reducing operations reduces latency. §4.2

Memory-bound

Time is dominated by waiting for operands; reducing operations changes nothing. §4.3

Peak activation memory

The maximum bytes of intermediate tensors that must be live simultaneously. The budget that decides whether a CNN fits on an MCU. §1.6

Arena

The single statically-sized buffer in which an MCU runtime places every intermediate tensor, planned offline as a packing problem. §4.15

Duty cycle

The fraction of time a system is awake. Converts an energy-per-inference figure into an average power. §1.6

Cascade

A chain of increasingly expensive models, each gating the next, so the expensive ones run rarely. §1.8

Dennard scaling

The pre-2004 regime in which shrinking transistors also allowed lower voltage, giving more speed at constant power. Its end is why this field exists. §1.2

Signals and features

Nyquist rate

Twice the highest frequency present. Sampling below it folds high frequencies onto low ones irreversibly. §2.4

Aliasing

The additive overlap of spectral replicas caused by undersampling. Cannot be undone digitally. §2.4

ENOB

Effective number of bits, (SINAD − 1.76)/6.02. What a converter actually delivers, as opposed to its nominal word length. §2.5

SQNR

Signal-to-quantization-noise ratio, ≈ 6.02N + 1.76 dB for a full-scale sine. Also the per-layer diagnostic for quantization bugs. §2.5, §3.29

Dither

Small noise added before quantization to decorrelate the error, converting structured distortion into benign hiss. §2.5

STFT

Short-time Fourier transform: windowed transforms of overlapping segments, giving a time–frequency picture. §2.12

Spectral leakage

Energy from one frequency appearing in neighbouring bins because the window truncates the signal. Controlled by the window shape. §2.12

Mel scale

m = 2595 log₁₀(1 + f/700), an approximation of human auditory frequency resolution. §2.13

MFCC

Mel-frequency cepstral coefficients: DCT of log mel-band energies. The DCT exists to decorrelate for diagonal-covariance GMMs and should usually be dropped for CNNs. §2.13

Mutual information

I(X;Y) = H(Y) − H(Y|X): how many nats knowing X saves you about Y. Detects non-linear dependence that correlation misses. §2.15

Polyphase / CIC

Decimation structures that compute only the samples that survive (polyphase) or need no multipliers at all (CIC). §2.7

Models

Support vector

A training point with non-zero dual coefficient. Kernel-SVM inference cost is proportional to how many there are. §3.8

Kernel trick

Replacing inner products with a kernel function to fit a linear model in a feature space never constructed. §3.8

Random features

An explicit finite map whose inner products approximate a kernel, removing the dependence on the number of support vectors. §3.8

Calibration

The property that a score of 0.3 means a 30 % chance. Required for any transferable threshold. §3.5

Depthwise separable convolution

A spatial depthwise stage plus a 1×1 channel-mixing stage; reduces arithmetic by 1/C_out + 1/k². §3.13

Inverted residual

Expand → depthwise → project, with the skip connecting the narrow ends so that only a small tensor stays live. §3.18

Linear bottleneck

Omitting the non-linearity after the narrow projection, because a ReLU in a low-dimensional space destroys information irrecoverably. §3.18

Squeeze-and-excitation

Channel-wise attention from a global average. Cheap in MACs, awkward in deployment because of the global reduction. §3.18

Receptive field

The span of input that influences one output. Must exceed the duration of the event being detected. §3.15

Streaming inference

Keeping per-layer ring buffers so overlapping windows recompute only the new columns. §3.15

Compression

Affine quantization

x ≈ s(q − z) with an integer zero-point so that real zero is exactly representable. §3.23

Zero-point

The integer that maps to real 0. Its exactness matters because padding introduces zeros everywhere. §3.23

Requantization

Rescaling the int32 accumulator to the output scale by M = s₁s₂/s₃, implemented as a fixed-point multiply and shift. §3.24

Per-channel quantization

One weight scale per output channel. The standard fix for depthwise layers, whose per-channel ranges differ by orders of magnitude. §3.25

PTQ / QAT

Post-training quantization (calibration only, no labels) versus quantization-aware training (fake quantization in the forward pass). §3.27, §3.28

Straight-through estimator

Defining the backward pass of a rounding node to be the identity, so gradients survive a non-differentiable forward. §3.28

Cross-layer equalisation

Rescaling consecutive layers by S and S⁻¹ — exact for ReLU — to equalise channel ranges before quantizing. §3.27

AdaRound

Learning per-weight rounding directions against a layer-wise reconstruction objective instead of rounding to nearest. §3.27

Unstructured / structured / N:M sparsity

Scattered zeros (best accuracy, no speedup), whole channels removed (smaller dense model), or exactly N of every M zero (fixed pattern hardware can decode). §3.30

Saliency

The predicted loss increase from deleting a parameter: ½h_iiw_i² under OBD’s diagonal-Hessian assumption. §3.31

Lottery ticket

A sparse subnetwork that, trained from its original initialisation, matches the dense network. §3.32

Break-even sparsity

σ < b_value/(b_value+b_index) — below it, a sparse format is larger than the dense one. §3.33

Knowledge distillation

Training a student on a teacher’s softened output distribution rather than on labels alone. §3.38

Dark knowledge

The teacher’s relative ranking over the wrong classes — the similarity structure a one-hot label cannot express. §3.37

Capacity gap

The point beyond which a stronger teacher yields a worse student, because the student cannot represent its function. §3.40

Hardware and systems

SIMD / SIMT

One instruction over several data lanes (CPU vector units) or over a lock-step group of threads (GPU warps). §4.1, §4.9

Warp divergence

Both sides of a branch executing in sequence with inactive lanes masked. Why data-dependent control flow is expensive on GPUs. §4.9

Systolic array

A grid of processing elements passing operands to neighbours, so N² MACs per cycle need only 2N external fetches. §4.10

Dataflow

Which quantity an accelerator pins in its processing elements — weights, outputs, or a row of each — and therefore which reuse it captures. §4.10

TCM

Tightly-coupled memory: small single-cycle SRAM addressed directly, with no cache logic. Placement is yours to decide. §4.1

DMA

A peripheral that moves data without the core, enabling double-buffered acquisition during inference. §4.6

im2col

Reshaping convolution into a matrix multiply by materialising overlapping patches — at a k² memory blow-up that a partial version avoids. §4.5

Winograd

Trading multiplications for additions and transforms; 2.25× fewer multiplies for F(2×2, 3×3), at some numerical cost. §4.5

Operator fusion

Computing several graph operations in one pass so intermediates never reach memory. Saves traffic, not arithmetic. §4.15

Analog in-memory computing

Storing weights as conductances so Ohm’s and Kirchhoff’s laws perform the matrix–vector product in place. §4.12

MLPerf Tiny

The community benchmark for tiny systems: keyword spotting, visual wake words, image classification, anomaly detection — with measured energy, plus a streaming task from v1.3. Topic D

Covariate / concept / label shift

The inputs move, the input–label relationship moves, or only the class priors move. Different symptoms, different fixes. §4.19

Federated learning

Training by averaging locally-computed updates so raw data never leaves the device. §4.20

Numbers and formulas worth knowing

Cost formulas

Performance

Signals

Quantization

Sparsity and factorisation

Energy, after Horowitz (45 nm)

System-level anchors

• CR2032 coin cell ≈ 2.4 kJ; one year of service ⇒ ≈ 77 µW average power ceiling (before derating).
• A cascade's expected energy: E₁ + p₁E₂ + p₁p₂E₃ + …
• DS-CNN keyword spotting, small configuration: ≈ 38.6 kB, ≈ 5.4 MOps per inference, ≈ 94.4 % on Google Speech Commands (Zhang et al., 2017).
• MLPerf Tiny tasks: keyword spotting, visual wake words, small-image classification, anomaly detection — plus a streaming wake-word benchmark from v1.3.