Finding a faint signal in a loud ocean: notes on recognition in high-noise audio

For detection and classification in heavy noise, the representation and the channel-fusion choice decide more than model size. Notes from underwater target recognition.

Some recognition problems are decided before the classifier ever runs. Take underwater target recognition from a multichannel hydrophone array: the signature you want is, most of the time, quieter than the noise around it, and the accuracy ceiling is set by how you represent and combine the signal long before any model sees it. Few labels, a hard compute budget, and a signal that hides in plain water. The reasoning below is general — the setting just makes it vivid.

The target is quieter than the floor

In a high-noise detection problem, almost every window you look at is noise. The event you care about is rare, faint, and often overlapping with louder, irrelevant sources. That sets up two compounding problems: severe class imbalance, and a signal-to-noise ratio low enough that a careless feature pipeline blurs the signature into the background before a model ever sees it. Once the representation has thrown the signal away, no amount of downstream capacity gets it back. So the first battle isn't the classifier — it's the input.

Representation before model

How you turn raw audio into a model input is a design decision, not a default. Window length, frequency resolution, linear vs. perceptual scaling, and whether you let the front end learn — each trades time resolution against frequency resolution, and the right point depends entirely on the signature you're hunting. A compact model on a representation matched to the signal will beat a much larger model on a generic one. We spent more time here than anywhere else, and it paid back the most.

Where you fuse the channels

Multichannel data adds a lever single-channel work doesn't have: where you combine the channels. Fuse late — train per-channel and average — and you throw away the cross-channel structure (relative phase, coherence) that often carries a faint signal up out of the noise. Fuse early and the model can exploit that structure directly. A light channel-attention mechanism on top lets it lean on the informative channels and down-weight the dead ones. In practice the fusion point moved the needle more than network depth did.

What didn't survive contact with the noise

The honest part. The first instinct was to spend the budget on capacity — a heavier, cleverer architecture. It looked great on the slice of data we had and then fell apart the moment the noise conditions shifted, because it had quietly memorised the scarce labels rather than learning the signature. The lesson stuck: capacity wasn't the bottleneck. Representation and data were. The fix was to make the model smaller and the inputs better.

Limited labels: lean on structure

When labelled events are expensive and noise is free, let the noise do work. Self-supervised pretraining — reconstructing masked spectrogram patches on the abundant unlabelled audio — gives the model a sense of "normal" before it ever sees a labelled example, so the few labels you have go toward the decision boundary instead of basic feature learning. Augmentation that mimics the real noise distribution does the rest. The label budget stretches a long way when the model isn't starting from scratch.

Performance vs. compute is the whole game

Because the model has to run on an edge device, every point of accuracy is paid for in watts and milliseconds. That reframes the objective: you're not chasing the top of the accuracy axis, you're finding the knee of the accuracy-versus-compute curve — the smallest model that still clears the bar. A capable model that can't run at the sensor is worth less than a modest one that can. Designing to the constraint, rather than treating it as an afterthought, is most of the engineering.

The transfer

None of this is really about the ocean. The same shape — a faint signal, multiple channels, heavy noise, scarce labels, a fixed compute budget — turns up in vibration-array condition monitoring, RF, structural health, and parts of biomedical sensing. Represent carefully, fuse early, pretrain on the noise, and design to the constraint. The playbook ports.

We take on a limited number of engagements per quarter. If your problem involves edge constraints, unusual sensors, or no labelled data — that's our specialty.