🔊 Audio Forensics Module — Real-Time Voice Deepfake Detection
Detects AI-cloned voices within the first 10 seconds of a live call.
This module analyzes the spectrogram of incoming audio to detect synthetic artifacts — micro-imperfections in pitch, frequency consistency, and missing microtremors that human ears miss but machine learning can spot.
How it works:
- Audio → WavLM Encoder — SSL features capture phoneme-level synthesis artifacts
- Sliding Window Analysis — 4-second windows at 1-second hops for temporal resolution
- CQT Spectrogram Heatmap — Highlights exact frequencies that triggered the alert
- Risk Score (0-100) — Instant classification with HIGH RISK flag within 10 seconds
Architecture:
Raw Audio → WavLM (CNN-Transformer) → Classifier → Risk Score + Spectrogram Heatmap
Model: Fine-tuned WavLM-base on ASVspoof 2019 LA | Paper: arxiv:2408.07414
Supported formats: WAV, MP3, FLAC, OGG, M4A
Optimal: 3-10 seconds of speech at 16kHz
Max analyzed: First 10 seconds
Upload or record audio to begin analysis.
Live Call Monitoring
Click the microphone to start streaming. The system analyzes audio in real-time and flags suspicious voice patterns within 10 seconds.
🏗️ Architecture: CNN-Transformer Hybrid
┌──────────────┐ ┌───────────────────────┐ ┌──────────────────┐ ┌─────────────┐
│ Raw Audio │ ──→ │ WavLM SSL Encoder │ ──→ │ Classifier Head │ ──→ │ Risk Score │
│ (16kHz) │ │ (CNN + Transformer) │ │ (Linear + Pool) │ │ (0-100) │
└──────────────┘ └───────────────────────┘ └──────────────────┘ └─────────────┘
│ │
│ ┌───────────────────────┐ ┌──────────────────┐ │
└────────→│ CQT Spectrogram │ ──→ │ Artifact Heatmap│──────────────→│
│ (84 bins, 8 octaves) │ │ (Red=suspicious)│ Final
└───────────────────────┘ └──────────────────┘ Report
🔬 What Synthetic Artifacts Does It Detect?
| Artifact Type | Description | Where in Spectrogram |
|---|---|---|
| Unnatural Pitch Consistency | AI voices have too-smooth F0 contours | Low frequency bands (80-400 Hz) |
| GAN Frequency Smoothness | Neural vocoders produce unnaturally smooth harmonics | Mid-high frequencies (1-4 kHz) |
| Missing Microtremors | Real voices have subtle involuntary tremors at 8-12 Hz | Fine temporal structure |
| Spectral Banding | Codec-based TTS shows regular spectral gaps | High frequencies (>4 kHz) |
| Phase Discontinuities | Neural TTS has unnatural phase relationships | Temporal transitions |
📊 Model Performance
- Training Data: ASVspoof 2019 LA (25,380 samples — real + 6 spoofing algorithms)
- Architecture: WavLM-base (94.6M params) + Sequence Classification head
- Expected Performance: ~99.5% accuracy, <1% EER on in-domain data
- Real-time Speed: ~30ms per 4-second window on GPU, ~150ms on CPU
📚 References
- WavLM Model Ensemble for Audio Deepfake Detection (arxiv:2408.07414)
- XLSR-Mamba: Dual-Column Bidirectional SSM (arxiv:2411.10027) — 0.93% EER
- Nes2Net: Lightweight Nested Architecture (arxiv:2504.05657)
- ASVspoof 2019: Automatic Speaker Verification Spoofing & Countermeasures Challenge
Built for Audio Forensics Hackathon | Model: WavLM-base fine-tuned on ASVspoof 2019 LA | Detects AI-cloned voices within 10 seconds