Media Processing Engine

MPE smoke run 21 September 2026

First complete run of the inherited pipeline on the LT GPU server: three recorded chapters through transcription, fraud check, metadata, silence trimming, zoom, eye-level framing, background replacement, stitching and the audio pass, 27 minutes end to end. Both players stream an HLS ladder (1080p, 720p, 480p) with an MP4 fallback. The context panel shows the title, summary, key concepts and research field the pipeline suggested. Compare the audio: the left player is the video before the noise reduction step, the right one is the final video as the pipeline delivers it today.

Processing time

Steps in pipeline order, one job of three chapters

# Step Agent image Runs Per run Total
1 Audio extraction per chapter ffmpeg 3 5 s 14 s
2 Language detection whisper 3 9 to 10 s 29 s
3 Transcription whisper 3 13 s 40 s
4 Fraud detection (Claude Opus 5) rubyllm 1 21 s 21 s
5 Suggest metadata (Claude Opus 5) rubyllm 1 20 s 20 s
6 Silence trimming silero 3 14 to 16 s 43 s
7 Zoom timeline (Claude Sonnet 5) rubyllm 3 24 to 30 s 78 s
8 Eye-level framing dlib 3 24 to 26 s 73 s
9 Background mask video-matting 3 223 to 251 s 700 s
10 Background adjust video-matting 3 87 to 97 s 270 s
11 Stitching with intro, wipes and outro ffmpeg 1 41 s 41 s
12 Audio extraction of the stitched video ffmpeg 1 5 s 5 s
13 Noise reduction noisereduce 1 11 s 11 s
14 Audio replace ffmpeg 1 5 s 5 s
Downloads from the bucket, volume setup minio-client 12 5 s 60 s
Uploads of every intermediate to the bucket minio-client 39 5 s 210 s
Total, wall clock, 81 steps 27 min 3 s

Measured on 21 September 2026 on the LT GPU server (RTX 4000 SFF Ada). Background mask and adjust take 60 percent of the run. Every step also carries about 4.5 seconds of orchestration overhead, and the transfers add 17 percent; both go away once the job runner on the server replaces the laptop-driven controller.