Advanced Motion-Compensated Denoising Using MVToolsMotion-compensated denoising is a powerful technique for removing noise from video while preserving temporal detail and minimizing motion-related artifacts. MVTools is a widely used Avisynth/ VapourSynth plugin that provides motion estimation and compensation primitives which form the backbone of many advanced temporal filters. This article covers the theory behind motion-compensated denoising, practical MVTools workflows, parameter tuning, performance tips, and common pitfalls with worked examples.
1. Why motion compensation matters
Spatial denoisers operate on single frames and can blur fine detail or leave temporal flicker when applied frame-by-frame. Temporal denoising averages information across frames to increase signal-to-noise ratio, but naive temporal filters (like simple temporal averaging) produce ghosting and smear when objects move. Motion-compensated denoising (MCD) uses motion vectors to align pixels from neighboring frames before combining them, allowing the filter to preserve moving details and reduce artifacts.
Key benefits:
- Better noise reduction at lower blurring cost compared to purely spatial filters.
- Preservation of temporal coherence, reducing flicker and jitter.
- Selective smoothing of stationary regions while preserving moving objects.
2. MVTools fundamentals
MVTools provides several functions; the primary ones used for denoising are:
- MAnalyse / MAnalyse1 / MAnalyse2 (motion estimation)
- MRecalculate (refine motion vectors)
- MBlock/ MDegrain / MFlowBlur (motion-compensated operations)
- MMask / MCreate / MIsBlock / MChroma for masks and analysis
Basic workflow:
- Convert source to a suitable format (typically YUV420/422 with a higher bit depth when available).
- Run motion analysis (MAnalyse) to produce forward/backward motion vector clips.
- Optionally postprocess vectors (MRecalculate, MFilter).
- Use MDegrain or custom reconstruction to produce the denoised clip.
- Blend, mask, or apply spatial denoising on residuals as needed.
3. Motion estimation: parameters and trade-offs
Motion estimation quality is crucial. MAnalyse has many parameters; the most important are:
- isb: whether vectors are backward (0) or forward (1) or both (2).
- blksize: block size (e.g., 8, 16). Smaller blocks capture fine motion but increase noise sensitivity and cost.
- overlap: block overlap in pixels. Higher overlap reduces blockiness.
- search: search radius (pixels). Larger searches capture fast motion but increase time and risk of incorrect matches.
- thsad / lambda: thresholds and penalties for motion smoothness; affect whether vectors are trusted.
Practical advice:
- Use blksize 8 or 16 with overlap = blksize – 2 or similar. For film-like motion, 8×8 with overlap 4–6 often balances detail and stability.
- For noisy sources, increase thsad to avoid matching on noise.
- Use multi-pass strategies: coarse estimation with larger blocks, then refine with smaller blocks (MRecalculate) to reduce false vectors.
4. Denoising with MDegrain
MDegrain is a high-level MVTools filter that performs motion-compensated temporal denoising using the vectors from MAnalyse.
Example VapourSynth (pseudocode) pipeline:
# Assume vs.Source -> clip from vapoursynth import core clip_y = core.resize.Bicubic(clip, format=vs.YUV420P8) # example vectors = core.mv.Analyse(clip_y, isb=2, blksize=8, overlap=4, search=4, thsad=200) den = core.mv.Degrain1(clip_y, vectors, thSAD=200, thSADC=120, thSAD2=140) MDegrain parameters to tune:
- thSAD / ThSADC: thresholds that control how strictly candidates are accepted. Lower values let more frames contribute; higher values demand better matches.
- chroma motion handling: MVTools primarily estimates motion on luma. Use MChroma or other methods to handle chroma if color artifacts appear.
- number of reference frames: more frames yield stronger denoising but greater risk of mis-registration. Commonly use 1–3 frames on each side (total 3–7 frames).
Example practical settings:
- Mild denoising: use 1 reference per side, thSAD around 200–400 (depends on bit depth and noise).
- Aggressive denoising: 2–3 references per side, thSAD lower if vectors are high quality, but include spatial denoising steps to avoid softness.
5. Combining spatial and temporal filters
Temporal MCD handles random noise well but may leave patterned noise, compression artifacts, or blocking on low-detail areas. A common approach:
- Apply moderate MDegrain to remove most temporal noise.
- Use a conservative spatial denoiser (e.g., FFT3DFilter, NLMeans, BM3D) on residual noise.
- Use masks to protect edges and preserve grain where desired.
Example:
- Run MDegrain with conservative settings.
- Create a detail mask from strengthened edges (Sobel/Laplacian) and limit spatial denoising to low-detail regions.
- Reconstruct final clip by merging spatially denoised and temporally denoised versions guided by masks.
6. Handling occlusions and uncovered areas
Occlusions (areas that appear/disappear between frames) challenge MCD. MVTools provides ways to detect unreliable vectors:
- MIsBlock can indicate blocks with unreliable motion.
- MAnalyse’s thSAD/flags can be tuned to mark poor matches.
- Use per-pixel or per-block confidence masks to reduce reliance on motion-compensated samples in occluded regions.
Strategy:
- Blend more from the current frame where motion confidence is low.
- Use single-frame spatial denoising for uncovered areas rather than temporal averaging.
7. Chrominance considerations
MVTools estimates motion on luma; chroma channels need careful handling to avoid color smearing:
- Up-sample chroma before motion estimation if chroma subsampling causes mismatch.
- Use MChroma to propagate vectors to chroma or apply separate chroma-aware filters.
- In difficult cases, denoise chroma only spatially and rely on luma-based temporal denoising for luminance detail.
8. Performance and optimization
MVTools can be CPU-intensive. Tips:
- Use appropriate blksize: larger blocks reduce computation.
- Limit the number of reference frames.
- Use multi-threaded builds of VapourSynth/Avisynth and ensure plugins are optimized.
- Pre-convert to higher bit depth only if necessary; denoising benefits from more bits but increases cost.
Batch processing: split long sources into segments; cache motion vectors when experimenting with parameters to avoid recomputing.
9. Common artifacts and fixes
- Ghosting/halos: tighten thSAD thresholds, reduce number of reference frames, or increase motion search accuracy.
- Blockiness: increase overlap, use smaller blocks, or apply post-filtering on motion vectors.
- Flicker: ensure temporal consistency by increasing vector trust or using additional temporal smoothing of offsets.
- Chroma smearing: handle chroma separately or use MChroma to transfer motion vectors.
10. Example end-to-end VapourSynth script
import vapoursynth as vs core = vs.core src = core.ffms2.Source("input.mkv") # Convert to 16-bit YUV for better headroom src_yuv = core.resize.Bicubic(src, format=vs.YUV420P16) # Analyze motion (both directions) vectors = core.mv.Analyse(src_yuv, isb=2, blksize=8, overlap=4, search=5, searchparam=2, pel=2, truemotion=True, thsad=400) # Temporal denoise degrain = core.mv.Degrain1(src_yuv, vectors, thSAD=400, thSADC=200) # Convert back to original format out = core.resize.Bicubic(degrain, format=src.format.id) out.set_output() Adjust thSAD and other values to taste; higher values are stricter and preserve detail, lower values denoise more aggressively.
11. Troubleshooting checklist
- Verify motion vectors visually (MVTools has visualization helpers) to ensure matches are sensible.
- Start with conservative settings; increase aggressiveness gradually.
- Compare frames at full resolution to check for ghosting or smearing.
- Use targeted masks rather than global heavy filtering.
12. Conclusion
Motion-compensated denoising with MVTools offers a flexible, high-quality method for reducing temporal noise while preserving motion and fine detail. The key is good motion estimation, conservative use of temporal blending, and combining temporal denoising with selective spatial filters and masks. With careful parameter tuning and vector inspection, MVTools can produce excellent results across noisy footage, scanned film, and compressed sources.
Leave a Reply