ϕ-Noise:
Training-Free Temporal Video Conditioning
via Phase-Based Noise Manipulation

Applications

I2V Motion Transfer

Motion Transfer (+ Structure Conditioning)

Cut & Drag

Abstract

Latent video diffusion models generate videos by progressively transforming Gaussian noise into realistic samples conditioned on text or visual inputs. However, existing conditioning methods often require additional training and computational overhead. Motivated by recent findings on the importance of frequency components in generative models, we propose a simple, training-free approach for motion-conditioned video generation by injecting low-frequency phase information from a reference video directly into the diffusion noise latents.

Our method transfers motion cues without modifying the model architecture or inference pipeline. Using several applications, we demonstrate effective control over both appearance and dynamics in generated videos, while achieving competitive or superior results compared to more complex conditioning approaches.

How Does It Work?

ϕ-Noise operates by decomposing both the reference video latent and the Gaussian noise latent into the frequency domain via the Discrete Fourier Transform (DFT). The low-frequency phase components of the noise — which encode coarse spatial structure and dominant temporal dynamics — are replaced with those from the reference. A Spectral-Temporal Energy Balancing Mask Φ is then applied to preserve total signal energy and ensure stable denoising.

ϕ-Noise injects low-frequency phase information from a reference video into Gaussian noise latents, enabling training-free motion and structural conditioning across diverse video generation tasks — with no changes to the diffusion model.

Analysis

Phase Substitution & Energy Balancing

Figure 2. Phase and Energy Analysis — phase distributions, latent energy evolution across denoising steps, and qualitative comparison with/without energy balancing.

Naïve phase injection disrupts the expected energy profile of the noise, causing saturation artifacts and out-of-distribution denoising. ϕ-Noise applies a frequency-dependent scaling mask Φ that scales down injected low-frequency magnitudes by 1/γ while compensating by scaling high frequencies by β, guaranteeing exact energy preservation: E(z̃_k ⊙ Φ) = E(z̃).

Because the DFT and its inverse are computationally negligible relative to a single diffusion step, ϕ-Noise introduces no additional runtime or memory overhead and requires no changes to the model architecture or inference pipeline.

Comparisons

ϕ-Noise supports three distinct video generation tasks within a single unified framework, all using WAN (Wan2.2-14B) as the base model with no additional training.

Quantitative Results

We evaluate on a diverse benchmark of 60 high-quality videos: 20 from the TTM dataset, 30 from LOVEU-TGVE-2023, and 10 in-the-wild videos for real-world generalization. ϕ-Noise achieves competitive or state-of-the-art results at negligible additional cost.

ϕ-Noise achieves the best Flow-Error (motion fidelity) and Dynamics-Degree scores — the two metrics most directly measuring motion transfer quality — while requiring no additional training.