Abstract: Diffusion models achieve state-of-the-art performance in generative modeling but are limited by their reliance on Gaussian noise and the high computational cost of iterative sampling. Star-shaped diffusion addresses the former by introducing a non-Markovian forward process, yet this comes at the expense of temporal coherence in the reverse process. We propose a novel framework that resolves this trade-off by learning a Markovian projection of a star-shaped forward process, and its reversal. This design enables learning over a broad class of exponential models and recovers DDPM as a special case. It is particularly well-suited for knowledge distillation, allowing few-step or even single-step generation. Experiments demonstrate the effectiveness and flexibility of our approach across multiple generative tasks.

DDPM
DDPM
Flow Matching
Flow Matching
Shortcut
Shortcut
M-Star (Ours)
M-Star
Steps: 256
One Step 128 Steps 256 Steps

Progressive Distillation: Few-Step Generation

M-Star excels at progressive distillation. By learning to predict clean data from any timestep, M-Star naturally supports compression to 4-step or even 1-step generation. The distilled models maintain high sample quality while dramatically reducing inference cost.

Algorithm 1: M-Star Training

Algorithm 2: M-Star Sampling

Training Process: The M-Star training procedure learns to predict clean data z₀ from the sufficient tail statistic st while simultaneously optimizing the interaction matrix Mt. At each iteration, the algorithm samples a random timestep and generates noisy states from the star-shaped forward process. The tail statistic st = Σk=tT Ak⊤ψ(zk) compresses the entire future trajectory into a fixed-size representation, enabling efficient learning across all timesteps. The learnable interaction matrix Mt balances local coherence from the current state with global guidance from the predicted clean data, minimizing prediction error through KL divergence optimization.

Sampling Process: The M-Star sampling procedure generates high-quality samples through iterative denoising from pure noise. Starting from zT ~ q(zT), the algorithm iteratively predicts the clean data ẑ₀ from the current tail statistic and samples the next state from the Markovian reverse distribution q̃(zt-1 | zt, z₀ = ẑ₀). The tail statistic is efficiently updated at each step by incorporating the sufficient statistic of the newly sampled state. This Markovian structure ensures temporal coherence between consecutive states, preventing the erratic trajectories that can occur in non-Markovian approaches.

M-Star generates high-quality images unconditionally. The Markovian projection maintains temporal coherence throughout the sampling process, resulting in superior sample quality compared to SS-DDPM. By learning to predict clean data from any timestep, M-Star is particularly well-suited for progressive distillation, enabling few-step or even single-step generation without sacrificing quality. The framework generalizes standard DDPMs to exponential family distributions while preserving the benefits of sequential generation, making it applicable to diverse data modalities including directional data, constrained geometries, and meteorological datasets.

Class-Conditional Image Generation: SS-DDPM vs M-Star

M-Star demonstrates superior quality in class-conditional generation. Click on the images below to compare SS-DDPM (left) with M-Star (right). The Markovian projection ensures consistent and high-fidelity samples across different classes.

Unconditional Image Generation

M-Star generates high-quality images unconditionally. The Markovian projection maintains temporal coherence throughout the sampling process, resulting in superior sample quality compared to SS-DDPM.