Video Foundation Model Researcher

Juncheng Wang

I work on video foundation models, with a focus on large-scale training, agentic generation, and multimodal evaluation.

Scaling decides how well a model renders. It cannot decide what was never specified.

Currently researching next-generation video foundation models at Tencent Hunyuan. Previously at AMD and Alibaba Wan.

Juncheng Wang
Ph.D. Candidate, The Hong Kong Polytechnic University

A Roadmap for Video Generation

Where the conditioning comes from — and who is responsible for writing it.

Scaling decides how well a model renders.
It cannot decide what was never specified.

A prompt is a lossy description of a physical scene — on VideoPhy-2 even the best models clear only about a third of joint physical accuracy. More compute sharpens the renderer; it cannot recover what the prompt never contained. So the thing that keeps moving is the conditioning, and who writes it.

  1. L1Text condition

    written by a human

    Say it

    The prompt is the whole contract.

    One sentence has to specify a long, coherent, high-rate signal, and all of that difficulty lands on the decoder. Siren spreads the load across residual codebooks instead of asking one model to carry all twelve.

    • Language Model Based Text-to-Audio Generation: Anti-Causally Aligned Collaborative Residual Transformers

      ArchitectureEMNLP 2025 · first author

  2. L2Reference condition

    written by a human, in pixels

    Show it

    Point at what language cannot describe.

    Layout, timing, identity — easier to show than to say, so conditioning turns structural. DenseControl places every individual in a dense crowd; MelQCD takes the condition from another modality entirely. Precision rises, and a human still has to supply the reference.

    • DenseControl: Instance-Level Controllable Synthesis of Dense Crowd Image

      Spatial controlIEEE TMM 2026 · first author

    • Synchronized Video-to-Audio Generation via Mel Quantization-Continuum Decomposition

      Cross-modal controlCVPR 2025 · first author

  3. L3Searched condition

    written by search, at inference

    Find it

    Buy the missing half with test-time compute.

    If the specification is incomplete, go looking for the rest. Gen-Searcher retrieves external evidence before it draws; my own work spends the budget internally — Guided by the Plan steers decoding to stay faithful to a plan, Think Before You Move reasons in latent motion space before the first pose. The ceiling rises while the weights stay put.

    • Guided by the Plan: Enhancing Faithful Autoregressive Text-to-Audio Generation with Guided Decoding

      Steered decodingEACL 2026 · first author

    • Think Before You Move: Latent Motion Reasoning for Text-to-Motion Generation

      Latent reasoningTPAMI, major revision · co-first author

  4. L4Composed condition

    written by a fixed pipeline

    Arrange it

    Orchestration becomes a component.

    Once text, images, audio and clips all condition one model, arrangement is the bottleneck — MiniMax H3 promotes it to its own module. But a loop cannot close on a metric it does not have: Beyond Time Shifts turns an omni-LLM into a reference-free critic for audio-visual quality and sync.

    • Beyond Time Shifts: Adapting Omni-LLM as a Reference-Free Evaluator for Generative Audio-Visual Models

      The verifierECCV 2026 · co-first author

  5. L5Self-improving condition

    written by a trained planner

    Learn to arrange it

    Give the orchestrator a gradient.

    NEWTON demotes generation to one action in an agent’s toolbox: a planner orchestrates physics-aware tools, a verifier closes the loop for re-planning. Only the planner trains — VideoPhy-2 joint accuracy 21.4 → 29.7 on LTX-Video and 30.7 → 37.4 on Veo-3.1, with both generators untouched. ROC-Agent runs the same loop one level up, on research itself. What improves is the harness, not the weights.

    • NEWTON: Agentic Planning for Physically Grounded Video Generation

      Trainable orchestrationPreprint, under review · co-first author

    • ROC-Agent: Research Orchestration via Cyclic Agents for Autonomous Deep Learning Experimentation

      The loop, one level upUnder review · first author

The next level gives the weights back what the harness learned.

Levels 1–4 changed who writes the conditioning; L5 made that writer trainable. What is missing is internalization — folding verified harness improvements back into the generator, until the model no longer needs the scaffolding. That is where I am working now, and I would happily be argued out of it. Get in touch.

Selected Work

Generation, planning, and feedback for multimodal systems.

NEWTON: planning, tool use, generation, and verification in one loop.

Agentic video generation · Preprint, under review · Co-first author

NEWTON: Agentic Planning for Physically Grounded Video Generation

A plan–execute–verify system that treats video generation as one action inside a larger workflow, orchestrating physics-aware tools and critic feedback.

Paper Project Code

Training framework for a reference-free audio-visual evaluator

Multimodal evaluation · ECCV 2026 · Co-first author

Beyond Time Shifts: Adapting Omni-LLM as a Reference-Free Evaluator for Generative Audio-Visual Models

A learned critic for fine-grained audio-visual quality and synchronization, designed to turn generated outputs into useful feedback.

Paper Project Code

MelQCD video-to-audio generation overview

Multimodal generation · CVPR 2025 · First author

Synchronized Video-to-Audio Generation via Mel Quantization-Continuum Decomposition

Decomposes audio into predictable discrete structure and expressive continuous detail for synchronized video-conditioned generation.

Paper Project Code

Siren collaborative residual transformer pipeline for autoregressive text-to-audio generation

Autoregressive audio generation · EMNLP 2025 Oral · First author

Language Model Based Text-to-Audio Generation: Anti-Causally Aligned Collaborative Residual Transformers

Introduces Siren, a collaborative residual-transformer architecture that resolves imbalanced RVQ prediction difficulty and improves autoregressive text-to-audio generation.

Paper

Industry Experience

2026.06 — Present

Tencent · Hunyuan-Video Group

Qingyun (Project Up) Intern

Pre-training research for the next generation of Hunyuan video foundation models.

Working with Dr. Jianwei Zhang

2026.01 — 2026.06

Advanced Micro Devices (AMD)

Student Research Fellow

Efficient flow-model research for video generation.

Mentored by Dr. Tong Shen and Dr. Emad Barsoum

2023.09 — 2025.09

Alibaba · Wan Group, Tongyi Lab

Research Intern

Post-training for foundational video generation and audio-centric generative modeling.

Mentored by Dr. Chao Xu, Lei Shang, and Dr. Liefeng Bo

Publications

† Equal contribution. * Junior student advisee.

  1. Under review

    SphereFlow: Missing Modality Imputation via Geometric Transport on Hypersphere

    Juncheng Wang et al.

  2. Under review

    ROC-Agent: Research Orchestration via Cyclic Agents for Autonomous Deep Learning Experimentation

    Juncheng Wang et al.

  3. ECCV 2026

    Beyond Time Shifts: Adapting Omni-LLM as a Reference-Free Evaluator for Generative Audio-Visual Models

    Yijie Qian*†, Juncheng Wang†, Chao Xu, et al.

    Paper

  4. ECCV 2026

    Progression as Latent Drift: Generative Forecasting of Slow-Evolving Pathologies

    Yuxiang Feng*†, Juncheng Wang†, Chao Xu, et al.

    Paper

  5. IEEE TMM 2026

    DenseControl: Instance-Level Controllable Synthesis of Dense Crowd Image

    Juncheng Wang, Lei Shang, Wang Lu, Baigui Sun, and Shujun Wang.

    Paper

  6. Preprint

    NEWTON: Agentic Planning for Physically Grounded Video Generation

    Yuxiang Feng†, Juncheng Wang†, Chao Xu, et al.

    Paper

  7. ICLR 2026 Oral

    Decentralized Attention Fails Centralized Signals: Rethinking Transformers for Medical Time Series

    Guoqi Yu*, Juncheng Wang, Chen Yang, Jing Qin, Angelica I. Aviles-Rivero, and Shujun Wang.

    Paper

  8. EACL 2026

    Guided by the Plan: Enhancing Faithful Autoregressive Text-to-Audio Generation with Guided Decoding

    Juncheng Wang, Zhe Hu, Chao Xu, Siyue Ren, Yuxiang Feng, Yang Liu, Baigui Sun, and Shujun Wang.

    Paper

  9. IJCV 2026

    Exploring Scale Shift in Crowd Localization under the Context of Domain Generalization

    Juncheng Wang, Lei Shang, Ziqi Liu, Wang Lu, Xixu Hu, Zhe Hu, Jindong Wang, and Shujun Wang.

    Paper

  10. TPAMI major revision

    Think Before You Move: Latent Motion Reasoning for Text-to-Motion Generation

    Yijie Qian†, Juncheng Wang†, Yuxiang Feng, Chao Xu, et al.

    Paper

  11. EMNLP 2025 Oral

    Language Model Based Text-to-Audio Generation: Anti-Causally Aligned Collaborative Residual Transformers

    Juncheng Wang, Chao Xu, Cheng Yu, Zhe Hu, Haoyu Xie, Guoqi Yu, Lei Shang, and Shujun Wang.

    Paper

  12. CVPR 2025

    Synchronized Video-to-Audio Generation via Mel Quantization-Continuum Decomposition

    Juncheng Wang, Chao Xu, Cheng Yu, Lei Shang, Zhe Hu, Shujun Wang, and Liefeng Bo.

    Paper

  13. IEEE TIP 2023

    Crowd Localization from Gaussian Mixture Scoped Knowledge and Scoped Teacher

    Juncheng Wang, Junyu Gao, Yuan Yuan, and Qi Wang.

    Paper