Physical Agents
Robota-Embed
VLA embeddings for experience retrieval, robot in-context learning, and task-local evolution.
FirstVLA embedding model
About Me
I am a final-year Ph.D. student in Computer Science at the Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University, advised by Prof. Li Yi.
I am currently a Research Intern at NVIDIA Research, where I work on Physical Agents and Physical RSI, mentored by Dr. Han Cai, with guidance from Prof. Song Han. Previously, I was a Researcher at Memories.ai, focusing on efficient video MLLMs, and a Research Intern with the Spatial Intelligence Group at Microsoft Research Asia, where I studied human–object interaction.
My research centers on Physical Agents and Physical RSI: building embodied systems that learn from physical experience, adapt in context, and continually improve both their task-execution capabilities and the mechanisms by which they learn.
- Physical Agents: developing action-aware representations and experience retrieval for embodied reasoning, in-context learning, and long-horizon interaction, including Robota-Embed.
- Physical RSI: building systems that turn interaction, experience, and feedback into better representations, context, data, and learning strategies, including Physis-Lang.
These directions build on nine representative first-authored papers spanning 4D world understanding and egocentric human–object interaction (HOI4D, CVPR 2022; LeaF, ICCV 2023; PhysReaction, ACM MM 2024 Oral), multimodal representation learning (TupleInfoNCE, ICCV 2021; CrossVideo, ICRA 2024; OmniRetriever, EMNLP 2026 Oral), and efficient multimodal and long-context intelligence (MAP, CVPR 2025; MARC, ICLR 2026; ST-Think, WACV 2026).
Recent Work
Physical RSI
Physis-Lang
Self-evolving physical language for video world model curation, training, and generation.
#1 & #2Physics-IQ
News
- Oct 2026 Released Robota-Embed, the first unified VLA embedding model for robot in-context learning and evolution.
- Sep 2026 Physis-Lang ranked #1 and #2 on the public Physics-IQ Verified leaderboard with Cosmos3-Super and Cosmos3-Nano.
- Sep 2026 MAGIC-Video accepted to NeurIPS 2026.
- Aug 2026 OmniRetriever accepted to EMNLP 2026 Main Conference (Oral).
- Aug 2026 HOI-JEPA accepted to ECCV-HuMoWM 2026.
- Jan 2026 MARC (RL-based Token Compression for VideoLLM) accepted to ICLR 2026.
- Nov 2025 PointNet4D and ST-Think accepted to WACV 2026.
- Aug 2025 OnlineHOI accepted to ACM MM 2025.
- Jul 2025 Culture3D accepted to ICCV 2025.
- Jul 2025 X-LeBench accepted to EMNLP 2025.
- May 2025 MutualNeRF accepted to UAI 2025.
- Feb 2025 MAP and MobileH2R accepted to CVPR 2025.
- Nov 2024 InterHuman accepted to 3DV 2025.
- Jul 2024 PhysReaction accepted to ACM MM 2024 (Oral).
- Feb 2024 Physics-aware HOI Denoising accepted to CVPR 2024.
- Jan 2024 CrossVideo accepted to ICRA 2024.
- Jul 2023 LeaF accepted to ICCV 2023.
- Feb 2023 C2P accepted to CVPR 2023.
- Jul 2022 PPTr accepted to ECCV 2022.
- Feb 2022 HOI4D accepted to CVPR 2022.
- Jul 2021 TupleInfoNCE accepted to ICCV 2021.
Publications
* denotes equal contribution, ^ denotes corresponding author.
Bridging Modalities, Spanning Time: Structured Memory for Ultra-Long Agentic Video Reasoning
Advances in Neural Information Processing Systems (NeurIPS), 2026
Introduces MAGIC-Video, a training-free framework that combines a multimodal memory graph with a narrative memory chain for cross-modal reasoning over videos spanning days to weeks.
OmniRetriever: Any-to-Any Audio-Video-Text Retrieval via Fusion-as-Teacher Distillation
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2026 (Main Conference, Oral Presentation)
Introduces fusion-as-teacher distillation and Tuple-InfoNCE for unified audio-video-text retrieval, surpassing Gemini Embedding 2 on 12-direction retrieval and delivering strong gains on audio-text benchmarks.
MARC: Memory-Augmented RL Token Compression for Efficient Video Understanding
International Conference on Learning Representations (ICLR), 2026
Proposes a reinforcement learning based token compression framework for VideoLLMs that significantly reduces visual token length while preserving performance, enabling more efficient long-video understanding.
ST-Think: How Multimodal Large Language Models Reason About 4D Worlds from Ego-Centric Videos
WACV, 2026
Investigating how multimodal large language models understand and reason about dynamic 4D environments from ego-centric video inputs.
PointNet4D: A Lightweight 4D Point Cloud Video Backbone for Online and Offline Perception in Robotic Applications
WACV, 2026
Proposes PointNet4D, a lightweight hybrid Mamba-Transformer 4D backbone with 4DMAP pretraining for efficient online and offline point cloud video perception in robotics, achieving strong results across 9 tasks and enabling 4D Diffusion Policy and 4D Imitation Learning on RoboTwin and HandoverSim benchmarks.
CULTURE3D: A Large-Scale and Diverse Dataset of Cultural Landmarks and Terrains for Gaussian-Based Scene Rendering
ICCV, 2025
Introduces a 10-billion-point ultra-high-resolution drone dataset of 20 culturally significant landmarks and terrains, enabling large-scale Gaussian-based 3D scene reconstruction and benchmarking.
X-LeBench: A Benchmark for Extremely Long Egocentric Video Understanding
EMNLP (Findings), 2025
Builds a life-logging simulation pipeline and benchmark with 432 ultra-long egocentric video logs (23 minutes–16.4 hours) to stress-test multimodal LLMs on temporal reasoning, memory, and context aggregation.
OnlineHOI: Towards Online Human-Object Interaction Generation and Perception
ACM Multimedia (ACM MM), 2025
Presents an online framework that jointly predicts future human motions and perceives object states for streaming HOI scenarios.
MutualNeRF: Improve the Performance of NeRF under Limited Samples with Mutual Information Theory
Uncertainty in Artificial Intelligence (UAI), 2025
Proposes MutualNeRF, a mutual-information-driven framework that improves NeRF under limited views via MI-guided sparse view sampling and few-shot regularization in both semantic and pixel spaces.
MAP: Unleashing Hybrid Mamba-Transformer Vision Backbone's Potential with Masked Autoregressive Pretraining
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025
Introduces Masked Autoregressive Pretraining that jointly optimizes Mamba and Transformer blocks via local MAE-style reconstruction and global autoregressive generation, yielding state-of-the-art hybrid vision backbones.
MobileH2R: Learning Generalizable Human to Mobile Robot Handover Exclusively from Scalable and Diverse Synthetic Data
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025
Scales 100K+ synthetic handover scenes and distills them through 4D imitation learning into safe, generalizable mobile robot policies that outperform baselines by at least 15% success in sim and real.
Interactive Humanoid: Online Full-Body Motion Reaction Synthesis with Social Affordance Canonicalization and Forecasting
International Conference on 3D Vision (3DV), 2025
Proposed a framework for synthesizing physically plausible humanoid reactions using forward dynamics guidance and social affordance forecasting.
PhysReaction: Physically Plausible Real-Time Humanoid Reaction Synthesis via Forward Dynamics Guided 4D Imitation
ACM Multimedia (ACM MM), 2024 (Oral Presentation)
Leveraging forward dynamics to guide 4D imitation learning for realistic humanoid reactions.
CrossVideo: Self-supervised Cross-modal Contrastive Learning for Point Cloud Video Understanding
IEEE International Conference on Robotics and Automation (ICRA), 2024
Introduces CrossVideo, a self-supervised cross-modal contrastive learning framework for point cloud video understanding.
Physics-aware Hand-object Interaction Denoising
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024
Proposes a physics-aware hand-object interaction denoising framework that leverages physical constraints to improve the accuracy of hand-object interaction predictions.
LeaF: Learning Frames for 4D Point Cloud Sequence Understanding
IEEE/CVF International Conference on Computer Vision (ICCV), 2023
Introduces learnable frame representations to effectively capture temporal dynamics in 4D point cloud sequences.
Complete-to-Partial 4D Distillation for Self-Supervised Point Cloud Sequence Representation Learning
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
Introduces a complete-to-partial 4D distillation framework that leverages self-supervised learning to improve the performance of point cloud sequence representation learning.
Point Primitive Transformer for Long-Term 4D Point Cloud Video Understanding
European Conference on Computer Vision (ECCV), 2022
Introduces a point primitive transformer for long-term 4D point cloud video understanding.
HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022
Releases HOI4D, a large-scale egocentric 4D HOI dataset with category-level annotations that enables learning robust human-object interaction perception and manipulation policies.
Contrastive Multimodal Fusion with TupleInfoNCE
IEEE/CVF International Conference on Computer Vision (ICCV), 2021
Introduces a contrastive multimodal fusion framework that leverages tuple-based contrastive learning to improve the performance of multimodal fusion.
Academic Services
- Conference Reviewer: CVPR, ICCV, ECCV, ACM MM, ICLR, NeurIPS, ICML, 3DV, WACV, AAAI
- Journal Reviewer: TMM
- Workshop Organizer: CVPR 2023 Workshop on 4D Hand Object Interaction