Xilin Wei

Xilin Wei

Ph.D. Student, Fudan University
Research Intern, Qwen / Alibaba Token Hub, Alibaba Group

I am a third-year Ph.D. student in Artificial Intelligence at the College of Computer Science and Artificial Intelligence, Fudan University, advised by Prof. Dahua Lin.

I am currently a research intern at Qwen / Alibaba Token Hub, Alibaba Group, advised by Shuai Bai and Hang Zhang. Previously, I worked with Jiaqi Wang and Yuhang Zang at Shanghai AI Laboratory.

My research focuses on visual agentic coding, multimodal agents, and efficient reasoning in LLMs and MLLMs. My recent work centers on agents that create and edit websites and 3D content, covering data construction and rollout, realistic evaluation, and user-preference alignment through reward modeling and reinforcement learning (RL). I also study video understanding and implicit chain-of-thought reasoning.

I welcome research collaborations. Feel free to reach out at wiselnn570@gmail.com.

2,488Citations 15h-index 15i10-index

Google Scholar · Updated

🔥 News

  • [2026.10] RefCraft and BlenderWorld are available: evaluating visual agents for reference-conditioned web development and 3D creation.
  • [2026] RNG-Bench is accepted to EMNLP 2026 Main, and EndoCoT to ECCV 2026.
  • [2026.05] I joined Qwen / Alibaba Token Hub, Alibaba Group as a research intern, focusing on webdev coding, evaluation, and user-preference alignment through reward modeling and RL.
  • [2026] SIM-CoT is accepted to ICLR 2026.
  • [2025.05] VideoRoPE is accepted to ICML 2025 as an Oral Presentation.
  • [2024.09] ShareGPT4Video and MMDU are accepted to NeurIPS 2024.

📝 Selected Publications

* denotes equal contribution. See Google Scholar for the full publication list and current citation counts.

RefCraft benchmark overview with visual references and iterative webpage development

RefCraft: Benchmarking Visual-Reference-Conditioned Agentic Generation through Web Development

Yuchi Wang, Rongyao Fang, Weihao Yu, Xilin Wei, Tiancheng Gu, Dingkang Yang, Hang Zhang, Dayiheng Liu, Hongsheng Li, Shuai Bai

arXiv · 2026

A benchmark of 300 realistic webpage-development tasks combining natural-language instructions with one to five visual references. Agents must interpret references, build webpages, and iteratively inspect and refine their outputs, evaluated with verifiable keypoints.

PaperProject

BlenderWorld examples of reference-conditioned 3D creation tasks

BlenderWorld: Benchmarking Visual Agents for 3D Creation from Real-World References

Xinyu Fang, Yuchi Wang, Weihao Yu, Rongyao Fang, Zhaohai Li, Zhixun Li, Tiancheng Gu, Xilin Wei, Yuhang Zang, Hongsheng Li, Dahua Lin, Guofeng Zhang, Shuai Bai, Zhibo Yang, Dayiheng Liu

arXiv · 2026

Evaluating visual agents on 223 realistic Blender creation tasks spanning architecture, products, natural environments, and scientific visualization. Multi-view renders are assessed with 2,651 task-specific rubrics and visual quality scores.

PaperProject

RNG-Bench Matching Pairs and 3D Maze environments for evaluating memory and reasoning

Beyond the Current Observation: Evaluating Multimodal Large Language Models in Controllable Non-Markov Games

Shengyuan Ding*, Xilin Wei*, Xinyu Fang*, Haodong Duan, Dahua Lin, Jiaqi Wang, Yuhang Zang

EMNLP 2026 · Main Conference

RNG-Bench tests whether multimodal models can reconstruct earlier observations and act on them in Matching Pairs and 3D Maze. Its Memory Gap metric separates forgetting from reasoning failures. I led the 3D Maze track and studied the interaction between memory and reasoning.

Paper

SIM-CoT step-level supervision for stable implicit reasoning

SIM-CoT: Supervised Implicit Chain-of-Thought

Xilin Wei, Xiaoran Liu, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Jiaqi Wang, Xipeng Qiu, Dahua Lin

ICLR 2026 · 79 citations

Step-level supervision stabilizes implicit chain-of-thought reasoning and prevents latent states from collapsing as the reasoning budget grows. An auxiliary decoder supplies interpretable supervision during training and is removed at inference, preserving efficiency.

PaperCode

VideoRoPE spatiotemporal rotary position embedding design

VideoRoPE: What Makes for Good Video Rotary Position Embedding?

Xilin Wei*, Xiaoran Liu*, Yuhang Zang, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Jian Tong, Haodong Duan, Qipeng Guo, Jiaqi Wang, Xipeng Qiu, Dahua Lin

ICML 2025 Oral Presentation · 79 citations

A spatiotemporal rotary position embedding framework for video LLMs, combining low-frequency temporal allocation, diagonal layout, and adjustable temporal spacing. VideoRoPE improves long-video understanding, retrieval, and hallucination robustness.

PaperCodeProject

VideoRoPE++ context extrapolation and V-RULER long-video benchmark

VideoRoPE++: Towards Better Video Rotary Position Embedding

Xilin Wei, Xiaoran Liu, Yuhang Zang, Shengyuan Ding, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Qipeng Guo, Jiaqi Wang, Xipeng Qiu, Dahua Lin

Under Review · 2025

Extending VideoRoPE with YaRN-V for training-free context extrapolation and V-RULER for more challenging long-video reasoning. The framework improves long-video retrieval, understanding, and hallucination suppression.

PaperCode

ShareGPT4Video captioning dataset and video understanding framework

ShareGPT4Video: Improving Video Understanding and Generation with Better Captions

Lin Chen*, Xilin Wei*, Jinsong Li*, Xiaoyi Dong*, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Bin Lin, Zhenyu Tang, Li Yuan, Yu Qiao, Dahua Lin, Feng Zhao, Jiaqi Wang

NeurIPS 2024 · 546 citations

High-quality video captions, a versatile video captioner, and the ShareGPT4Video-8B video-language model. Fine-grained captioning improves both video understanding and text-to-video generation across videos of different durations and resolutions.

PaperCodeProject

More Publications & Collaborations

Ongoing Research

  • Co-Design Bench: Multimodal Collaborative Coding Evaluation for Agentic Models — Under Review, 2026. Evaluating multimodal coding agents on real user edit requests from 26,314 agentic-coding sessions, including voice notes, design documents, and screen recordings. A reference-free verifier framework checks task completion using executable runtime assertions against the pre-edit state, with adversarial re-verification in a live browser. Project lead.
  • User Preference Reward Modeling and RL for Web Development — 2026. Developing user preference reward models for web development, improving consistency across candidate orderings and incorporating rendering feedback. Investigating RL for preference alignment and whether reward gains translate into better webpage quality and generalization. Project lead.

💼 Research Experience

Research Intern · Qwen / Alibaba Token Hub, Alibaba Group May 2026 – Present

Hangzhou, China · Advisors: Shuai Bai and Hang Zhang

Working on webdev coding in the Qwen team. I contribute to data construction and trajectory rollout, lead Co-Design Bench and webdev reward model training, and work on user-preference alignment through reward modeling and RL. I also contribute to cowork projects.

Research Intern · Shanghai AI Laboratory Nov 2023 – Apr 2026

Shanghai, China · Advisors: Jiaqi Wang and Yuhang Zang

Member of the InternLM-XComposer and InternVL teams, working on multimodal perception and interaction, efficient model architectures, and multimodal reasoning.

🛠 Selected Projects

  • Qwen3.7–Qwen4.0: webdev coding and multimodal agents. Worked on the data-to-evaluation-to-alignment pipeline, including front-end design data, trajectory rollout, Co-Design Bench, user preference reward modeling, and RL. Qwen
  • InternLM-XComposer2: free-form text-image composition. Responsible for text-writing data construction, including design, cleaning, and quality assurance.
  • InternLM-XComposer2.5-OmniLive: streaming audio-video interaction. Contributed to the design and training of explicit and implicit tasks for real-time multimodal interaction.
  • VLMEvalKit: multimodal model evaluation. Contributed to integrating selected visual question answering (VQA) tasks into the toolkit.
  • WildClawBench: real-world agent evaluation. Contributed to the Social Interaction sub-module, developing tasks involving realistic collaboration and multi-party coordination.
  • Intern-S1-Pro: scientific multimodal foundation model. Contributed to the trillion-parameter model for general and specialized scientific tasks.

📖 Education

  • Ph.D. in Artificial Intelligence · Fudan University, Shanghai, China. Sep 2024 – Present. Advised by Prof. Dahua Lin.
  • B.Eng. in Computer Science · Sichuan University, Chengdu, China. Sep 2020 – Jun 2024. GPA: 3.94/4.0; rank: 1/318.

🎖 Academic Service & Awards

Academic service. Reviewer for ICML, NeurIPS, ICLR, COLM, CVPR, ECCV, ACL, EMNLP, AAAI, IJCAI, and TMLR.

Awards.

  • National Scholarship, 2021–2023.
  • Project Lead, National Undergraduate Innovation and Entrepreneurship Training Program, 2023.