Hi there Welcome to my Homepage!

I am a first-year PhD student at MMLab, The University of Hong Kong, supervised by Prof. Xihui Liu and Prof. Ping Luo. I am also a co-founder of Interactive & Active AI (IAAI), which I am building together with friends.

Before joining HKU, I studied Big Data and Management Science at Shanghai Jiao Tong University, where I ranked 1st out of 40 students with a GPA of 93.69/100. During my undergraduate years, I was fortunate to collaborate with Prof. Cheng Hua, Prof. Zhuoran Yang, and Prof. Tao Yao.

News

  • Wrote Don't feed back the last block: a deep intermediate feedback source gives lower loss at the same parameter count and compute.
  • Wrote a blog post, The Benefit of Looping Twice: looping the middle layers twice saves about a third of the parameters for little added compute.
  • Released NextTransformer, an open framework and fair benchmark for next-generation Transformer architectures. Code · Website · Docs
  • Posted Truncate or Not? on arXiv. Project page · arXiv · Code
  • QTPT was accepted to NeurIPS 2026.
  • Started my PhD at HKU MMLab.

Industry Experience

Kling

Research Intern · Advisors: Dr. Ziyuan Huang and Dr. Xintao Wang

Education

The University of Hong Kong

PhD Student at MMLab@HKU

Supervisors: Prof. Xihui Liu and Prof. Ping Luo

Shanghai Jiao Tong University

Bachelor's Program in Big Data and Management Science

Rank 1/40 · GPA 93.69/100 (4.10/4.30)

Selected Publications & Projects

NextTransformer: the Transformer observatory, toward the next generation of Transformers Project 2026

NextTransformer: An Open Framework and Fair Benchmark for Next-Generation Transformer Architectures

One framework and one compute-matched protocol for attention, sparse, linear, looped, residual and hybrid Transformers, with an open observatory of audited results.

Co-Initiator

Project 2026

Overview of looped transformer scaling with truncated backpropagation arXiv 2026

Truncate or Not? A Scaling Study of Looped Transformers with Truncated Backpropagation

Xuyuan Xiong*, Siyu Chen*, Zihan Shao, Yue Su, Weiyang Jin, Ping Luo, Zhuoran Yang, Xihui Liu

* Equal contribution

A unified scaling law over nearly 600 runs favors full BPTT through fewer loops, and two fully trained loops beat a larger non-looped model with fewer parameters.

arXiv preprint, 2026

QTPT and supervised pretraining comparison NeurIPS 2026

From Weak Data to Strong Policy: Q-Targets Enable Provable In-Context Reinforcement Learning

Yichen Lin*, Xuyuan Xiong*, Xue Wang, Xiangfu Meng, Mike Mingcheng Wei, Tao Yao

* Equal contribution

We propose QTPT, which replaces supervised behavior cloning with Bellman Q-target pretraining for in-context reinforcement learning, provably and empirically learning strong policies even from random or suboptimal offline trajectories.

NeurIPS 2026

LithoGRPO inverse lithography illustration ICML 2026

LithoGRPO: Fast Inverse Lithography via GRPO Reinforced Flow Matching

Yao Lai, Xuyuan Xiong, Zeyue Xue, Guojin Chen, Jing Wang, Xihui Liu, Rui Zhang, Robert Mullins, Bei Yu, Ping Luo

LithoGRPO unifies flow matching with GRPO reinforcement learning and a 130× faster shot-counting evaluator to optimize high-fidelity, manufacturable photomasks under physical lithography constraints.

ICML 2026

Grouped Bias Adaptation algorithm illustration ICLR 2026

Taming Polysemanticity in LLMs: Theory-Grounded Feature Recovery via Sparse Autoencoders

Siyu Chen, Heejune Sheen, Xuyuan Xiong, Tianhao Wang, Zhuoran Yang

We formulate polysemanticity as a sparse mixture model and propose Group Bias Adaptation (GBA), the first Sparse Autoencoder training algorithm with provable monosemantic feature recovery guarantees.

ICLR 2026

SPOT and OMDT branching strategy comparison NeurIPS 2025

SPOT: Scalable Policy Optimization with Trees for Markov Decision Processes

Xuyuan Xiong, Pedro Chumpitaz-Flores, Kaixun Hua, Cheng Hua

SPOT formulates decision-tree policy optimization in MDPs as a mixed-integer program with reduced-space parallel branch-and-bound, computing compact, interpretable optimal policies an order of magnitude faster.

NeurIPS 2025

Blog

Feedback from the last block compared with feedback from a deep intermediate block Blog 2026

Don't feed back the last block

In a sweep at two model sizes, a deep intermediate block provides the best feedback source at the same parameter count and compute. We explore why it may retain useful context while being less specialised for the output head.

Nov. 16, 2026 · 10 min read

Honors and Awards

  • Hong Kong PhD Fellowship Scheme (HKPFS)
  • National Scholarship
  • Shaoqiu Scholarship

Skills

Python (PyTorch), C++, Julia, Linux, HTML, LaTeX