Hi there Welcome to my Homepage!
I am a first-year PhD student at MMLab, The University of Hong Kong, supervised by Prof. Xihui Liu and Prof. Ping Luo. I am also a co-founder of Interactive & Active AI (IAAI), which I am building together with friends.
Before joining HKU, I studied Big Data and Management Science at Shanghai Jiao Tong University, where I ranked 1st out of 40 students with a GPA of 93.69/100. During my undergraduate years, I was fortunate to collaborate with Prof. Cheng Hua, Prof. Zhuoran Yang, and Prof. Tao Yao.
News
- Wrote Don't feed back the last block: a deep intermediate feedback source gives lower loss at the same parameter count and compute.
- Wrote a blog post, The Benefit of Looping Twice: looping the middle layers twice saves about a third of the parameters for little added compute.
- Released NextTransformer, an open framework and fair benchmark for next-generation Transformer architectures. Code · Website · Docs
- Posted Truncate or Not? on arXiv. Project page · arXiv · Code
- QTPT was accepted to NeurIPS 2026.
- Started my PhD at HKU MMLab.
Industry Experience
Kling
Research Intern · Advisors: Dr. Ziyuan Huang and Dr. Xintao Wang
Education
The University of Hong Kong
PhD Student at MMLab@HKU
Supervisors: Prof. Xihui Liu and Prof. Ping Luo
Shanghai Jiao Tong University
Bachelor's Program in Big Data and Management Science
Rank 1/40 · GPA 93.69/100 (4.10/4.30)
Selected Publications & Projects
Project 2026
NextTransformer: An Open Framework and Fair Benchmark for Next-Generation Transformer Architectures
One framework and one compute-matched protocol for attention, sparse, linear, looped, residual and hybrid Transformers, with an open observatory of audited results.
Co-Initiator
Project 2026
arXiv 2026
Truncate or Not? A Scaling Study of Looped Transformers with Truncated Backpropagation
* Equal contribution
A unified scaling law over nearly 600 runs favors full BPTT through fewer loops, and two fully trained loops beat a larger non-looped model with fewer parameters.
arXiv preprint, 2026
NeurIPS 2026
From Weak Data to Strong Policy: Q-Targets Enable Provable In-Context Reinforcement Learning
* Equal contribution
We propose QTPT, which replaces supervised behavior cloning with Bellman Q-target pretraining for in-context reinforcement learning, provably and empirically learning strong policies even from random or suboptimal offline trajectories.
NeurIPS 2026
ICML 2026
ICLR 2026
Taming Polysemanticity in LLMs: Theory-Grounded Feature Recovery via Sparse Autoencoders
We formulate polysemanticity as a sparse mixture model and propose Group Bias Adaptation (GBA), the first Sparse Autoencoder training algorithm with provable monosemantic feature recovery guarantees.
ICLR 2026
NeurIPS 2025
SPOT: Scalable Policy Optimization with Trees for Markov Decision Processes
SPOT formulates decision-tree policy optimization in MDPs as a mixed-integer program with reduced-space parallel branch-and-bound, computing compact, interpretable optimal policies an order of magnitude faster.
NeurIPS 2025
Blog
Blog 2026
Don't feed back the last block
In a sweep at two model sizes, a deep intermediate block provides the best feedback source at the same parameter count and compute. We explore why it may retain useful context while being less specialised for the output head.
Nov. 16, 2026 · 10 min read
Blog 2026
The Benefit of Looping Twice: Parameter Savings with Little Added Compute
Looping the middle layers of a transformer twice reaches the same loss with about a third fewer parameters for a few percent more training compute, in dense and mixture-of-experts models; a scaling law fitted to nearly 600 runs shows why two loops are the sweet spot.
Nov. 2, 2026 · 15 min read
Honors and Awards
- Hong Kong PhD Fellowship Scheme (HKPFS)
- National Scholarship
- Shaoqiu Scholarship
Skills
Python (PyTorch), C++, Julia, Linux, HTML, LaTeX