Research pillar
LLM and agentic cloud systems
Serving and scaling LLMs and agents, from efficient reasoning to workload-aware AI infrastructure.
Agentic workloads create systems challenges across both model execution and cloud resource management. This direction studies supporting foundations for efficient reasoning and workload-aware AI infrastructure, treating algorithms and the surrounding cloud stack as a co-design problem rather than optimizing them in isolation.
Current examples include thought pruning for efficient self-consistency scaling, control-plane design that manages workloads rather than treating an AI cluster as a single undifferentiated resource, and ARIES, our open-source experimentation framework for agent serving systems.
Open-source systems
Related publications
FASER: Fine-Grained Phase Management for Speculative Decoding in Dynamic LLM Serving
Wenyan Chen, Chengzhi Lu, Yanying Lin, and Dmitrii Ustiugov
arXiv preprint, 2026
@misc{chen2026faserfinegrainedphasemanagement, title = {FASER: Fine-Grained Phase Management for Speculative Decoding in Dynamic LLM Serving}, author = {Wenyan Chen and Chengzhi Lu and Yanying Lin and Dmitrii Ustiugov}, year = {2026}, eprint = {2604.20503}, archiveprefix = {arXiv}, primaryclass = {cs.DC}, url = {https://arxiv.org/abs/2604.20503} }TokenScale: Timely and Accurate Autoscaling for Disaggregated LLM Serving with Token Velocity
Ruiqi Lai, Hongrui Liu, Chengzhi Lu, Zonghao Liu, Siyu Cao, Siyang Shao, Yixin Zhang, Luo Mai, and Dmitrii Ustiugov
ACM Symposium on Cloud Computing (SoCC), 2026
@inproceedings{lai2026tokenscale, title = {TokenScale: Timely and Accurate Autoscaling for Disaggregated LLM Serving with Token Velocity}, author = {Ruiqi Lai and Hongrui Liu and Chengzhi Lu and Zonghao Liu and Siyu Cao and Siyang Shao and Yixin Zhang and Luo Mai and Dmitrii Ustiugov}, booktitle = {ACM Symposium on Cloud Computing (SoCC)}, year = {2026} }Rethinking AI Cloud Infrastructure for Agentic Serving Systems with the Aries Experimentation Framework
Leonid Kondrashov, Hongrui Liu, JooYoung Park, Boxi Zhou, Zonghao Liu, Chengzhi Lu, Riccardo Mancini, Esha Choukse, Haris Javaid, German Sviridov, Tao Peng, Chen Zhao, Anastasia Avdeeva, Aleksei Gusev, Marios Kogias, Luo Mai, and Dmitrii Ustiugov
ACM Symposium on Cloud Computing (SoCC), 2026
@inproceedings{kondrashov2026rethinkingaicloudinfrastructure, title = {Rethinking AI Cloud Infrastructure for Agentic Serving Systems with the Aries Experimentation Framework}, author = {Leonid Kondrashov and Hongrui Liu and JooYoung Park and Boxi Zhou and Zonghao Liu and Chengzhi Lu and Riccardo Mancini and Esha Choukse and Haris Javaid and German Sviridov and Tao Peng and Chen Zhao and Anastasia Avdeeva and Aleksei Gusev and Marios Kogias and Luo Mai and Dmitrii Ustiugov}, booktitle = {ACM Symposium on Cloud Computing (SoCC)}, year = {2026} }Slim-SC: Thought Pruning for Efficient Scaling with Self-Consistency
Colin Hong, Xinyu Guo, Chaanan Anand Singh, Esha Choukse, and Dmitrii Ustiugov
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2025 · Oral presentation at the main conference
@inproceedings{hong2025slimsc, title = {{Slim-SC}: Thought Pruning for Efficient Scaling with Self-Consistency}, author = {Hong, Colin and Guo, Xinyu and Singh, Chaanan Anand and Choukse, Esha and Ustiugov, Dmitrii}, booktitle = {Conference on Empirical Methods in Natural Language Processing (EMNLP)}, year = {2025} }Manage the Workloads, not the Cluster: Designing a Control Plane for Large-Scale AI Clusters
Ruiqi Lai, Siyu Cao, Long Li, Long Mai, and Dmitrii Ustiugov
EuroMLSys, 2025 · Poster
PDF2 citations@inproceedings{lai2025workloads, title = {Manage the Workloads, not the Cluster: Designing a Control Plane for Large-Scale {AI} Clusters}, author = {Lai, Ruiqi and Cao, Siyu and Li, Long and Mai, Long and Ustiugov, Dmitrii}, booktitle = {EuroMLSys}, year = {2025} }