About Me

I am an Assistant Professor in the Department of Computer Science at Rice University. Prior to this, I was a Postdoctoral Scholar in the Sky Computing Lab at UC Berkeley working with Prof. Ion Stoica. I earned my Ph.D. in Computer Science from Rice University and my bachelor's degree from Shandong University.

I am seeking self-motivated interns and PhD students in the directions of LLM and agentic systems. If you're interested, feel free to reach out via email ['jxing AT rice.edu'].

Research

My group builds practical and open-source systems for evolving AI workloads (e.g., LLMs and agents) to run efficiently on GPUs. We currently focus on the following directions:

• GPU "OS": We aim to build an efficient and practical GPU "OS" that manages GPU resources for sharing across various AI workloads. This becomes increasingly important as GPUs become larger and workloads become more diverse and dynamic. See our vision paper for more thoughts. Our recent progress includes kvcached Fetching stars... , Prism, and ConServe.
• LLM/Agent routing: We aim to build an open-source community for LLM/Agent routing including evaluation, leaderboards, training, and new routers. Checkout our leaderboard website https://routeworks.github.io/ to understand the current state-of-the-art. To evaluate your routers, refer to our github RouterArena Fetching stars... . Datasets and routers are released on our huggingface. Read our RouterArena paper for more details.
• LLM/Agentic systems: We also work on general LLM/Agentic systems, such as large scale serving (SkyLB Eurosys'26), offline inference (BlendServe ASPLOS'26), code generation (S* EMNLP'25), deterministic inference (TBIK).

Older projects:

• Programmable hardware for systems/security: We leverage programmable switches and SmartNICs to build secure and efficient networked systems. Projects include NetWarden (USENIX Security'20), Ripple (USENIX Security'21), Bedrock (USENIX Security'22), RDMI (USENIX Security'23), FlexCore (NSDI'22), Pipeleon (SIGCOMM'23), Atlas (MobiCom'23).

Group

I am fortunate to work with a group of amazing students. Feel free to email if you are interested in working with us.

Current PhD students

• Xingqi Cui (2025-)
• Rixin Liu (2025-)
• Louie Lu (2025-)

Current graduate/undergraduate RAs

• George Zhang (undergraduate)
• Shen Zhang (undergraduate)