RL for generative models

MIT-IBM Watson AI Lab · May 2025–May 2026. RL fine-tuning of flow models for molecules and crystals, and the diversity analysis behind Tailor (ACL 2026).

MIT-IBM Watson AI Lab, May 2025 – May 2026. Researcher in Chuang Gan’s group, working on reinforcement learning for generative models.

RL fine-tuning of flow models. Investigated using RL to fine-tune flow-matching models for molecule and crystal generation, where the reward comes from downstream chemical and structural objectives rather than likelihood.
Tailored primitive initialization. Co-author on (Yao et al., 2026), which shows that RL fine-tuning of LLM reasoners depends heavily on the warm-start data it begins from, and proposes Tailor, a method for constructing a warm-start set from tailored reasoning primitives. I built the embedding-similarity pipeline used for the paper’s diversity analysis — the evidence that Tailor’s warm-start data is more reasoning-diverse than the Standard-CoT and 4-STaR baselines.

References

2026

ACL
Tailored primitive initialization is the secret key to reinforcement learning

Y. Yao, G. Zeng, R. Wu, and 4 more authors

In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2026

Abs DOI Bib HTML

Introduces Tailor, a warm-start strategy that initializes RL fine-tuning of LLM reasoners from a tailored set of reasoning primitives. I built the embedding-similarity pipeline used for the paper’s diversity analysis, showing that Tailor’s warm-start data is more reasoning-diverse than Standard-CoT and 4-STaR baselines.
@inproceedings{yao2026tailored, title = {Tailored primitive initialization is the secret key to reinforcement learning}, author = {Yao, Y. and Zeng, G. and Wu, R. and Zhang, Y. and Zhao, D. and Hong, Z.-W. and Gan, C.}, booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)}, pages = {33300--33318}, address = {San Diego, CA}, year = {2026}, doi = {10.18653/v1/2026.acl-long.1537}, }