Sketch-to-3D retrieval for generative car design

MIT DeCoDE Lab, January 2025. Researcher in Faez Ahmed’s group on generative AI for engineering design.

The lab was building a multi-agent framework in which LLM-driven agents collaborate on aesthetic and aerodynamic car design (Elrefaie et al., 2025). My piece was the sketch-to-3D retrieval pipeline: given a 2D sketch of a car, retrieve the nearest 3D car mesh from a large dataset so the downstream agents can start from real geometry rather than from scratch.

Built the retrieval pipeline end-to-end (sketch embedding → mesh-view embedding → nearest-neighbour lookup).
Benchmarked a range of retrieval models and similarity metrics to pick the ones that best matched sketches to meshes.

References

2025

ASME IDETC
AI agents in engineering design: A multi-agent framework for aesthetic and aerodynamic car design

M. Elrefaie, J. Qian, R. Wu, and 3 more authors

In Proceedings of the ASME 2025 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC-CIE), Volume 3B: 51st Design Automation Conference, 2025

Abs DOI Bib HTML

A multi-agent framework in which LLM-driven agents collaborate on aesthetic and aerodynamic car design. I built the sketch-to-3D retrieval pipeline (2D sketch to nearest 3D car mesh) and benchmarked retrieval models and similarity metrics.
@inproceedings{elrefaie2025agents, title = {{AI} agents in engineering design: A multi-agent framework for aesthetic and aerodynamic car design}, author = {Elrefaie, M. and Qian, J. and Wu, R. and Chen, Q. and Dai, A. and Ahmed, F.}, booktitle = {Proceedings of the ASME 2025 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC-CIE), Volume 3B: 51st Design Automation Conference}, pages = {V03BT03A048}, address = {Anaheim, CA}, year = {2025}, doi = {10.1115/DETC2025-169682}, }