A Generative Foundation Model for Multimodal Histopathology

Abstract

We introduce MUPAD, a generative foundation model designed to integrate histological, molecular, and clinical information across multiple modalities. The model was trained on 100 million histology image patches, 1.6 million text-histology pairs, and 10.8 million RNA-histology pairs spanning 34 human organs. Key achievements include significant improvements in image synthesis quality and the ability to perform cross-modal translation tasks like converting standard staining to immunohistochemistry imaging, demonstrating that a single, unified generative model pretrained across heterogeneous pathology modalities can substantially outperform specialized alternatives.

Jinxi Xiang

Postdoctoral Fellow in Medical AI

My research interests include computer vision and medical image analysis.