Talk Title
From Histology to Spatial Tumor Ecosystems: Multimodal AI for Cancer Analysis
Abstract
Routine H&E histology is among the most scalable windows into cancer, preserving tissue architecture across large retrospective cohorts. However, molecular states, cellular programs, and cell-cell interactions are only indirectly encoded in tissue morphology. Spatial transcriptomics can reveal these programs but remains limited in scale, cost, and availability. In this keynote, I will present our work toward multimodal computational pathology systems that connect histology with molecular and clinical information. I will first briefly trace a research trajectory from efficient whole-slide navigation and biomarker prediction to vision-language pathology reporting and multimodal foundation representations. I will then focus on Phoenix, a pan-cancer framework for virtual spatial transcriptomics from routine H&E images. Phoenix infers spatially resolved gene-expression programs and cell-state distributions, allowing archived histology cohorts to be studied as virtual spatial atlases. I will show how this framework can recover established cancer biology, characterize recurrent tumor-microenvironment ecotypes, and reveal spatial interactions associated with treatment response - signals that may be missed by slide-level prediction or cell-abundance analysis alone. Finally, I will discuss the challenges that determine whether virtual molecular profiling becomes scientifically and clinically useful: generalization across cohorts, organs, staining protocols, and gene panels; validation against experimental spatial measurements; interpretability of morphology-molecular associations; and the distinction between biologically supported inference and unsupported molecular predictions. Together, these efforts aim to move computational pathology from isolated prediction tasks toward reliable multimodal models of tumor ecosystems.