ICCV2023 TEMPOR The dynamics of biomolecules are crucial for our understanding of their functioning in living systems. However, current 3D imaging techniques, such as cryogenic electron microscopy (cryo-EM), require freezing the sample, which limits the observation of their conformational changes in real time. The innovative liquid-phase electron microscopy (liquid-phase EM) technique allows molecules to be placed in the native liquid environment, providing a unique opportunity to observe their dynamics. In this paper, we propose TEMPOR, a Temporal Electron MicroscoPy Object Reconstruction algorithm for liquid-phase EM that leverages an implicit neural representation (INR) and a dynamical variational auto-encoder (DVAE) to recover time series of molecular structures. We demonstrate its advantages in recovering different motion dynamics from two simulated datasets, 7bcq and Cas9. To our knowledge, our work is the first attempt to directly recover 3D structures of a temporally-varying particle from liquid-phase EM movies. It provides a promising new approach for studying molecules' 3D dynamics in structural biology.
paper [ICCV2023] notebook [Notebook(Chinese)] code [Code]
TEMPOR utilizes a dynamic VAE to reconstruct real-time 3D dynamics of molecules from liquid-phase EM movies. First, movie is transformed into the Fourier domain and passed through the encoder, a temporal inference model, to infer the molecule's latent states. At each moment the latent state depends on the input movie and the previous latent state, and thus captures the low-dimensional dynamics of a molecule's conformational changes. These latent states are then used by the INR-based volume generator to create temporally-varying volumes and corresponding projection images from views estimated by pose estimator. The parameters are jointly optimized by minimizing the difference between input and output images, resulting in a reconstructed time series of a molecule's 3D structures.
We demonstrated the performance of TEMPOR on 7bcq (rigid linear motion) and Cas9 (non-rigid conformational motion) dataset by comparing it with the baseline approach, cryoDRGN. In this section, we assume that poses for each projection is known.
7bcq dataset:
Cas9 dataset:
We also utilized an amortized inference model to infer poses and tested our approach on Cas9 dataset without known poses. In this section, the baseline is HetEM. Our approach, TEMPOR, generates volumes with higher resolution and fewer errors in most cases.
Ye E, Wang Y, Zhang H, et al. Recovering a Molecule's 3D Dynamics from Liquid-phase Electron Microscopy Movies[C]. ICCV 2023: International Conference on Computational Vision. 2023.