Machine Learning Techniques for Data Reduction/Compression

This work was partially supported by DOE RAPIDS2 DE-SC0021320 and DOE DE-SC0022265

Scientific simulations generate data at scales that challenge storage, communication, and analysis. My group develops machine learning and hybrid compression methods that reduce scientific data while controlling reconstruction error and preserving downstream quantities of interest.

Our methods combine learned representations with algorithms that enforce constraints on the reconstructed data. This includes guaranteed and constrained autoencoders, attention-based models, and scalable hybrid pipelines. We evaluate these methods on data from fusion, climate, and computational fluid dynamics simulations.

Recent work extends these methods to foundation and generative models for spatiotemporal scientific data, including CAESAR and generative latent diffusion. The research studies how learned representations and correction methods can provide useful reduction while meeting specified scientific error requirements.

The work is supported by a continuing program of DOE research, including RAPIDS2 and RAPIDS3. The coauthored book Machine Learning Methods for Scientific Data Compression is listed in the July 2026 vitae as forthcoming from CRC Press in 2026.

Selected Publications

  1. T. Banerjee, J. Choi, J. Lee, Q. Gong, J. Chen, S. Klasky, A. Rangarajan, and S. Ranka, Scalable hybrid learning techniques for scientific data compression. IEEE Transactions on Parallel and Distributed Systems, 37(1), 2026, pp. 29–44.
  2. X. Li, L. Zhu, J. Lee, R. Sengupta, S. Klasky, S. Ranka, and A. Rangarajan, CAESAR: A Unified Framework for Foundation and Generative Models for Efficient Compression of Scientific Data. Applied Sciences, 15(16), 2025, Art. no. 8977.
  3. Q. Gong, M. Ainsworth, J. Chen, X. Liang, L. Zhu, E. Klasky, T. Athawale, Q. Liu, S. Klasky, S. Ranka, and A. Rangarajan, Stability-preserving Lossy Compression for Large-scale Partial Differential Equations. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC), 2025, pp. 1992–2005.
  4. X. Li, L. Zhu, A. Rangarajan, and S. Ranka, Generative Latent Diffusion for Efficient Spatiotemporal Data Reduction. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC), 2025, pp. 1980–1991.
  5. X. Li, J. Lee, A. Rangarajan, and S. Ranka, Foundation Model for Lossy Compression of Spatiotemporal Scientific Data. Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD), 2025, pp. 368-380.
  6. X. Li, J. Lee, A. Rangarajan, and S. Ranka, Attention Based Machine Learning Methods for Data Reduction with Guaranteed Error Bounds. 2024 IEEE International Conference on Big Data (BigData), 2024, pp. 1039-1048.
  7. J. Lee, A. Rangarajan, and S. Ranka. Guaranteeing Error Bounds with Preservation of Derived Quantities in Compressive Autoencoders. Proceedings of DCC 2024, pp. 567.
  8. J. Lee, A. Rangarajan, and S. Ranka. Nonlinear-by-Linear: Guaranteeing Error Bounds in Compressive Autoencoders, Proceedings of 2023 International Conference on Contemporary Computing, 2023, pp. 552–561.
  9. T. Banerjee, J. Lee, J. Choi, Q. Gong, J. Chen, CS. Chang, S. Klasky, A. Rangarajan, and S. Ranka. Online and Scalable Data Compression Pipeline with Guarantees on Quantities of Interest. Proceedings of e-Science 2023, pp. 1–10.