Projects (selected)

• cuSZ-I: High-Fidelity Error-Bounded Lossy Compression for Scientific Data on GPUs [Paper Link] [Code Link]
  • We develop a GPU-customized interpolation-based data predictor G-interp with highly parallelized efficient interpolation, which can present excellent data prediction accuracy, thus leading to a high overall compression ratio.
  • We design and implement a highly lightweight interpolation auto-tuning kernel for GPU interpolation to optimize both performance and compression quality of cuSZ-I.
  • We improve the implementation of GPU-based Huffman encoding and import a new lossless module NVIDIA Bitcomp to further reduce its encoding redundancy.
  • cuSZ-I improves compression ratio over other state-of-the-art GPU-based scientific lossy compressors extremely, by up to about 500% under the same error bound or PSNR. Meanwhile, it preserves a compression throughput of the same magnitude as other compressors.


• SRN-SZ: Scientific Error-bounded Lossy Compression with Super-resolution Neural Networks [Paper Link]
  • We propose a scientific error-bounded lossy compressor SRN-SZ, in which the compression is performed with a hybrid of super-resolution networks and interpolations.
  • Leveraging the Hybrid Attention Transformer (HAT) network, we designed a specialized training pipeline with several adaptive techniques to optimize the super-resolution quality of scientific data.
  • According to the experimental results, SRN-SZ has achieved up to 75% compression ratio improvements under the same error bound and up to 80% compression ratio improvements under the same PSNR.


• QoZ 2.0 (HPEZ): High-performance Effective Scientific Error-bounded Lossy Compression with Auto-tuned Multi-component Interpolation [Paper Link] [Code Link]
  • Founded on theoretical analysis and algorithmic optimizations, QoZ 2.0 substantially upgrades the most critical step in the quality-oriented compression -- interpolation prediction, leading to an immensely improved data prediction accuracy.
  • QoZ 2.0 develops a series of optimization strategies including block-wise interpolation tuning, dynamic dimension freezing, and Lorenzo tuning, which can substantially improve the adaptability of the auto-tuning for the compression across a broad spectrum of inputs.
  • QoZ 2.0 significantly outperforms state-of-the-art error-bounded lossy compressors in terms of rate-distortion, while still having a satisfactory speed. It preserves a leading speed compared to other high-ratio compressors. Consequently, it achieves the best throughput in distributed data transfer over WAN based on our experiments. QoZ 2.0 exhibits the least time cost in data transfer for most scientific datasets with up to 40% time reduction.


• FAZ: A flexible auto-tuned modular error-bounded compression framework for scientific data [Paper Link]
  • FAZ is a flexible and adaptive error-bounded lossy compression framework, which projects a fairly high capability of adapting to diverse datasets.
  • FAZ contains different compression modules which having same or similar functionalities, and dynamically selects and auto-tunes the best-fit modules in online compression to establish the optimized compression pipeline for each input data.
  • After careful selection of compression techniques and design of pipeline auto-tuning methods, the current version of FAZ is a hybrid-framework leveraing both prediction-based and transform-based compression techniques.
  • Experiments show that, compared to the second best compressor, FAZ can improve the compression ratio by up to 120%, 190%, and 75% when setting the same error bound, the same PSNR and the same SSIM, respectively.


• QoZ: Dynamic Quality Metric Oriented Error Bounded Lossy Compression for Scientific Datasets [Paper Link] [Code Link]
  • QoZ is the first dynamic quality-metric-oriented scientific error-bounded lossy compressor, which can online optimize its compression according to user-specified preferred quality metrics (e.g. PSNR, SSIM, Autocorrelation).
  • Based on SZ3 interpolation predictor, QoZ develops and leverages an advanced interpolation-based predictor with high flexibility for auto-tuning, and an effective quality-metric driven auto-tuning algorithm.
  • Compared with the second best lossy compressor, QoZ can achieve up to 70% compression ratio improvement under the same error bound, up to 150% compression ratio improvement under the same PSNR, or up to 270% compression ratio improvement under the same SSIM.


• AE-SZ: Exploring Autoencoder-based Error-bounded Compression for Scientific Data [Paper Link]
  • AE-SZ is a deep-learning (autoencoder) based scientific data lossy compressor, which is also a pioneering practice in its time of leveraging deep learning techniques in scientific data compression.
  • Like traditional lossy compressors such as SZ, AE-SZ has full support of error-bounding requirements.
  • SWAE (Sliced-Wasserstein Autoencoder) and GDN (Generalized Divisive Normalization) is utilized in AE-SZ for generating effective compressed representation of the original input data. The compressed latent vectors are further quantized to maximally improve the compression ratio.
  • Based on SZ3 interpolation predictor, QoZ develops and leverages an advanced interpolation-based predictor with high flexibility for auto-tuning, and an effective quality-metric driven auto-tuning algorithm.
  • AE-SZ can obtain a much better compression quality (100%-800% improvement in compression ratio with the same data distortion) compared with SZ2.1 and ZFP in cases with high compression ratios.