Output reading tools and scripts

Python

Environment Setup

To read *.bp with Python you can either set up a Conda environment with ADIOS2
(example on Perlmutter at NERSC)

module load conda
conda init
conda create --name adios2env python=3.[X].[Y]
conda activate adios2env
conda install -c conda-forge numpy scipy matplotlib adios2
python
>>> import adios2

Alternatively, you can set up your own ADIOS2 installation with a Python virtual environment:

module load python/[3.X.Y]
python -m venv [path of your choice]
source [path of your choice]/bin/activate
pip install numpy scipy matplotlib mpi4py

Then, you configure ADIOS2 with -DADIOS2_USE_Python=ON, make and install. In the installation directory of your ADIOS2 configuration, locate the ADIOS2 Python bindings, e.g.,

[path to ADIOS2]/lib/python3.[X]/site-packages/adios2

and soft-link (ln -s) that directory in the package directory of your Python virtual environment, e.g.,

[path of your choice]/lib/python3.[X]/site-packages/

Or you can create a dummy file pyproject.toml

# pyproject.toml - minimal, PEP-621-style
# ---------------------------------------
[build-system]
# Even if you have no native code to build, PEP 517 requires a backend.
# Setuptools + wheel is the lightest pair that pip understands everywhere.
requires      = ["setuptools>=61", "wheel"]      # two tiny wheels
build-backend = "setuptools.build_meta"

[project]
name            = "adios2"                 # whatever users will `pip install`
version         = "2.10.2"
description     = "Adios2 Python Bindings"
requires-python = ">=3.9"

# Pure‑Python runtime dependencies go here.  Because you already compiled
# ADIOS-2 yourself, you just declare that the Python bindings must be
# importable at runtime - pip will not try to build them again.
dependencies = [
  "numpy>=2.2.0"
]

[tool.setuptools.packages.find]                  # <-- one-liner autor-discovers your .py files

in [path to ADIOS2]/lib/python3.[X]/site-packages/adios2 and use

pip install [path to ADIOS2]/lib/python3.[X]/site-packages/adios2

Either method should allow you to import the ADIOS2 Python bindings with

>>> inport adios2

If that does not work, you can try adding the location of the ADIOS2 Python bindings to the environment variable PYTHONPATH:

export PYTHONPATH=${PYTHONPATH}:[path to ADIOS2 Python bindings]

XGC-Analysis Python Package

We provide the (not yet complete) Python package XGC-Analysis for reading and processing XGC data. It is available on Github and will be installable with pip soon.

Documentation for the XGC-Analysis package is available here.

MATLAB

To read *.bp with MATLAB, set the environment variable MATLABPATH to the directory of your ADIOS2 install. In the MATLAB script you can import the data using the functions ADIOSOPEN, ADIOSREAD and ADIOSCLOSE.

file = adiosopen('bp_file_path');
data = adiosread(file,'variable_path_in_bp_file');
adiosclose(file);
% do something with data

Here are instructions for specific machines.

ml PrgEnv-gnu
ml matlab/R2021b
module use -a /global/cfs/cdirs/m499/perlmutter/gcc/modulefiles
ml adios2/devel-nompi
LD_PRELOAD=/opt/cray/pe/gcc/11.2.0/snos/lib64/libstdc++.so matlab

module use -a /home/jongc/sw/modulefiles
module load adios2-nompi/devel
module load matlab/R2022b
matlab