Output reading tools and scripts¶
MATLAB¶
To read *.bp with MATLAB, set the environment variable MATLABPATH to the directory of your ADIOS2 install.
In the MATLAB script you can import the data using the functions ADIOSOPEN, ADIOSREAD and ADIOSCLOSE.
file = adiosopen('bp_file_path');
data = adiosread(file,'variable_path_in_bp_file');
adiosclose(file);
% do something with data
Here are instructions for specific machines.
Cori¶
module swap PrgEnv-intel PrgEnv-gnu
module load matlab/R2020b
module use -a /project/projectdirs/m499/Software/modulefiles
module load adios2/DEFAULT-nompi
LD_PRELOAD=/opt/gcc/11.2.0/snos/lib64/libstdc++.so.6 matlab
Perlmutter¶
ml PrgEnv-gnu
ml matlab/R2021b
module use -a /global/cfs/cdirs/m499/perlmutter/gcc/modulefiles
ml adios2/devel-nompi
LD_PRELOAD=/opt/cray/pe/gcc/11.2.0/snos/lib64/libstdc++.so matlab
Stellar¶
module use -a /home/jongc/sw/modulefiles
module load adios2-nompi/devel
module load matlab/R2022b
matlab
Example scripts (to come?)¶
Python¶
To read *.bp with Python you can either set up a Conda environment with ADIOS2
(example on Cori NERSC)
module load python
conda init
conda create --name adios2env python=3.7
conda activate adios2env
conda install -c conda-forge numpy scipy matplotlib adios2
python
>>> import adios2
or you need an ADIOS2 install configured with the flag -DADIOS2_USE_Python=ON.
This ADIOS2 setting typically only works with the GNU compiler, so if you run XGC with ADIOS2 built using another compiler you may have to make a separate build of ADIOS2 for reading in Python.
To read ADIOS2 data in Python you can set the environment variable PYTHONPATH and use import.
export PYTHONPATH=/PATH_TO_INSTALL/adios2
import adios2
However, if you have separate ADIOS2 installs for running XGC and reading ADIOS2 data in Python there can be a conflict. The following works on Traverse where XGC is built with the nvhpc (NVIDIA) Compiler and we have a separate ADIOS2 build with the GNU compiler for Python.
import os
if os.environ['XGC_PLATFORM'].lower() == 'traverse':
from ctypes import *
cdll.LoadLibrary("PATH_TO_INSTALL/adios2-gcc/lib64/libadios2_core.so.2")
import adios2
To read data in Python it is possible to do:
with adios2.open("bp_file_path", "r") as fh:
data = fh.read("variable_path_in_bp_file")
The read data will typically be a Numpy array with the corresponding dimensions to the XGC variable. However, depending on how the variable is stored in XGC it is sometimes necessary to read it by stepping to obtain all data:
import numpy as np
import adios2
data_list = []
with adios2.open("bp_file_path", "r") as fh:
for fstep in fh:
data_list.append(fstep.read("variable_path_in_bp_file"))
data = np.array(data_list)