Building Variorum

Variorum can be built from source with CMake or with spack. Building Variorum creates the libvariorum library, the powmon monitoring tool, and Variorum examples.

Build Dependencies

The build dependencies for a minimal build with no parallel components require the following:

• C

• hwloc

• jansson

• Access to vendor-specific drivers or modules (see Supported Platform Documentation).

For a parallel build with MPI/OpenMP, Variorum depends on:

• rankstr (only required for MPI/OpenMP builds)

The CMake variables (ENABLE_MPI={ON,OFF} and ENABLE_OPENMP={ON,OFF}) control the building of parallel examples. If ENABLE_MPI=ON, an MPI compiler is required.

hwloc (Required)

hwloc is an open-source project providing a portable abstraction of the hierarchical topology of modern architectures.

Variorum leverages hwloc for detecting hardware topology. When reading/writing a register on a particular hardware thread, hwloc can map that to the correct physical socket.

jansson (Required)

jansson is an open-source C library for encoding, decoding and manipulating JSON data.

Variorum leverages JANSSON to provide a JSON-based API that can retrieve power data for external tools/software.

rankstr (Optional)

rankstr is an open-source C library providing functions that identify unique strings across an MPI communicator.

Variorum leverages rankstr to split a communicator into subcommunicators by hostname. The allows for a single control or monitor process in Variorum to for example, enforce a power or frequency limit on a node or to print the hardware counters once on a node.

Building with CMake

Variorum can be built and installed as follows after cloning from GitHub:

# install hwloc and jansson dependencies
sudo apt-get install libhwloc15 libhwloc-dev libjansson4 libjansson-dev

git clone https://github.com/llnl/variorum

cd variorum
mkdir build install
cd build

cmake -DCMAKE_INSTALL_PREFIX=../install ../src
make -j8
make install

Host Config Files

To handle build options, third party library paths, etc., we rely on CMake’s initial-cache file mechanism. We call these initial-cache files host-config files, as we typically create a file for each platform or specific hosts if necessary. These can be passed to CMake via the -C command line option as shown below:

cmake {other options} -C ../host-configs/{config_file}.cmake ../src

An example is provided in host-configs/boilerplate.cmake to create your own configuration file. Example configuration files named by machine hostname, the SYS_TYPE environment variable, and platform name (via uname) are also provided in the host-configs directory. These files use standard CMake commands. CMake set commands need to specify the root cache path as follows:

set(CMAKE_VARIABLE_NAME {VALUE} CACHE PATH "")

CMake Build Options

Variorum’s build system supports the following CMake options:

• HWLOC_DIR - Path to an HWLOC install.

• JANSSON_DIR - Path to a JANSSON install.

• SPHINX_EXECUTABLE - Path to sphinx-build binary (required for documentation).

• VARIORUM_WITH_AMD_CPU (default=OFF) - Enable Variorum build for AMD CPU architecture.

• VARIORUM_WITH_NVIDIA_GPU (default=OFF) - Enable Variorum build for Nvidia GPU architecture.

• VARIORUM_WITH_IBM_CPU (default=OFF) - Enable Variorum build for IBM CPU architecture.

• VARIORUM_WITH_ARM_CPU (default=OFF) - Enable Variorum build for ARM CPU architecture.

• VARIORUM_WITH_INTEL_CPU (default=ON) - Enable Variorum build for Intel CPU architecture.

• VARIORUM_WITH_INTEL_GPU (default=OFF) - Enable Variorum build for Intel discrete GPU architecture.

• ENABLE_FORTRAN (default=ON) - Enable Fortran compiler for building example integration with Fortran application, Fortran compiler must exist.

• ENABLE_PYTHON (default=ON) - Enable Python wrappers for adding PyVariorum examples.

• ENABLE_MPI (default=OFF) - Enable MPI compiler for building MPI examples, MPI compiler must exist.

• ENABLE_OPENMP (default=ON) - Enable OpenMP extensions for building OpenMP examples.

• ENABLE_WARNINGS (default=OFF) - Build with compiler warning flags -Wall -Wextra -Werror, used primarily by developers.

• BUILD_DOCS (default=ON) - Controls if the Variorum documentation is built (when sphinx and doxygen are found).

• BUILD_SHARED_LIBS (default=ON) - Controls if shared (ON) or static (OFF) libraries are built.

• BUILD_TESTS (default=ON) - Controls if unit tests are built.

• VARIORUM_DEBUG (default=OFF) - Enable Variorum debug statements, useful if values are not translating correctly.

• USE_MSR_SAFE_BEFORE_1_5_0 (default=OFF) - Use msr-safe prior to v1.5.0, dependency of Intel architectures for accessing counters from userspace.

Building on Multiple Architectures

Several HPC systems have compute nodes that are composed of components or hardware from different vendors. An example is the Sierra supercomputer at LLNL, where the CPUs are based on IBM platform (Power9), and the GPUs are from NVIDIA (Volta). Because power and energy dials are vendor-specific, we need to enable a multi-architecture build for such platforms. As of version 0.7, we have added support for such builds within Variorum. These can now be specified with turning the CMake options for the associated devices, for example. VARIORUM_WITH_NVIDIA_GPU and VARIORUM_WITH_IBM_CPU can both be set to ON during the same build.

Support for multi-architecture builds is still new and limited, especially with the JSON-based APIs. We are working toward adding more features in the near future.

Building with Spack

To install Variorum with all options (and also build all of its dependencies as necessary) run:

spack install variorum

The Variorum spack package provides several variants that customize the options and dependencies used to build Variorum (see table below). Variants are enabled using + and disabled using ~.

Variant	Description	Default
shared	Build Variorum as shared library	ON (+shared)
docs	Build Variorum’s Documentation	OFF (~docs)
log	Enable Variorum’s logging infrastructure	OFF (~log)
build_type	Specify build type	Release with Debug Info (build_type=RelWithDebugInfo)

Debugging

Setting the VARIORUM_LOG environment variable at runtime to VARIORUM_LOG=1 will print out debugging information.