monoprop

Building from Source

Build the Python bindings and the C++ unit-test tree, with or without MPI.

monoprop has one supported from-source build workflow:

  • the Python bindings — the nanobind extension behind import monoprop, built with scikit-build-core and driven by uv (or pip);
  • the C++ unit tests — built within the same uv (or pip) invocation.

MPI is off by default in every build path; you enable it explicitly. The mechanism differs by build:

BuildEnable MPI with
Python bindings and C++ tests (scikit-build / uv / pip)monoprop_ENABLE_MPI=ON in the environment

The prebuilt wheels published to PyPI (pip install monoprop) are also built without MPI, so a from-source build is required for multi-rank runs.

Prerequisites

  • a C++23-compliant compiler; on Linux the minimum supported versions are GCC 14 and Clang 18
  • CMake and Ninja
  • Python 3.11 or newer and the uv package manager (for the bindings)
  • an MPI implementation such as Open MPI (only for MPI builds)
  • hwloc (version 2.9+) and pkg-config (required so CMake can locate hwloc)

The repository ships a DevContainer with all of the above pre-configured; opening the folder in VS Code and rebuilding the container is the quickest route to a working environment.

Building with Nix

The repository is also a Nix flake:

CommandWhat it gives you
nix developa shell with every prerequisite above, plus uv, just and Node.js
nix build .#monopropthe Python package, built without MPI
nix build .#monoprop-mpithe same package with monoprop_ENABLE_MPI=ON
nix runa Python interpreter with monoprop importable

Inside nix develop the uv sync and just workflows below apply unchanged. The shell sets two variables that only matter on NixOS: UV_PYTHON_PREFERENCE=only-system, because uv's managed interpreters expect a loader NixOS does not provide, and LD_LIBRARY_PATH, so that manylinux wheels can resolve libstdc++.

The packaged build deviates from the uv build in three places, all in nix/monoprop.nix:

  • the C++ unit tests are disabled, because they resolve msgpack-cxx through a git fetch that the build sandbox denies — build them from the dev shell instead;
  • monoprop_ENABLE_ARCH_FLAGS is off, since a store path may be substituted onto a machine other than the one that built it; pass .override { enableArchFlags = true; } for a native build;
  • the version comes from the tracked root VERSION file, because setuptools-scm cannot read git metadata inside the sandbox; update it to the stable version when preparing a release. The release workflow rejects tags that do not match it.

Using monoprop downstream with Nix

Use the exported overlay when monoprop must share a downstream project's Python and native dependency package set. Following the downstream nixpkgs input avoids introducing a second Python, Boost, hwloc, or MPI closure:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    monoprop = {
      url = "github:Algorithmiq/monoprop";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = { nixpkgs, monoprop, ... }:
    let
      system = "x86_64-linux";
      pkgs = import nixpkgs {
        inherit system;
        overlays = [ monoprop.overlays.default ];
      };
      python = pkgs.python312.withPackages (ps: [
        pkgs.monoprop
        ps.numpy
      ]);
    in {
      devShells.${system}.default = pkgs.mkShell {
        packages = [ python ];
      };
    };
}

The overlay provides pkgs.monoprop and pkgs.monoprop-mpi. For consumers that do not need to compose package sets, the pre-built flake outputs remain available as monoprop.packages.${system}.monoprop and monoprop.packages.${system}.monoprop-mpi. Consume these entrypoints from the Git repository flake; flake.nix, flake.lock, and nix/ are deliberately excluded from the Python source distribution.

Building the Python bindings

uv creates a virtual environment, installs the Python dependencies, and compiles the nanobind extension in editable mode. Re-run the sync command whenever the dependency graph or the C++ sources change.

Without MPI (default)

uv sync --all-extras -v

This produces a single-process build with no MPI dependency.

With MPI

Set monoprop_ENABLE_MPI=ON in the environment:

monoprop_ENABLE_MPI=ON uv sync --all-extras -v

It works the same with pip when installing from a checkout:

monoprop_ENABLE_MPI=ON pip install .

That one variable does two things, matched by the [[tool.scikit-build.overrides]] block in pyproject.toml: it sets the monoprop_ENABLE_MPI CMake option, and it adds mpi4py to the build requirements. mpi4py is a build-time input — the bindings need its headers to hand an MPI communicator across the nanobind boundary — but only for MPI builds, so it is injected per-build through PEP 517's get_requires_for_build_wheel rather than sitting in build-system.requires where every build would pay for it.

Because it is an environment variable, it does not participate in uv's build cache key. Add --reinstall-package monoprop --no-cache when flipping MPI on or off in an existing environment, or uv may reuse the previous build.

Driving it with config-settings instead

A scikit-build-core override can only match environment variables, never config-settings, so cmake.define alone will not pull in mpi4py — the build would fail at configure time with a message telling you as much. Pass both settings to use this route:

uv sync --all-extras -v \
    --config-settings=cmake.define.monoprop_ENABLE_MPI=ON \
    --config-settings=build.requires=mpi4py>=4.1.0

In a workspace, prefer the per-package form so the settings do not reach the sibling distributions in packages/:

uv sync --all-extras -v \
    --config-settings-package="monoprop:cmake.define.monoprop_ENABLE_MPI=ON" \
    --config-settings-package="monoprop:build.requires=mpi4py>=4.1.0"

The cost of this route is that the mpi4py pin is repeated on the command line instead of living once in pyproject.toml; the environment switch is preferred for that reason.

Verify the install

uv run python -c "import monoprop as mp; print(mp.__version__)"

Running the bindings

A serial run is just a normal Python invocation:

uv run python your_script.py

For a multi-rank run, launch the same script under mpiexec (requires an MPI build) and pass comm=MPI.COMM_WORLD to the simulator:

mpiexec -n 8 uv run python your_script.py

See Parallelism and distribution for the communicator options and the operator-partitioning controls.

Building the C++ unit tests

The supported C++ workflow reuses the build tree produced by uv sync. Do not run cmake --preset ... to configure this project directly: the top-level CMake configuration expects scikit-build-core to provide Python, nanobind, and related cache variables. Instead, first create the tree with uv sync, then invoke ctest directly to run the C++ unit tests.

Release tree

uv sync --all-extras -v
ctest --test-dir build/editable/Release

This uses the scikit-build-core Release tree at build/editable/Release and runs bin/monoprop_unit_tests.x there, along with bin/monoprop_link_export_probe.x — a link-time check that the installed shared monoprop library exports every detail/** free function reachable from the public template chain. The installed CMake target records whether monoprop itself was built with MPI; a serial package does not acquire MPI compile definitions or link dependencies merely because the consuming project already has an MPI::MPI_CXX target.

just test-find-package goes one step further out: it builds cpp/tests/find_package_smoke — a standalone project that consumes the installed package through find_package(monoprop CONFIG) — and runs it. That is the only leg that exercises monopropConfig.cmake, so it is what catches a missing find_dependency or a usage requirement the package fails to export. It compiles the same source as the in-tree probe, so the two cannot drift.

Debug tree

uv sync --all-extras -v --config-settings=cmake.build-type=Debug
ctest --test-dir build/editable/Debug

Sanitizer trees

The QA workflow builds two sanitizer profiles, selected with monoprop_SANITIZER (none, asan-ubsan, or tsan). Both require Linux and either GCC or Clang; CI uses the default system GCC, which is what builds the released wheels.

Sanitizer trees intentionally skip the _core.pyi typing stubs — generating them means importing the instrumented extension, which is not worth arranging for a tree that is never packaged.

ASan + LSan + UBSan

SKBUILD_CMAKE_BUILD_TYPE=AsanUbsan \
SKBUILD_CMAKE_DEFINE="monoprop_SANITIZER=asan-ubsan" \
uv sync --group workspace-test --all-extras --reinstall-package monoprop --no-cache -v

--reinstall-package monoprop --no-cache matters locally. uv decides whether to rebuild from the cache-keys in pyproject.toml, which hash source files — not environment variables. So if you already have an editable install, switching only the build type leaves the sources unchanged, uv sync does nothing, and build/editable/AsanUbsan is never created (the ctest below then fails with "No such file or directory"). CI omits these flags because each job starts on a clean runner with nothing installed.

The C++ test binary is fully instrumented, so every check applies to it, including leak detection:

just test-cpp-asan

The Python tests are a different situation: an instrumented _core is loaded into an ordinary CPython, so the ASan runtime must be preloaded, and the checks that assume the whole process is instrumented have to be switched off.

just test-py-asan

Both preloads are required. Without libasan.so, the first import monoprop aborts with ASan runtime does not come first in initial library list. Without libstdc++.so.6, ASan initialises before libstdc++ is loaded — python is not linked against it — so ASan's __cxa_throw interceptor never resolves the real symbol, and the first C++ exception thrown out of the engine dies with CHECK failed: real___cxa_throw != 0. monoprop reports validation errors as C++ exceptions, so that affects a large part of the suite.

Leak detection is off here because LSan cannot get a clean baseline from an uninstrumented interpreter — the C++ leg above is what covers leaks.

Sanitizer reports are hidden by pytest

pytest replaces the stderr file descriptor, so a sanitizer report printed from native code is swallowed and the run looks like a bare exit code 1 with no diagnostic. just test-py-asan therefore sets log_path, routing the reports to sanitizer-log.*; read them with:

just sanitizer-reports

The QA workflow runs that recipe on failure.

ThreadSanitizer

SKBUILD_CMAKE_BUILD_TYPE=Tsan \
SKBUILD_CMAKE_DEFINE="monoprop_SANITIZER=tsan" \
uv sync --group workspace-test --all-extras --reinstall-package monoprop --no-cache -v

just test-cpp-tsan

TSan is scoped to the concurrency surface — the partition and ShmComm tests — which is where its reports are worth the runtime cost.

There is no ThreadSanitizer equivalent of the ASan Python leg. TSan does not support dlopen of instrumented libraries, so loading an instrumented _core into a stock interpreter crashes during TSan initialisation; exercising the Python API under TSan would require a TSan-instrumented CPython.

  • Use just code-coverage for the coverage build.
  • Use just test-cpp and just test-cpp-mpi for the two C++ legs, or ctest --test-dir build/editable/Release -L mpi-2 to filter further.
  • Use just build [uv sync args…] when you want the build CI performs. It adds --all-extras and, for a serial build, drops the mpi extra — importing mpi4py.MPI without an MPI runtime raises. monoprop_ENABLE_MPI drives all of it, so monoprop_ENABLE_MPI=ON just build --group workspace-test is the MPI build and a bare just build is the serial one.
  • The platform packages the build needs are listed in tools/packages/, one per line: apt.txt / brew.txt, plus apt-mpi.txt / brew-mpi.txt for an MPI build. .github/actions/setup and .devcontainer/Dockerfile both install from those files, so there is one list per platform to keep current.

See also

On this page