CI Test Reports

CI Test Reports#

Every push runs the small suite as a gate; the large / regression suite runs weekly. The results below cover three groups:

  • Zoomy — the superproject’s own tests/, run per backend.

  • Library — each library/zoomy_* submodule’s own tests/.

  • Notebooks — the published tutorial notebooks, executed end to end.

Summary#

Latest run: 2026-08-23 16:53:14 UTC

Small (every push)#

Zoomy — superproject tests/

Suite

Tests

Failures

Errors

Skipped

Duration (s)

Core / NumPy

112

37 ⚠️

0

25

5.7

JAX

16

0

0

1

114.0

AMReX

12

0

0

12

1.5

OpenFOAM

7

0

0

7

1.1

Library — each submodule’s own tests/

Suite

Tests

Failures

Errors

Skipped

Duration (s)

zoomy_core

217

49 ⚠️

0

1

1195.7

zoomy_jax

80

32 ⚠️

4

0

366.8

zoomy_amrex

0

0

0

0

0.0

zoomy_foam

31

1 ⚠️

0

18

2.1

zoomy_prepost

15

0

0

0

0.8

zoomy_server

9

0

0

0

8.0

Notebooks — published tutorials, executed

Suite

Tests

Failures

Errors

Skipped

Duration (s)

Core notebooks

2

0

0

0

53.1

AMReX notebooks

1

1 ⚠️

0

0

4.4

Large / regression (weekly)#

Zoomy — superproject tests/

Suite

Tests

Failures

Errors

Skipped

Duration (s)

Core / NumPy

1

0

0

1

1.1

JAX

1

0

0

1

1.5

AMReX

1

0

0

1

1.3

OpenFOAM

1

0

0

1

1.1

Library — each submodule’s own tests/

Suite

Tests

Failures

Errors

Skipped

Duration (s)

zoomy_core

6

2 ⚠️

0

0

105.9

zoomy_jax

11

11 ⚠️

0

0

627.8

zoomy_amrex

0

0

0

0

0.0

zoomy_foam

15

0

0

15

0.8

zoomy_prepost

0

0

0

0

0.4

zoomy_server

0

0

0

0

0.0

Notebooks — published tutorials, executed

Suite

Tests

Failures

Errors

Skipped

Duration (s)

Core notebooks

—

—

—

—

—

AMReX notebooks

—

—

—

—

—

A ⚠️ marks a suite with failures. A dash means that suite produced no report in the latest run (a stub is shown in its section below).

Detailed HTML reports#

Embedded pytest-html reports. Scroll horizontally on narrow viewports.

Zoomy — superproject tests/#

Core / NumPy#

Small (every push)

Large / regression (weekly)

JAX#

Small (every push)

Large / regression (weekly)

AMReX#

Small (every push)

Large / regression (weekly)

OpenFOAM#

Small (every push)

Large / regression (weekly)

Library — each submodule’s own tests/#

zoomy_core#

Small (every push)

Large / regression (weekly)

zoomy_jax#

Small (every push)

Large / regression (weekly)

zoomy_amrex#

Small (every push)

Large / regression (weekly)

zoomy_foam#

Small (every push)

Large / regression (weekly)

zoomy_prepost#

Small (every push)

Large / regression (weekly)

zoomy_server#

Small (every push)

Large / regression (weekly)

Notebooks — published tutorials, executed#

Core notebooks#

Small (every push)

Large / regression (weekly)

AMReX notebooks#

Small (every push)

Large / regression (weekly)

How to use this page#

Local build with real reports#

From the repository root:

# one suite (repeat per group/unit you care about)
python tests/reporting/generate_test_report.py \
  --markers "small and core" --pytest-paths "tests" \
  --output-dir artifacts/test-reports/small/zoomy/core
# then assemble + build
python docs/scripts/generate_ci_test_report.py
jupyter-book build docs/book

docs/scripts/generate_ci_test_report.py reads artifacts/test-reports/<tier>/<group>/<unit>/** and writes the summary (docs/_generated/test_report_summary.md), the embeds (docs/_generated/test_report_embeds.md), the per-unit docs/book/_static/pytest-report-<tier>-<group>-<unit>.html, and mirrors the published tutorials into docs/book/tutorials/ipynb/. A stub is written wherever a run produced no report.html.

Adding or changing tests#

Markers are mostly automatic — the root conftest.py applies them:

  • A test under library/zoomy_<stack>/tests (or tests/**/zoomy_<stack>/) gets the <stack> marker from its path (foam → openfoam).

  • A test with no size marker becomes small; anything taking more than 5 minutes individually must be marked large explicitly.

  • library/zoomy_<stack>/tests is collected only when import zoomy_<stack> resolves, so each container runs exactly the suites it can.

A tutorial notebook joins the Notebooks group by being added to tests/notebooks/test_tutorials.py::PUBLISHED (path + the backend marker it needs) and to PUBLISHED_TUTORIALS in the docs generator — it is then executed as a @pytest.mark.notebook test and mirrored onto the site.

Docs-only changes#

Pushing documentation alone still rebuilds the site; Render Webpage re-downloads the latest Smart Tests artifacts from main when it runs.

Architecture#

Scope. CI runs three groups across four backend containers (GHCR images with the checked-out tree pip install -e’d over them, so reports match the commit):

Group

What runs

Where

Zoomy

superproject tests/, filtered per backend

core / jax / amrex / foam images

Library

each library/zoomy_*/tests/

its own backend image (prepost/server in the core image)

Notebooks

tests/notebooks/ (@pytest.mark.notebook)

core (+ amrex) images

Two tiers.

Tier

Runs on

Markers

Gates?

small

every push and PR

small and not regression and …

yes — a failing test fails the build

large

weekly (Sun 03:00 UTC) + manual

(large or benchmark or regression) and …

yes

Chain. Containers publishes the images → Smart Tests (.github/workflows/tests-report.yml) runs both tiers and merges the per-unit HTML/JUnit into test-reports-small-bundle and test-reports-large-bundle → Render Webpage downloads those bundles, runs docs/scripts/generate_ci_test_report.py, and builds this page. Smart Tests checks out with submodules: recursive — library/* are submodules and pip install -e fails without them.

Contents