WarpX is designed to study kinetic effects and relativistic physics from desktop simulations to the latest Exascale supercomputers and was awarded the 2022 Gordon Bell Prize. WarpX is primarily implemented in modern C++ (currently C++17) with extensive bindings to Python. WarpX relies on the C++ performance portability layer in AMReX #5 to implement first-class support for NVIDIA, AMD, and Intel GPUs as well as CPUs, relying on multi-level parallelism (MPI+OpenMP/CUDA/ROCm/SYCL).
WarpX has by now over 100 contributors from research labs, academia and private industry, located in North America, Europe and beyond. WarpX is the flagship code of the Beam, Plasma & Accelerator Simulation Toolkit (BLAST) and its inclusion in HPSF will potentially serve as a blueprint for other ("smaller") BLAST codes to follow later on. BLAST codes are widely used by the community (institutional usage map) and growing rapidly. WarpX development is sustained by a diverse set of funding structures from the US, Europe, subcontracts and private in-kind contributions.
3. Statement on Alignment with High Performance Software Foundation's Mission
Like its predecessor Warp, WarpX has bee committed to open science and thus implements a fully open, community-based development model since its inception in the US DOE Exascale Computing Project. In WarpX, a large set of software technology comes together and needs to work at the same time to solve a science problem, which makes collaborations core to our strategy. WarpX already aims to target users beyond the HPC scope by actively deploying also for desktops on macOS, Linux and Windows platforms, attracting users at all scales and at the same time shipping its application (and all of its dependencies!) to a diverse set of target platforms (desktop/edge, cloud, HPC).
10. Please list all external dependencies and their license
WarpX itself is open-source under BSD-3-Clause-LBNL license: LICENSE.txt. Please see the notices in NOTICE.txt.
WarpX depends on dozens of direct and hundreds of indirect dependencies, all under compatible licenses. WarpX prefers OSI-approved dependencies, but also depends on other licenses where unavoidable (e.g., Nvidia CUDA).
This, combined with an MPI implementation can be used for many routines, benchmarking or development on new exotic architectures. In practice, one wants more physics modules, accelerated computing (e.g., CUDA), accelerated math libraries (on-device FFTs and on-device linear algebra), scalable I/O, in situ visualization, extra physics modules, ... and Python bindings, for which we rely on dependencies that are linked above.
11. Please describe your release methodology and mechanics
We currently heavily influence AMReX features, sometimes breaking, and thus release in lock-step with monthly releases. The core feature stack AMReX + pyAMReX + WarpX + dependencies is after GitHub tag and release shipped via various package managers, primarily: Spack and Conda-Forge. We use continuous deployment where possible.
Our releases are like our development tested by CI (compilation of CPU and GPU platforms, run of >280 tests on CPU).
Jean-Luc Vay (chair), Remi Lehe and Axel Huebl form the SC. Decisions are primarily made in the TC, the SC currently acts as last resort to avoid stalemates in extraordinary situations and we aim to eventually not need it anymore in the future.
13. Please list the project members with access to commit to the mainline of the project
There are no direct pushes to WarpX mainline. The Technical Committee (TC) list above has access to commit to mainline. Merge to mainline has to undergo review and approval by at least one committee member (no self approval). Contributors need approval by at least 1 TC member. Only TC members can merge.
14. Please describe the project's decision-making process
The technical committee (TC) is the core governance body, where, under normal operations, most ideas are discussed and decisions are made. These decisions are usually made at developer meetings. Individual TC members can approve and merge code changes. TC members are expected to seek approval of another maintainer for their own changes, except under exigent circumstances. TC members lead and weigh in on technical discussions and, if needed, can call for a vote of the TC for a technical decision. TC members can merge/close PRs and issues, and moderate (including blocking or muting) bad actors.
The steering committee (SC) is responsible for providing overall direction and guidance, but for the most part allows normal development decisions to be made by the TC. The SC can veto decisions made by the TC, but only does so in extreme circumstances. Moreover, a veto can be overridden by a 2/3 vote of the TC.
The TC can also nominate “Contributors”, who have the ability to triage and review pull requests, but who cannot merge changes themselves.
15. What is the maturity level of your project?
We expect to join at the “Established” stage. Currently, we do not meet the requirement that no more than 50% of our leadership team come from the same institution for “Core” membership.
16. Please list the project's official communication channels
Issues and discussions take place on Github.
WarpX also has a Slack (https://ecpwarpx.slack.com) where users and developers can have more fast-paced discussions, either in the general channel or in topic-specific channels such as “build”, “gpu”, “load_balancing”, etc. Users can be added to the slack channel(s) by emailing a member of the steering committee.
17. Please list the project's social media accounts
N/A
18. Please describe any existing financial sponsorships
WarpX developers are funded by their respective institutions, currently primarily in the US and Europe. Currently, there are several WarpX developers whose efforts are supported by projects funded by the US Department of Energy’s Office of Science. This support requires regular renewal, however, WarpX is currently used in many DOE-supported projects, so it is unlikely that this support will completely collapse in the near future. Additional support comes from subcontracts and in-kind contributions with industry.
19. Please describe the project's infrastructure needs or requests
We are very interested in support for CI, particularly access to GPU resources. Supporting our slack channel is very useful as well, as it is a major means of communication for our developers. We are also interested in having a user meeting at some point in the future. We also have a yearly hackathon for developers that we currently self-organize or tag along established organizations like NERSC (e.g. Open Hackathons), where financial support would be awesome to grow further. We also need help creating a logo, a prerequisite so that we can make stickers!
1. Name of Project
WarpX
2. Project Description
WarpX is an advanced electromagnetic & electrostatic Particle-In-Cell code. WarpX implements explicit and implicit time integrators that enable its use in many physics domains, from laser-plasma physics, particle accelators, fusion applications, space physics to microelectronics.
WarpX is designed to study kinetic effects and relativistic physics from desktop simulations to the latest Exascale supercomputers and was awarded the 2022 Gordon Bell Prize. WarpX is primarily implemented in modern C++ (currently C++17) with extensive bindings to Python. WarpX relies on the C++ performance portability layer in AMReX #5 to implement first-class support for NVIDIA, AMD, and Intel GPUs as well as CPUs, relying on multi-level parallelism (MPI+OpenMP/CUDA/ROCm/SYCL).
WarpX has by now over 100 contributors from research labs, academia and private industry, located in North America, Europe and beyond. WarpX is the flagship code of the Beam, Plasma & Accelerator Simulation Toolkit (BLAST) and its inclusion in HPSF will potentially serve as a blueprint for other ("smaller") BLAST codes to follow later on. BLAST codes are widely used by the community (institutional usage map) and growing rapidly. WarpX development is sustained by a diverse set of funding structures from the US, Europe, subcontracts and private in-kind contributions.
3. Statement on Alignment with High Performance Software Foundation's Mission
Like its predecessor Warp, WarpX has bee committed to open science and thus implements a fully open, community-based development model since its inception in the US DOE Exascale Computing Project. In WarpX, a large set of software technology comes together and needs to work at the same time to solve a science problem, which makes collaborations core to our strategy. WarpX already aims to target users beyond the HPC scope by actively deploying also for desktops on macOS, Linux and Windows platforms, attracting users at all scales and at the same time shipping its application (and all of its dependencies!) to a diverse set of target platforms (desktop/edge, cloud, HPC).
4. Project Website
Landing page: https://ecp-warpx.github.io Manual: https://warpx.readthedocs.io GitHub repo: https://github.com/ECP-WarpX/WarpX GitHub org: https://github.com/ECP-WarpX
5. Code of Conduct (please provide a link)
6. Governance Practices (please provide a link)
7. Sponsor from the High Performance Software Foundation's Technical Advisory Committee
Andrew Myers (LBNL) @atmyers Todd Gamblin (LLNL) @tgamblin
8. What is the project's solution for source control?
Git / GitHub
9. What is the project's solution for issue tracking?
GitHub issues (and discussions)
10. Please list all external dependencies and their license
WarpX itself is open-source under
BSD-3-Clause-LBNL
license: LICENSE.txt. Please see the notices in NOTICE.txt.WarpX depends on dozens of direct and hundreds of indirect dependencies, all under compatible licenses. WarpX prefers OSI-approved dependencies, but also depends on other licenses where unavoidable (e.g., Nvidia CUDA).
Full list: https://warpx.readthedocs.io/en/latest/install/dependencies.html#install-dependencies Spack representation: https://packages.spack.io/package.html?name=warpx Conda representation: https://conda-metadata-app.streamlit.app/?q=conda-forge/warpx
Adding new dependencies to WarpX undergoes a review process by the TAC to ensure source license compatibility and long-term maintainability.
WarpX can be built with a minimal core list of dependencies for reduced functionality:
BSD-3-Clause
BSD-3-Clause-LBNL
This, combined with an MPI implementation can be used for many routines, benchmarking or development on new exotic architectures. In practice, one wants more physics modules, accelerated computing (e.g., CUDA), accelerated math libraries (on-device FFTs and on-device linear algebra), scalable I/O, in situ visualization, extra physics modules, ... and Python bindings, for which we rely on dependencies that are linked above.
11. Please describe your release methodology and mechanics
We currently heavily influence AMReX features, sometimes breaking, and thus release in lock-step with monthly releases. The core feature stack AMReX + pyAMReX + WarpX + dependencies is after GitHub tag and release shipped via various package managers, primarily: Spack and Conda-Forge. We use continuous deployment where possible.
Our releases are like our development tested by CI (compilation of CPU and GPU platforms, run of >280 tests on CPU).
12. Please list the project's leadership team
WarpX's open governance structure has a “Steering Committee (SC)”, “Technical Committee (TC)”, and “Contributors”. The “Technical Committee” corresponds to people with the ability to merge pull requests, those people are initially:
On Oct 8th, we have a vote to add two more members to our TAC:
Jean-Luc Vay (chair), Remi Lehe and Axel Huebl form the SC. Decisions are primarily made in the TC, the SC currently acts as last resort to avoid stalemates in extraordinary situations and we aim to eventually not need it anymore in the future.
13. Please list the project members with access to commit to the mainline of the project
There are no direct pushes to WarpX mainline. The Technical Committee (TC) list above has access to commit to mainline. Merge to mainline has to undergo review and approval by at least one committee member (no self approval). Contributors need approval by at least 1 TC member. Only TC members can merge.
14. Please describe the project's decision-making process
The technical committee (TC) is the core governance body, where, under normal operations, most ideas are discussed and decisions are made. These decisions are usually made at developer meetings. Individual TC members can approve and merge code changes. TC members are expected to seek approval of another maintainer for their own changes, except under exigent circumstances. TC members lead and weigh in on technical discussions and, if needed, can call for a vote of the TC for a technical decision. TC members can merge/close PRs and issues, and moderate (including blocking or muting) bad actors.
The steering committee (SC) is responsible for providing overall direction and guidance, but for the most part allows normal development decisions to be made by the TC. The SC can veto decisions made by the TC, but only does so in extreme circumstances. Moreover, a veto can be overridden by a 2/3 vote of the TC.
The TC can also nominate “Contributors”, who have the ability to triage and review pull requests, but who cannot merge changes themselves.
15. What is the maturity level of your project?
We expect to join at the “Established” stage. Currently, we do not meet the requirement that no more than 50% of our leadership team come from the same institution for “Core” membership.
16. Please list the project's official communication channels
Issues and discussions take place on Github. WarpX also has a Slack (https://ecpwarpx.slack.com) where users and developers can have more fast-paced discussions, either in the general channel or in topic-specific channels such as “build”, “gpu”, “load_balancing”, etc. Users can be added to the slack channel(s) by emailing a member of the steering committee.
17. Please list the project's social media accounts
N/A
18. Please describe any existing financial sponsorships
WarpX developers are funded by their respective institutions, currently primarily in the US and Europe. Currently, there are several WarpX developers whose efforts are supported by projects funded by the US Department of Energy’s Office of Science. This support requires regular renewal, however, WarpX is currently used in many DOE-supported projects, so it is unlikely that this support will completely collapse in the near future. Additional support comes from subcontracts and in-kind contributions with industry.
19. Please describe the project's infrastructure needs or requests
We are very interested in support for CI, particularly access to GPU resources. Supporting our slack channel is very useful as well, as it is a major means of communication for our developers. We are also interested in having a user meeting at some point in the future. We also have a yearly hackathon for developers that we currently self-organize or tag along established organizations like NERSC (e.g. Open Hackathons), where financial support would be awesome to grow further. We also need help creating a logo, a prerequisite so that we can make stickers!