AMReX-Codes / pyamrex

GPU-Enabled, Zero-Copy AMReX Python Bindings including AI/ML
http://pyamrex.readthedocs.io
Other
32 stars 15 forks source link

add PlotFileData bindings #320

Closed BenWibking closed 1 day ago

BenWibking commented 1 month ago

Enable reading plotfiles with pyAMReX. Also adds a script in tools/ to create images from plotfiles.

Example use-case: https://github.com/BenWibking/plotfile-viewer.

Closes https://github.com/AMReX-Codes/pyamrex/issues/317.

ax3l commented 1 month ago

@BenWibking thanks a lot, looking good! Ping us when you are ready with the unit test and we can include it maybe in this month's release even?

BenWibking commented 1 month ago

@ax3l I've added a unit test to read a 3D plotfile with a single level. I put the binary data in the PR, but I can move that to some more appropriate place if needed.

When I try to add a test for a 2D plotfile, I get this error:

tests/test_plotfiledata_2d.py:8: in <module>
    import amrex.space2d as amr
../myenv/lib/python3.12/site-packages/amrex/space2d/__init__.py:22: in <module>
    from . import amrex_2d_pybind
E   ImportError: generic_type: type "AMReX" is already registered!

Do 2D tests need to have some special configuration?

BenWibking commented 1 month ago

@ax3l Aside from the 2D test issue, everything should be working now.

baperry2 commented 3 weeks ago

This is awesome! I was hoping for this capability. In general, to use pyamrex for pre/postprocessing workflows having as much I/O capability as possible is super helpful.

Another example would be reading/writing individual FAB data (appears to be commented out here: https://github.com/AMReX-Codes/pyamrex/blob/63cae480781e38f8bd6d5d98e790ad15c21a5ae6/src/Base/FArrayBox.cpp#L38), which would have been useful for something I was looking at doing a while ago, but am not longer planning on.

BenWibking commented 2 weeks ago

This is awesome! I was hoping for this capability. In general, to use pyamrex for pre/postprocessing workflows having as much I/O capability as possible is super helpful.

Another example would be reading/writing individual FAB data (appears to be commented out here:

https://github.com/AMReX-Codes/pyamrex/blob/63cae480781e38f8bd6d5d98e790ad15c21a5ae6/src/Base/FArrayBox.cpp#L38

), which would have been useful for something I was looking at doing a while ago, but am not longer planning on.

That would be useful. I'm not sure why that's commented out. Maybe @ax3l knows?

In the future, I would like to add the ability to lazy-load FABs for out-of-core analysis, but that might require additional changes in AMReX.

ax3l commented 2 weeks ago

Awesome, yes those function should be added, too, but maybe in a separate PR.

There are some comments in #82 and tests/test_farraybox.py how to bind C++ iostreams and Python io. I simply had no time to test it and no need yet, but PRs are very welcome on it (with a test please) :pray:

ax3l commented 2 days ago

Hi @BenWibking , do you have a chance to finish this for the release slayed for today? :)

BenWibking commented 2 days ago

@ax3l Thanks for the reminder. Let me know how this looks.

BenWibking commented 2 days ago

It looks like the CI failed due to "no disk space left on device":

  Error: Error running analysis for cpp: Encountered a fatal error while running "/opt/hostedtoolcache/CodeQL/2.17.6/x64/codeql/codeql database run-queries --ram=14567 --threads=4 /home/runner/work/_temp/codeql_databases/cpp --expect-discarded-cache --min-disk-free=1024 -v --intra-layer-parallelism". Exit code was 2 and error was: A fatal error occurred: Could not write a relation to a file
  (eventual cause: IOException "No space left on device") See the logs for more details. For more information, see https://gh.io/troubleshooting-code-scanning/out-of-disk-or-memory
ax3l commented 1 day ago

The CI error is from static code analysis. I think @WeiqunZhang know how we can exclude part of the code (the AMReX part) to get it into the limits of CI?

ax3l commented 1 day ago

Hi @BenWibking,

Thank you for your contribution! Please feel free to add yourself to our .zenodo.json so we can credit you in releases.