LLNL / Caliper

Caliper is an instrumentation and performance profiling library
http://software.llnl.gov/Caliper/
BSD 3-Clause "New" or "Revised" License
343 stars 63 forks source link

Adds Python bindings to Caliper with pybind11 #573

Open ilumsden opened 2 months ago

ilumsden commented 2 months ago

This PR adds initial Python bindings to Caliper using the pybind11 library. The resulting Python library (named pycaliper) contains bindings for the following Caliper APIs:

These bindings have been tested using the annotations added to Hatchet and Thicket in LLNL/hatchet#142 and LLNL/thicket#195.

Remaining Work:

These bindings are complete and could be merged as-is. However, it would be ideal if I added docstrings to the Python module before merge.

daboehme commented 2 months ago

Great work, thanks @ilumsden! In addition to the documentation it would be great to add an example like the c/cxx/fortran-example apps in the examples/apps directory for the Python annotations. A unit test would also be great. The Caliper unit tests are actually run through Python already so it shouldn't be too difficult to add; check out the tests/ci_app_tests directory. I'm happy to help if needed.

ilumsden commented 6 days ago

Outstanding work:

ilumsden commented 5 days ago

@daboehme this PR is now ready for review.

daboehme commented 3 days ago

Hi @ilumsden, I added a few fixes in a PR to your branch.

I can't run the tests via make test because the pycaliper module is not found. Did you ever get that to work?

It works for me when I add the installed location to PYTHONPATH and run the ci_test_py_ann program or py-example directly. It still doesn't work through the test_python_api.py script, even with PYTHONPATH set.

I got a type mismatch error with the config_preset call, I had to comment that out in the test. Doesn't seem to like the lambda.

I'd also like to move the sources to src/interface/, where the Fortran/C bindings already are.

ilumsden commented 3 days ago

@daboehme thanks for the feedback. I've moved the bindings to src/interface, and I've fixed the tests so that PYTHONPATH will be set correctly. Only problem right now is that my way of setting the path uses configure_file, meaning the Python test files will not be automatically updated by make.

I also fixed a few bugs in the test file where I was not using the Python API correctly. The test should now work. I'm currently messing with the GitHub Actions runner to see if I can get that to work.