GlobalArrays / ga

Partitioned Global Address Space (PGAS) library for distributed arrays
http://hpc.pnl.gov/globalarrays/
Other
97 stars 38 forks source link

conda package #251

Closed ltalirz closed 1 month ago

ltalirz commented 2 years ago

Hi, I just wanted to mention that I'm preparing a conda package for globalarrays on conda-forge (as I need it as a dependency for another package).

https://github.com/conda-forge/staged-recipes/pull/17924

As I'm not familiar with the package, I've started with a minimal build. The build runs and most of the tests pass - one test says it's going to run 90 minutes, so I guess one will either need to accept that it will fail or skip it somehow.

Just wondering whether you have any suggestions concerning the build script, and whether anyone would like to be added as a co-maintainer of the feedstock

P.S. I noticed that https://github.com/GlobalArrays/ga/issues/239 was about building globalarrays in a conda environment - perhaps prebuilding the package could be useful in this regard as well.

edoapra commented 2 years ago

You seem to be using version 5.8, while 5.8.1 is out https://github.com/GlobalArrays/ga/releases https://github.com/ltalirz/staged-recipes/blob/71116b890e0402400d34d8b4c2ca5dc247bb9779/recipes/globalarrays/meta.yaml#L2

ltalirz commented 2 years ago

Thanks Edoardo, just pushed the update.

About the tests, is there a way to run just a subset of them?

marcindulak commented 2 years ago

fedora runs the following subset of tests https://src.fedoraproject.org/rpms/ga/blob/5e189740cd1f0f1a0adfcca8c205de3077393326/f/ga.spec#_193-196, which executes about 10 tests in total. Is this still good enough?

export OMPI_MCA_btl=^uct
export OMPI_MCA_btl_base_warn_component_unused=0
make NPROCS=2 VERBOSE=1 check-ma check-travis
make NPROCS=2 TESTS="global/testing/test.x global/testing/testc.x global/testing/testmatmult.x global/testing/patch.x global/testing/simple_groups_comm.x global/testing/elempatch.x" check-TESTS VERBOSE=1

debian seems to run 81 tests in total, (see the "Result" log at https://buildd.debian.org/status/logs.php?pkg=ga&arch=amd64&suite=sid) and the build, including tests (there are 6 failures) takes less than 30 minutes. I'm note sure the tests are run in parallel since there are errors like "Program requires 4 GA processes",

Comparing debian to your test results of https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=453855&view=logs&j=6f142865-96c3-535c-b7ea-873d86b887bd&t=22b0682d-ab9e-55d7-9c79-49f3c3ba4823 it looks like debian did not execute the long test you mention random_parallel. I can't see immediately that debian skips any tests https://sources.debian.org/src/ga/5.8-1/debian/rules/.

ltalirz commented 2 years ago

Thanks a lot for looking into this @marcindulak !

I think the 10 tests from Fedora would suffice.

Do you happen to know how to forward the TESTS variable to the test run when following the cmake route?

I tried

CTEST_OUTPUT_ON_FAILURE=ON TESTS="global/testing/test.x global/testing/testc.x global/testing/testmatmult.x global/testing/patch.x global/testing/simple_groups_comm.x global/testing/elempatch.x" make test

but that still ended up running all the tests.

edoapra commented 2 years ago

@ltalirz I think that the last test should be replaced by check-TESTS

CTEST_OUTPUT_ON_FAILURE=ON TESTS="global/testing/test.x global/testing/testc.x global/testing/testmatmult.x global/testing/patch.x global/testing/simple_groups_comm.x global/testing/elempatch.x" make check-TESTS
marcindulak commented 2 years ago

I think the problem is that "check-TESTS" is not available when using the makefiles generated using cmake. Maybe it's better to switch to the regular make?

edoapra commented 2 years ago

My comments were assuming use of autoconf. I do use and try to maintain the autoconf ga machinery.

ajaypanyala commented 1 month ago

Closing due to inactivity, feel free to re-open if needed!