search

cylondata / cylon

Cylon is a fast, scalable, distributed memory, parallel runtime with a Pandas like DataFrame.
https://cylondata.org
Apache License 2.0
298 stars 48 forks source link

test: cylon on summet, test, cache can not be created #622

Open laszewsk opened 2 years ago

laszewsk commented 2 years ago

jsrun -n 4 pytest --with-mpi -q python/pycylon/test/test_dist_rl.py
........ [100%] [100%] [100%] [100%] =============================== warnings summary =============================== ../../../../ccs/home/gregorvl/CYLON/lib/python3.7/site-packages/_pytest/cacheprovider.py:433 /ccs/home/gregorvl/CYLON/lib/python3.7/site-packages/_pytest/cacheprovider.py:433: PytestCacheWarning: could not create cache path /autofs/nccs-svm1_home1/gregorvl/cylon/python/pycylon/.pytest_cache/v/cache/nodeids config.cache.set("cache/nodeids", sorted(self.cached_nodeids))

../../../../ccs/home/gregorvl/CYLON/lib/python3.7/site-packages/_pytest/stepwise.py:52 /ccs/home/gregorvl/CYLON/lib/python3.7/site-packages/_pytest/stepwise.py:52: PytestCacheWarning: could not create cache path /autofs/nccs-svm1_home1/gregorvl/cylon/python/pycylon/.pytest_cache/v/cache/stepwise session.config.cache.set(STEPWISE_CACHE_DIR, [])

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html =============================== MPI Information ================================ rank: 1 size: 4 MPI version: 3.1 MPI library version: Open MPI v10.4.0.03rtm4, package: IBM Spectrum MPI, ident: IBM Spectrum MPI, repo rev: IBM_SPECTRUM_MPI_10.04.00.03_2021.01.12_RTM4, Unreleased developer copy � py rc: r: � mpi4py config: mpicc: /sw/summit/spack-envs/base/opt/linux-rhel8-ppc64le/gcc-9.3.0/spectrum-mpi-10.4.0.3-20210112-6depextb6p6ulrvmehgtbskbmcsyhtdi/bin/mpicc 2 passed, 2 warnings in 1.� =============================== warnings summary =============================== ../../../../ccs/home/gregorvl/CYLON/lib/python3.7/site-packages/_pytest/cacheprovider.py:433 /ccs/home/gregorvl/CYLON/lib/python3.7/site-packages/_pytest/cacheprovider.py:433: PytestCacheWarning: could not create cache path /autofs/nccs-svm1_home1/gregorvl/cylon/python/pycylon/.pytest_cache/v/cache/nodeids config.cache.set("cache/nodeids", sorted(self.cached_nodeids))

../../../../ccs/home/gregorvl/CYLON/lib/python3.7/site-packages/_pytest/stepwise.py:52 /ccs/home/gregorvl/CYLON/lib/python3.7/site-packages/_pytest/stepwise.py:52: PytestCacheWarning: could not create cache path /autofs/nccs-svm1_home1/gregorvl/cylon/python/pycylon/.pytest_cache/v/cache/stepwise session.config.cache.set(STEPWISE_CACHE_DIR, [])

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html =============================== MPI Information ================================ rank: 3 size: 4 MPI version: 3.1 MPI library version: Open MPI v10.4.0.03rtm4, package: IBM Spectrum MPI, ident: IBM Spectrum MPI, repo rev: IBM_SPECTRUM_MPI_10.04.00.03_2021.01.12_RTM4, Unreleased developer copy � py rc: r: � mpi4py config: mpicc: /sw/summit/spack-envs/base/opt/linux-rhel8-ppc64le/gcc-9.3.0/spectrum-mpi-10.4.0.3-20210112-6depextb6p6ulrvmehgtbskbmcsyhtdi/bin/mpicc 2 passed, 2 warnings in 1.� =============================== warnings summary =============================== ../../../../ccs/home/gregorvl/CYLON/lib/python3.7/site-packages/_pytest/cacheprovider.py:433 /ccs/home/gregorvl/CYLON/lib/python3.7/site-packages/_pytest/cacheprovider.py:433: PytestCacheWarning: could not create cache path /autofs/nccs-svm1_home1/gregorvl/cylon/python/pycylon/.pytest_cache/v/cache/nodeids config.cache.set("cache/nodeids", sorted(self.cached_nodeids))

../../../../ccs/home/gregorvl/CYLON/lib/python3.7/site-packages/_pytest/stepwise.py:52 /ccs/home/gregorvl/CYLON/lib/python3.7/site-packages/_pytest/stepwise.py:52: PytestCacheWarning: could not create cache path /autofs/nccs-svm1_home1/gregorvl/cylon/python/pycylon/.pytest_cache/v/cache/stepwise session.config.cache.set(STEPWISE_CACHE_DIR, [])

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html =============================== MPI Information ================================ rank: 2 size: 4 MPI version: 3.1 MPI library version: Open MPI v10.4.0.03rtm4, package: IBM Spectrum MPI, ident: IBM Spectrum MPI, repo rev: IBM_SPECTRUM_MPI_10.04.00.03_2021.01.12_RTM4, Unreleased developer copy � py rc: r: � mpi4py config: mpicc: /sw/summit/spack-envs/base/opt/linux-rhel8-ppc64le/gcc-9.3.0/spectrum-mpi-10.4.0.3-20210112-6depextb6p6ulrvmehgtbskbmcsyhtdi/bin/mpicc 2 passed, 2 warnings in 1.� =============================== warnings summary =============================== ../../../../ccs/home/gregorvl/CYLON/lib/python3.7/site-packages/_pytest/cacheprovider.py:433 /ccs/home/gregorvl/CYLON/lib/python3.7/site-packages/_pytest/cacheprovider.py:433: PytestCacheWarning: could not create cache path /autofs/nccs-svm1_home1/gregorvl/cylon/python/pycylon/.pytest_cache/v/cache/nodeids config.cache.set("cache/nodeids", sorted(self.cached_nodeids))

../../../../ccs/home/gregorvl/CYLON/lib/python3.7/site-packages/_pytest/stepwise.py:52 /ccs/home/gregorvl/CYLON/lib/python3.7/site-packages/_pytest/stepwise.py:52: PytestCacheWarning: could not create cache path /autofs/nccs-svm1_home1/gregorvl/cylon/python/pycylon/.pytest_cache/v/cache/stepwise session.config.cache.set(STEPWISE_CACHE_DIR, [])

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html =============================== MPI Information ================================ rank: 0 size: 4 MPI version: 3.1 MPI library version: Open MPI v10.4.0.03rtm4, package: IBM Spectrum MPI, ident: IBM Spectrum MPI, repo rev: IBM_SPECTRUM_MPI_10.04.00.03_2021.01.12_RTM4, Unreleased developer copy � py rc: r: � mpi4py config: mpicc: /sw/summit/spack-envs/base/opt/linux-rhel8-ppc64le/gcc-9.3.0/spectrum-mpi-10.4.0.3-20210112-6depextb6p6ulrvmehgtbskbmcsyhtdi/bin/mpicc 2 passed, 2 warnings in 1.�(CYLON) bash-4.4$

laszewsk commented 2 years ago

the following fixes it jsrun -n 4 pytest -p no:cacheprovider --with-mpi -q python/pycylon/test/test_dist_rl.py