DASDAE / dascore

A python library for distributed fiber optic sensing
Other
72 stars 16 forks source link

conda cache string to include architecture #312

Closed d-chambers closed 9 months ago

d-chambers commented 9 months ago

Description

This PR adds the system architecture to the cache string in the CI systems. Occasionally the OSX CI will fail and issue an "illegal operation error". From some precursory googling this can be due to the system using the wrong binaries. I suspect Github CI may be introducing some Mx chips into the OSX pool, so if the cache is created with a x86 machine then a M2 machine tries to use the complied bits we could get this error.

Of course, this is just speculation but having this extra part in the cache string wont hurt anything so its worth a shot.

Checklist

I have (if applicable):

codecov[bot] commented 9 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (7374718) 99.30% compared to head (1ceab02) 99.30%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #312 +/- ## ======================================= Coverage 99.30% 99.30% ======================================= Files 86 86 Lines 6796 6796 ======================================= Hits 6749 6749 Misses 47 47 ``` | [Flag](https://app.codecov.io/gh/DASDAE/dascore/pull/312/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DASDAE) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/DASDAE/dascore/pull/312/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DASDAE) | `99.30% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DASDAE#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

d-chambers commented 9 months ago

I guess it isn't the caching as the macos builds here are also failing. For completeness, here is the log output:

Run pytest -s --cov dascore --cov-append --cov-report=xml
============================= test session starts ==============================
platform darwin -- Python 3.10.[13](https://github.com/DASDAE/dascore/actions/runs/7223230814/job/19681855738?pr=312#step:11:14), pytest-7.4.3, pluggy-1.3.0
rootdir: /Users/runner/work/dascore/dascore
configfile: pyproject.toml
plugins: cov-4.1.0, pytest_codeblocks-0.17.0
collected 1769 items / 1 skipped

Fatal Python error: Illegal instruction

Current thread 0x000000010c328600 (most recent call first):
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/tables/node.py", line 246 in __init__
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/tables/leaf.py", line 264 in __init__
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/tables/array.py", line 172 in __init__
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/tables/group.py", line 1158 in _g_load_child
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/tables/file.py", line 411 in get_node
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/tables/file.py", line 1550 in _get_node
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/tables/group.py", line 685 in _f_get_child
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/tables/group.py", line 28 in _get_value_from_container
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/tables/misc/proxydict.py", line 23 in __getitem__
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/tables/group.py", line 713 in _f_iter_nodes
  File "/Users/runner/work/dascore/dascore/dascore/io/dasdae/utils.py", line 125 in <listcomp>
  File "/Users/runner/work/dascore/dascore/dascore/io/dasdae/utils.py", line 125 in _get_coords
  File "/Users/runner/work/dascore/dascore/dascore/io/dasdae/utils.py", line 160 in _read_patch
  File "/Users/runner/work/dascore/dascore/dascore/io/dasdae/core.py", line 111 in read
  File "/Users/runner/work/dascore/dascore/dascore/io/core.py", line 295 in _wraper
  File "/Users/runner/work/dascore/dascore/dascore/io/core.py", line 506 in read
  File "/Users/runner/work/dascore/dascore/dascore/clients/filespool.py", line 67 in _load_patch
  File "/Users/runner/work/dascore/dascore/dascore/core/spool.py", line 382 in _patch_from_instruction_df
  File "/Users/runner/work/dascore/dascore/dascore/core/spool.py", line 372 in _get_patches_from_index
  File "/Users/runner/work/dascore/dascore/dascore/core/spool.py", line 343 in __getitem__
  File "/Users/runner/work/dascore/dascore/dascore/examples.py", line 180 in _example_event_1
  File "/Users/runner/work/dascore/dascore/dascore/examples.py", line 304 in get_example_patch
  File "/Users/runner/work/dascore/dascore/tests/test_examples.py", line 25 in test_example_1
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/_pytest/python.py", line 194 in pytest_pyfunc_call
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/pluggy/_callers.py", line 77 in _multicall
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/pluggy/_manager.py", line 115 in _hookexec
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/pluggy/_hooks.py", line 493 in __call__
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/_pytest/python.py", line 1792 in runtest
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/_pytest/runner.py", line 169 in pytest_runtest_call
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/pluggy/_callers.py", line 77 in _multicall
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/pluggy/_manager.py", line 115 in _hookexec
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/pluggy/_hooks.py", line 493 in __call__
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/_pytest/runner.py", line 262 in <lambda>
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/_pytest/runner.py", line 341 in from_call
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/_pytest/runner.py", line 261 in call_runtest_hook
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/_pytest/runner.py", line 222 in call_and_report
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/_pytest/runner.py", line 133 in runtestprotocol
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/_pytest/runner.py", line 1[14](https://github.com/DASDAE/dascore/actions/runs/7223230814/job/19681855738?pr=312#step:11:15) in pytest_runtest_protocol
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/pluggy/_callers.py", line 77 in _multicall
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/pluggy/_manager.py", line 1[15](https://github.com/DASDAE/dascore/actions/runs/7223230814/job/19681855738?pr=312#step:11:16) in _hookexec
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/pluggy/_hooks.py", line 493 in __call__
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/_pytest/main.py", line 350 in pytest_runtestloop
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/pluggy/_callers.py", line 77 in _multicall
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/pluggy/_manager.py", line 115 in _hookexec
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/pluggy/_hooks.py", line 493 in __call__
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/_pytest/main.py", line 325 in _main
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/_pytest/main.py", line 271 in wrap_session
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/_pytest/main.py", line 318 in pytest_cmdline_main
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/pluggy/_callers.py", line 77 in _multicall
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/pluggy/_manager.py", line 115 in _hookexec
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/pluggy/_hooks.py", line 493 in __call__
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/_pytest/config/__init__.py", line [16](https://github.com/DASDAE/dascore/actions/runs/7223230814/job/19681855738?pr=312#step:11:17)9 in main
  File "/Users/runner/miniconda3/envs/dascore/lib/python3.10/site-packages/_pytest/config/__init__.py", line 192 in console_main
  File "/Users/runner/miniconda3/envs/dascore/bin/pytest", line 8 in <module>

Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, matplotlib._c_internal_utils, PIL._imaging, matplotlib._path, kiwisolver._cext, tables._comp_lzo, tables._comp_bzip2, tables.utilsextension, numexpr.interpreter, tables.hdf5extension, tables.linkextension, tables.lrucacheextension, tables.tableextension, tables.indexesextension, pandas._libs.tslibs.np_datetime, pandas._libs.tslibs.dtypes, pandas._libs.tslibs.base, pandas._libs.tslibs.nattype, pandas._libs.tslibs.timezones, pandas._libs.tslibs.ccalendar, pandas._libs.tslibs.fields, pandas._libs.tslibs.timedeltas, pandas._libs.tslibs.tzconversion, pandas._libs.tslibs.timestamps, pandas._libs.properties, pandas._libs.tslibs.offsets, pandas._libs.tslibs.strptime, pandas._libs.tslibs.parsing, pandas._libs.tslibs.conversion, pandas._libs.tslibs.period, pandas._libs.tslibs.vectorized, pandas._libs.ops_dispatch, pandas._libs.missing, pandas._libs.hashtable, pandas._libs.algos, pandas._libs.interval, pandas._libs.lib, pandas._libs.ops, pandas._libs.arrays, pandas._libs.tslib, pandas._libs.sparse, pandas._libs.indexing, pandas._libs.index, pandas._libs.internals, pandas._libs.join, pandas._libs.writers, pandas._libs.window.aggregations, pandas._libs.window.indexers, pandas._libs.reshape, pandas._libs.groupby, pandas._libs.json, pandas._libs.parsers, pandas._libs.testing, scipy._lib._ccallback_c, scipy.special._ufuncs_cxx, scipy.special._ufuncs, scipy.special._specfun, scipy.special._comb, scipy.linalg._fblas, scipy.linalg._flapack, scipy.linalg.cython_lapack, scipy.linalg._cythonized_array_utils, scipy.linalg._solve_toeplitz, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_sqrtm_triu, scipy.linalg.cython_blas, scipy.linalg._matfuncs_expm, scipy.linalg._decomp_update, scipy.sparse._sparsetools, _csparsetools, scipy.sparse._csparsetools, scipy.sparse.linalg._isolve._iterative, scipy.sparse.linalg._dsolve._superlu, scipy.sparse.linalg._eigen.arpack._arpack, scipy.sparse.csgraph._tools, scipy.sparse.csgraph._shortest_path, scipy.sparse.csgraph._traversal, scipy.sparse.csgraph._min_spanning_tree, scipy.sparse.csgraph._flow, scipy.sparse.csgraph._matching, scipy.sparse.csgraph._reordering, scipy.linalg._flinalg, scipy.special._ellip_harm_2, scipy.interpolate._fitpack, scipy.interpolate.dfitpack, scipy.optimize._minpack2, scipy.optimize._group_columns, scipy._lib.messagestream, scipy.optimize._trlib._trlib, scipy.optimize._lbfgsb, _moduleTNC, scipy.optimize._moduleTNC, scipy.optimize._cobyla, scipy.optimize._slsqp, scipy.optimize._minpack, scipy.optimize._lsq.givens_elimination, scipy.optimize._zeros, scipy.optimize.__nnls, scipy.optimize._highs.cython.src._highs_wrapper, scipy.optimize._highs._highs_wrapper, scipy.optimize._highs.cython.src._highs_constants, scipy.optimize._highs._highs_constants, scipy.linalg._interpolative, scipy.optimize._bglu_dense, scipy.optimize._lsap, scipy.spatial._ckdtree, scipy.spatial._qhull, scipy.spatial._voronoi, scipy.spatial._distance_wrap, scipy.spatial._hausdorff, scipy.spatial.transform._rotation, scipy.optimize._direct, scipy.interpolate._bspl, scipy.interpolate._ppoly, scipy.interpolate.interpnd, scipy.interpolate._rbfinterp_pythran, scipy.interpolate._rgi_cython, scipy.ndimage._nd_image, _ni_label, scipy.ndimage._ni_label, scipy.signal._sigtools, scipy._lib._uarray._uarray, scipy.signal._max_len_seq_inner, scipy.signal._upfirdn_apply, scipy.signal._spline, scipy.integrate._odepack, scipy.integrate._quadpack, scipy.integrate._vode, scipy.integrate._dop, scipy.integrate._lsoda, scipy.signal._sosfilt, scipy.signal._spectral, scipy.special.cython_special, scipy.stats._stats, scipy.stats.beta_ufunc, scipy.stats._boost.beta_ufunc, scipy.stats.binom_ufunc, scipy.stats._boost.binom_ufunc, scipy.stats.nbinom_ufunc, scipy.stats._boost.nbinom_ufunc, scipy.stats.hypergeom_ufunc, scipy.stats._boost.hypergeom_ufunc, scipy.stats.ncf_ufunc, scipy.stats._boost.ncf_ufunc, scipy.stats.ncx2_ufunc, scipy.stats._boost.ncx2_ufunc, scipy.stats.nct_ufunc, scipy.stats._boost.nct_ufunc, scipy.stats.skewnorm_ufunc, scipy.stats._boost.skewnorm_ufunc, scipy.stats.invgauss_ufunc, scipy.stats._boost.invgauss_ufunc, scipy.stats._biasedurn, scipy.stats._levy_stable.levyst, scipy.stats._stats_pythran, scipy.stats._statlib, scipy.stats._sobol, scipy.stats._qmc_cy, scipy.stats._mvn, scipy.stats._rcont.rcont, scipy.signal._peak_finding_utils, matplotlib._image, scipy.io.matlab._mio_utils, scipy.io.matlab._streams, scipy.io.matlab._mio5_utils (total: [17](https://github.com/DASDAE/dascore/actions/runs/7223230814/job/19681855738?pr=312#step:11:18)7)
/Users/runner/work/_temp/7f927edb-1366-45cd-ace4-9a55eb7a65ee.sh: line 1:  [18](https://github.com/DASDAE/dascore/actions/runs/7223230814/job/19681855738?pr=312#step:11:19)15 Illegal instruction: 4  pytest -s --cov dascore --cov-append --cov-report=xml
d-chambers commented 9 months ago

Ok, this seems to have fixed it (fingers crossed). Will merge for now since this only affects the CI and none of the changes will hurt anything even if they didn't fix the issue.