Justin-J-Miller
commented
7 months ago Hi @justinrporter,
Sorry for the delay- I started poking at this today. As of numpy 1.24.0 ragged array creation without specifying dtype=object results in a ValueError.
E.g. In a fresh environment with python 3.9 (3.9.19) and numpy 1.26.4:
import numpy as np
from enspara import ra
a = [np.arange(i) for i in range(1,6)]
b = ra.RaggedArray(a)Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/export/seas-home/justinjm/software/enspara/enspara/ra/ra.py", line 502, in __init__
array = np.array(list(array))
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (5,) + inhomogeneous part.Converting the RA creation to: np.array(list(array),dtype=object) will let this proceed. The nested arrays retain their dtype (e.g. int64 in this case), but the entire array becomes an object array. I imagine switching all array creations to np.array(___, dtype=object) could fix our issues, but there will be loads of other sanity checks that need to be rewritten.
I'm not familiar enough with the rest of enspara's ra handling- do we think making everything object arrays will result in a huge performance loss? @lgsmith and I were chatting about this earlier- do you know how much performance we gain by putting this into a np array as opposed to leaving it as a list?
Less immediately related, in the above environment I get the following after running nosetests enspara: 2 skips, 66 errors, and 2 failures.
~33 errors are the above ValueErrors.
~33 errors are deprecations of np.int and np.bool (suggested to replace with int or bool unless wishing to specify precision (int) or scalar type (bool, alternatively use np.bool_. I've got a version of this commit now with all instances of np.bool replaced with bool and all instances of np.int replaced with int.
Happy to push more on this but wanted to see what the hive mind had to say about handling these two errors.
lgsmith
justinrporter
mizimmer90
This PR fixes a number of tests that were broken under newer versions of our various libraries as well as in python 3.9.
@lgsmith could you run the tests in this branch and verify?
md.load(bug is now fixed inmasterbut not released)implied_timescales.infer_timestepthat resulted from MDTraj changing the error type it raises when a file is not a valid type formd.loadfromValueErrortoOSError(we now catch both)setup.pyto python 3.9exposonscode to provide more information when aRecursionErroris reached bytop.select. I only encountered this error in testing, I couldn't reproduce it outside of a test runner (both nose and pytest), so I'm not digging it too much more.np.floattofloat). In numpy 1.20,np.floatwill no longer be an alias for the builtinfloatso we are just retaining old behavior with this.random_statetoAffinityPropagationwhich caused test reversions. We now userandom_state=0to maintain results that match the publication.scikit-learn>=0.23.3(see above)DummyCommanddummy_mpi4pyobjects to handle running the tests without mpi4py installed. Some tests still error out, though.