TODO: un-lock scipy

[mathematicalmichael]

we need to get to the bottom of what the problem is in scipy==1.3.1 ...

Using ripgrep, run rg scipy, pass -C 2 for context around regex matches (or -B for before, -A for after).

I would call this relatively high priority (despite many common projects version-locking scipy), and will put in a bit of time on it myself.

The goal would obviously be backwards-compatibility, and if possible... I would like to avoid going through the detailed changelog between the two versions.

[mathematicalmichael]

bash script I'm using for debugging test_sample.py. Since it has been very helpful, I will paste it here for posterity (can be readily adapted for other purposes, and I often forget how to invoke single tests in nose):

#!/bin/bash

for f in `seq 2 6`; do
  echo "Running tests for $f processors";
  mpirun -n $f nosetests test.test_sample:$1;
done

Usage: ./runtest.sh Test_sample_set

[mathematicalmichael]

I figured out what broke our implementation. @smattis

I tracked it to scipy.io.loadmat , and went about building scipy from source checking out commits one at a time that impacted this directory.

here is the list of "suspects": https://github.com/scipy/scipy/commits/v1.3.0/scipy/io/matlab

This is the last one we pass on: https://github.com/scipy/scipy/commit/a2546b91faf0c74b5f51b048856f24dd9db3eb0d#diff-da53e8dd97abe920dd48a332b398a05f

This breaks it: https://github.com/scipy/scipy/commit/af0c74c89f6dee868f65630b13681460bd04cb26#diff-da53e8dd97abe920dd48a332b398a05f

[mathematicalmichael]

it appears that the behavior change is that files aren't left open, the with .. as .. structure is used to ensure they get closed out asap.

for some reason this poses a problem for us?

[smattis]

We should fix this for version 3. I suggest we move to pickle, which should take care of the problem

[mathematicalmichael]

upgrading SciPy requires python 3.5 Python 2 is EOL very soon, so I'm going to charge ahead with removing 2.7 from Travis

[mathematicalmichael]

fixed for v3 since scipy.io is replaced with pickle.