Notes about dependency versions and compatibility

[smmaurer]

To have this written down:

Python

• v2.7 was used in the last cycle
• latest is v3.8, and v3.6+ is considered "current"
• BAUS works in 3.6 (after a lot of updating to support Python 3); still need to test newer versions

Python itself doesn't change much in new 3.x releases (and we're not using any cutting edge features) -- the main issue is that compiled numerical libraries need to provide new binaries for each Python version. Big ones like NumPy and Pandas are updated quickly, but it can take longer for niche libraries like Pandana or PyTables (which UrbanSim uses for HDF i/o) to provide support.

Operating systems

• BAUS (and Python in general) should run equivalently on Mac, Linux, and Windows
• Mac and Linux are almost identical from a Python perspective; the only difference is in getting appropriate binary versions of numerical libraries
• Windows has additional variations in file i/o, but BAUS is written to support it

One cross-platform issue we run into is that Pandas sometimes chooses different precision levels for data (like int32 vs int64) depending on the environment it's running in. This can trip up Orca when data tables are being updated -- a good fix is to add the cast=True argument to the update_col_from_series() call. (If the mismatch is more than just precision level, like an int vs. float issue, you should dig deeper to make sure the code is doing the right thing.)

NumPy

• v1.10 was used in the prior cycle
• latest is v1.17, last version with support for Python 2.7 is v1.16
• compatibility issues are not common

A couple of bugs related to changes in NumPy/Pandas default behaviors did come up last year, see PR #99 and issue #96.

Pandas

• something around v0.20 was used in the prior cycle
• latest is v1.0, last version with support for Python 2.7 is v0.24
• BAUS/UrbanSim uses some syntax that was removed in Pandas v1.0 -- the latest version that works right now is v0.25

Statsmodels

• latest is v0.11, last version with support for Python 2.7 is v0.10

Statsmodels is installed as a sub-dependency of UrbanSim, for OLS estimation and simulation. On my machine, pip install statsmodels in Python 2.7 tries to install v0.11, which causes errors, so you need to install v0.10 manually. I'll try to get this fixed in an update to UrbanSim.

Scikit-learn

• latest is v0.22, last version with support for Python 2.7 is v0.20

This is installed as a sub-dependency of Pandana.

Orca

• v1.5 was used in the last cycle [confirm]
• latest is v1.5

Orca_test

• not used in the last cycle
• latest is v0.1

Pandana

• v0.2 was used in the last cycle
• latest is v0.4

More Pandana notes TK.

UrbanSim

• v3.0 was used in the last cycle
• latest is v3.1, which is not yet compatible with Pandas 1.0
• can only be installed (a) from pip or (b) from the udst conda channel -- not conda-forge
• installs statsmodels, matplotlib, and pytables as dependencies (which can sometimes cause problems themselves)

UrbanSim_Defaults

• v0.1 was used in the last cycle
• latest is v0.2
• only installs from pip, not conda

[fscottfoti]

My big question would be whether Python2.7 is no longer supported and thus it's ok to use Python3-only features. I guess I'd hope that we'd be there soon.