vanvalenlab / deepcell-tf

Deep Learning Library for Single Cell Analysis
https://deepcell.readthedocs.io
Other
413 stars 95 forks source link

Upgrade (or widen) supported `tensorflow` version #714

Closed sam-goodwin closed 6 months ago

sam-goodwin commented 6 months ago

deepcell relies on 2.8.0 of tensorflow whereas the latest version is 2.16.1. Are there any plans to either upgrade or widen the allowed version of tensorflow? This is creating a "dependency hell" situation where I can't run deepcell on MacOS because there are no builds for tensorflow that old.

Are there known issues preventing us from widening that to >=2.8.0?

msschwartz21 commented 6 months ago

This isn't a robustly tested solution, but here is how I create a deepcell environment on my M1 mac. I hope this helps!

mamba create --name deepcell python=3.8 -y
mamba activate deepcell
mamba install -c apple tensorflow-deps
pip install tensorflow-macos~=2.8.0 tensorflow-metal==0.4.0 tensorflow-addons~=0.16.1 "numpy>=1.16.6" "scipy>=1.2.3,<2" "scikit-image>=0.19.3" "scikit-learn>=0.20.4" matplotlib tqdm "deepcell-tracking~=0.6.1" "deepcell-toolbox>=0.12.1" networkx lxml joblib
pip install --no-deps spektral~=1.0.4 deepcell
sam-goodwin commented 6 months ago

Thanks for the tip @msschwartz21, it is helpful! Do you know if this is dependent on mambda or can it be achieved with any package manager like pip, poetry or uv?

I'd also like to continue the conversation on upgrading tensorflow version. The longer deepcell lags, the more difficult it becomes to use deepcell along with other models.

msschwartz21 commented 6 months ago

@sam-goodwin my impression was that the only way to install tensorflow on Macs was using mamba, but I haven't tried it with pip alone.

rossbar commented 6 months ago

Are there any plans to either upgrade or widen the allowed version of tensorflow?

Yes, the project is of course interested in supporting the latest versions of dependencies. Contributions toward this goal are strongly encouraged!

Are there known issues preventing us from widening

Yes, it is not trivial to do. deepcell-tf currently relies on tf/keras features that have since been deprecated and removed.

Do you know if this is dependent on mamba or can it be achieved with any package manager like pip, poetry or uv?

Long story short: this will only work for conda/mamba. The conda/mamba forge channel(s) consists of community-supported installation recipes that may have broader support windows than what the tensorflow project officially supports.

The official support windows from tensorflow are listed here. There is no explicit information about support for the M{1-3} platforms in that table, so you'd likely have to find a different source to figure out exactly what is officially supported. The release history on PyPI will tell you which versions explicitly provide wheels for which platforms. It looks like tensorflow 2.13 is the first version for which there are explicit arm64 wheels on macos.


I'm going to go ahead and close this as duplicate of #711 . As always, contributions related to bumping tensorflow version would certainly get reviewed! However, the development of new segmentation methods is currently higher in the priority queue.

sam-goodwin commented 5 months ago

This isn't a robustly tested solution, but here is how I create a deepcell environment on my M1 mac. I hope this helps!

mamba create --name deepcell python=3.8 -y
mamba activate deepcell
mamba install -c apple tensorflow-deps
pip install tensorflow-macos~=2.8.0 tensorflow-metal==0.4.0 tensorflow-addons~=0.16.1 "numpy>=1.16.6" "scipy>=1.2.3,<2" "scikit-image>=0.19.3" "scikit-learn>=0.20.4" matplotlib tqdm "deepcell-tracking~=0.6.1" "deepcell-toolbox>=0.12.1" networkx lxml joblib
pip install --no-deps spektral~=1.0.4 deepcell

Unfortunately, this doesn't work for python 3.10.

rossbar commented 5 months ago

Unfortunately, this doesn't work for python 3.10.

To be clear, the Python version is specified explicitly in the above example. In other words, the mamba environment itself downloads and installs Python 3.8. Note also that the apple channel (as indicated by -c apple) may not be the best/only source for tensorflow dependencies on aarch64.