Install MedCAT with pytorch cpu

[jenniferjiangkells]

MedCAT currently install with torch>=1.13.0, which installs the full wheel of pytorch, with CUDA and all. In a Docker image, pytorch takes around ~5GB of storage. It's way too heavy for deployments that only requires CPU inference.

Pytorch offers a slimmed-down cpu-only version that is 5 times smaller than the full wheel - pip install torch --index-url https://download.pytorch.org/whl/cpu. It would be nice if MedCAT also offers a cpu-only installation option which is smaller in size and more practical for the purposes of production deployment.

[tomolopolis]

@jenniferajiang - I second this as well. A cogstack-nifi 'slim' version was suggested in our prev call for the SLaM folks also. Ticket created on our dev board. CU-8692wktvq

[adam-sutton-1992]

@jenniferajiang - Due to how PyPI & pip are configured, along with how pytorch is deployed we cannot have a cpu only wheel of MedCAT.

PyPI is set up so it only downloads wheels from "https://pypi.org/simple", unless changed explicitly from the user (such as --index-url like you've done here). This is for security purposes so you can always be sure that the wheel is coming from a trusted source, or the user consciously changed the source yourself.

Pytorch also do not offer CPU-only wheels on PyPI, they're available at https://download.pytorch.org/whl/cpu. In setup.py you're very limited in changing the source of the wheel you want. You provide a specific wheel via:

setup
(
...
install_requires = ["torch@https://download.pytorch.org/whl/cpu/
torch-2.1.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl"]
...
)

which isn't ideal as there are multiple versions depending on versions, python, os, architecture, etc...

I've looked at multiple work arounds for this, and all of them offering this functionality have been purposely closed off (requirements.txt, setup.cfg, pyproject.toml). I think it's better for us to accept the constraints rather than find another work around which would likely be removed.

My first suggestion is we document how to install a CPU only version - using the options provided to us by pip:

pip install medcat --extra-index-url https://download.pytorch.org/whl/cpu/

this will install the most relevant up to date version of pytorch for the cpu given our constraints provided in setup.py (currently torch>=1.13.0). This solution seems the most common among people I've seen trying to do the same as us here.

The second suggestion would be a wrapper bash script that would install MedCAT along with whatever extra parameters are required. This could be called when starting up a docker so the preferred version is installed saving network / disk usage.

I've added the extra documentation to the README.md and added a bash wrapper setup.sh for the pip install command (https://github.com/CogStack/MedCAT/tree/cpu-only).

Added conversations regarding PyPI and Pytorch:

Pytorch - https://github.com/pytorch/pytorch/issues/26340 PyPI - https://discuss.python.org/t/specifying-extra-index-url-in-setup-cfg-option-dependencies/19377