glasgowcompbio / vimms

A programmable and modular LC/MS simulator in Python
MIT License
19 stars 6 forks source link

Unable to use 64-bit packages #223

Closed joewandy closed 1 year ago

joewandy commented 3 years ago

To run IAPI library, we need to use Python in 32-bit: https://github.com/thermofisherlsms/iapi/issues/2.

However it seems impossible to install ax-platform, stable-baselines3 or any package that depends on pyTorch .. because pyTorch isn't supported on 32-bit Python.

https://github.com/pytorch/pytorch/pull/40321 There's isnt even any 32-bit whl https://download.pytorch.org/whl/torch/

TensorFlow also does not support 32-bit platforms so I don't think we can use any DL methods in the controller -- at least not straightforward to do this. This will be a problem when @mcbrider5002 wants to do real-time peak picking in the future, as it's going to be a DL model.

To get around this, we could perhaps run the controller logic on a separate localhost service, as discussed a long time ago? So there's a separate controller service running on localhost that contains the actual fragmentation strategy. A shell controller processes scans, forwards it to the controller service, and receives back the scan commands to perform which it sends to IAPI.

Alternatively @RonanDaly suggested that people have had success in using multiprocessing to do this - https://stackoverflow.com/questions/52919107/a-python-script-in-32-bit-invoking-multiprocessing-running-in-64-bit-virtualenv. Obviously need to pass data via whatever multiprocessing uses - named pipes perhaps - so not as quick as shared memory, but if it's a small amount it shouldn't be an issue. This could perhaps also be a speedup, as code in other process can run in parallel - we just have to design it carefully.

For now, just so we don't need to keep commenting codes for every git pull on the instrument PC, I'll move the experimental codes in the files below that rely on 64-bit packages to its own separate repo:

joewandy commented 1 year ago

The latest version of IAPI now supports 64 bit, so we just need to upgrade to it.