search

kirchhausenlab / incasem

Automated Segmentation of cellular substructures in Electron Microscopy
BSD 3-Clause "New" or "Revised" License
16 stars 1 forks source link

Removing MongoDB dependency? #13

Open GenevieveBuckley opened 11 months ago

GenevieveBuckley commented 11 months ago

Following up on an earlier conversation with @bentaculum about removing MongoDB as a dependency.

What exactly would be involved with something like this? Eg: Is a replacement required or can it be removed entirely, what technical issues might this cause, etc. I see that pymongo is used in the train, predict, and predict_multiple scripts, but I don't have a lot of insight beyond that.

GenevieveBuckley commented 11 months ago

(For extra context: our HPC team have said that this is a blocker for use (a persistent service requiring execution from a privileged account is not a good fit for shared use HPC systems, and there are possible cybersecurity risks that limit our options. So it would be nice if we could work around that issue altogether)

bentaculum commented 11 months ago

Hi Genevieve, @patrickstock is working on making jupyter notebooks (potentially even cloud hosted on Colab/Sagemaker) for inference (WIP) and training (coming soon) that don't talk to an experiment tracking database, see here. This will result in getting rid of the mongoDB dependency entirely. We'll be happy to notify you once this is ready for trying it out.

GenevieveBuckley commented 11 months ago

Very cool, we'd love to try that out when it's ready