choosehappy / QuickAnnotator

An open-source digital pathology based rapid image annotation tool
BSD 3-Clause Clear License
74 stars 27 forks source link

modify cuda_11 install and create cuda_12 and cup docker install, remove cuda_10 install #38

Closed nanli-emory closed 5 months ago

nanli-emory commented 7 months ago

create a new cuda_12 folder for cuda12 docker install change ./requirments.txt and ./Dockerfile for cup only docker install

jacksonjacobs1 commented 5 months ago

Just tested this PR, nicely done @nanli-emory

This is ready to be merged in @choosehappy


This is a bit of a tangent, but I found the QA docker installation instructions unclear.

Why do we mount -v /data/$CaseID/QuickAnnotator:/opt/QuickAnnotator ? I omitted the mount in my docker run and it doesn't have an effect.

choosehappy commented 5 months ago

Why do we mount -v /data/$CaseID/QuickAnnotator:/opt/QuickAnnotator ? I omitted the mount in my docker run and it doesn't have an effect.

i think if you don't do this, then the images + masks internally stored within quickannotator will remain in the docker and not in the base o/s available for downstream usage?

further, i think there is a strong motivation for having project files in the base o/s (where they can be backedup and stored) versus the docker container which may accidently be deleted, taking annotation work with it?

i think basically the docker container is to provide the environment, but the data is kept externally?

jacksonjacobs1 commented 5 months ago

Got it.

Is there an I/O performance hit associated with this? Since QuickAnnotator needs to rapidly read and write in the mounted directory. https://stackoverflow.com/questions/62493402/is-read-write-performance-better-with-docker-volumes-on-windows-inside-of-a-doc

Asking also because I/O will be even more important in V2.

choosehappy commented 5 months ago

fascnating -- didn't know about or even consider this

hmmm is this relevant only for windows, i think so?

some other resources:

https://docs.docker.com/desktop/wsl/best-practices/ https://github.com/docker/for-win/issues/6742

https://docs.docker.com/storage/volumes/

for any seriously scalable stuff, i think it'll have to accept that it will be linux based - giving us some freedom i think in how we design v2

it looks like we should consider using volumes instead of bind mounts and/or not mounting things from the WSL2 /mnt/c/ share and instead using internal linux directories which also seem to provide good speed

if i understand correctly, the worst case, is that you're on the windows CLI cmd, and run a docker -v

if you go into wsl2, at the bash command line and do a docker -v /opt/data:/opt/data

since its not in the windows o/s, the speed should be ideal? but volumes provide a way of being totally agnostic

that said, in this use we may have to think about it more and provide better documentation, it seems like a lot of this will depend on how the user configures thing? that said, i don't think QA is doing a lot of rapid reading/writing? its mostly writes when people annotate - i suspect we're talking about a few seconds difference

there may, however, be an impact when doing DL training, since the reads require rapid reading -- but in the case of QA in general the datasets are sufficiently small (since they start from 0 and actively learning build up), that after the first read they're in memory cache anyway -- so I'm not sure we ever pay that time consequence unless the DBs get larger than ram?