Closed nanli-emory closed 7 months ago
Just tested this PR, nicely done @nanli-emory
This is ready to be merged in @choosehappy
This is a bit of a tangent, but I found the QA docker installation instructions unclear.
Why do we mount -v /data/$CaseID/QuickAnnotator:/opt/QuickAnnotator
? I omitted the mount in my docker run
and it doesn't have an effect.
Why do we mount -v /data/$CaseID/QuickAnnotator:/opt/QuickAnnotator ? I omitted the mount in my docker run and it doesn't have an effect.
i think if you don't do this, then the images + masks internally stored within quickannotator will remain in the docker and not in the base o/s available for downstream usage?
further, i think there is a strong motivation for having project files in the base o/s (where they can be backedup and stored) versus the docker container which may accidently be deleted, taking annotation work with it?
i think basically the docker container is to provide the environment, but the data is kept externally?
Got it.
Is there an I/O performance hit associated with this? Since QuickAnnotator needs to rapidly read and write in the mounted directory. https://stackoverflow.com/questions/62493402/is-read-write-performance-better-with-docker-volumes-on-windows-inside-of-a-doc
Asking also because I/O will be even more important in V2.
fascnating -- didn't know about or even consider this
hmmm is this relevant only for windows, i think so?
some other resources:
https://docs.docker.com/desktop/wsl/best-practices/ https://github.com/docker/for-win/issues/6742
https://docs.docker.com/storage/volumes/
for any seriously scalable stuff, i think it'll have to accept that it will be linux based - giving us some freedom i think in how we design v2
it looks like we should consider using volumes instead of bind mounts and/or not mounting things from the WSL2 /mnt/c/ share and instead using internal linux directories which also seem to provide good speed
if i understand correctly, the worst case, is that you're on the windows CLI cmd, and run a docker -v
if you go into wsl2, at the bash command line and do a docker -v /opt/data:/opt/data
since its not in the windows o/s, the speed should be ideal? but volumes provide a way of being totally agnostic
that said, in this use we may have to think about it more and provide better documentation, it seems like a lot of this will depend on how the user configures thing? that said, i don't think QA is doing a lot of rapid reading/writing? its mostly writes when people annotate - i suspect we're talking about a few seconds difference
there may, however, be an impact when doing DL training, since the reads require rapid reading -- but in the case of QA in general the datasets are sufficiently small (since they start from 0 and actively learning build up), that after the first read they're in memory cache anyway -- so I'm not sure we ever pay that time consequence unless the DBs get larger than ram?
create a new cuda_12 folder for cuda12 docker install change ./requirments.txt and ./Dockerfile for cup only docker install