ImagingDataCommons / CloudSegmentator

Medical imaging segmentation workflows for FireCloud (Terra) and Seven Bridges Cancer Genomics Cloud
Apache License 2.0
3 stars 2 forks source link

Resolve Issue #9 (Reorganizing dockerfiles and docker images CI/CD) #26

Closed vkt1414 closed 1 year ago

vkt1414 commented 1 year ago
  1. reorganize the docker files to name them more intuitively and include description of the functionality

    • Dockerfiles are now reorganized by their task names instead of task numbers.
    • Old vs New Folders
      • task1 = imagingdatacommons/download_convert task2 =imagingdatacommons/inference_totalseg task3 =imagingdatacommons/radiomics tasks1and2 =imagingdatacommons/download_convert_inference_totalseg end_to_end =imagingdatacommons/download_convert_inference_totalseg_radiomics
  2. set up automated docker image build process (probably, using GitHub Actions) and deposit each functionally distinctive image into an intuitively named DockerHub repo, tagged with the GitHub hash as a version

    • for each dockerfile, a github actions script is employed to build dev or latest tags depending on the branch (dev or main), whenever the corresponding dockerfile or the weight_download script or the github actions script itself is modified.
    • https://hub.docker.com/?namespace=imagingdatacommons
    • A short description and overview is written for each docker image repository
    • The docker image will have a label layer, containing the maintainer, base image used, githash, and the url to the dockerfile corresponding to the git commit hash
    • The docker base images are now used with sha256 digest instead of the tag directly as the tag may be overwritten.

The last four commits spilled are related to other issues. Will isolate branches going forward.

fedorov commented 1 year ago

@vkt1414 nice PR summary!

When you reference the corresponding issue, it's usually done by using hash sign followed by issue number in the commit message - this will establish the connection - such as #9.

fedorov commented 1 year ago

@vkt1414 sounds good - we should describe those points in the documentation under the respective folder of the repo. Will you update the other workflows or should I do it?

vkt1414 commented 1 year ago

@vkt1414 sounds good - we should describe those points in the documentation under the respective folder of the repo. Will you update the other workflows or should I do it?

I can add/update the readme.md file with in the cwl files folder, to explain how the CWL file is generated, what we chose to keep and what we stripped.

fedorov commented 1 year ago

We can do this separately, no need to hold this PR.

Can you make the other changes I asked - updating workflows as the one I modified, and also pushing the images only when the commit is made to the main branch? Let me know if you need help.

vkt1414 commented 1 year ago

We can do this separately, no need to hold this PR.

Can you make the other changes I asked - updating workflows as the one I modified, and also pushing the images only when the commit is made to the main branch? Let me know if you need help.

Sure. I just made changes to github actions files, the same way you edited one yesterday, to rebuild images on main branch only with a pull request.

I also renamed the radiomics to dicom_seg_pyradiomics_sr to describe more closely what we are using the docker image for.. I believe these are all the changes related to dockerfiles.