64 project template potentially outdated

[mikewoodward94]

Summary of changes

I've updated various aspects of this repo with relation to model training. Tidied up imports in all scripts.

scripts/train.py

• Added mlflow set up, xnat_config, and a few more params to the start of the script.
• Added more comprehensive datamodule and network set ups.
• Added callbacks with testing of test dataset.

project/DataModule.py

• Added transform composing.
• Changed flow to be clear, get_data -> validate_data -> data split
• Added visualise data functionality.
• Added caching.
• Added example xnat fetch actions.
• Added dataset_stats function.

project/Network.py

• Added create_model example.
• Added metrics example.
• Added test_end figure generation.

config/config_files.cfg

• Added xnat and params sections

transforms/init.py

• Added example transform lists

requirements.txt

• Cleaned up requirements to bare minimum

Dockerfile

• Simplified to pull from python image

Reason for changes

Utilised this repo recently and found that a lot was missing/out of date. Will hopefully be useful for future users!

[mikewoodward94]

4. You've added a lot to the DataModule.py and Network.py files, which is great, albeit it takes it away from being a 'general' project template. Again, I think it's worth making it very clear in the README.md that the template is suggested code and that users should expect to have to change significant chunks of the code for their project. What's provided is really an approximate framework that people must edit it.

I agree, I was in two minds on whether to leave it more general or add a lot. Decided to go with adding a lot with the thinking being it's easier to have examples and then delete what's not needed than not have examples and need to figure out how to add what's needed? Happy to go the other way if you think it's more useful!

[tomaroberts]

it's easier to have examples and then delete what's not needed than not have examples and need to figure out how to add what's needed

Yeah, I agree with this. The only downside is people taking for granted what you have and not realising what should be edited and what should be left. For me, the general 'structure' of having DataModule and Network classes are broadly consistent between projects, whereas the methods and functions are editable. Just need to make that clear in README really.

Another thought actually – in the README, can you add a section to link to our actual projects that use MLOps too, e.g. ScaphX, CARNAX, dental, etc. Gradually we'll build up a list of examples.