mila-iqia / milatools

Tools to connect to and interact with the Mila cluster
MIT License
60 stars 11 forks source link

Feature suggestion: config file #92

Open vict0rsch opened 7 months ago

vict0rsch commented 7 months ago

Would be great to specify an identifier (like a project name or something) to map a project to a path, possibly per cluster. Something like

# ~/.milatools.yaml
myproject:
  mila: /home/mila/s/schmidtv/project/code/repos/repoName
  beluga: /home/vsch/scratch/project/repos/repoName

myproject2:
  mila: /home/mila/s/schmidtv/thatthing/foo/proj2
  beluga: /home/vsch/scratch/thatotherthing/bar/proj2-github

and then specify an identifier or project with -p myproject2 (or -i myproject2)


You could even push this feature further to be more complex and precise with alloc defaults

# ~/.milatools.yaml
myproject:
  mila: 
    path: /home/mila/s/schmidtv/project/code/repos/repoName
    alloc:
      cpus: 2
      memory: 12G
  beluga: 
    path: /home/vsch/scratch/project/repos/repoName

myproject2:
  mila: 
    path: /home/mila/s/schmidtv/thatthing/foo/proj2
    alloc:
      memory: 16G
  beluga: 
    path: /home/vsch/scratch/thatotherthing/bar/proj2-github

And potentially overwrite defaults with something like --remember-alloc

vict0rsch commented 7 months ago

And milatools would always create a config file with an example structure, either in comments or as an unused dummy project, if it does not already exist