search

valeriupredoi / bgcval2

Package for BGCVal v2.0
3 stars 0 forks source link

Automate downloading on mass-cli1. #69

Closed ledm closed 1 year ago

ledm commented 1 year ago

The recent Mass PR works (#63), yay, but there are some issues, notably with permissions, and downloading via cron.

Permissions:

We need to make downloaded data accessible from anyone. At the moment, whoever first downloads a jobID, creates and owns the directory in /gws/nopw/j04/ukesm/BGC_data/. However, this means that if someone else wants to update that jobID download with recent data, they can not. The solution is to put `chmod g+w; command in the python script.

Downloading via crontab

So, for some reason, it looks like the mass-cli1 cronjob is run by root, who can't run moo get, so we need to source /etc/bash. Following the suggestion of Alan, from the jasmin help desk. (Thanks Alan!)

This is already done and will be submitted in a PR soon.

valeriupredoi commented 1 year ago

hey bud @ledm Happy New Year (here too haha)!

if someone else wants to update that jobID download with recent data, they can not. The solution is to put `chmod g+w; command in the python script.

I don't quite get it - the data downloaded in the repo (which is already group writeable to) is readable, but not x-ecutable or w-riteable to (as it should be); are you saying you want a chmod g+x be run on the dir eg /gws/nopw/j04/ukesm/BGC_data/u-xyyyzz?