search

E3SM-Project / zppy

E3SM post-processing toolchain
BSD 3-Clause "New" or "Revised" License
6 stars 15 forks source link

[Bug]: Change permissions and ownership for the temp directories to be the same as the parent directory #636

Open forsyth2 opened 1 month ago

forsyth2 commented 1 month ago

What happened?

When checking disk usage, errors like du: cannot read directory 'ac.benedict/E3SMv3_dev/20231012.v3alpha04_bigrid_L80_QBO1.F20TR.chrysalis/post/scripts/tmp.407421.rCki': Permission denied. This is because of a permissions issues with the temp directories zppy creates.

What machine were you running on?

Chrysalis

Environment

N/A

What command did you run?

du

Copy your cfg file

N/A

What jobs are failing?

No response

What stack trace are you encountering?

No response

xylar commented 1 month ago

@forsyth2, while you're at it, could you give the tmp directories a more useful name (say including ts or climo or whatever the task is)? That would also help with debugging if something goes wrong.

It might be easiest to find something else you can call to generate a random hash for the end of the directory name, but then just use a normal mkdir call so permissions are unaffected.

xylar commented 1 month ago

mktemp --dry-run might be helpful for just generating the hash:

hash=`mktemp --dry-run -d XXXX`
work_dir=`mkdir tmp.ilamb.${id}.${hash}`
forsyth2 commented 3 weeks ago

@chengzhuzhang also noted that tmp directories with ... can't get deleted either. @xylar noted one possible cause for that is a regex parsing issue.

xylar commented 3 weeks ago

I think you probably need to delete them manually, finding the appropriate escape characters. Sometimes tab completion can tell you what they are.