UNM-CARC / QuickBytes

Development of short tutorials for UNM's Center for Advanced Research Computing
18 stars 16 forks source link

Use Scratch on Taos #75

Closed gmfricke closed 3 years ago

gmfricke commented 4 years ago

The previous version had all the IO on the home directory

gmfricke commented 4 years ago

@liphardt Please try running this and see if it works. You might want to do something similar for the wheeler script.

liphardt commented 4 years ago

@gmfricke I am testing this now to make sure it works the way we want. I added an additional couple of steps to create a directory for the additional output of an Orca job, copy that additional output to the directory, and then clean the scratch directory after. Does this seem reasonable? I can modify the Wheeler PBS script to do the same.

gmfricke commented 4 years ago

@liphardt It does seem reasonable. I think we are working from the same scratch PBS example :) It always makes me nervous to include anything in a quickbyte that automates the deletion of a directory. There is a chance, even if remote, that scratch/$prefix already exists and contains data the user cares about. I think you should add a scratch directory already exists check. Something like if [ -d $scratch_dir/$prefix/ ] then exit with an error. Alternatively, you can use mktemp on scratch which makes some (maybe?) guarantees about directory uniqueness.

liphardt commented 4 years ago

@gmfricke Yeah, that is a really good point. I will work on setting that up.

gmfricke commented 3 years ago

@liphardt Just checking the status on this.

liphardt commented 3 years ago

@gmfricke Sorry. I will get it done this week.

liphardt commented 3 years ago

@gmfricke I added the temporary directory in scratch to work out of.