search

COSIMA / master_index

Scripts to generate COSIMA master index
0 stars 0 forks source link

Exclude directory from indexing #30

Closed micaeljtoliveira closed 1 year ago

micaeljtoliveira commented 1 year ago

Add a mechanism to exclude some directories from the indexing and use it to (temporarily, I hope) exclude one directory that has been making the indexing fail.

micaeljtoliveira commented 1 year ago

@aekiss Does this look reasonable? This directory has been preventing the database update to run smoothly for quite some time now.

aekiss commented 1 year ago

I've reminded Paul about his corrupt file /g/data/ik11/outputs/access-om2-01/01deg_jra55v140_iaf_cycle4_MWpert/restart882/ocean/ocean_velocity_advection.res.nc

micaeljtoliveira commented 1 year ago

@aekiss Thanks for reminding Paul and for the comment (good to refresh my bash). I'll merge it as it is, as I don't think there's the need to spend too much time with this. I would rather spend some time improving the python indexing script to avoid these issues altogether.

aekiss commented 1 year ago

great, thanks

aekiss commented 1 year ago

@micaeljtoliveira Paul has just renamed the offending file to 

/g/data/ik11/outputs/access-om2-01/01deg_jra55v140_iaf_cycle4_MWpert/restart882/ocean/ocean_velocity_advection.res.nc-CORRUPT

so that will (I think) automatically exclude it.

So I guess we should now set

EXCLUSIONS=( )

?

micaeljtoliveira commented 1 year ago

Ups, should have waited a little longer.

micaeljtoliveira commented 1 year ago

@aekiss See #31