nasa-jpl / its_live_production

A NASA MEaSUREs project to provide automated, low latency, global glacier flow and elevation change datasets
MIT License
17 stars 6 forks source link

S2 transfer from ASF and fix filename on the fly #4

Closed jhkennedy closed 2 months ago

jhkennedy commented 2 years ago

From a Sentinel-2 pair like

S2A_6VXP_20210408_0_L1C S2A_6VXP_20210418_0_L1C

autoRIFT currently produces a netCDF with the name

S2A_20210408_L1C_X_S2A_20210418_L1C_G0120V02_P072.nc

However, this name is not necessarily unique as multiple Sentinel-2 acquisitions will have the same time at the precision used in the file name. If instead the autoRIFT files included the whole Sentinel-2 filenames

S2A_6VXP_20210408_0_L1C_X_S2A_6VXP_20210418_0_L1C_G0120V02_P072.nc

the "tile" section would prevent clobbering.

This PR modifies the ingest script to:

NOTE: I have not tested this! This is also a one-off to fix problem in already generated files and likely shouldn't be merged as the filenames should be fixed upstream.

mliukis commented 2 years ago

@jhkennedy Thank you for proposed code changes! Is it going to be a permanent change to the job ID file? Will it always provide output filename for the granule?

jhkennedy commented 2 years ago

@mliukis I would expect that I/we will modify the code to product the correct output file going forward -- this would be use just once to re-ingest the current S2 files with correct names

mliukis commented 2 years ago

@jhkennedy I will add it as conditional fix then. Thanks!

jhkennedy commented 2 years ago

@mliukis here's the associated CSV for this PR

s2_mini_ak_sv_jk_succeeded_pairs.csv