For our EUMETCAST GDS processing, granules may arrive very late, sometimes more than 90 minutes after the initial measurement. Currently, gatherer measures the timeliness only by comparing processing time with measurement time, so if timeout is set to 90 minutes and a message arrives about a newly arrived granule that measured 92 minutes ago, gatherer will conclude a timeout immediately. Currently, this means that either gatherer ships out too many short granules (defeating the point of gatherer) or the user needs to set the timeout very long, which leads to other problems. Currently I have the timeout set to two hours, which sometimes leads to high-latitude granules being gathered between subsequent orbits.
I would like to see an option in gatherer in which the timeliness is measured not by "current time - measurement time" but by "current time - time of first granule". As an added bonus, this will also make it somewhat easier to test a setup by dropping in old files.
For our EUMETCAST GDS processing, granules may arrive very late, sometimes more than 90 minutes after the initial measurement. Currently, gatherer measures the timeliness only by comparing processing time with measurement time, so if timeout is set to 90 minutes and a message arrives about a newly arrived granule that measured 92 minutes ago, gatherer will conclude a timeout immediately. Currently, this means that either gatherer ships out too many short granules (defeating the point of gatherer) or the user needs to set the timeout very long, which leads to other problems. Currently I have the timeout set to two hours, which sometimes leads to high-latitude granules being gathered between subsequent orbits.
I would like to see an option in gatherer in which the timeliness is measured not by "current time - measurement time" but by "current time - time of first granule". As an added bonus, this will also make it somewhat easier to test a setup by dropping in old files.