Refactor the bottle neck in combining the geovals

JCSDA-internal / ioda-converters

Various converters for getting obs data in and out of IODA

9 stars 4 forks source link

Refactor the bottle neck in combining the geovals #1352

Closed danholdaway closed 1 year ago

danholdaway commented 1 year ago

Description

The combine_obsspace code for contains a nasty bottle neck when combining GeoVaLs, where there is a rather inefficient nested loop. For GMAO when converting all the of the observations from GSI to IODA format and including the GeoVaLs it was taking around 50 minutes. After this change it's more like 8 minutes.

danholdaway commented 1 year ago

Much cleaner, yes I never liked the loops and it's not surprising that this would be much more efficient. Thanks for this!