search

coecms / ARCCSSive

ARCCSS Data Access Tools
Apache License 2.0
3 stars 1 forks source link

Improve the speed of compare_ESGF #26

Open ScottWales opened 6 years ago

ScottWales commented 6 years ago

compare_ESGF is slow, what can be done to make it faster?

paolap commented 6 years ago

Hi Scott, since we're moving to the mas database in a short time this isn't worth it. The slow point is retrieving the dataset information, i.e. pyesgf. I run lots of tests in the past, I think we discussed this before but decided to leave it aside since requests should be getting more and more rare and we're moving to a new system. The only change I did was to make it more efficient if you have more than one variable but this was at the expense of searching for a single variable.