Should only load GeoIP2 database once.

ukwa / ukwa-heritrix

The UKWA Heritrix3 custom modules and Docker builder.

9 stars 7 forks source link

Should only load GeoIP2 database once. #1

Closed anjackson closed 10 years ago

anjackson commented 10 years ago

I just noticed that the current GeoIP2 lookup module re-load the GeoIP2 database from a file every single time a URL is looked up.

https://github.com/ukwa/bl-heritrix-modules/blob/master/src/main/java/uk/bl/wap/modules/deciderules/ExternalGeoLookup.java#L52

This is almost certainly rather inefficient. Instead, the database could be set up during construction/initialisation once, and then re-used across lookups.

PsypherPunk commented 10 years ago

Fixed in 3b607cfadb0957a4d835fad68f50902ea5bd0938.