Version 2.4.11 introduces a regression

[secastro]

Hi team:

Thanks for all the work you do on ENTRADA.

This ticket reports a regression introduced by a recent change push to docker with image 2.4.11.

The change was intended to avoid a container instance from downloading the MaxMind database again if the existing version on disk was fresh enough, but the change prevents the container from starting.

Below is the error message docker is reporting when trying to start.

    : Create directory: /entrada/data/conf
2024-08-06 10:13:38.426  INFO 1 --- [main] n.s.e.enrich.geoip.GeoIPServiceImpl
    : Using Maxmind database location: /entrada/data/work/maxmind
2024-08-06 10:13:38.682  WARN 1 --- [main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'countryEnrichment' defined in URL [jar:file:/entrada/bin/entrada.jar!/BOOT-INF/classes!/nl/sidnlabs/entrada/enrich/geoip/CountryEnrichment.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'geoIPServiceImpl': Invocation of init method failed; nested exception is java.lang.NullPointerException: Cannot invoke "java.util.Date.after(java.util.Date)" because "lastModified" is null

Happy to attach more comprehensive logs if needed.

[mwullink]

Hi,

Are you sure you are using the correct Maxmind API endpoint and the correct credentials? Looking at the code i see that this error can only happen when http HEAD request returns <> 200

example url: https://download.maxmind.com/app/geoip_download?edition_id=GeoIP2-Country&suffix=tar.gz&license_key=xxxxxx_key_here_xxxxx

maybe try the http head request using curl and check the status code.

i can add some additional logging to show error when HEAD request failed.

- Maarten

Op 6 aug 2024, om 12:22 heeft Sebastian Castro @.***> het volgende geschreven:

Hi team:

Thanks for all the work you do on ENTRADA.

This ticket reports a regression introduced by a recent change push to docker with image 2.4.11.

The change was intended to avoid a container instance from downloading the MaxMind database again if the existing version on disk was fresh enough, but the change prevents the container from starting.

Below is the error message docker is reporting when trying to start.

: Create directory: /entrada/data/conf

2024-08-06 10:13:38.426 INFO 1 --- [main] n.s.e.enrich.geoip.GeoIPServiceImpl : Using Maxmind database location: /entrada/data/work/maxmind 2024-08-06 10:13:38.682 WARN 1 --- [main] ConfigServletWebServerApplicationCont ext : Exception encountered during context initialization - cancelling refresh a ttempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'countryEnrichment' defined in URL [jar:file:/entrada/bi n/entrada.jar!/BOOT-INF/classes!/nl/sidnlabs/entrada/enrich/geoip/CountryEnrichm ent.class]: Unsatisfied dependency expressed through constructor parameter 0; ne sted exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'geoIPServiceImpl': Invocation of init method failed; n ested exception is java.lang.NullPointerException: Cannot invoke "java.util.Date .after(java.util.Date)" because "lastModified" is null

Happy to attach more comprehensive logs if needed.

— Reply to this email directly, view it on GitHubhttps://github.com/SIDN/entrada/issues/200, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AALSFIFZEIGWEY2IDCPSB5DZQCPWRAVCNFSM6AAAAABMCARR2WVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQ2TANJRGQ4DGMQ. You are receiving this because you are subscribed to this thread.Message ID: @.***>

[secastro]

Using curl to retrieve the MaxMind database file

curl "https://download.maxmind.com/app/geoip_download?edition_id=GeoIP2-Country&suffix=tar.gz&license_key=<LICENSE_KEY>"
Daily GeoIP database download limit reached

and even using the HEAD request, returns an error

curl --head "https://download.maxmind.com/app/geoip_download?edition_id=GeoIP2-Country&suffix=tar.gz&license_key=<LICENSE_KEY>"
HTTP/2 429
date: Tue, 06 Aug 2024 13:21:06 GMT
content-type: text/plain;charset=UTF-8
content-length: 44
cache-control: private, max-age=0
expires: Tue, 06 Aug 2024 13:21:06 GMT
strict-transport-security: max-age=31536000; includeSubDomains; preload
server: cloudflare
cf-ray: 8aef53ff9f096403-LHR

complaining there are too many requests.

This could mean your solution might not work as expected.

[mwullink]

Yes, the MM ratelimit is anoying, i made a change in 2.4.12 to not download anything if the HEAD check fails. But it may take some time for the MM ratelimiter counters to reset to allow you to download again.