Building from source; ending with Failed to create "/opt/arkime/etc/GeoLite2-ASN.mmdb" in "arkime"

[vanyell]

scripts/build.sh: line 130: ((: > 0 : syntax error: operand expected (error token is "> 0 ") Failed to create "/opt/arkime/etc/GeoLite2-ASN.mmdb" in "arkime"

Already added MAXMIND key in the dockerfile.

Is this fatal?

[mmguero]

Just FYI if you don't want to have to modify the dockerfile, if you run the ./scripts/build.sh (with no arguments to build all services or with the name(s) of the services you want to build images for, e.g., ./scripts/build.sh arkime) it will take the value of MAXMIND_GEOIP_DB_LICENSE_KEY from an environment variable.

As far as it not getting that file, I just tried a build on a couple of different machines on two networks and didn't have that problem. Is the problem you're having consistently reproducible? I.e., if you run ./scripts/build.sh arkime does it fail again in the same way?

[vanyell]

Thanks for the reply...

FYI. I entered the MAXMIND key both in arkime-secrets.env and the Dockerfile.

Still the same, unless I'm doing something wrong. I already tried manually doing the curl download part.. It does work though.

28 [arkime stage-1 19/21] RUN [ ${#MAXMIND_GEOIP_DB_LICENSE_KEY} -gt 1 ] && for DB in ASN Country City; do cd /tmp && curl -s -S -L -o "GeoLite2-$DB.mmdb.tar.gz" "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-$DB&license_key=$MAXMIND_GEOIP_DB_LICENSE_KEY&suffix=tar.gz" && tar xf "GeoLite2-$DB.mmdb.tar.gz" --wildcards --no-anchored '.mmdb' --strip=1 && mkdir -p $ARKIME_DIR/etc/ $ARKIME_DIR/rules/ $ARKIME_DIR/logs/ && mv -v "GeoLite2-$DB.mmdb" $ARKIME_DIR/etc/; rm -f "GeoLite2-$DB"; done; curl -s -S -L -o $ARKIME_DIR/etc/ipv4-address-space.csv "https://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.csv" && curl -s -S -L -o $ARKIME_DIR/etc/oui.txt "https://www.wireshark.org/download/automated/data/manuf"

28 DONE 2.9s

31 [arkime] exporting to image

31 exporting layers

31 exporting layers 4.9s done

31 writing image sha256:451674527c304353b806f4e3136f958fd161dd79c5b7d1c87e376b52c58c1c46 done

31 naming to ghcr.io/idaholab/malcolm/arkime:23.12.1 0.0s done

31 DONE 4.9s

scripts/build.sh: line 130: ((: > 0 : syntax error: operand expected (error token is "> 0 ") Failed to create "/opt/arkime/etc/GeoLite2-ASN.mmdb" in "arkime"

[mmguero]

Can you just test locally and make sure your key is accepted? Run this script, replace XXXXXXXXXXX with your key or have MAXMIND_GEOIP_DB_LICENSE_KEY as a local environment variable.

#!/bin/bash

MAXMIND_GEOIP_DB_LICENSE_KEY=${MAXMIND_GEOIP_DB_LICENSE_KEY:-XXXXXXXXXXX}

pushd /tmp >/dev/null 2>&1
[ ${#MAXMIND_GEOIP_DB_LICENSE_KEY} -gt 1 ] && for DB in ASN Country City; do
    curl -s -S -L -o "GeoLite2-$DB.mmdb.tar.gz" "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-$DB&license_key=$MAXMIND_GEOIP_DB_LICENSE_KEY&suffix=tar.gz" && \
    tar xf "GeoLite2-$DB.mmdb.tar.gz" --wildcards --no-anchored '*.mmdb' --strip=1
    rm -f "GeoLite2-$DB.mmdb.tar.gz"
done
ls -l ./GeoLite*.mmdb
popd >/dev/null 2>&1

Do you end up with output like:

-rw-r--r-- 1 user user  8327628 Jan  2 07:16 ./GeoLite2-ASN.mmdb
-rw-r--r-- 1 user user 67564232 Jan  2 07:34 ./GeoLite2-City.mmdb
-rw-r--r-- 1 user user  6453602 Jan  2 07:37 ./GeoLite2-Country.mmdb

[vanyell]

Yes I do end up with the said output

[mmguero]

One thing I saw in the output you pasted (and perhaps it's just github's formatting) was --wildcards --no-anchored '.mmdb' which does not match what the Dockerifle has. Could this have gotten changed somehow in your local file?

I don't have any other ideas why it wouldn't be working for you. I guess you could put a set -x at the beginning of that part of the dockerfile (e.g., RUN set -x && [ ${#MAXMIND_GEOIP_DB_LICENSE_KEY} -gt 1 ] && ...) and watch the verbose bash output?

[vanyell]

It seems that MAXMIND_GEOIP_DB_LICENSE_KEY is evaluating to 0. Isn't arkime-secret.env supposed to be read here?

with your previous script, MAXMIND_GEOIP_DB_LICENSE_KEY is evaluating to 40. Weird

[mmguero]

so that's the length of the string, which is making sense why you're not getting anything. the arkime-secret.env file is used at runtime, not build time.

I think if you build it like this:

$ MAXMIND_GEOIP_DB_LICENSE_KEY=XXXXXXX ./scripts/build.sh arkime

It will work.

[vanyell]

Thanks. That did it