helium / blockchain-etl

Blockchain follower that follows and stores the Helium blockchain
Apache License 2.0
64 stars 37 forks source link

Adds Docker Support #152

Closed chrisabruce closed 3 years ago

chrisabruce commented 3 years ago

This pull request takes the work Evan already did on the validator to support Docker and modifies it to work for ETL.

dansku commented 3 years ago

can't wait! :)

nmharmon8 commented 3 years ago

It seem that rebar3 as docker_etl tar hits github's rate limit for cloning. Is there a way to throttle rebar3?

===> Failed to fetch and copy dep: {git,"https://github.com/helium/erlang-h3.git",
                                  {ref,
                                      "cdc3a10fb5ad79e1ba01893bebcd28d6944851f1"}}
The command '/bin/sh -c rebar3 as docker_etl tar' returned a non-zero code: 1

Each time I try and build the dockerfile it fails on a different dep. That is leading me to believe that github it interrupting the clones. https://stackoverflow.com/a/50567020/3757091

madninja commented 3 years ago

Not that I know and that'd be the first time I've seen that happen

On Wed, Apr 14, 2021 at 5:21 PM Nathan Harmon @.***> wrote:

It seem that rebar3 as docker_etl tar hits github's rate limit for cloning. Is there a way to throttle rebar3?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/helium/blockchain-etl/pull/152#issuecomment-819911499, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACZBRPI3KCJ7VSTDDNUQDTIYPQTANCNFSM42VCYZHQ .

nmharmon8 commented 3 years ago

I was able to get past it by replacing ``rebar3 as docker_etl tar ``` with a script:

until rebar3 as docker_etl tar 
do
  echo "Try again"
done

Not a good solution. It failed with "Failed to fetch and copy dep" quite a few time. It succeeded to download what it failed on the next time.

chrisabruce commented 3 years ago

@madninja are we good to merge this now?