sopstool installation failing, mozilla/sops crit unable to find

[Mikelew88]

Current Behavior

When installing sopstool on a large number of worker nodes on a spark cluster, there are times when sopstool fails to install on some nodes, leading to issues for the entire cluster. mozilla/sops crit unable to find is seemingly a throttling issue.

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 1937 100 1937 0 0 23258 0 --:--:-- --:--:-- --:--:-- 23337
mozilla/sops info checking GitHub for latest tag
mozilla/sops crit unable to find '' - use 'latest' or see https://github.com/mozilla/sops/releases for details
mv: cannot stat ���/tmp/bin/sopstool���: No such file or directory
mv: cannot stat ���/tmp/bin/sops���: No such file or directory
Completed 1.5 KiB/3.9 KiB (26.6 KiB/s) with 3 file(s) remaining
download: .../.ds_util_config.sops.yml to .../.ds_util_config.sops.yml
Completed 1.5 KiB/3.9 KiB (26.6 KiB/s) with 2 file(s) remaining
Completed 2.7 KiB/3.9 KiB (44.0 KiB/s) with 2 file(s) remaining
download: .../pip.sops.conf to .../pip.sops.conf
Completed 2.7 KiB/3.9 KiB (44.0 KiB/s) with 1 file(s) remaining
Completed 3.9 KiB/3.9 KiB (42.0 KiB/s) with 1 file(s) remaining
download: .../.sops.pypirc to .../.sops.pypirc
Completed 183 Bytes/183 Bytes (449 Bytes/s) with 1 file(s) remaining
download: .../.sops.yaml to ./.sops.yaml
/usr/lib/hadoop/logs/ds_bootstraps/default_spark.sh: line 85: sopstool: command not found
Collecting ds_util
Could not find a version that satisfies the requirement ds_util (from versions: )
No matching distribution found for ds_util

Expected Behavior

Most of the nodes in the cluster spin up correctly:

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 1937 100 1937 0 0 22920 0 --:--:-- --:--:-- --:--:-- 23059
mozilla/sops info checking GitHub for latest tag
mozilla/sops info found version 3.0.5 for 3.0.5/linux/amd64
mozilla/sops info downloading from https://github.com/mozilla/sops/releases/download/3.0.5/sops-3.0.5.linux
mozilla/sops info installed ./bin/sops
Ibotta/sopstool info checking GitHub for latest tag
Ibotta/sopstool info found version: 0.3.0 for v0.3.0/linux/amd64
Ibotta/sopstool info installed ./bin/sopstool
Both sops and sopstool installed
Completed 1.5 KiB/3.9 KiB (17.3 KiB/s) with 3 file(s) remaining
download: .../.ds_util_config.sops.yml to .../.ds_util_config.sops.yml
Completed 1.5 KiB/3.9 KiB (17.3 KiB/s) with 2 file(s) remaining
Completed 2.7 KiB/3.9 KiB (16.5 KiB/s) with 2 file(s) remaining
download: .../.sops.pypirc to .../.sops.pypirc
Completed 2.7 KiB/3.9 KiB (16.5 KiB/s) with 1 file(s) remaining
Completed 3.9 KiB/3.9 KiB (2.9 KiB/s) with 1 file(s) remaining 
download: .../pip.sops.conf to .../pip.sops.conf
Completed 183 Bytes/183 Bytes (550 Bytes/s) with 1 file(s) remaining
download: .../ds_util/.sops.yaml to ./.sops.yaml
Collecting ds_util

Steps to reproduce

Spamming installation seems to reproduce the issue.

[blimmer]

GitHub is very likely IP rate-limiting you. Can you try using the HTTP mirror instead?

[Mikelew88]

That could work, but I worry I'll still get rate-limited by github for sops, do you have any suggestion for how I can get a binary of sops?

[onyxraven]

https://github.com/Ibotta/sopstool#download-sops-from-our-https-mirror the HTTP mirror is on S3. So doing a download of the url as noted in that readme won't be limited.

[Mikelew88]

Ah, I see now, that's exactly what I needed. Thank you!

[blimmer]

No problem - if there's a documentation change that could help clarify this use case, please feel free to submit a PR.