Closed velll closed 2 months ago
š Hi Pavel! Hope you're doing well!
This looks similar to https://github.com/docker/scout-cli/issues/89#issuecomment-1932367067. Our internal conversations say this isn't anything we recognise our side, maybe a proxy or something similar?
We did a bit of a github search for the error message, which didn't throw up much but we did see this unless its related to your registry https://github.com/kubernetes/registry.k8s.io/blob/218104931605223efecdbbf36f338cac3f6b3b12/docs/debugging.md?plain=1#L10
I'm pretty sure the v1/graphql
is a call to the Scout API hosted on GCP and there are reports online of Google giving that 403 error for various reasons like a blocked IP address, calls from a sanctioned country (e.g. Iran), so perhaps there is something going on there. Does it always fail for the same image e.g. if you retry the action? Or is it sporadic even for the exact same call? I'm wondering if there is some kind of github <-> google rate limiting going on. My guess is that you've successfully pulled the image, extracted the SBOM then the Scout CLI connects to our API to match the SBOM packages to the latest CVE information and thats the call thats failing.
You said you're self hosting the runners. How many runners do you have, is it possible there is one of them with a dodgy IP address that's being mistaken for somewhere sanctioned?
Thank you for the quick reply Mike! ā”
We do self host runners for these workflows in our Azure cloud in EU šŖšŗ I will try reproducing this directly from the runners next week, and see if changing the IPs helps.
The action does actually consistently fail for the same image, so I don't immediately suspect network. But there very much could be something interesting there.
Thank you for all the details! I will continue next week and will post the findings
Took us a while to get to it, but we basically confirmed that this error is related to a banned IP.
We build our images on self hosted runners, and for one of the workflows we had only one machine operating runners with the specific set of labels. This machine was unlucky to get an IP that was banned, that's why we were seeing the errors consistently.
We confirmed this by running docker scout
commands directly from the machines hosting the runners (docker/scout-cli
image was extremely useful for running commands on a server while having a minimal footprint).
We're confident, that the only difference in the configuration is the IP address. We replaced the old machine, and didn't encounter these errors since.
I'm closing the issue. Thank you for the support! Hope this thread helps the next team running into a similar issue.
Hi everyone! š
We are using the
docker/scout-action
in our CD pipeline, and we started getting 403 errors from thecves
command today. The error looks like this:This does not happen on every image. I am getting cves for some of our images (logs, comments, the works), and errors for others. Couldn't find any system. This makes me think that it's not an issue with our secrets (PAT).
ā Could you help me understand what could lead to this error? Which service does the action tries to reach in this case? (Our registry, something internal in scout, github? What should I double check in our access credentials?)
What I tried?
I tried running
docker scout cves
with the same configuration on the same images locally on v1.9.3 ā the command runs, and I am receiving correct output in all cases. This makes me think the error is related to the actions specifically. š¤I also tried setting the version of the action to v1.9.1, this did not change the error.
Runners
We run our actions on self-hosted x86 ubuntu
Output
Here is the full output of the action including the configuration
Please let me know if you need additional information