Failing to pull base image (possibly permission issues)

[doggopadre]

I have setup the Dockerfile using this template here, When I do docker build im getting this error below:

I even tried removing the trailing :VERSION to let docker resolve to the latest version of the image but still the same.

Note that I have created a PAT token with all necessary permissions as mentioned here, as well as sign in to the Container registry service as shown in the shared screenshot here.

I could docker pull other public images just fine, which gives me reason to believe perhaps only members of the org this image is hosted from have the PATs that are allowed to pull this image..... I could be wrong but I need some assistance with setup, if anyone managed to successfully pull the image please assist.

[beiertu-mms]

Hi @donaldedwin, could you tried to pull with an actual version of the image, for example

docker pull ghcr.io/mediamarktsaturn/technolinator:1.45.1

?

[doggopadre]

[doggopadre]

Works fine for other images like here:

[beiertu-mms]

Thank you @donaldedwin, I will need to check with our security team, and will come back to you as soon as possible.

[beiertu-mms]

@donaldedwin FYI, the packages settings is currently set to internal. I have created a support ticket to change it to public as well. Will update you, when that is the case.

[doggopadre]

Thanks a lot, Kinda had a hunch that could've been probably the case. I'm thinking I was missing something also on my end 🫠... I probably am, but I'll wait for your possible resolution.

[doggopadre]

@beiertu-mms any good news from the mighty Sec team? :)

[beiertu-mms]

@beiertu-mms any good news from the mighty Sec team? :)

ticket is approved, but it's not yet in processing... :(

will try to move it forward by pocking at the responsible team :)

[beiertu-mms]

@donaldedwin our platform team notice me, that this is more complex than expected because it's not possible to selectively allow public package(s). It's an all or nothing operation and so it conflict with our policy to keep stuffs internal.

@heubeck can we also publish the image to docker hub or something similar?

[doggopadre]

It should be possible at least according to the docs here to granulise permissions for select packages, but I suppose docker hub works well also :)

[heubeck]

thx @beiertu-mms , will ask our dockerhub admin for a namespace.

[heubeck]

Pardon us, @donaldedwin. For allowing you to proceed, I mirrored the latest container image to my personal quay.io profile: https://quay.io/repository/heubeck/mediamarktsaturn/technolinator?tab=tags Sorry for the lost time... hope you can now make good progress.

[doggopadre]

Thanks a lot @heubeck as always helpful👏

[doggopadre]

Hello @heubeck and team.

Thanks again for setting up the image in your private Quail.

So here is what i have done:

• I modified my dockerfile to listen to port 8082 because the default 8080 is used by Dependency Track Frontent.
• I also modified the dtrack_URL to 0.0.0.0:8081 because i suppose we need the base URL for the API with port 8081 not the frontend with 8080.
• I then set up the webhook secret because when i built the image first time without it, it errored out saying it was missing,so i created the key and set it up in the repo i created for the app as well as the app settings itself, and then added the secret to the dockerfile i have my smee.io webhook also set and ready and configured in the app I also setup my dtrack as well and all is running.

Here are issues im currently having:

• I am getting push events successfully sent to the smee.io webhook but it's only for push events for the repo i created for the app but not for other repos.
• Even though i am getting these events for this one repo sent to the smee.io webhook, i am not getting them consumed by my local app (container).
• The logs are not showing anything. Is there anywhere i need to set the webhook URL in the docker file? Because from the image logs, it appears that the app is listening on 0.0.0.0:8082 which i believe is not reachable from the internet.
• My understanding from our last chat was that the App image will pull the events sent to the webhook but it appears it's not. Am i missing something? and do you think this is something i should put as an issue on GH or its something silly im doing that you can easily help me with?

Thanks a lot in advance 👏

[heubeck]

Hey @donaldedwin,

great hearing from you.

also modified the dtrack_URL to 0.0.0.0:8081 because i suppose we need the base URL for the API with port 8081 not the frontend with 8080.

guess you mean http://localhost:8081 (or whatever hostname is appropriate) as dtrack url? ;)

Even though i am getting these events for this one repo sent to the smee.io webhook, i am not getting them consumed by my local app (container).

have you set QUARKUS_GITHUB_APP_WEBHOOK_PROXY_URL to your smee url?: https://docs.quarkiverse.io/quarkus-github-app/dev/developer-reference.html#quarkus-github-app_quarkus.github-app.webhook-proxy-url

My understanding from our last chat was that the App image will pull the events sent to the webhook but it appears it's not. Am i missing something? and do you think this is something i should put as an issue on GH or its something silly im doing that you can easily help me with?

the QUARKUS_GITHUB_APP_WEBHOOK_PROXY_URL will cause the app to pull the events from smee instead of expecting them via it's own webhook.

[doggopadre]

guess you mean http://localhost:8081/ (or whatever hostname is appropriate) as dtrack url? ;)

Yes as the track URL. The default for the image is also 8080 so I thought there will be a conflict if they want to both bind the same port since im testing locally

have you set QUARKUS_GITHUB_APP_WEBHOOK_PROXY_URL to your smee url?:

I missed this most probably because I was following the deployment documentation here and it's not there.

How about other push events from my other repos not being listened to by the web hook?

[doggopadre]

Here are my logs im not getting anything

[heubeck]

How about other push events from my other repos not being listened to by the web hook?

does smee retrieve them? for me, it worked with all events, I enabled for the github app config. but technolinator only reacts on push and pull-request what are you missing?

[heubeck]

Here are my logs im not getting anything

that looks good in terms of smee, doing a redelivery at smee doesn't trigger anything?

[doggopadre]

does smee retrieve them?

No it does not only get from the home repo the app is deployed, this is something I can deal with my end.

that looks good in terms of smee, doing a redelivery at smee doesn't trigger anything?

No it doesn't trigger anything, is it because the app is prefering to listen on 0.0.0.0:8080 instead of the see web hook?

if possible @heubeck I can share my dockerfile privately?

[heubeck]

No it doesn't trigger anything, is it because the app is prefering to listen on 0.0.0.0:8080 instead of the see web hook?

that doesn't matter for the smee thing, as the events are pulled from smee instead of pushed by github like regular.

if possible @heubeck I can share my dockerfile privately?

sure, just send it over via mail

[heubeck]

Thanks to our awesome @MediaMarktSaturn/cloud-platform team, the package https://github.com/MediaMarktSaturn/technolinator/pkgs/container/technolinator is now public.

Can you please test and confirm @donaldedwin ?

[doggopadre]

Great, I am now able to pull the image from ghcr.io/mediamarktsaturn/technolinator:1.46.3

[doggopadre]

However 🤣

A few developements:

Seems like the app has some issues with either the content-type or the transfer-encoding of the POST request from Github. I have setup ngrok to directly portforward the image's listening port from my local setup, and im getting a 415 error as you can see below, which is where I got the first assumption that it could be the content-type that's the possible issue.

Also, when I check the logs from the image itself I get this error "Unable to extract installation id from payload". Which I assume could be tied to the above, since the content type was not the expected one then it won't parse anything from the payload and thus can't get the installation id. Here is the screenshot of the logs, as well as a snippet of the said log.

{"timestamp":"2023-08-25T12:03:05.038Z","sequence":56,"loggerClassName":"org.jboss.logging.Logger","loggerName":"io.quarkus.vertx.http.runtime.QuarkusErrorHandler","level":"ERROR","message":"HTTP Request to / failed, error id: 270f49b0-7a81-4b81-b420-704a71bf5917-1","threadName":"vert.x-eventloop-thread-0","threadId":32,"mdc":{},"ndc":"","hostName":"88da4b0fcd61","processName":"quarkus-run.jar","processId":1,"exception":{"refId":1,"exceptionType":"java.lang.IllegalStateException","message":"Unable to extract installation id from payload","frames":[{"class":"io.quarkiverse.githubapp.runtime.Routes","method":"extractInstallationId","line":159},{"class":"io.quarkiverse.githubapp.runtime.Routes","method":"handleRequest","line":121},{"class":"io.quarkiverse.githubapp.runtime.Routes_RouteHandler_handleRequest_516ab5af9409426c5dbdf44ccd172a61ce906df9","method":"invoke"},{"class":"io.quarkus.vertx.web.runtime.RouteHandler","method":"handle","line":97},{"class":"io.quarkus.vertx.web.runtime.RouteHandler","method":"handle","line":22},{"class":"io.vertx.ext.web.impl.BlockingHandlerDecorator","method":"lambda$handle$0","line":48},{"class":"io.vertx.core.impl.ContextBase","method":"lambda$null$0","line":137},{"class":"io.vertx.core.impl.ContextInternal","method":"dispatch","line":264},{"class":"io.vertx.core.impl.ContextBase","method":"lambda$executeBlocking$1","line":135},{"class":"io.quarkus.vertx.core.runtime.VertxCoreRecorder$14","method":"runWith","line":577},{"class":"org.jboss.threads.EnhancedQueueExecutor$Task","method":"run","line":2513},{"class":"org.jboss.threads.EnhancedQueueExecutor$ThreadBody","method":"run","line":1538},{"class":"org.jboss.threads.DelegatingRunnable","method":"run","line":29},{"class":"org.jboss.threads.ThreadLocalResettingRunnable","method":"run","line":29},{"class":"io.netty.util.concurrent.FastThreadLocalRunnable","method":"run","line":30},{"class":"java.lang.Thread","method":"run","line":1623}]}}

Did anyone else run into a similar issue or perhaps know how I can get around this?

I tried to change the content type between the only 2 options available application/x-www-form-urlencoded and applicationl/json in the test repo under web hooks but I get the same error.

[heubeck]

wait... webhook? haven't you created a github app in your user profile or your organization and enabled it for your repo? the regular repo webhooks will not work as their purpose is different.

I followed this guide back then: https://docs.quarkiverse.io/quarkus-github-app/dev/register-github-app.html

[doggopadre]

Hello Team. Redid everything from scratch and everything is now in sync. I will document my setup experience and will share with you and perhaps reference it as some sort of setup documentation

[beiertu-mms]

Hello Team. Redid everything from scratch and everything is now in sync. I will document my setup experience and will share with you and perhaps reference it as some sort of setup documentation

yes, that would be great :)

[heubeck]

awesome, thank you @donaldedwin