401 Unauthorized on some API endpoints when using OIDC

SimonAlling commented 2 years ago

Expected behavior and actual behavior

I expect to be able to access all API endpoints that my user has access to via e.g. curl, using my CLI secret for authentication.

However, when using an OIDC user account, I cannot access some endpoints:

I can GET the list of projects, but I cannot GET a specific project.
I can GET the list of repositories in a project, but I cannot GET a specific repository.
I can GET the list of artifacts in a repository, but I cannot GET a specific artifact (or e.g. its vulnerabilities).

Specifically, when using curl, I get a 401 response with this body:

{"errors":[{"code":"UNAUTHORIZED","message":"unauthorized"}]}

I can access all endpoints with curl using a robot account or the admin account (which do not use OIDC).

I can also access all endpoints in the web GUI and the API Explorer (/devcenter-api-2.0) using my OIDC account.

Steps to reproduce the problem

HARBOR_USERNAME="bob"
 HARBOR_PASSWORD="…"
HARBOR_URL="https://example.com"
HARBOR_PROJECT="foo"
HARBOR_REPO="bar"
HARBOR_TAG="baz"
API_PATHS=(
    "/projects"
    "/projects/${HARBOR_PROJECT:?}"
    "/projects/${HARBOR_PROJECT:?}/repositories"
    "/projects/${HARBOR_PROJECT:?}/repositories/${HARBOR_REPO:?}"
    "/projects/${HARBOR_PROJECT:?}/repositories/${HARBOR_REPO:?}/artifacts"
    "/projects/${HARBOR_PROJECT:?}/repositories/${HARBOR_REPO:?}/artifacts/${HARBOR_TAG:?}"
    "/projects/${HARBOR_PROJECT:?}/repositories/${HARBOR_REPO:?}/artifacts/${HARBOR_TAG:?}/additions/vulnerabilities"
)

for apiPath in "${API_PATHS[@]}"; do
    curl --silent --output /dev/null --write-out "%{http_code} %{url_effective}\n" -u ${HARBOR_USERNAME:?}:${HARBOR_PASSWORD:?} "${HARBOR_URL:?}/api/v2.0${apiPath}"
done

Output:

200 https://example.com/api/v2.0/projects
401 https://example.com/api/v2.0/projects/foo
200 https://example.com/api/v2.0/projects/foo/repositories
401 https://example.com/api/v2.0/projects/foo/repositories/bar
200 https://example.com/api/v2.0/projects/foo/repositories/bar/artifacts
401 https://example.com/api/v2.0/projects/foo/repositories/bar/artifacts/baz
401 https://example.com/api/v2.0/projects/foo/repositories/bar/artifacts/baz/additions/vulnerabilities

Lines like this one show up in the log:

2022-04-04T14:00:00Z [ERROR] [/server/middleware/security/basic_auth.go:40][requestID="…"]: failed to authenticate bob: not supported

The error is thrown here.

Additional information

Notably, it turns out that in the web GUI and the API Explorer, a cookie named sid is included in each request. That cookie alone turns out to be enough for authentication and authorization – the Authorization header (i.e. curl's -u flag) is not even needed:

 HARBOR_SID="deadbeefdeadbeefdeadbeefdeadbeef"
curl --silent --output /dev/null --write-out "%{http_code} %{url_effective}\n" --cookie "sid=${HARBOR_SID:?}" "${HARBOR_URL:?}/api/v2.0/projects/${HARBOR_PROJECT:?}"

As previously stated, the cookie is necessary for some endpoints (when accessed by an OIDC user). However, the API Explorer generates commands like this one, without any cookie:

curl -X 'GET' \
  'https://example.com/api/v2.0/projects/foo' \
  -H 'accept: application/json' \
  -H 'X-Is-Resource-Name: false' \
  -H 'authorization: Basic Ym9iOm15IENMSSBzZWNyZXQK'

All in all, there seems to be some inconsistency regarding OIDC authentication:

When the API is accessed by an OIDC user, some endpoints (e.g. /projects/foo) require an OIDC cookie, while others (e.g. /projects) do not.
In the API Explorer, commands without necessary authentication information are displayed.

Versions

Harbor version: 2.4.1
docker engine version: N/A
docker-compose version: N/A

SimonAlling commented 2 years ago

Probably related: #16398

MinerYang commented 2 years ago

Please refer to this https://github.com/goharbor/harbor/issues/16398#issuecomment-1074695316 Or refer to FAQ https://github.com/goharbor/harbor/wiki/Harbor-FAQs in API OIDC part

lindhe commented 2 years ago

Or refer to FAQ https://github.com/goharbor/harbor/wiki/Harbor-FAQs in API OIDC part

For anyone else getting confused by this: search for just "OIDC", not "API OIDC", on that Wiki page.

lindhe commented 2 years ago

My Harbor is configured to use OIDC for authentication, how do I access Harbor's API?

[A] In such case, you have to use the OIDC token as bearer token to access Harbor's API.

That's clearly not true. You don't have to use the OIDC token. That is only required sometimes, depending on which API endpoint is used for the call.

MinerYang commented 2 years ago

Correction: search for OIDC on https://github.com/goharbor/harbor/wiki/Harbor-FAQs. Appreciate!

SimonAlling commented 2 years ago

Please refer to this #16398 (comment) Or refer to FAQ https://github.com/goharbor/harbor/wiki/Harbor-FAQs in API OIDC part

Why has this issue been closed when neither point of concern below has been addressed?

When the API is accessed by an OIDC user, some endpoints (e.g. /projects/foo) require an OIDC cookie, while others (e.g. /projects) do not.
In the API Explorer, commands without necessary authentication information are displayed.

github-actions[bot] commented 2 years ago

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

lindhe commented 2 years ago

Please keep this open.

github-actions[bot] commented 2 years ago

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

cesarb1392 commented 2 years ago

This issue is still relevant, please keep it open

j14s commented 2 years ago

Still an issue on 2.5.1

lindhe commented 2 years ago

When I ran into this yesterday, it was on a Docker API. Can that be a clue?

github-actions[bot] commented 1 year ago

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

lindhe commented 1 year ago

Ping

till commented 1 year ago

Same error, is this a bug or no bug?

When I log into Harbor via browser and go to API docs, it gives me an option to authorize, and I do this using basic auth. Once authorized, I can run example requests on the Swagger part. When I try something similar with a command line client or with Go, I am stuck with UnAuthorize instead?

till commented 1 year ago

So this seems like a bug:

I was trying to do a /users/current (with curl/go) — that one is broken with my (OIDC) user (used username and CLI secret), but a /projects works fine. The request works (as others have mentioned) on the website (Swagger).

lindhe commented 1 year ago

I think this is clearly a bug.

github-actions[bot] commented 1 year ago

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

till commented 1 year ago

Please don't close this. I know it's a lot to work on. But it's been confirmed by various people that this is a bug, maybe it can be rectified eventually?

majusmisiak commented 1 year ago

@SimonAlling I have run exactly in the same issue that you described, with auth working with Authorization: Basic on some endpoints and with Cookie: sid on others.

In my case, this makes using Terraform provider for Harbor impossible.

Did you find any workaround for this issue?

Edit: (tag @Vad1mo )

Ketec commented 1 year ago

Same issue. After an hour of debugging why requesting artifacts works fine but I cannot get tags of a specific one I end up here.

I have a user in Jenkins credential storage I use to make the curl requests.

This is not for just people to login and play around with postman or something.

This is programmatic use in pipeline. It should have a single credential/api key you can use to access API, stored securely with credentials.

That excess requirement here makes ZERO sense - you can access ALL artifacts with user:password but not individual?

github-actions[bot] commented 1 year ago

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

till commented 1 year ago

Please un-stale.

SimonAlling commented 1 year ago

@majusmisiak:

Did you find any workaround for this issue?

IIRC, unfortunately not. I changed jobs a few weeks after creating this issue and haven't used Harbor since.

naveenb29 commented 1 year ago

still an issue - v2.7.1-6015b3ef

Ketec commented 1 year ago

YEt again i end up on this very same issue trying to deal with getting actual image version for latest tag from harbor.

All i can do is query ALL artifacts and loop through every time just to find the 'latest' tag.

But it looks like noone in Harbor team cares at all. Zero response.

ryanmac8 commented 1 year ago

Still an issue

lindhe commented 1 year ago

Would be nice to try this now after #8723 was purportedly resolved. I don't have a Harbor instance available right now, but if someone could test it that would be nice.