search

cs3org / reva

WebDAV/gRPC/HTTP high performance server to link high level clients to storage backends
https://reva.link
Apache License 2.0
167 stars 113 forks source link

OCM: provider authorizer problems #4791

Closed MahdiBaghbani closed 2 months ago

MahdiBaghbani commented 2 months ago

These are my finding about the "open" driver for ocmprovider:

2024-08-02 18:16:58.006 TRC ../reva-git/pkg/auth/scope/scope.go:51 > scope is valid pid=49 pkg=http resource=/sciencemesh/accept-invite scope={"resource":{"decoder":"json","value":"eyJyZXNvdXJjZV9pZCI6eyJzdG9yYWdlX2lkIjoic3RvcmFnZS1pZCIsIm9wYXF1ZV9pZCI6Im9wYXF1ZS1pZCJ9LCJwYXRoIjoic29tZS9maWxlL3BhdGgudHh0In0="},"role":1} traceid=11571ce8-50ad-4afe-b9a1-18e087437310
2024-08-02 18:16:58.006 DBG ../reva-git/pkg/rhttp/rhttp.go:265 > http routing: url=sciencemesh pid=49 pkg=http
2024-08-02 18:16:58.006 DBG ../reva-git/internal/http/services/experimental/sciencemesh/sciencemesh.go:130 > sciencemesh routing path=/accept-invite pid=49 pkg=http traceid=11571ce8-50ad-4afe-b9a1-18e087437310
2024-08-02 18:16:58.006 TRC ../reva-git/pkg/auth/scope/scope.go:51 > scope is valid pid=49 resource={"domain":"revaowncloud1.docker"} scope={"resource":{"decoder":"json","value":"eyJyZXNvdXJjZV9pZCI6eyJzdG9yYWdlX2lkIjoic3RvcmFnZS1pZCIsIm9wYXF1ZV9pZCI6Im9wYXF1ZS1pZCJ9LCJwYXRoIjoic29tZS9maWxlL3BhdGgudHh0In0="},"role":1} traceid=11571ce8-50ad-4afe-b9a1-18e087437310
2024-08-02 18:16:58.007 TRC ../reva-git/pkg/auth/scope/scope.go:51 > scope is valid pid=49 resource={"domain":"revaowncloud1.docker"} scope={"resource":{"decoder":"json","value":"eyJyZXNvdXJjZV9pZCI6eyJzdG9yYWdlX2lkIjoic3RvcmFnZS1pZCIsIm9wYXF1ZV9pZCI6Im9wYXF1ZS1pZCJ9LCJwYXRoIjoic29tZS9maWxlL3BhdGgudHh0In0="},"role":1} traceid=800544ac-ac43-4c2c-a078-95c66d772771
2024-08-02 18:16:58.007 ERR ../reva-git/internal/grpc/services/ocmproviderauthorizer/ocmproviderauthorizer.go:107 > error getting provider info error="error probing OCM services at remote server: error doing request: Get \"revaowncloud1.docker/ocm-provider\": unsupported protocol scheme \"\"" pid=49 traceid=800544ac-ac43-4c2c-a078-95c66d772771
2024-08-02 18:16:58.007 DBG ../reva-git/internal/grpc/interceptors/log/log.go:67 > unary code=OK end="02/Aug/2024:18:16:58 +0000" from=tcp://[::1]:33756 pid=49 start="02/Aug/2024:18:16:58 +0000" time_ns=269400 traceid=800544ac-ac43-4c2c-a078-95c66d772771 uri=/cs3.ocm.provider.v1beta1.ProviderAPI/GetInfoByDomain user-agent=
2024-08-02 18:16:58.007 DBG ../reva-git/internal/grpc/interceptors/log/log.go:67 > unary code=OK end="02/Aug/2024:18:16:58 +0000" from=tcp://127.0.0.1:54892 pid=49 start="02/Aug/2024:18:16:58 +0000" time_ns=979069 traceid=11571ce8-50ad-4afe-b9a1-18e087437310 uri=/cs3.gateway.v1beta1.GatewayAPI/GetInfoByDomain user-agent=
2024-08-02 18:16:58.007 ERR ../reva-git/internal/http/services/reqres/reqres.go:64 > grpc forward invite request failed error="error getting provider info" pid=49 pkg=http traceid=11571ce8-50ad-4afe-b9a1-18e087437310
2024-08-02 18:16:58.008 ERR ../reva-git/internal/http/interceptors/log/log.go:104 > processed http request host=172.24.0.5 method=POST pid=49 pkg=http status=500 traceid=11571ce8-50ad-4afe-b9a1-18e087437310 uri=/sciencemesh/accept-invite
2024-08-02 18:16:58.008 TRC ../reva-git/internal/http/interceptors/log/log.go:111 > http end="02/Aug/2024:18:16:58 +0000" host=172.24.0.5 method=POST pid=49 pkg=http proto=HTTP/2.0 req_headers={"Accept":["*/*"],"Authorization":["Basic bWFoZGk6dVdkTGNseXY5QWRBMDF5NUlDRUd4MU5Bd1pYUUdpUjA="],"Content-Length":["101"],"Content-Type":["application/json"]} res_headers={"Content-Type":["application/json"],"Vary":["Origin"],"X-Access-Token":["eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzIyNzA5MDE3LCJpYXQiOjE3MjI2MjI2MTcsImlzcyI6InJldmFvd25jbG91ZDIuZG9ja2VyIiwidXNlciI6eyJpZCI6eyJpZHAiOiJyZXZhb3duY2xvdWQyLmRvY2tlciIsIm9wYXF1ZV9pZCI6Im1haGRpIn0sInVzZXJuYW1lIjoibWFoZGkiLCJkaXNwbGF5X25hbWUiOiJtYWhkaSJ9LCJzY29wZSI6eyJ1c2VyIjp7InJlc291cmNlIjp7ImRlY29kZXIiOiJqc29uIiwidmFsdWUiOiJleUp5WlhOdmRYSmpaVjlwWkNJNmV5SnpkRzl5WVdkbFgybGtJam9pYzNSdmNtRm5aUzFwWkNJc0ltOXdZWEYxWlY5cFpDSTZJbTl3WVhGMVpTMXBaQ0o5TENKd1lYUm9Jam9pYzI5dFpTOW1hV3hsTDNCaGRHZ3VkSGgwSW4wPSJ9LCJyb2xlIjoxfX19.P6Dr0FDB5U_jr95fcBoQe3O8stBkVKdbZ8cX0cW_te4"]} size=79 start="02/Aug/2024:18:16:57 +0000" status=500 time_ns=85988001 traceid=11571ce8-50ad-4afe-b9a1-18e087437310 uri=/sciencemesh/accept-invite

The error happens because of an error probing OCM services at the remote server: 

error doing request: Get \"revaowncloud1.docker/ocm-provider\": unsupported protocol scheme \"\"

since the provider domain doesn't have http:// or https:// scheme prefix.

but if the ocmprovider is set to json driver:

2024-08-02 18:23:05.463 INF ../reva-git/internal/http/interceptors/auth/auth.go:269 > core access token generated pid=49 pkg=http traceid=0de5d8fe-595c-4507-96e0-6b50f6a92d64
2024-08-02 18:23:05.463 TRC ../reva-git/pkg/auth/scope/scope.go:51 > scope is valid pid=49 pkg=http resource=/sciencemesh/accept-invite scope={"resource":{"decoder":"json","value":"eyJyZXNvdXJjZV9pZCI6eyJzdG9yYWdlX2lkIjoic3RvcmFnZS1pZCIsIm9wYXF1ZV9pZCI6Im9wYXF1ZS1pZCJ9LCJwYXRoIjoic29tZS9maWxlL3BhdGgudHh0In0="},"role":1} traceid=0de5d8fe-595c-4507-96e0-6b50f6a92d64
2024-08-02 18:23:05.463 DBG ../reva-git/pkg/rhttp/rhttp.go:265 > http routing: url=sciencemesh pid=49 pkg=http
2024-08-02 18:23:05.463 DBG ../reva-git/internal/http/services/experimental/sciencemesh/sciencemesh.go:130 > sciencemesh routing path=/accept-invite pid=49 pkg=http traceid=0de5d8fe-595c-4507-96e0-6b50f6a92d64
2024-08-02 18:23:05.464 TRC ../reva-git/pkg/auth/scope/scope.go:51 > scope is valid pid=49 resource={"domain":"revaowncloud1.docker"} scope={"resource":{"decoder":"json","value":"eyJyZXNvdXJjZV9pZCI6eyJzdG9yYWdlX2lkIjoic3RvcmFnZS1pZCIsIm9wYXF1ZV9pZCI6Im9wYXF1ZS1pZCJ9LCJwYXRoIjoic29tZS9maWxlL3BhdGgudHh0In0="},"role":1} traceid=0de5d8fe-595c-4507-96e0-6b50f6a92d64
2024-08-02 18:23:05.465 TRC ../reva-git/pkg/auth/scope/scope.go:51 > scope is valid pid=49 resource={"domain":"revaowncloud1.docker"} scope={"resource":{"decoder":"json","value":"eyJyZXNvdXJjZV9pZCI6eyJzdG9yYWdlX2lkIjoic3RvcmFnZS1pZCIsIm9wYXF1ZV9pZCI6Im9wYXF1ZS1pZCJ9LCJwYXRoIjoic29tZS9maWxlL3BhdGgudHh0In0="},"role":1} traceid=bd05cd64-ac90-48e4-9afb-ab010233bda3
2024-08-02 18:23:05.465 DBG ../reva-git/internal/grpc/interceptors/log/log.go:67 > unary code=OK end="02/Aug/2024:18:23:05 +0000" from=tcp://[::1]:32828 pid=49 start="02/Aug/2024:18:23:05 +0000" time_ns=151529 traceid=bd05cd64-ac90-48e4-9afb-ab010233bda3 uri=/cs3.ocm.provider.v1beta1.ProviderAPI/GetInfoByDomain user-agent=
2024-08-02 18:23:05.465 DBG ../reva-git/internal/grpc/interceptors/log/log.go:67 > unary code=OK end="02/Aug/2024:18:23:05 +0000" from=tcp://127.0.0.1:37986 pid=49 start="02/Aug/2024:18:23:05 +0000" time_ns=1960700 traceid=0de5d8fe-595c-4507-96e0-6b50f6a92d64 uri=/cs3.gateway.v1beta1.GatewayAPI/GetInfoByDomain user-agent=
2024-08-02 18:23:05.466 TRC ../reva-git/pkg/auth/scope/scope.go:51 > scope is valid pid=49 resource={"invite_token":{"token":"482cbe93-91d7-4823-ad7a-f3cd0edfa067"},"origin_system_provider":{"domain":"revaowncloud1.docker","services":[{"endpoint":{"path":"https://revaowncloud1.docker/ocm/","type":{"name":"OCM"}},"host":"revaowncloud1.docker"},{"endpoint":{"path":"https://owncloud1.docker/remote.php/webdav/","type":{"name":"Webdav"}},"host":"owncloud1.docker"}]}} scope={"resource":{"decoder":"json","value":"eyJyZXNvdXJjZV9pZCI6eyJzdG9yYWdlX2lkIjoic3RvcmFnZS1pZCIsIm9wYXF1ZV9pZCI6Im9wYXF1ZS1pZCJ9LCJwYXRoIjoic29tZS9maWxlL3BhdGgudHh0In0="},"role":1} traceid=0de5d8fe-595c-4507-96e0-6b50f6a92d64
2024-08-02 18:23:05.467 TRC ../reva-git/pkg/auth/scope/scope.go:51 > scope is valid pid=49 resource={"invite_token":{"token":"482cbe93-91d7-4823-ad7a-f3cd0edfa067"},"origin_system_provider":{"domain":"revaowncloud1.docker","services":[{"endpoint":{"path":"https://revaowncloud1.docker/ocm/","type":{"name":"OCM"}},"host":"revaowncloud1.docker"},{"endpoint":{"path":"https://owncloud1.docker/remote.php/webdav/","type":{"name":"Webdav"}},"host":"owncloud1.docker"}]}} scope={"resource":{"decoder":"json","value":"eyJyZXNvdXJjZV9pZCI6eyJzdG9yYWdlX2lkIjoic3RvcmFnZS1pZCIsIm9wYXF1ZV9pZCI6Im9wYXF1ZS1pZCJ9LCJwYXRoIjoic29tZS9maWxlL3BhdGgudHh0In0="},"role":1} traceid=8851f0d0-35da-4708-9d77-9a2a440a1ec6
2024-08-02 18:23:05.588 DBG ../reva-git/internal/grpc/interceptors/log/log.go:67 > unary code=OK end="02/Aug/2024:18:23:05 +0000" from=tcp://[::1]:48982 pid=49 start="02/Aug/2024:18:23:05 +0000" time_ns=120624546 traceid=8851f0d0-35da-4708-9d77-9a2a440a1ec6 uri=/cs3.ocm.invite.v1beta1.InviteAPI/ForwardInvite user-agent=
2024-08-02 18:23:05.588 DBG ../reva-git/internal/grpc/interceptors/log/log.go:67 > unary code=OK end="02/Aug/2024:18:23:05 +0000" from=tcp://127.0.0.1:37986 pid=49 start="02/Aug/2024:18:23:05 +0000" time_ns=122186507 traceid=0de5d8fe-595c-4507-96e0-6b50f6a92d64 uri=/cs3.gateway.v1beta1.GatewayAPI/ForwardInvite user-agent=
2024-08-02 18:23:05.589 INF ../reva-git/internal/http/services/experimental/sciencemesh/token.go:200 > invite forwarded pid=49 pkg=http provider=revaowncloud1.docker token=482cbe93-91d7-4823-ad7a-f3cd0edfa067 traceid=0de5d8fe-595c-4507-96e0-6b50f6a92d64
2024-08-02 18:23:05.589 INF ../reva-git/internal/http/interceptors/log/log.go:104 > processed http request host=172.26.0.5 method=POST pid=49 pkg=http status=200 traceid=0de5d8fe-595c-4507-96e0-6b50f6a92d64 uri=/sciencemesh/accept-invite
2024-08-02 18:23:05.589 TRC ../reva-git/internal/http/interceptors/log/log.go:111 > http end="02/Aug/2024:18:23:05 +0000" host=172.26.0.5 method=POST pid=49 pkg=http proto=HTTP/2.0 req_headers={"Accept":["*/*"],"Authorization":["Basic bWFoZGk6Y2ZIcmtxck1EeFNBeTNJVDNtQ3hsMzVvWEJ4dTllSEc="],"Content-Length":["101"],"Content-Type":["application/json"]} res_headers={"Vary":["Origin"],"X-Access-Token":["eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzIyNzA5Mzg1LCJpYXQiOjE3MjI2MjI5ODUsImlzcyI6InJldmFvd25jbG91ZDIuZG9ja2VyIiwidXNlciI6eyJpZCI6eyJpZHAiOiJyZXZhb3duY2xvdWQyLmRvY2tlciIsIm9wYXF1ZV9pZCI6Im1haGRpIn0sInVzZXJuYW1lIjoibWFoZGkiLCJkaXNwbGF5X25hbWUiOiJtYWhkaSJ9LCJzY29wZSI6eyJ1c2VyIjp7InJlc291cmNlIjp7ImRlY29kZXIiOiJqc29uIiwidmFsdWUiOiJleUp5WlhOdmRYSmpaVjlwWkNJNmV5SnpkRzl5WVdkbFgybGtJam9pYzNSdmNtRm5aUzFwWkNJc0ltOXdZWEYxWlY5cFpDSTZJbTl3WVhGMVpTMXBaQ0o5TENKd1lYUm9Jam9pYzI5dFpTOW1hV3hsTDNCaGRHZ3VkSGgwSW4wPSJ9LCJyb2xlIjoxfX19.lxuWnNzM9WCCaPvD2T8dsn4H7D_Bdpg5hX67Uj3rWrQ"]} size=0 start="02/Aug/2024:18:23:05 +0000" status=200 time_ns=293586247 traceid=0de5d8fe-595c-4507-96e0-6b50f6a92d64 uri=/sciencemesh/accept-invite

Reva will read the correct endpoint with https:// or http:// from json file and connect to it.

MahdiBaghbani commented 2 months ago

When trying to get provider information from https://revaowncloud1.docker/ocm-provider it redirects to https://revaowncloud1.docker/.wellknown/ocm and for some reason /.wellknown/ocm is a protected path 😨

Confirming it with curl:

17f0f43e8de2:/# curl -vvvv -L https://revaowncloud1.docker/.wellknown/ocm
* Host revaowncloud1.docker:443 was resolved.
* IPv6: (none)
* IPv4: 172.20.0.6
*   Trying 172.20.0.6:443...
* Connected to revaowncloud1.docker (172.20.0.6) port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / X25519 / RSASSA-PSS
* ALPN: server accepted h2
* Server certificate:
*  subject: C=RO; ST=Bucharest; L=Bucharest; O=IT; CN=revaowncloud1.docker
*  start date: Mar  3 11:48:34 2024 GMT
*  expire date: Feb  8 11:48:34 2124 GMT
*  subjectAltName: host "revaowncloud1.docker" matched cert's "revaowncloud1.docker"
*  issuer: C=RO; ST=Bucharest; L=Bucharest; O=IT; CN=dev-stock
*  SSL certificate verify ok.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://revaowncloud1.docker/.wellknown/ocm
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: revaowncloud1.docker]
* [HTTP/2] [1] [:path: /.wellknown/ocm]
* [HTTP/2] [1] [user-agent: curl/8.9.0]
* [HTTP/2] [1] [accept: */*]
> GET /.wellknown/ocm HTTP/2
> Host: revaowncloud1.docker
> User-Agent: curl/8.9.0
> Accept: */*
> 
* Request completely sent off
< HTTP/2 401 
< vary: Origin
< www-authenticate: Basic realm="revaowncloud1.docker"
< www-authenticate: Bearer realm="revaowncloud1.docker"
< content-length: 0
< date: Sun, 04 Aug 2024 13:42:26 GMT
< 
* Connection #0 to host revaowncloud1.docker left intact

logs on revaowncloud1.docker:

2024-08-04 13:42:26.24 WRN reva-git/internal/http/interceptors/auth/auth.go:216 > core access token not set or invalid pid=49 pkg=http traceid=e28ebe61-d548-4eed-9e56-8de2f5b9a61b
2024-08-04 13:42:26.241 DBG reva-git/internal/http/interceptors/auth/auth.go:225 > error retrieving credentials error="no basic auth provided" pid=49 pkg=http traceid=e28ebe61-d548-4eed-9e56-8de2f5b9a61b
2024-08-04 13:42:26.241 DBG reva-git/internal/http/interceptors/auth/auth.go:225 > error retrieving credentials error="no bearer auth provided" pid=49 pkg=http traceid=e28ebe61-d548-4eed-9e56-8de2f5b9a61b
2024-08-04 13:42:26.241 DBG reva-git/internal/http/interceptors/auth/auth.go:225 > error retrieving credentials error="no public token provided" pid=49 pkg=http traceid=e28ebe61-d548-4eed-9e56-8de2f5b9a61b
2024-08-04 13:42:26.241 WRN reva-git/internal/http/interceptors/log/log.go:104 > processed http request host=172.20.0.7 method=GET pid=49 pkg=http status=401 traceid=e28ebe61-d548-4eed-9e56-8de2f5b9a61b uri=/.wellknown/ocm
2024-08-04 13:42:26.241 TRC reva-git/internal/http/interceptors/log/log.go:111 > http end="04/Aug/2024:13:42:26 +0000" host=172.20.0.7 method=GET pid=49 pkg=http proto=HTTP/2.0 req_headers={"Accept":["*/*"],"User-Agent":["curl/8.9.0"]} res_headers={"Vary":["Origin"],"Www-Authenticate":["Basic realm=\"revaowncloud1.docker\"","Bearer realm=\"revaowncloud1.docker\""]} size=0 start="04/Aug/2024:13:42:26 +0000" status=401 time_ns=304190 traceid=e28ebe61-d548-4eed-9e56-8de2f5b9a61b uri=/.wellknown/ocm

but somehow this doesn't make sense, since in this file: https://github.com/cs3org/reva/blob/dde65a44013db3c4a8e8e5219a7707674838e410/internal/http/services/wellknown/wellknown.go#L74-L80

this path has been marked as unprotected with the prefix .wellknown and here they are being applied: https://github.com/cs3org/reva/blob/dde65a44013db3c4a8e8e5219a7707674838e410/cmd/revad/runtime/http.go#L83-L90

MahdiBaghbani commented 2 months ago

It turns out that the .wellknown service is listening on port: 42333 instead of 443:

2024-08-05 10:42:51.872 INF reva-git/pkg/rhttp/rhttp.go:185 > http service enabled: ocs@/ocs pid=49 pkg=http
2024-08-05 10:42:51.872 INF reva-git/pkg/rhttp/rhttp.go:185 > http service enabled: dataprovider@/data pid=49 pkg=http
2024-08-05 10:42:51.872 INF reva-git/pkg/rhttp/rhttp.go:185 > http service enabled: sciencemesh@/sciencemesh pid=49 pkg=http
2024-08-05 10:42:51.872 INF reva-git/pkg/rhttp/rhttp.go:185 > http service enabled: datagateway@/datagateway pid=49 pkg=http
2024-08-05 10:42:51.872 INF reva-git/pkg/rhttp/rhttp.go:185 > http service enabled: prometheus@/metrics pid=49 pkg=http
2024-08-05 10:42:51.872 INF reva-git/pkg/rhttp/rhttp.go:185 > http service enabled: appprovider@/app pid=49 pkg=http
2024-08-05 10:42:51.872 INF reva-git/pkg/rhttp/rhttp.go:185 > http service enabled: metrics@/register_metrics pid=49 pkg=http
2024-08-05 10:42:51.872 INF reva-git/pkg/rhttp/rhttp.go:185 > http service enabled: ocm@/ocm pid=49 pkg=http
2024-08-05 10:42:51.872 INF reva-git/pkg/rhttp/rhttp.go:185 > http service enabled: ocdav@/ pid=49 pkg=http
2024-08-05 10:42:51.872 DBG reva-git/cmd/revad/runtime/runtime.go:460 > spawned http server for services listening at tcp:[::]:443 pid=49 services=["ocdav","dataprovider","sciencemesh","ocs","datagateway","prometheus","appprovider","metrics","ocm"]
2024-08-05 10:42:51.877 INF reva-git/pkg/rhttp/rhttp.go:185 > http service enabled: dataprovider@/data pid=49 pkg=http
2024-08-05 10:42:51.878 DBG reva-git/cmd/revad/runtime/runtime.go:460 > spawned http server for services listening at tcp:[::]:36069 pid=49 services=["dataprovider"]
2024-08-05 10:42:51.88 INF reva-git/pkg/rhttp/rhttp.go:185 > http service enabled: wellknown@//.well-known pid=49 pkg=http
2024-08-05 10:42:51.88 DBG reva-git/cmd/revad/runtime/runtime.go:460 > spawned http server for services listening at tcp:[::]:42333 pid=49 services=["wellknown"]

This was due to a misconfiguration:

[http.services.wellknown.ocmprovider]
address = ":443"

@glpatcern suggested that it should be like:

[http.services.wellknown]
address = ":443"

[http.services.wellknown.ocmprovider]
... all other settings ...