P2P not working with harbor

fengwang-db commented 5 months ago

What happened in your environment?

This is more like a question than a bug. We deploy harbor using azure blob store as the backend storage. During image pull, harbor returns a signed URL to Azure blob store. When we turn on the P2PConfig to route the overlaybd blob download requests to a local server, we found the overlaybd requests are Azure URLs, e.g.

/registryfs/registryfs.cpp:132|GET:p2p_url: 127.0.0.1:8081/https://abced.blob.core.windows.net/harbor//docker/registry/v2/blobs/sha256/fc/fc830920cdbfd4bbb850c2c11a7932c2371f3916ed7709ecf168729d13326d01/data?se=2024-04-26T06%3A18%3A35Z&sig=zAFnql%2BX%2FJcDREL2tuiA%2FRuShaRQz5zSr8UDbf205W0%3D&sp=r&sr=b&sv=2016-05-31

What did you expect to happen?

Ideally we would like the request to be harbor, something like: /v2/main/hello/test/blobs/sha256:ba81c3880903ca64f19e7dd0a6f7e52e90b68c2e8e0e3e63d143b1c6d4d60221?ns=harbor.test.com So we can use something like Kraken and reduce the harbor load. It also seems inefficient for overlaybd to fetch a signed-url from harbor for each request.

Is this an expected behavior? How difficult is it to change the behavior to the above?

How can we reproduce it?

enable P2P in overlaybd.json:

      "p2pConfig": {
          "enable": true,
          "address": "127.0.0.1:8081"
      },

What is the version of your Overlaybd?

0.6.17

What is your OS environment?

Ubuntu 20.04

Are you willing to submit PRs to fix it?

[X] Yes, I am willing to fix it.

BigVan commented 5 months ago

Did you mean that you want overlaybd send http(s) request through http proxy during the entire lazy-loading progress

fengwang-db commented 5 months ago

@BigVan That would also work. Is it something supported?

fengwang-db commented 5 months ago

Another question, what's the difference between registryfs and registryfs_v2? Is there a preference which one should be used?

fengwang-db commented 5 months ago

https://github.com/containerd/overlaybd/blob/v0.6.17/src/overlaybd/registryfs/registryfs.cpp#L125-L126 Basically we would prefer not to set the url to the re-direct url.

fengwang-db commented 5 months ago

Is something like https://github.com/containerd/overlaybd/pull/327 acceptable? Or any better ideas?

BigVan commented 5 months ago

So we can use something like Kraken and reduce the harbor load. It also seems inefficient for overlaybd to fetch a signed-url from harbor for each request.

So Kraken didn't work for the blob-signed URL?

fengwang-db commented 5 months ago

@BigVan correct. kraken doesn't work with pre-signed url. it expects the request to be registry API compliant.

BigVan commented 5 months ago

I think maybe setting an HTTP proxy is better than 'ignore redirect URL'?

It there any possible?

containerd / overlaybd