C2D: Handle multiple container services

[alexcos20]

Right now, we are only supporting dockerhub for container services. In order to add more, we need to refactor the checksum check flow.

Proposal:

• add new endpoint on provider that will accept container service details (url, etc), docker image & tag/digest. It will verify (fetch digest for tag based images) and return it. Our frontend will use that endpoint, instead or custom proxy
  • expand provider with multiple services (right now, only dockerhub api is supported). Will start with AWS ECR, next on list: GCP, Azure

[calina-c]

Integrated some preliminary work here: https://github.com/oceanprotocol/provider/pull/525https://github.com/oceanprotocol/provider/pull/525

namely moving container validation separately for better reusability. That impacted the messages, meaning now we send validation message for each type of container type tried (for now only docker, but open for more). I also added the endpoint to validate the container separately.

[alexcos20]

It's clear that expanding provider with multiple services will take a long time. But we don't need to.

Proposed new flow:

• at publish, docker digest is required. Dapps may try to use a proxy to fetch the digest from dockerhub, but if the image is served by another container service, it will fail. Then user must fill in that value
• provider, when starting a new job, has to check if the digest is defined in the algo ddo, container section and if dataset has publisherTrustedAlgorithms, then it needs to check the container hash. That's it. No image checks, no nothing else.
• c2d engine will try to use the digest to start a job. If the image does not exists, job will fail

How to handle private images (that requires authentication at docker pull ...)

• TBD in a new issue (doable, with some restrictions)