search

elixir-cloud-aai / tesk-api

GA4GH TES API Service that translates tasks into Kubernetes Batch API calls
Apache License 2.0
7 stars 18 forks source link

feature request: support for private docker images in task executors #55

Open pavelnikonorov opened 5 months ago

pavelnikonorov commented 5 months ago

Summary At GENXT, we develop security addons for Kubernetes that leverage Confidential Computing technology, which is a protection of data while it's in use in remote infrastructures by performing computations in hardware-based, isolated, and verifiable Trusted Execution Environments (TEEs). TESK is a perfect fit as a task execution system that may work inside TEE to provide remote users both with a verifiable, cryptographically-backed and unique software “fingerprint” and proof of memory isolation (confidentiality and integrity assurances for both code and data). Within this context, we've identified a critical feature gap in TESK: the ability to use private Docker images for task executors. This functionality is essential for scenarios that require working with proprietary software.

Requirement The current implementation only supports public Docker images. For enhanced security and flexibility, we need the capability to authenticate against private Docker registries.

Proposed Solution

  1. Extend the TESK API to include Docker registry credentials in the task definition.
  2. Use Kubernetes secrets for storing Docker credentials.
  3. Modify the job creation logic to incorporate imagePullSecrets for private Docker image retrieval.

Preliminary Discussion This need has been preliminarily discussed within our team and @uniqueg. We've outlined a potential approach involving modifications to the TesExecutor class and the job creation process to support this feature.

Request: We propose the creation of a special branch for the development and testing of this feature. Our team is ready to lead the implementation and testing phases, including by involving the GA4GH GSoC OSS community – @aaravm is ready to commit the first version.

Looking forward to further discussions and contributions towards integrating this feature into TESK.

uniqueg commented 4 months ago

Thanks a lot @pavelnikonorov. As discussed elsewhere, we will gladly consider a PR for this.

However, we would appreciate contributions to the open issue and ideally a PR (does not need to be a great one that covers all angles, just something concrete that gets the discussion going within the community) that generalizes the used approach for (a future version of) the TES specification that will ensure that all implementations will use the same general approach to support pulling images from private container image registries.