Spike: Implement `POST /v3/packages/:guid/upload`

[tcdowney]

Background

CF for VMs context

During cf push the CLI will package up the local app source code into a zip file (referred to as app bits) and send it to the Cloud Controller API. The Cloud Controller API will do some additional processing (a big portion of this is a feature called resource matching which we can ignore for now) and send the app bits to the packages bucket in the blobstore. The associated CF Package for these bits will be updated to have the state READY to indicate that the source code has been fully uploaded.

CF for K8s context

In CF for K8s we use kpack for application staging and provide the app source code as an OCI image. In this flow the CLI will still package the local app source code as a zip file and sent it to the Cloud Controller API. Instead of sending it to a blobstore, though, Cloud Controller will pass the zip off to another component called registry-buddy which converts the zip file into a single-layer OCI image that contains the app source. It then pushes that image to an image registry. Once this is done the associated CF Package is updated to have the state READY and Cloud Controller implicitly knows where the image was pushed to based on the configured image registry / package guid.

What to do

The CF API shim will be performing the roles of Cloud Controller and registry-buddy in this case. Implement the POST /v3/packages/:guid/upload endpoint (see docs) on the shim and have it convert the zip file into an OCI Image that kpack can use as a source. Then update the associated Package to refer to it (see https://github.com/cloudfoundry/cf-crd-explorations/issues/6 for how this looks and is used).

Acceptance Criteria

GIVEN I have the shim deployed to a cluster WHEN I run the following curl

curl "http://api-shim.example.org/v3/packages/[guid]/upload" \
  -X POST \
  -F bits=@"path-to-app-source.zip" \

AND It finished uploading / conversion THEN I can kubectl get packages [guid] -o yaml to see where the app source image was pushed to AND I can docker pull the image to verify that it contains the app source code*

*Alternatively based on #6 you may be able to use the CF Package with CF Build to get kpack to stage it.

---

Dev Notes

• The registry-buddy code copy pasted some code from the https://github.com/buildpacks/pack (see https://github.com/buildpacks/pack/issues/814). They've updated that code to be public so we may be able to just consume it directly.
• Zip to OCI Image conversion uses a lot of memory in cf-for-k8s. For the spike the simplest thing to do is just have the CF API shim do the conversion directly, but it might be valuable to have the converter / shim be independently scalable.
• REST API request would be a MULTIPART upload.

[matt-royal]

After discussion, @tcdowney and I have decided not to consider resource matching or making this component independently scalable for this work.

[tcdowney]

Tasks

• [x] Implement multipart form handler (nginx portion)
• [x] Figure out how to configure shim to talk to a registry
• [x] Use packages from https://github.com/buildpacks/pack/issues/814 to convert the app source code to an image
• [x] Upload image to registry
• [x] Set Package State in response based on Status Conditions

[akrishna90]

CF Package response is missing Package.State info that needs to be derived based on the status conditions

[akrishna90]

From @Birdrock : Should state be a separate status? Or do we derive state from the other statuses directly? Output:

• States are derived from Conditions

• Helper functions that take the Conditions and return a State string

• A dedicated Condition should be avoided since it is a calculated value from other Conditions

• State is not a useful construct to a kubectl power user.

[akrishna90]

Miro link for Package Conditions to State mapping table : https://miro.com/app/board/o9J_lFiI8CU=/?moveToWidget=3074457361338757508&cot=14

[matt-royal]

We were able to verify that the acceptance criteria were met.

[akrishna90]

During implementation of this Spike, few open questions popped up about image registries and how access information for these registries would stored and shared. We are capturing those questions here.

1. CF for VMs traditionally had and supported a single blobstore. In Kubernetes world there is no such restriction when it comes to image registries. This opens a question whether we confine to a single registry with our implementation or do we support multiple registries? (Note - current spike implementation has 2 image registries - a package registry and kpack registry)
2. Access credentials for the registry (image registry host url, repository within the registry, username, password) are passed to the shim using namespace scoped secret object. How to make this secret available on arbitrary namespaces?
   • Make the secret a cluster scoped resource?
   • Add secret to the namespace during namespace creation/initialization? Note: Problem with Namespace scoped objects goes beyond just secrets - read issue/59 for more info.