Closed chuckbelisle closed 1 year ago
[x] Create a pro-b
and unclassified
version of the application for aws-js-s3-explorer. Reason is purely for UX improvement - the unclassified s3proxy instance that is exposed to the kubeflow UI does not mount the protected-b volume. However, there is only one UI for aws-js-s3-explorer, and the protected-b option is still shown even though there does not exist a protected-b volume mounted to the unclassified s3proxy instance. To improve UX, I propose we just add a line to the init container for each deployment to use sed
to remove the irrelevant menu options in each case (see https://github.com/StatCan/aws-js-s3-explorer/blob/main/index.html#L78-L82). E.g. add something like sed -i '/option data-i18n base-value="unclassified"/d' index.html
to the init container for copying the static files to the nginx container.
[ ] Verify that the appropriate volumes are mounting correctly to unclassified and protected-b instances of s3proxy.
[ ] Need default virtualservice for the "Storage" button on the kubeflow interface.
[ ] Need to figure out how to enable downloads from the s3 aws js explorer UI when the user is accessing the UI through the Kubeflow iFrame
Case 1: Profile is created from https://github.com/StatCan/aaw-kubeflow-profiles
I created the collin-test
profile for this test.
In the jsonnet
file for the profile, wrap the profile in the addS3
function (see example for my profile here: https://github.com/StatCan/aaw-kubeflow-profiles/commit/a5d4883e678f04c6e245ef9e413c3c5c2952bbcf)
User logs into kubeflow (dev) with usual SSO
kubeflow central dashboard configmap populates "Storage" option on the left menu of Kubeflow
User clicks "Storage" and is presented the aws-js-s3-explorer UI. User must select Unclassified, Unclassified read only, or Protected-b
Note: On the first page load, the service worker may not get loaded into the browser. This can lead to a false "403" forbidden error message. As described in the s3proxy developer docs, we have a service worker that removes an erroneous authorization
header that is added automatically by the client-side s3 javascript library used by the aws-js-s3-explorer tool.
Case 1.1: Select "Unclassified" bucket
User creates test
prefix (folder) in s3 explorer, folder is created successfully.
User uploads moderately large single file (122 MB), file is uploaded successfully.
Note: The UI is a bit finicky with file uploads; the progress bar shows 0% progress and turns green when the file upload completes. The file is uploaded correctly, but the UI appears as though no progress has been made.
Note: the aws-js-s3-explorer UI does not currently allow users to download files through the kubeflow interface. The issue is that the request URL in the aws js s3 explorer application does not have the /s3/<user namespace>
prefix; this works correctly in a noVNC notebook when requests are made directly against the s3proxy-web
service in the cluster. For example, in a noVNC notebook, users can visit s3proxy-web/s3/<user-namespace>/
(or s3proxy-protected-b-web/s3/<user-namespace>/
in a protected b environment) and connect directly to the s3proxy pod within the cluster.
In relation to Collin's comment here. ✔️ means good to go ❌ means problem (and in this case so does ❗, but was not in Collin's initial comment ). ⭕ means task to be completed.
The PR just needed slight tweaking as initially it complained about not being able to find the index.html
. Fixing it up results in this. As a result when navigating to it on the kubeflow interface you only get unclassified
. What we do need to verify is how to get to the protected b version while in novnc and we should only see unclassified-ro
and protectedb
. --> We accessed this during tech elab using the internal service of s3proxy-protected-b-web
and were able to verify seeing those two volumes.
To test this I started up a few notebooks. The PVC's of aaw-protected-b
, aaw-unclassified-ro
only got mounted to protected b pods and aaw-unclassified
was only mounted to non protected b pods.
However, note that if you delete
the s3proxy pod in your namespace (or restart the deployment) any files that were uploaded are gone.
We are able to access kubeflow central dashboard from the protected b remote desktop and as a result, can interact with the unclassified
volume. You are also able to upload to the unclassifed-ro volume.
Notes from debugging session with @Jose-Matsuda and @mathis-marcotte : https://github.com/StatCan/daaas-private/issues/80
Discussion includes some networking details which is why it is outlined in a private repo.
Closing this issue. The work will progress through other issues, including
Refer to Blob-CSI epic https://github.com/StatCan/daaas/issues/1001 and move forward with go-live of Blob-CSI and S3 proxy features.
Follow the confirmation of the feature functioning in Prod we will need to action the following