search

NASA-IMPACT / veda-jupyterhub

VEDA JupyterHub technical planning and documentation
1 stars 1 forks source link

Deploy `jupyterhub-fancy-profiles` to VEDA hub #34

Closed batpad closed 1 month ago

batpad commented 4 months ago

We have been working in jupyterhub-fancy-profiles to modernize the Jupyterhub spawner frontend - i.e. the screen where one sees profiles choices after login. This is also a dependency for dynamic image building.

Over the past few weeks @oliverroick has worked to build upon the initial implementation by @yuvipanda -

Apart from paving the way to dynamic image building, this should offer several UI benefits on the post-login profile selection screen, and allow us to build out the frontend to accommodate more features in the future.

We need to figure out the final steps of testing and deploying, making sure it doesn't break current profile options, etc.

@yuvipanda shall we use this ticket to talk about next steps here and what we'd need to do to test thoroughly and then deploy to a staging environment?

yuvipanda commented 4 months ago

Yes, I'd say next steps are:

  1. Get https://github.com/yuvipanda/jupyterhub-fancy-profiles/pull/36 merged
  2. Make a release
  3. Get it into 2i2c-org/infrastructure
  4. Deploy it only to VEDA staging
  5. Iterate.

I don't think any more testing here is needed right now, as VEDA isn't using dynamic image building - that's the part that needs more work.

yuvipanda commented 4 months ago

I merged the PR, made a release, and tried it on the staging hub. Ran into https://github.com/yuvipanda/jupyterhub-fancy-profiles/issues/38 which needs to be fixed before we can proceed further.

yuvipanda commented 4 months ago

Issues to be fixed before we can put this back on staging:

Issues to be fixed before we can put this on prod:

batpad commented 3 months ago

@yuvipanda all the above issues have been fixed, plus:

I created a release for v0.3.4

Whenever you're able, I think we can do another deploy to staging and test this out. Thanks!

Thanks to @oliverroick for most of the work getting these done!

batpad commented 3 months ago

@yuvipanda any luck here getting to this staging VEDA hub?

Thanks!

yuvipanda commented 3 months ago

@batpad I've deployed it on staging.hub.openveda.cloud now!

image image

I have noticed the following issues:

I haven't tested unlisted_choice but otherwise this looks good!

I'd also love for some additional folks to test these combinations out!

Looking good though! Excited to push this forward.

batpad commented 3 months ago

cc @oliverroick - I'll take a look at this tomorrow, but might ping you.

The background for marking something as selected should probably be some kinda grey that's already used in the jupyterhub UI. Currently it's a green that's not used anywhere in the UI.

That makes sense - I just used the bg-success or so class, but can see the best place to set that.

It displays 'build your own image' even though that functionality is currently not available. Perhaps fancy-profiles is showing it every time unlisted_choice is present?

@oliverroick might be good to take a look at this together to make sure we disable the Build your own Image bits for now / make it explicitly configurable, independent of Bring your Own Image.

Thanks @yuvipanda !

batpad commented 3 months ago

It displays 'build your own image' even though that functionality is currently not available. Perhaps fancy-profiles is showing it every time unlisted_choice is present?

Unfortunately I realized this ran into a subtle bug where the drop-down does not display if you have no choices selected but have both dynamic_image_building and unlisted_choice enabled.

And in debugging that, I discovered there maybe some bugs with the unlisted_choice data getting sent to the backend 😭 .

I'll ticket both these - should be relatively simple things and hopefully these are the last of the blockers before this can get to production 🤞

batpad commented 3 months ago

Working through enabling dynamic image building via a profile options here: https://github.com/yuvipanda/jupyterhub-fancy-profiles/pull/58

Filed an issue for the form submission for Bring your own Image being broken on staging: https://github.com/yuvipanda/jupyterhub-fancy-profiles/issues/59

batpad commented 2 months ago

@yuvipanda both the above issues should be fixed + a loading indicator added for the initial form load.

Can we deploy this update to the VEDA staging hub so I can test? Thanks!

yuvipanda commented 2 months ago

@batpad deployed as https://github.com/2i2c-org/infrastructure/pull/4437.

I opened https://github.com/yuvipanda/jupyterhub-fancy-profiles/issues/65 too :) but i haven't been able to put it through its paces, and would need someone else to do that.

batpad commented 2 months ago

This has been on staging for a bit now and seems to be working basically fine.

@wildintellect would you be able to help test and let us know if there's any concerns or anything looks broken.

Steps to test:

Please let us know any feedback or if there are any questions, etc. Issues can be filed in https://github.com/yuvipanda/jupyterhub-fancy-profiles/issues/ or you can drop comments here.

We will hope to get this deployed to production in the next weeks and then work on integrating the dynamic image building component and implement URL handling for profile states.

+cc @oliverroick for visibility

maxrjones commented 2 months ago

I was using the staging deployment last week and Alex asked if I had any feedback, so I figured I drop it here. Overall, the new profile selector drop-downs are great! I immediately noticed when going to the staging Hub for the first time that the server options page was super visually appealing and concise.

Quick caveat that I don't know if these few pieces of feedback actually fit within the jupyterhub-fancy-profiles discussion or elsewhere.

So excited to see all these improvements coming out of IMPACT!

wildintellect commented 2 months ago

@batpad looks better to me. Still some open questions

yuvipanda commented 2 months ago

@maxrjones re:

I'd be interested in learning more about the pros/cons of allowing VEDA users to use any image versus restricting to specific namespaces.

In general, I think of images as generally untrustworthy and all security should be enforced via runtime container options rather than by restriction what images can run. So the overall security posture doesn't change based on what images people use.

We do want to make the UX around custom images better though!

batpad commented 2 months ago

@maxrjones thanks so much for the testing and feedback!

Would it be possible to optionally disallow spaces in the "Custom image" entry box? I know you technically can have Docker images with spaces in the name, but it might be nice to disincentivize that

Haha nice catch :-) . This should be a simple configuration fix - one can pass in a validation_regex to validate the docker image, and we probably just need to improve the regex here: https://github.com/2i2c-org/infrastructure/blob/main/config/clusters/nasa-veda/staging.values.yaml#L38

I wonder if it would be worthwhile to have some safeguards on the custom image options?

So I know some hubs only allow images from a particular namespace, etc. This is configurable via the same regex above. So far, it's been useful to have the flexibility, but this can all be configured per needs: for example, we could have different profile options based on permissions / use groups, where some users can only use images from a particular org, etc. if that's needed.

I think it could be nice to only be able to click "Add New Server" if you have typed something into the "Name your server" box since it otherwise behaves the same as the "My Server" button.

That seems like a good catch! This screen is separate from fancy-profiles, but this definitely seems like a good one to file anyway. @yuvipanda do you know where I should file issues like this with the Hub Control Panel or so?

maxrjones commented 2 months ago

Thanks both, really helpful to understand better the design choices behind the custom images. Long-term I'd be excited to learn more about how security is handled via runtime container options to be able to help out more in this space.

yuvipanda commented 2 months ago

@batpad ah, that should probably be filed in the jupyterhub repo itself!

maxrjones commented 2 months ago

Still really appreciating the custom docker images! it would be cool if there was an option to cache the names of images that people use so that it's not necessary to copy/paste it each time. I'm not sure if where the custom images code lives, but if this sounds interesting to other people I could raise a separate issue about it.

yuvipanda commented 2 months ago

@maxrjones the code is in https://github.com/yuvipanda/jupyterhub-fancy-profiles! I definitely 100% agree that would be a super useful feature to add. Open an issue?

maxrjones commented 2 months ago

@maxrjones the code is in https://github.com/yuvipanda/jupyterhub-fancy-profiles! I definitely 100% agree that would be a super useful feature to add. Open an issue?

ah nice, looks like it would be covered by https://github.com/yuvipanda/jupyterhub-fancy-profiles/issues/43. I'll be sure to skip the issues in jupyterhub-fancy-profiles before posting in the future

batpad commented 2 months ago

This is deployed to the production VEDA hub! I'll leave this open for a couple days in case we get any critical feedback.

batpad commented 1 month ago

Closing this! I think we can capture improvements, etc. as separate tickets.