Host a publicly available ChRIS UI with authentication and registration system.

[NicolasRannou]

First version of the login form can be found at: https://github.com/FNNDSC/cube-auth

[NicolasRannou]

@mairin maybe we can just work in the https://github.com/FNNDSC/cube-design repo to work on the designs?

I think it is not clear yet what is the exact purpose of the page:

• create user accounts?
• advertise a bit?
• do we want to "log" into the system to see currently running jobs and so on? (very much like )

[mairin]

@NicolasRannou sounds good. I am going to try to get some of @danmcp 's time today to get a little more context around this so we have a better idea of the purpose / priorities we'll need to keep in mind with the design. Should we use this ticket for hashing that out, and the cube-design repo for storing the mockups and more granular design issues?

[NicolasRannou]

sounds good to me @mairin !

[mairin]

@NicolasRannou here's a flowchart I drew up of the hackfest participant's workflow based on my convo with @danmcp yesterday. I think we need to flesh out a bit more how the participants are going to learn to containerize - is that a workshop someone is giving or some documentation we'd want to have links to in the UI? I think we should also talk about the lifetime of this UI, if there's going to be multiple hackfests what's the long term roadmap for it?

Anyway here's the flow chart:

[mairin]

Data to ask at account signup:

• email
• name
• project name (so we can know which account submitted which plugin?)
• source URLs (since dockerhub doesn't track this)
• (license?) (not sure)

[NicolasRannou]

Actually I think the plugin registration part should happen at a later stage, once the user is logged into the app. It could be a simple form.

The reason is that we probably want people to be able to submit multiple plugins.

[mairin]

@NicolasRannou I was thinking #6 would be registration, if we could do it in a webform in the app that would be great. The submit to dockerhub in #3 before they engage with the app was meant to show they have to take the step of submitting to docker hub before they could register the plugin. I didn't know how long that process might take, so assumed they might want to do it before they go into the UI so they'd have the docker hub URL ready for the registration step.

Does that make sense, or am I misunderstanding the process?

[NicolasRannou]

Yes I think that makes sense -

What got me confused is the following:

Data to ask at account signup:

email name project name (so we can know which account submitted which plugin?) source URLs (since dockerhub doesn't track this) (license?) (not sure)

Project name, url and license may not be part of the sign up workflow and should probably be provided by the user once they are logged into the app, if they want to "register" a new plugin in ChRIS.

Do you see what I mean or am I misunderstanding something?

[mairin]

@NicolasRannou ahhhh ok! Yes I get the confusion now, makes sense. :)

[jbernal0019]

Hey @mairin and @NicolasRannou This is a very interesting conversation and we need to think about it for the Hackaton. Here are some of my thoughts:

The plugin registration process/workflow is independent of the chris user registration process. In principle there should be three types of people interacting with Chris "Chris administrator", "Chris plugin developer" and "Chris user". Now given those entities we can/should think what's the best workflow to make it easier and secure the plugin registration process. This is not an obvious thing as there are many variables to consider here. From the top of my head:

Should we actually enable an API endpoint to register new plugins? If so should plugin developers be constrained to be Chris users too? Can all Chris users be plugin developers? Should a plugin be marked with an owner in the internal DB at all?

Having an owner for a plugin in the DB can really make things complex because that can imply many things in the long run. For instance the ultimate goal of this system is to allow users to compose, store, retrieve and run pipelines (ordered collections) of registered plugins no matter who the owner of a plugin in the pipeline would be.

In opposition shouldn't plugins be just registered by the Chris administrator and not marked with any ownership relationship? How plugin developers submit (not register) plugin definitions for registration to the Chris administrator? Should it be an independent app or just another CUBE API endpoint? Again, should the admin keep track of plugin ownership in the DB when she actually registers the plugin?

It seems to me that for security and other reasons (avoiding bad practices, like bitcoin mining for instance) the plugin registration workflow should include a human-based revision of the plugin app code by the Chris administrator anyway.

In summary this issue needs thorough thinking. We want a system that can scale easily with many new plugins but at the same time properly dealing with security and bad practices. We probably don't want to create plugin ownership relationships in the DB too.

[mairin]

@jbernal0019 hi Jorge! thanks for bringing up these points, it's super helpful. here's some thoughts:

1. Should we actually enable an API endpoint to register new plugins?

We don't necessarily need to. If there is already a way for an admin logged into the system to register a plugin directly in the system, then, we shouldn't need this for a minimum viable product assuming a CHRIS admin is present at the event. The simplest mechanism to have registration happen would be to have the URL written on a white board or passed on a piece of paper to the admin. (Not saying we need to do this, but it'd be the minimum we could do.)

2. Should plugin developers be constrained to be CHRIS users too?

I think I don't fully understand this question, but in the context of the CHRIS hackathon events, I think part of the purpose behind the event is to allow the plugin developers to see a bit of what CHRIS has to offer to get them excited about it as a platform and to help motivate them to create / maintain plugins for it. So in that context I'd answer yes.

But in the broader context, should plugin developers necessarily be CHRIS users (user in the sense of, using CHRIS to earnestly run jobs, not for plugin testing purposes?) I think the answer is no, they probably should have access to a CHRIS system somehow to try out their plugins / test them, right?

3. Can all Chris users be plugin developers?

It is my understanding (which may be dated, or incomplete, or wrong) that while a CHRIS user could be a plugin developer, there is not an expectation that all CHRIS users are plugin developers. Then again, it depends what you mean by CHRIS user, right? Do you mean CHRIS the entire system, or just this front end into it (noting that there are other front ends, such as the radiology viewer?) I would answer no to this assuming the question refers to the entire CHRIS system regardless the interface the user is interacting with, but I sense that may not be the context you were thinking of? :-/ ?

4. Should a plugin be marked with an owner in the internal DB at all? / Should plugins just be registered by CHRIS admins without ownership?

For the context of the hackathons - the short term case - I would answer a definite yes here; again I believe part of the goal of the hackathons is to get upstream developers excited about creating CHRIS plugins, and as part of that it would be helpful to associate developers with plugins so if we need to follow up or want to check in on that developer's experience with CHRIS, or even if it turns out the plugin has some great use case and we want to figure out who created it - I think it's essential info to track.

For a longer term context, I still think maintaining a contact per plugin could be important, but maybe that's too heavily influence by my background? I am coming from the Red Hat platform dept context (with previous experience working on the sw / patch management sw we use) where in our OSes we package and maintain 10,000+ pieces of software from various upstreams, it's important when something goes wrong or if there is a support question to be able to track down ownership across a lot of contexts. (e.g. RHT owner = person who is assigned internal bugs against package, responsible for security updates and enhancements and packaging them, etc. Upstream owner = point of contact for collaboration on issues and to submit upstream fixes, etc.)

So I'm kind of applying the things i know from Red Hat and Fedora about the infrastructure for downstream code packaging, distribution, and maintainership conceptually on CHRIS plugins when various factors (I'm thinking particularly of scale & distribution) may vary pretty wildly.

In case it's helpful though, an example of a relevant infrastructure component I'm thinking of here we have in Fedora claled anitya. It's a message bus backed upstream code monitoring system that periodically checks upstream maintainers to see if new code has been released, and flags / notifies downstream maintainers to repackage the code (see https://release-monitoring.org/about and see https://release-monitoring.org/project/5506/ for an example of it in action for a specific piece of software). Having an automated sort of notification system here relies on having both identified downstream and upstream contacts in the metadata for the package for the system to know where to look for updates and who to notify to update the downstream packaging. A driver for this system is, for example, making sure maintainers stay on top of upstream security patches and get them downstream as quickly as possible.

Where I'm extrapolating this to CHRIS in my head is thinking down the road, you might have some kind of repo or even docker hub like kind of infrastructure specific to CHRIS plugins, perhaps an instance of this infrastructure internal to a hospital (to maintain specific versions researchers rely on, similar to how Red Hat Satellite is used if you're familiar with that product) and to enable that you'd need this sort of metadata around plugins.

I know this is a huge tangent, but hope it helps illustrate a perspective that might be helpful in thinking about CHRIS plugins long term?

5. In opposition shouldn't plugins be just registered by the Chris administrator and not marked with any ownership relationship?

This is a good question. I think I don't understand fully what "submit" vs "register" means. Perhaps when I labeled the diagram "register" I should have used the term "submit"? E.g. by "submit" you mean, submit for a CHRIS admin to approve of, otherwise it sits in a queue? And by "register" do you mean, an approved plugin that then shows up in the UI (say in a drop down) ready for jobs by any CHRIS user with access to said plugin in? If yes to both, I think we're on the same page but maybe I haven't been thinking about submit vs register as clearly as I should have been.

I don't think any developer logged into a system should be able to have a plugin automatically available for running and visible to that (or other) users in UI. Absolutely an admin should have the ability to approve or reject plugin submissions, for security reasons and others IMHO.

6. How plugin developers submit (not register) plugin definitions for registration to the Chris administrator?

Short-term, could be as simple as a post it note handed to admin on-site at the hackathon, or writing on a white board for on-site admin to process. Or something extremely simple, like a basic form in a UI provided just for the hackathons that emails admins who will be active either on site or at the time of the hackathon with the dockerhub url and the plugin devel's name / email. The email trail could be enough for the follow-up piece I mentioned earlier; the emails could be saved and manually put into a spreadsheet at some point. (might even be able to use a google form that automatically creates the spreadsheet from the results?)

Long-term / high-level / bluesky / whatnot, I'd suspect that the administrator of any particular CHRIS system (assuming there are multiple CHRIS systems at different sites at different orgs) might not be in the business of registering plugins directly from a developer in a manual sense and instead might be registering them after selecting those of relevance to his or her user community from some kind of common shared catalog? (e.g. like a dockerhub for CHRIS plugins that doesn't exist yet that maybe could someday.)

Mid-term, then, I don't know if it would make sense to have a CUBE API endpoint or some kind of formal plugin submission system. But I don't know what your thinking is in terms of say a possible ecosystem developing around CHRIS plugins. Do you know what I mean?