HHNB: register model in the Model Catalog from the HHNB model simulation page

lbologna commented 4 years ago

Use Case Update

Aspect	Detail
Summary	In the Hodgkin-Huxley Neuron Builder, after the model has been optimized (or uploaded) and the user has accessed the simulation page, she/he is given the opportunity to insert the model's metadata via an ad-hoc form and register the model in the Model Catalog
Expert	@lbologna @appukuttan-shailesh
Deadline	31/03/2020
Dependencies	Model Catalog
Link for testing	https://collab.humanbrainproject.eu/#/collab/1256/nav/29703

List of additional/changed features

The final step of the Hodgkin-Huxley Neuron Builder Use Case is the model simulation. This can be done after the optimization process has been undergone or after the user has uploaded her/his own model. In the simulation page (where the user can monitor the model behavior) we added a Model Registration button that allows the user to register the model in the Model Catalog after inserting the model's info. priority: nice-to-have for the user, important for use case integration purposes

Tasks

[x] frontend operations for inserting model metadata in the HHNB and managing info/wait/error popups
[x] backend operations to save a file in a repository collab storage using a refresh token of one of the member of the collab (so has to prevent the user to have write permissions on the repository collab itself
[x] backend operations to fetch metadata of base model used for the optimization (if any)
[x] backend operations to save the model in the Model Catalog and linking it to the repository collab storage
[ ] developers' testing
[ ] testers' testing
[ ] update Guidebook pages

Acceptance Criteria

From the simulation page of the HHNB, the user can register the model in the Model Catalog. The model is correctly displayed in the Model Catalog and correctly linked to the repository collab storage (from the Model Catalog)

Extra Requirements

System

No additional system requirements

Performance

No additional performance required

lbologna commented 4 years ago

Hello @alex4200, @appukuttan-shailesh,

I finalized the auto-fill for the model fields and handled possible multiple submissions. It seems to be working for me. Could you please have a look to see if everything is fine before putting to prod? Thank you.

alex4200 commented 4 years ago

@lbologna When I try to register I still get a 500 error (POST to https://bspg.pa.ibf.cnr.it/hh-neuron-builder/register-model-catalog/tab_20200319145610/3a0c1f99-5f61-4589-a915-f4b8c4ce1289).

Also, what exactly is the 'Author' and what is the 'Owner'?

lbologna commented 4 years ago

@lbologna When I try to register I still get a 500 error (POST to https://bspg.pa.ibf.cnr.it/hh-neuron-builder/register-model-catalog/tab_20200319145610/3a0c1f99-5f61-4589-a915-f4b8c4ce1289).

Also, what exactly is the 'Author' and what is the 'Owner'?

Hello @alex4200, I see in the log the error that you are getting. I am not getting it anymore but used to (while debugging). Please try to refresh the cache and run in incognito window. We can also have a brief call on skype to see it in real time.

Concerning your question: Also, what exactly is the 'Author' and what is the 'Owner'? I think @appukuttan-shailesh would answer this question better, because these are default fields of model catalog entries

appukuttan-shailesh commented 4 years ago

@alex4200 : Model author can include several people who might have been involved in the creation of that model. Model owner is typically a single person who is in-charge of that model. Somewhat like a publication having a list of authors and a corresponding author.

As an example, see here: https://collab.humanbrainproject.eu/#/collab/343/nav/41587?state=model.080559b0-f2e4-4897-b88c-c76517a18b80

appukuttan-shailesh commented 4 years ago

@lbologna : I tested HHNB just now, and it works great! Everything went through smoothly.

I just have one more suggestion. Once the model is registered, and we close the dialog box, it would be nice to disable the "Register in Model Catalog" button , or we could change the button to "Show model catalog entry" which would open the model entry in a new browser window.

(This would reduce accidental duplication of entries to some extent.)

alex4200 commented 4 years ago

@lbologna I cleared the cache and used a private firefox session, but I still get a 500 Server Error. (Time of test: 9:02)

lbologna commented 4 years ago

@lbologna : I tested HHNB just now, and it works great! Everything went through smoothly.

I just have one more suggestion. Once the model is registered, and we close the dialog box, it would be nice to disable the "Register in Model Catalog" button , or we could change the button to "Show model catalog entry" which would open the model entry in a new browser window.

(This would reduce accidental duplication of entries to some extent.)

Hello @appukuttan-shailesh, I am glad the workflow worked for you. Thanks for you suggestion, I just implemented the second option ("Show Model Catalog entry"). As we commented earlier, this will not prevent for a double registration of the same model but will make it a bit more complicated (the user will have to go back to the workflow and click the run simulation button (again) in order to be given the registration option a second(third/...) time)

lbologna commented 4 years ago

@lbologna I cleared the cache and used a private firefox session, but I still get a 500 Server Error. (Time of test: 9:02)

@alex4200, I checked the logs again and after commenting with @appukuttan-shailesh we realized that the token to be used was a collab member's one, in that only collab members are allowed to insert entries in the Model Catalog. I fixed this and tested with a non-member user and it worked for me. Could you please check again? Thanks

@appukuttan-shailesh, @alex4200 Concerning the collab permissions, while testing I got aware that another operation non-members cannot do is editing the model. We are storing the models and creating a new entry in the model catalog with a user which is a member of the collab, but this prevents non-members to edit the metadata if they wish. On the other side, giving non-members write access will allow them to delete entries from the storage and modify entries in the MC (even delete the MC catalog item from the Collab menu). A workaround for the storage would be to separate it from the MC (i.e. create two different collabs, one for storing the models and the other for hosting the MC) but the problem would be only partially solved.

I see a few solutions: 1) ask the users in which collab they want to add the model catalog instance (if it is not already present and if they already are members of at least one collab). Or ask them to create a new collab (similarly to what is being done at the moment when cloning the use cases). This will allow the users to change the metadata if they wish. The model file will be anyway safely stored in the actual collab (so the users will not be able to delete it because they will not be members of this collab). I think this is the most desirable solution but will take a longer in terms of development. 2) siimilar to the previous one, but creating a new collab for every model registered from the HHNB.
3) leave the model file uneditable to the users and suggest them to contact the support if they want to modify the model metadata. As far as I understand the CollabV2 will have multiple permission level (unit/group/...) so possibly this issue will be solved more easily once we will migrate the application to CollabV2.

Do you have any other suggestions? Thanks for your feedback

appukuttan-shailesh commented 4 years ago

@lbologna : you have brought up a valid issue here. The other day when we were discussing this I had imagined this problem but thought we could just with your option 3.

But now after seeing your options, I think we could do something along option 1. Instead of asking the user about the desired Collab, we could maybe simply pick the Collab from which the UseCase is running. Since the UseCase would have been cloned to their choice of Collab, they would be a member of this chosen Collab.

Like you have said, the model will continue to be stored in our Collab with the dummy user account, thereby preventing them from deleting the model source files. But they would continue to be in-charge of the model catalog entry, and can edit as they desire.

We would need to alter the mc_clb_id value here, to refer to the Collab in which the usecase is being run by the user.

I think this is the most desirable solution but will take a longer in terms of development.

Do you think the above would require much dev efforts, if we auto-identify the running Collab, and not asking the user separately for their choice?

lbologna commented 4 years ago

@lbologna : you have brought up a valid issue here. The other day when we were discussing this I had imagined this problem but thought we could just with your option 3.

But now after seeing your options, I think we could do something along option 1. Instead of asking the user about the desired Collab, we could maybe simply pick the Collab from which the UseCase is running. Since the UseCase would have been cloned to their choice of Collab, they would be a member of this chosen Collab.

Like you have said, the model will continue to be stored in our Collab with the dummy user account, thereby preventing them from deleting the model source files. But they would continue to be in-charge of the model catalog entry, and can edit as they desire.

We would need to alter the mc_clb_id value here, to refer to the Collab in which the usecase is being run by the user.

I think this is the most desirable solution but will take a longer in terms of development.

Do you think the above would require much dev efforts, if we auto-identify the running Collab, and not asking the user separately for their choice?

Hello @appukuttan-shailesh, thanks for your prompt feedback. Auto-identification of the running collab would be fast so I would definitely go with your suggestion of adding the MC (if not present) to the current Collab. There is still the possibility that the user who is running the use case is not a member of the Collab (e.g. a group of student using a single collab created by the professor): in that case, for the time being, we can proceed by adding the model in the MC instance of the destination collab (as we are doing now) and prompt a message to tell the user. At a later time we can add a Collab user's choice as mentioned before.

appukuttan-shailesh commented 4 years ago

Yeah a detailed message on the pop-up dialog explaining it is good enough. Alternatively, if its not much more extra work we can have the pop-up info showing first, and if the user says "Yes" to registering in our dummy collab then we proceed to register it.

So something like this, when the user clicks on the button:

We check if the user belongs to the running Collab
If not, then we show a pop-up explaining this, and that we can proceed to register in another collab but that the model would not be editable. They would have to contact support. Alternatively, run the use case from a collab that they are a member of.
If user clicks "yes" on the dialog, we register in our dummy collab.
If user clicks "no", we do not proceed with registration.

alex4200 commented 4 years ago

Hi,

I re-tested the HHNB use case, and for my normal user it seems to work fine. I can also open the link after registering the model.

When I run the usecase as a test user I still can register the model, but following the link does not seem to work. I get a page with a big "Welcome to the HBP Model Validation Service" on it:

Screenshot 2020-03-24 at 17 05 34

Is that the desired behavior?

appukuttan-shailesh commented 4 years ago

With this test user can you check if you can access any model catalog in any other collab..... Such as BSP.

On Tue, 24 Mar, 2020, 5:06 pm alex4200, notifications@github.com wrote:

Hi,

I re-tested the HHNB use case, and for my normal user it seems to work fine. I can also open the link after registering the model.

When I run the usecase as a test user I still can register the model, but following the link does not seem to work. I get a page with a big "Welcome to the HBP Model Validation Service" on it:

[image: Screenshot 2020-03-24 at 17 05 34] https://user-images.githubusercontent.com/12729291/77448992-bf2a7c80-6df1-11ea-944c-5fb501fbdb41.png

Is that the desired behavior?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cnr-ibf-pa/hbp-bsp-issues/issues/530#issuecomment-603333061, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF5W5VICP2UXIT7IRUKWLJTRJDLAVANCNFSM4LMBYYZA .

alex4200 commented 4 years ago

@appukuttan-shailesh I tried to access the Model Catalog by the test user, and I was able doing so after I authorized for "Model Validation Catalog" (or something similar). After I did that, I also was able to see the HHNB model after registration. So for those cases it seems that the authorization step is missing .

Other question related to the Register form:

How to find out the Owner of the model? The user just knows the owner of a model?
The author - is that myself, the person who registers the model?

appukuttan-shailesh commented 4 years ago

It depends. E.g. The owner is the user itself if he/she has created the model. Or it could be the user's supervisor.... it depends on them.

Typically all the people involved in the model development will be listed as "authors", and the PI of the project might be listed at the "owner". (Again it depends on the team.)

lbologna commented 4 years ago

Hello @alex4200,

the registration issue should be solved by clicking on the Model Catalog item in the left menu. This will load the MC home for that Collab. In principle this should display the authorization request form if the authorization hasn't been given. Could you please test with one of your test users which has not given the authorization to the MC yet? I can detail this behaviour in the final message, where the url is shown. The MC page is by default opened in a new tab so the url will still be visible in the original tab in case the user needs to reload the MC as stated above.

lbologna commented 4 years ago

Hello @alex4200 @appukuttan-shailesh

I finalized the integration with the following points:

check whether the user is member of the current collab:
- yes: create a MC instance (if not present) and save the model in the current collab
- no: save the model in the collab where the model are stored and warn the user that she will not be able to modify it (in case she needs to, write to support) or that she needs to run the hhnb in a collab she is a member of.
after the registration, give some indication on page reload and url fetching in case the model catalog authorization has not been given yet.

You may want to run the test in the following collab with non-member user account: https://collab.humanbrainproject.eu/#/collab/80481/nav/544957

Could you please check? If everything is fine I will move to prod.

Thank you!

alex4200 commented 4 years ago

Hi @lbologna I had another go and it looks better with the updated message. I tested as a member and non-member.

I suggest to put it in PROD (maybe next week Monday)

appukuttan-shailesh commented 4 years ago

Hey @lbologna , I too tested it now as both a member and non-member. I was about to test the above two cases, with the model catalog in the Collab deleted. As a member it was fine. But I accidentally deleted the HHNB app while testing the latter Could you add the HHNB app again here? I couldn't find it on the app list. Sorry about that.

lbologna commented 4 years ago

Hello @alex4200, @appukuttan-shailesh, thanks for testing. (@appukuttan-shailesh, just added the hhnb on the menu)

appukuttan-shailesh commented 4 years ago

Hi @lbologna , it works perfectly. All good to go into production now :+1:

lbologna commented 4 years ago

Hello @alex4200,

the app is in prod (with @appukuttan-shailesh we solved the problems I was mentioning to you earlier today).

Could you please test whether everything is fine for you as well?

Here's a link to one instance of the app in a Collab you are a member of: https://collab.humanbrainproject.eu/#/collab/1256/nav/42586

Here's one in a Collab you are not a member of: https://collab.humanbrainproject.eu/#/collab/80481/nav/546616

Thank you.

alex4200 commented 4 years ago

Hi @lbologna I took a look at the HHNB use cases, and they seem to work for the different scenarios.

lbologna commented 4 years ago

Hello @alex4200,

I integrated the guidebook page with the new functionality:

https://lbologna.github.io/hbp-sp6-guidebook/online_usecases/single_cell_building/hippocampus/p1_hh_neuron_builder/p1_hh_neuron_builder.html

Could you please have a look before moving the guidebook to PROD and closing the ticket?

Thank you!

alex4200 commented 4 years ago

Hi @Luca,

I had a look at the guidebook page and I have the following comments:

In the section "Single Cell Simulation Run" a link does not work (Phrase below the "Download jo results": "For an example of how the .zip file content must be structured, please refer to the following link "
- In the new section "Adding model": "where the user will be able to inserted the model’s metadata" -> "where the user will be able to insert the model’s metadata"
- In the new section "Adding model": "with the Model Catalog url of the optimized model." -> "with the Model Catalog URL of the optimized model."

lbologna commented 4 years ago

Hello @alex4200,

thanks for the comments. I made the changes/corrections.

Could you please have a look before putting to prod? Here's the link: https://lbologna.github.io/hbp-sp6-guidebook/online_usecases/single_cell_building/hippocampus/p1_hh_neuron_builder/p1_hh_neuron_builder.html

Thank you!

alex4200 commented 4 years ago

@lbologna I had a look at the documentation, and it looks fine to me. Please move it to PROD.

lbologna commented 4 years ago

Thanks @alex4200, just put in PROD. Closing.

cnr-ibf-pa / hbp-bsp-issues