search

cnr-ibf-pa / hbp-bsp-issues

Ticketing system for developers/testers and power users of the Brain Simulation Platform of the Human Brain Project
4 stars 0 forks source link

Hippocampus models container #437

Closed clupascu closed 5 years ago

clupascu commented 5 years ago

Details description of the task

We moved all the optimized models (including all the releases) from the github repository to the following container

https://object.cscs.ch/v1/AUTH_c0a333ecf7c045809321ce9d9ecdfdea/hippocampus_optimization/

Description of how to test it

Test/change model catalog, validation use cases, single cell building use cases and hippocampus models available for the use cases by fetching the data from the container so that we can permanently dismiss the github repository once everything works well with the CSCS container.

Known use cases using (Hippocampus) models in some way:

Use Case Migrated to CSCS?
Morphology Analysis Using Model Catalog
Rebuild an existing single hippocampal cell Using zip file
Build your own single hippocampal cell model using HBP data Using zip file
Single cell in silico experiment under current clamp Using Model Catalog

Any use cases missing?

appukuttan-shailesh commented 5 years ago

@clupascu : Could you describe what restructuring was done to the folders. I tested a few random entries and found that their CSCS paths were still accurate. Could you give me examples of Model catalog entries that have incorrect URLs now?

clupascu commented 5 years ago

@appukuttan-shailesh I took a look at the model catalog. It seems that now there are two type of models: the ones used by bluepymm (collab id 8123) and the initial ones (collab id 9821). Let's talk for now about the ones related to collab id 9821.

In the container all the releases from the github repository are now in the folder rat/CA1. The latest one is v.4.0.4. For example the models from this latest version are not in the model catalog.

Moreover, for some of the models in the model catalog the name is 1.0 it is not related to the version from github isn't it? The morphology points to the Migliore_2018_CA1 container, while the download location points to hippocampus_optimization container, more precisely to the optimizations folder (which is just one of the releases from the github repository). I found this for example in CA1_int_cNAC_990111HP2_20180118155424, but I think most of them are like this.

Can we add the missing models and decide what download locations to display and how to include the various releases?

appukuttan-shailesh commented 5 years ago

The latest one is v.4.0.4. For example the models from this latest version are not in the model catalog.

Yes, these are to be registered on the model catalog. I shall send you the script for doing this, so that it could be included as part of your workflow for future releases.

Moreover, for some of the models in the model catalog the name is 1.0 it is not related to the version from github isn't it?

The name of the model (let's take the example model) is "CA1_int_cNAC_990111HP2_20180118155424". The name "1.0" is currently just used to indicate the version/iteration of the developed model. This versioning isn't linked to Github, and I believe this shouldn't be the case since we intend to move away from GitHub for model storage. Do let us know if you can think of a more meaningful/useful versioning name for these model instances.

The morphology points to the Migliore_2018_CA1 container, while the download location points to hippocampus_optimization container, more precisely to the optimizations folder (which is just one of the releases from the github repository).

I shall update this immediately. (This scenario came up because the models had already been registered, without their morphologies, prior to the creation of the live paper container).

Can we add the missing models and decide what download locations to display and how to include the various releases?

Will update the above models, and send you the script for adding the models from the latest release. You could take a look and see if any changes are required.

appukuttan-shailesh commented 5 years ago

The URLs for the live paper models have been updated. The models and morphologies, both now point to the Migliore_2018_CA1 container. @clupascu: Do verify and let me know if any changes required.

The script for registering models on the Model Catalog (customized for your workflow) can be found in the Jupyter notebook inside the Collab: https://collab.humanbrainproject.eu/#/collab/8123/nav/373602

Do the following: 1) Download/copy the jupyter notebook to the Hippocampus collab 2) Verify the metadata in the initial cells 3) Run the entire notebook

The script will register all models with specified version label (e.g. "v.4.0.4"). For testing, I have run the script to register a single model from the repository: https://collab.humanbrainproject.eu/#/collab/8123/nav/68489?state=model.cf8d9d9f-8baf-4710-a9f1-04a77dea3860

Known issues: 1) Morphology files for models uploaded under "v.4.0.4" are not available. Hence it is not possible to launch these in the morphology viewer. @clupascu : I would suggest making the morphologies available as standalone files within each model's base directory.

2) Similarly, config/morph.json files are no longer available. Therefore, the model instances "parameters" field had been left blank.

3) Model not loading properly under BlueNaaS, example: https://collab.humanbrainproject.eu/#/collab/8123/nav/68489?state=model.cf8d9d9f-8baf-4710-a9f1-04a77dea3860 Error:

NEURON: Model loading failed local variable 'index' referenced before assignment

@clupascu, @genric : Could you investigate this?

Note: Do NOT run the notebook more than once! This would create duplicates. (I will delete the above test entry.... so you are free to run the entire notebook entirely once)

clupascu commented 5 years ago

@appukuttan-shailesh Regarding the second known issue, I noticed that in the live paper container we unzipped all the zip files corresponding to each model. If we unzip this will solve also the morphology viewer problem? It will be enough to display the morphology directly from the morphology folder of each model? Or for each version should I create a morphology folder with all the morphologies used in that version, as we did for the live paper?

appukuttan-shailesh commented 5 years ago

I prefer having the morphology files within the model directory (rather than a separate morphology directory for all models combined). Yes, the problem would be resolved if you unzip all the models like we did for the live paper. This might be duplication of a lot of files (zipped and unzipped for each model), so ideally just keeping the morphology file (.asc) at the same level as the image files (_responses.jpeg or _morph.jpeg) would suffice.

clupascu commented 5 years ago

Now I am trying to filter again the models in the model catalog using HBP-SP6 organization and I would like to select collab id 9821, but 9821 is not anymore in the list of the collab ids... Is the same for you?

clupascu commented 5 years ago

I prefer having the morphology files within the model directory (rather than a separate morphology directory for all models combined). Yes, the problem would be resolved if you unzip all the models like we did for the live paper. This might be duplication of a lot of files (zipped and unzipped for each model), so ideally just keeping the morphology file (.asc) at the same level as the image files (_responses.jpeg or _morph.jpeg) would suffice.

When unzipping I wanted to solve also the config/morph.json issue...

appukuttan-shailesh commented 5 years ago

I noticed recently that the drop-down box doesn't have all the right options........ but you can overcome this by using the search bar and entering "9821"..... (it is a generic search across all filter fields)

appukuttan-shailesh commented 5 years ago

When unzipping I wanted to solve also the config/morph.json issue...

I understand. I should point out that this file isn't too important with regards to registering on the model catalog. The parameters field basically had info such as: {"CA1_PC_cAC_sig": "oh140807_A0_idJ.asc"} which I believe isn't very important to see on the model catalog entry. Hence we could avoid having the "morph.json" file if required. I will let you choose.

clupascu commented 5 years ago

I noticed that some of the models from the live paper container are missing from the model catalog. For example CA1_int_bAC_060314AM2_20180120154128 and other 3. Can you check this too?

appukuttan-shailesh commented 5 years ago

They all exist, I believe. E.g. CA1_int_bAC_060314AM2_20180120154128: https://collab.humanbrainproject.eu/#/collab/1655/nav/75901?state=model.2d809d28-1f94-4ef6-acfc-083644afdcbf

You can follow the model catalog links in the "Optimizations" tab of the live paper: https://humanbrainproject.github.io/hbp-bsp-live-papers/2018/migliore_et_al_2018/migliore_et_al_2018.html

clupascu commented 5 years ago

When I searched "9821" I got only 43 models. I do not see 4 of the 47 models from the live paper container. CA1_int_bAC_060314AM2_20180120154128 is one of the 4 models I do not see. I do not understand why...

clupascu commented 5 years ago

When unzipping I wanted to solve also the config/morph.json issue...

I understand. I should point out that this file isn't too important with regards to registering on the model catalog. The parameters field basically had info such as: {"CA1_PC_cAC_sig": "oh140807_A0_idJ.asc"} which I believe isn't very important to see on the model catalog entry. Hence we could avoid having the "morph.json" file if required. I will let you choose.

I agree. The parameters field it doesn't seem so useful on the model catalog entry.

appukuttan-shailesh commented 5 years ago

When I searched "9821" I got only 43 models. I do not see 4 of the 47 models from the live paper container. CA1_int_bAC_060314AM2_20180120154128 is one of the 4 models I do not see.

I find this on the third page of the search results: image

clupascu commented 5 years ago

immagine immagine immagine immagine immagine immagine

I only see 5 bAC...

appukuttan-shailesh commented 5 years ago

try searching by that specific model name......

clupascu commented 5 years ago

When I search by that specific model name I get only the emodel from 8123 collab

immagine

appukuttan-shailesh commented 5 years ago

I am not sure what the problem is..... I would suggest checking using a different browser and/or private/incognito mode.... just to be sure it isn't caching issues.

That aside, I would suggest lets also move onto fixing the other issues and revisit this? E.g. morphology, and BlueNaaS?

p.s. Users can always access the live paper model catalog entries by accessing the links from the live paper.

clupascu commented 5 years ago

I prefer having the morphology files within the model directory (rather than a separate morphology directory for all models combined). Yes, the problem would be resolved if you unzip all the models like we did for the live paper. This might be duplication of a lot of files (zipped and unzipped for each model), so ideally just keeping the morphology file (.asc) at the same level as the image files (_responses.jpeg or _morph.jpeg) would suffice.

For version v.4.0.4 the morphology file (.asc) are now at the same level too as the image files. @appukuttan-shailesh Can you please test if it is possible now to launch these in the morphology viewer?

appukuttan-shailesh commented 5 years ago

@clupascu : I updated the sample model entry I had created (https://collab.humanbrainproject.eu/#/collab/8123/nav/68489?state=model.cf8d9d9f-8baf-4710-a9f1-04a77dea3860) with the morphology URL now available. I tested that the morphology viewer works fine now.

You can now run the script (Jupyter notebook shared above) for registering all the new models. This could be included as part of your workflow for releasing optimized models.

p.s. The BlueNaaS problem persists.

clupascu commented 5 years ago

p.s. The BlueNaaS problem persists.

I took a look at the BlueNaaS problem but I have no idea where is it coming from...

appukuttan-shailesh commented 5 years ago

@genric could you take a look and help @clupascu debug this?

SteMasoli commented 5 years ago

Hi. Just passing by to say that BlueNaaS is still stuck for me too.

clupascu commented 5 years ago

@appukuttan-shailesh I registered the models in the model catalog. Please check if everything is ok. I followed the steps you suggested, even if this step "Verify the metadata in the initial cells" was not so clear... What metadata should I verify in the initial cells? Do you mean the version_label update?

appukuttan-shailesh commented 5 years ago

@clupascu : Could you provide me access to the Collab 12027 so that I can verify the new entries?

By the metadata, I was intending this part:

    model_uuid = modelCatalog.register_model(
                                app_id = str(MCapp_navID),
                                name = model_name,
                                # alias = "", # optional; ensure uniqueness                                
                                author = "Rosanna Migliore",
                                organization = "HBP-SP6",
                                private = False,
                                cell_type = cell_type,
                                model_scope = "single cell",
                                abstraction_level = "spiking neurons",
                                brain_region = "hippocampus",
                                species = "Rattus norvegicus",
                                description = model_desc,
                                instances = [{"source" : base_path + model_url + "?bluenaas=true",
                                            "version" : "1.0",
                                            "parameters" : "",
                                            "code_format" : "asc, py, hoc, mod, json",
                                            "morphology" : str([base_path+morph for morph in repoItems if model_name in morph and version_label in morph and morph.endswith(".asc")]),
                                            "description" : ""}],
                                images = [{"url" : base_path + [img for img in repoItems if model_name in img and version_label in img and img.endswith("_morph.jpeg")][0],
                                         "caption" : "Morphology"},
                                        {"url" : base_path + model_url.replace(".zip", "_responses.png"),
                                         "caption" : "Responses"}])

(apologies... by the time I completed the script, this cell was no longer amongst the initial cells, and I should have pointed to it more clearly)

appukuttan-shailesh commented 5 years ago

Except for BlueNaaS functionality, everything else seems proper for these 39 new entries.

clupascu commented 5 years ago

@appukuttan-shailesh I provided access to @shailesh to the collab. You have 3 accounts on the collab, I don't know if I provided access to the right account. Please let me know. About the metadata, I am wondering if you can take the author, brain_region and species values from the _meta.json file contained in each model folder in order to make it more automatic and also because the author sometimes changes. Take a look for example at CA1_pyr_cACpyr_oh140807_A0_idJ_20190328142723 model in the catalog.

appukuttan-shailesh commented 5 years ago

@clupascu You have added the correct account.

We certainly can make use of the "_meta.json" file. You would have noticed that we already save all the contents of this file to fill the model description field. I have now updated the Jupyter notebook to take the author from this file ("contributors" param).

With regards to brain_region and species, is it necessary to extract data from this file? The problem that arises is that these fields require values that match KG terminology (e.g. not "Rat" but "Rattus norvegicus"). If you can provide a complete list of exact valid values, then I can incorporate mapping to KG values. But on the other hand, if we don't expect these two fields to be altered much, the same could be adjusted manually whenever required.

clupascu commented 5 years ago

Ok. We will stick to the KG terminology then. We will have "hippocampus" only and "mouse" and "rat" species.

appukuttan-shailesh commented 5 years ago

I have updated the notebook to assign proper KG species terminology based on your input of "mouse" or "rat". Also, the models have been updated to correctly reflect the author info as per their "_meta.json" files.

jdcourcol commented 5 years ago

@appukuttan-shailesh just to let you know that @genric will not look into that before June 9th as he is out of the office.

clupascu commented 5 years ago

@genric you had some time to look into this issue?

Model not loading properly under BlueNaaS, example:
https://collab.humanbrainproject.eu/#/collab/8123/nav/68489?state=model.cf8d9d9f-8baf-4710-a9f1-04a77dea3860

Error:
NEURON: Model loading failed
local variable 'index' referenced before assignment
genric commented 5 years ago

@clupascu I've updated the BlueNaaS error message to be more clear and @alex4200 registered the new issue: #442

clupascu commented 5 years ago

@appukuttan-shailesh I updated the models and registered them using the script. The new version label is "v4.0.5". I would like to remove from the model catalog though the models with the version label "v.4.0.4". How should I do this?

appukuttan-shailesh commented 5 years ago

@clupascu : The models with v.4.0.4 have now been deleted. Do verify and let me know if any issues.

clupascu commented 5 years ago

@appukuttan-shailesh ok. thanks. it seems ok. How should I do in the future to delete myself older versions? I can do it myself?

appukuttan-shailesh commented 5 years ago

As things are setup currently, the users are not permitted to delete data and they are required to contact us for this.

But in this particular scenario, what we could have done is to edit the existing entries to replace v.4.0.4 with v4.0.5. Editing of entries can be done by model owners and collaborators.

clupascu commented 5 years ago

So when I edit existing entries, I can edit anything from the entry: version, download location and so on...?

appukuttan-shailesh commented 5 years ago

You can as long as no validation test results have been registered for those model instances on the validation framework. Once a test result has been registered then it would be unsafe to allow changing the model instance info. In our case, this wasn't an issue as no test results had been registered for them till now.

antonelepfl commented 5 years ago

@appukuttan-shailesh when I try to fetch the list of the Hippocampus models I get Bad Gateway and that is why I cannot update the list of the models for Single Cell In Silico Experiments. I just make a request using this script could you help me with that?

appukuttan-shailesh commented 5 years ago

To my best knowledge, this is to do with the server getting timed-out while all the records are being fetched. There seems to have been a drastic increase in the number of hippocampus entries on the model catalog, as the same problem doesn't arise for other brain regions.

For now, I have deleted all the "e-model" entries that I had created earlier (as I can introduce them later when the system is more stable). I tested the REST API now for your query and it works fine.

@apdavison : we need to devise a long term solution for this problem.

Note: This also resolved issue #423

antonelepfl commented 5 years ago

Yes now it works correctly. Tomorrow the new list of the models should be out there

appukuttan-shailesh commented 5 years ago

I suppose there is nothing pending on this ticket. @alex4200 : If so, we could close it.

antonelepfl commented 5 years ago

Just to make sure, there are 90 Hippocampus models in total, @clupascu does this sounds correct?

clupascu commented 5 years ago

@antonelepfl there should be 46 (from the live paper models)+39 (from the latest version of the optimizations)=85 hippocampus models. 2 models from the live paper are not present in the Model Catalog because not used for the circuit, so at least there should be 83, not 90.

appukuttan-shailesh commented 5 years ago

@antonelepfl : I should point out that the model catalog will contain Hippocampus models other than those produced from Michele's team. For example, the paper on HippoUnit (under preparation) makes use of prior models from literature which would satisfy your current search criteria: ?brain_region=hippocampus&organization=HBP-SP6&model_scope=single%20cell&species=Rattus%20norvegicus

To target only the subset of models from Michele's team, you should either filter by author or something else appropriate. For e.g. most of their models have authors as "Rosanna Migliore" or "Paola Vitale". @clupascu : Correct me if I am wrong... and/or suggest better ways to identify your models.

Also, @clupascu : I remember that some models (e.g. CA1_pyr_cACpyr_oh140807_A0_idJ_20190328142723) were supposed to have author as "Paola Vitale" but it shows "Rosanna Migliore". Did you use the updated jupyter notebook (https://collab.humanbrainproject.eu/#/collab/8123/nav/373602) that we had changed following your previous comment?

antonelepfl commented 5 years ago

Ok thank you Shailesh for the clarification. Carmen if you can provide a query that I can use to fetch the models that you want to appear on the list.

clupascu commented 5 years ago

The query I am thinking of to fetch the hippocampus models I want to appear on the list would be "Collab ID"=12027 and you get 39 models (the latest ones) and "Collab ID"=9821 & "Author(s)"="Rosanna Migliore" and you get 43 models (from the live paper). 3 models from the live paper are missing but we will check this later.

clupascu commented 5 years ago

@appukuttan-shailesh yes. Some models e.g. CA1_pyr_cACpyr_oh140807_A0_idJ_20190328142723) were supposed to have author as "Paola Vitale" but it shows "Rosanna Migliore". I used https://collab.humanbrainproject.eu/#/collab/12027/nav/376102. I thought you updated this one. Should I clone it again and try to update the models?