search

bioimage-io / collection-bioimage-io

(deprecated in favor of bioimage-io/collection) RDF collection for BioImage.IO
5 stars 9 forks source link

Update filoTips default #636

Closed bioimageiobot closed 1 year ago

bioimageiobot commented 1 year ago

This is an automatic PR created by the @bioimageiobot regarding changes to the resource item 10.5281/zenodo.8346993. The following version(s) will be added:

Please review the changes and make sure the new item or version(s) pass the following check list:

Maintainers: @eddin022

Note: If you updated or re-uploaded another version for the current item on Zenodo, this PR won't be changed automatically. To proceed, you can do the following:

  1. Block this version, but keep looking for future versions: Edit the current resource.yaml and keep the top-level status field as accepted, but change the status under the current version to blocked.
  2. Accept this version and keep looking for future versions: Merge this PR for now.

  3. Keep proposed version(s) (and this resource in general if it is new) as pending: Close this PR without merging.

    Then wait for the CI on the main branch to complete. It should detect the new version(s) and create another PR for the new version(s).

Previous PRs of this resource: none

FynnBe commented 1 year ago

Thank you for contributing @eddin022 !

The zenodo record is missing all files linked in the rdf.yaml. @oeway , maybe there was an issue with bioimage.io/upload ?

For whatever reasons the files are missing, @eddin022 please add them. Let us know if you encounter any issues using bioimage.io/upload . Note that new zenodo record versions can also be created directly at zenodo.org.

One other thing I already noticed:

esgomezm commented 1 year ago

@eddin022, I think you used the ZeroCostDL4Mic notebook for this one. When you run the export to the bioimage model zoo, the notebook generates a zip file with the model and all the content that @FynnBe addressed. Can you see that zip file? Could you please send us a screenshot of its content? Let us know if we can help you upload the model correctly :)

eddin022 commented 1 year ago

Dear esgomezm,

Thank you for your reply. Yes, ZeroCostDL4Mic was used for this. The screenshot is below. I was a bit confused about the upload process. I believe I just uploaded the .yaml file (oops!). The .zip package is what should have been uploaded though correct? To fix this, should I edit the upload on Zenodo or Bioimage (or both)?

Thanks again!

Regards, Casey

[image: Screenshot 2023-10-04 at 6.11.57 PM.png]

On Wed, Oct 4, 2023 at 6:14 AM esgomezm @.***> wrote:

@eddin022 https://github.com/eddin022, I think you used the ZeroCostDL4Mic notebook for this one. When you run the export to the bioimage model zoo, the notebook generates a zip file with the model and all the content that @FynnBe https://github.com/FynnBe addressed. Can you see that zip file? Could you please send us a screenshot of its content? Let us know if we can help you upload the model correctly :)

— Reply to this email directly, view it on GitHub https://github.com/bioimage-io/collection-bioimage-io/pull/636#issuecomment-1746660457, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQDO45IIUQNNWGPAO5EF5I3X5VAINAVCNFSM6AAAAAA45LUEFWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBWGY3DANBVG4 . You are receiving this because you were mentioned.Message ID: @.***>

FynnBe commented 1 year ago

To fix this, should I edit the upload on Zenodo or Bioimage (or both)?

Whatever you prefer. You are updating the zenodo record either way. bioimage.io/#/upload may be more convenient to just drop the whole zip file.

I'll block this pending version 1 and look forward to see your updated version 2!

bioimageiobot commented 1 year ago

preview-collection-json

eddin022 commented 1 year ago

Fynn,

Sorry, I'm new to the platform. Is there a way to edit the upload, or should I just start a new upload?

Thanks again, Casey

On Thu, Oct 5, 2023 at 3:13 AM Fynn Beuttenmüller @.***> wrote:

To fix this, should I edit the upload on Zenodo or Bioimage (or both)?

Whatever you prefer. You are updating the zenodo record either way. bioimage.io/#/upload may be more convenient to just drop the whole zip file.

I'll block this pending version 1 and look forward to see your updated version 2!

— Reply to this email directly, view it on GitHub https://github.com/bioimage-io/collection-bioimage-io/pull/636#issuecomment-1748344645, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQDO45L4IEAK3A62TDHBJLTX5ZT2XAVCNFSM6AAAAAA45LUEFWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBYGM2DINRUGU . You are receiving this because you were mentioned.Message ID: @.***>

FynnBe commented 1 year ago

No worries, very happy to see that you contribute!

Either on zenodo.org or through bioimage.io you'll create another version of the record. Each Zenodo record has a concept DOI that is fixed and each version of that record has their own version specific DOI, see https://help.zenodo.org/faq/#versioning.

To add a new verison to a record on zenodo navigate to it while being logged in and you'll see a "new version" button below "edit" (edit is for chaning title, descripiton etc, keeping the files untouched): image

The bioimage.io/#/upload form also allows you such an update.

here in collection-bioimage-io we now know of your filoTips model and only blocked the current, first version, not the "concept" of it. So a new version will trigger a new PR.

So, yes please upload a new version of your record.

eddin022 commented 1 year ago

Dear Fynn,

Thanks for the help so far! I have uploaded a new version of this model successfully.

I'm now trying to upload my other models via bioimage.io. When trying to upload these models, the validation passes and "OK" is available to click. When I click "OK" I get the following error message: "Failed to find similar items, please try it again." I couldn't find a similar error online so I'm not sure how to fix it. I tried logging out and re-uploading, but received the same error message.

Regards, Casey

On Thu, Oct 5, 2023 at 3:30 PM Fynn Beuttenmüller @.***> wrote:

No worries, very happy to see that you contribute!

Either on zenodo.org or through bioimage.io you'll create another version of the record. Each Zenodo record has a concept DOI that is fixed and each version of that record has their own version specific DOI, see https://help.zenodo.org/faq/#versioning.

To edit a record on zenodo navigate to it while being logged in and you'll see an edit button: [image: image] https://user-images.githubusercontent.com/15139589/273031734-39041d2b-c01a-4457-ac74-28ced274f591.png

The bioimage.io/#/upload form also allows you such an update.

here in collection-bioimage-io we now know of your filoTips model and only blocked the current, first version, not the "concept" of it. So a new version will trigger a new PR.

So, yes please "edit" your upload by uploading a new version.

— Reply to this email directly, view it on GitHub https://github.com/bioimage-io/collection-bioimage-io/pull/636#issuecomment-1749598351, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQDO45IOWBR7TPUP27EPNVTX54KGXAVCNFSM6AAAAAA45LUEFWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBZGU4TQMZVGE . You are receiving this because you were mentioned.Message ID: @.***>

FynnBe commented 1 year ago

There is an issue with the new version you uploaded: Somehow the zenodo record holds a zip archive instead of the individual files. We currently do not support this. Maybe @oeway already knows what's going wrong. There might be an issue with the uploader.

oeway commented 1 year ago

There is an issue with the new version you uploaded: Somehow the zenodo record holds a zip archive instead of the individual files. We currently do not support this. Maybe @oeway already knows what's going wrong. There might be an issue with the uploader.

I haven’t looked into the uploader yet, but with the upgrade in zenodo, I doubt it will work properly.

I suggest that hold on the task and maybe take the oppotunity to migrate the uploading system to the new one. @FynnBe This however does mean that we will need to let the upload stop for a while. What do you think?

esgomezm commented 1 year ago

Hi @oeway and @FynnBe I have just checked and the uploader is not working at all. It does not upload any file anywhere. @oeway I agree with you that maybe it should be temporarily blocked until is solved.

@eddin022 I apologise for all the mess. I checked your repo and I think you directly updated the zenodo version by uploading the zip file, am I right? You did as we told you and we were not very clear so, our apologies. Now, the content of the zip file is perfectly fine (I double-checked)! =) We use the zip to drag and drop the content when uploading a model through the bioimage model zoo web page. If you want to update Zenodo directly as you did, you need to upload the files inside the zip independently (without being compressed). So all these files: Contours2InstanceSegmentation.ijm, sample_input_0.tif, README.md, sample_output_0.tif, cover.png, test_input.npy, keras_weights.hdf5, test_output.npy, per_sample_scale_range.ijm, tf_weights.zip, rdf.yaml, training_evaluation.csv. It will look like this:

Screenshot 2023-10-17 at 10 37 38

Please, update it and we will follow up!

eddin022 commented 1 year ago

Dear all,

Ahh, okay. I have uploaded as you requested. Thanks again for the help!

Unfortunately, I think I might need more... I'm trying to upload a total of 3 additional models via bioimage.io. I click upload, login to zenodo, drag and drop my zip file, I'm able to validate the form and the "OK" button becomes active. However, after clicking "OK" I receive an error "Failed to find similar items, please try it again". Is this the uploader error you were referring to? How should I proceed with uploading the other 3 models?

Regards, Casey

On Tue, Oct 17, 2023 at 3:38 AM esgomezm @.***> wrote:

Hi @oeway https://github.com/oeway and @FynnBe https://github.com/FynnBe I have just checked and the uploader is not working at all. It does not upload any file anywhere. @oeway https://github.com/oeway I agree with you that maybe it should be temporarily blocked until is solved.

@eddin022 https://github.com/eddin022 I apologise for all the mess. I checked your repo and I think you directly updated the zenodo version by uploading the zip file, am I right? You did as we told you and we were not very clear so, our apologies. Now, the content of the zip file is perfectly fine (I double-checked)! =) We use the zip to drag and drop the content when uploading a model through the bioimage model zoo web page. If you want to update Zenodo directly as you did, you need to upload the files inside the zip independently (without being compressed). So all these files: Contours2InstanceSegmentation.ijm, sample_input_0.tif, README.md, sample_output_0.tif, cover.png, test_input.npy, keras_weights.hdf5, test_output.npy, per_sample_scale_range.ijm, tf_weights.zip, rdf.yaml, training_evaluation.csv. It will look like this: [image: Screenshot 2023-10-17 at 10 37 38] https://user-images.githubusercontent.com/43643518/275767490-861d3f37-da48-4c49-98a3-a5257ba56bef.png

Please, update it and we will follow up!

— Reply to this email directly, view it on GitHub https://github.com/bioimage-io/collection-bioimage-io/pull/636#issuecomment-1765943726, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQDO45MQTXSJXMPTBKI3723X7Y7Z5AVCNFSM6AAAAAA45LUEFWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRVHE2DGNZSGY . You are receiving this because you were mentioned.Message ID: @.***>

FynnBe commented 1 year ago

There is an issue with the new version you uploaded: Somehow the zenodo record holds a zip archive instead of the individual files. We currently do not support this. Maybe @oeway already knows what's going wrong. There might be an issue with the uploader.

I haven’t looked into the uploader yet, but with the upgrade in zenodo, I doubt it will work properly.

I suggest that hold on the task and maybe take the oppotunity to migrate the uploading system to the new one. @FynnBe This however does mean that we will need to let the upload stop for a while. What do you think?

I wouldn't mind. The option to upload directly to zenodo is unaffected.

FynnBe commented 1 year ago

How should I proceed with uploading the other 3 models?

@eddin022 Please go directly to https://zenodo.org, login there, (go to the record you want edit) and create the (new verison of the) record as laid out by @esgomezm.

eddin022 commented 1 year ago

Fynn,

I have created a new version for the first model. I'm trying to upload 3 other models. Since they are different, my plan was to upload them separately and I'm receiving an error message when trying to upload a new model on bioimage.io. I'm happy to upload them directly on zenodo, but I would also like them to be uploaded on bioimage.io if possible. If I upload them directly on zenodo, can we later have them available on bioimage.io as well?

Casey

On Tue, Oct 17, 2023 at 12:04 PM Fynn Beuttenmüller < @.***> wrote:

How should I proceed with uploading the other 3 models?

@eddin022 https://github.com/eddin022 Please go directly to https://zenodo.org, login there, (go to the record you want edit) and create the (new verison of the) record as laid out by @esgomezm https://github.com/esgomezm.

— Reply to this email directly, view it on GitHub https://github.com/bioimage-io/collection-bioimage-io/pull/636#issuecomment-1766826157, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQDO45MDMHR37PW56D2OFSTX723CRAVCNFSM6AAAAAA45LUEFWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRWHAZDMMJVG4 . You are receiving this because you were mentioned.Message ID: @.***>

FynnBe commented 1 year ago

I'm happy to upload them directly on zenodo, but I would also like them to be uploaded on bioimage.io if possible. If I upload them directly on zenodo, can we later have them available on bioimage.io as well?

Dear Casey, apologies for our unclear communication -- we've been talking amongst ourselves so much it is hard sometimes to imagine a fresh pair of eyes on the matter. As we currently use zenodo.org as our storage space for all models, all the bioimage.io uplaoder would do is upload to zenodo.org in your name and make sure the keyword "bioimage.io" is present. Through this keyword our bot detects relevant zenodo records and opens a Pull Request (PR) like this one.

The upload to zenodo.org via bioimage.io has caused confusion before; therefore we decided to change this part in the near future. For now, please uplaod manually to zenodo.org and make sure to tag your recrods with the "bioimage.io" keyword. Once a PR such as this is merged they will show up on bioimage.io !

eddin022 commented 1 year ago

Got it, thanks!

Casey

On Tue, Oct 17, 2023 at 3:11 PM Fynn Beuttenmüller @.***> wrote:

I'm happy to upload them directly on zenodo, but I would also like them to be uploaded on bioimage.io if possible. If I upload them directly on zenodo, can we later have them available on bioimage.io as well?

Dear Casey, apologies for our unclear communication -- we've been talking amongst ourselves so much it is hard sometimes to imagine a fresh pair of eyes on the matter. As we currently use zenodo.org as our storage space for all models, all the bioimage.io uplaoder would do is upload to zenodo.org in your name and make sure the keyword "bioimage.io" is present. Through this keyword our bot detects relevant zenodo records and opens a Pull Request (PR) like this one.

The upload to zenodo.org via bioimage.io has caused confusion before; therefore we decided to change this part in the near future. For now, please uplaod manually to zenodo.org and make sure to tag your recrods with the "bioimage.io" keyword. Once a PR such as this is merged they will show up on bioimage.io !

— Reply to this email directly, view it on GitHub https://github.com/bioimage-io/collection-bioimage-io/pull/636#issuecomment-1767096248, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQDO45LUVJGLFINK3LIAUCDX73RAZAVCNFSM6AAAAAA45LUEFWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRXGA4TMMRUHA . You are receiving this because you were mentioned.Message ID: @.***>

esgomezm commented 1 year ago

@eddin022 just to double confirm with you: Each of your new models should be in a completely new Zenodo repository. So, each Zenodo repository (same as for papers, code or whatever), represents "A MODEL". If you change the model (as you did until now to update it or correct the format), then you need to update its corresponding Zenodo repository with a new version. If you want to upload a new model, then you need to create a new repository by clicking on + (new upload) in Zenodo. Once you create it, make sure as @FynnBe said, that you add the bioimage.io as a community. See the shots below Hope it's getting easy!

eddin022 commented 1 year ago

Got it. Thanks. I think the most recent issue was not being able to upload new models (not versions but different models) via bioimage.io recently. I was able to upload via zenodo directly though and I selected the bioimage.io community for each one.

Casey

On Wed, Oct 18, 2023 at 4:49 AM esgomezm @.***> wrote:

@eddin022 https://github.com/eddin022 just to double confirm with you: Each of your new models should be in a completely new Zenodo repository. So, each Zenodo repository (same as for papers, code or whatever), represents "A MODEL". If you change the model (as you did until now to update it or correct the format), then you need to update its corresponding Zenodo repository with a new version. If you want to upload a new model, then you need to create a new repository by clicking on + (new upload) in Zenodo. Once you create it, make sure as @FynnBe https://github.com/FynnBe said, that you add the bioimage.io as a community. See the shots below Hope it's getting easy!

  • New upload to create a new zenodo repository.

[image: Screenshot 2023-10-18 at 11 46 41] https://user-images.githubusercontent.com/43643518/276200155-938ee70c-0f2b-49ca-bff4-0011548f5e41.png

— Reply to this email directly, view it on GitHub https://github.com/bioimage-io/collection-bioimage-io/pull/636#issuecomment-1768086904, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQDO45JGSTYFFB4IVKCJEODX76Q3BAVCNFSM6AAAAAA45LUEFWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRYGA4DMOJQGQ . You are receiving this because you were mentioned.Message ID: @.***>

eddin022 commented 1 year ago

Dear all,

I hope you have been well.

I uploaded a total of 3 models and 2 datasets directly to Zenodo (listing bioimage.io as the community) mid-Sept and mid-Oct. 1 model has been accepted, but the rest are in review. Is there anything else that needs to be done for them to be accepted?

Thanks again!

Casey

On Wed, Oct 18, 2023 at 11:45 AM Casey Eddington @.***> wrote:

Got it. Thanks. I think the most recent issue was not being able to upload new models (not versions but different models) via bioimage.io recently. I was able to upload via zenodo directly though and I selected the bioimage.io community for each one.

Casey

On Wed, Oct 18, 2023 at 4:49 AM esgomezm @.***> wrote:

@eddin022 https://github.com/eddin022 just to double confirm with you: Each of your new models should be in a completely new Zenodo repository. So, each Zenodo repository (same as for papers, code or whatever), represents "A MODEL". If you change the model (as you did until now to update it or correct the format), then you need to update its corresponding Zenodo repository with a new version. If you want to upload a new model, then you need to create a new repository by clicking on + (new upload) in Zenodo. Once you create it, make sure as @FynnBe https://github.com/FynnBe said, that you add the bioimage.io as a community. See the shots below Hope it's getting easy!

  • New upload to create a new zenodo repository.

[image: Screenshot 2023-10-18 at 11 46 41] https://user-images.githubusercontent.com/43643518/276200155-938ee70c-0f2b-49ca-bff4-0011548f5e41.png

— Reply to this email directly, view it on GitHub https://github.com/bioimage-io/collection-bioimage-io/pull/636#issuecomment-1768086904, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQDO45JGSTYFFB4IVKCJEODX76Q3BAVCNFSM6AAAAAA45LUEFWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRYGA4DMOJQGQ . You are receiving this because you were mentioned.Message ID: @.***>

FynnBe commented 1 year ago

Dear Casey, I'm working on fixing the issues in https://github.com/bioimage-io/collection-bioimage-io/pull/656 And I haven't seen the datasets you're mentioning. Would you be so kind to share the zenodo links or DOIs of the resources you want to contribute, then I'll check each one myself.

I'm sorry for the delay and inconvenience. We are working on a series of internal updates and recently Zenodo changes their API which threw us a bit into turmoil.

eddin022 commented 1 year ago

Fynn,

No problem. Below are the DOIs:

10.5281/zenodo.10019447 10.5281/zenodo.10015653 10.5281/zenodo.10015613 10.5281/zenodo.8356811

Casey

On Mon, Nov 13, 2023 at 2:34 AM Fynn Beuttenmüller @.***> wrote:

Dear Casey, I'm working on fixing the issues in #656 https://github.com/bioimage-io/collection-bioimage-io/pull/656 And I haven't seen the datasets you're mentioning. Would you be so kind to share the zenodo links or DOIs of the resources you want to contribute, then I'll check each one myself.

I'm sorry for the delay and inconvenience. We are working on a series of internal updates and recently Zenodo changes their API which threw us a bit into turmoil.

— Reply to this email directly, view it on GitHub https://github.com/bioimage-io/collection-bioimage-io/pull/636#issuecomment-1807676147, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQDO45JEWPCJ6IMHDBMRRB3YEHLSJAVCNFSM6AAAAAA45LUEFWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBXGY3TMMJUG4 . You are receiving this because you were mentioned.Message ID: @.***>