upload bat files from sandbox to production

[myrmoteras]

Hi Donat and @flsimoes ,

Manuel finished last week the Sandbox upload of the ~230 records bats collection from the Google Sheet Felipe and Juliana shared in the last Arcadia sprint.

• Everything was uploaded in a separate Zenodo community at https://sandbox.zenodo.org/communities/bats_project/search. If you agree with this method of organisation, we could do the same when we perform the final upload on the production system. You can create the Zenodo community under the BLR Zenodo account and add any extra information in the description. As discussed before we can include the records also in the classic BLR community.
• The Google sheet used for the import and upload via Lycophron is at https://docs.google.com/spreadsheets/d/19T3S6kKJyVJpe7lgd5GKYKLfZXMCTZhgo6d0N8EhfIk/edit

Some pending action items before we go ahead with uploading to production:

• You’ll notice that the community includes 225 records, while the Google sheet has 239 rows (240 - 1 header).
  • 5 records with existing DOIs were already uploaded from previous tries
  • 5 records had an invalid affiliation value, but we can clean this up in the Google sheet
• 45 entries in the Google sheet were missing DOIs, but we uploaded them anyways to verify the metadata: https://sandbox.zenodo.org/communities/bats_project/search?q=exists:conceptdoi
• The metadata on the records (or at least a sample of them) has to be verified, just to make sure we didn’t mess up anything in the process. These are metadata-only updates which is fine to also perform later on (even after the production upload).

Let me know if you have any questions, I’ll be off next week on holidays, but we can re-route to the rest of the team so they can help.

Cheers, Alex

PS: I couldn’t find Juliana’s email address, so feel free to forward this or add her in the loop.

[myrmoteras]

https://sandbox.zenodo.org/record/1179813#.ZC2SGXZBz8A

• missing author

[myrmoteras]

https://sandbox.zenodo.org/record/1179813#.ZC2S03ZBz8B

having tags (e.g. <i> in the title seems to be recurring issue

(Boa constrictor orophias)

remove tags in title

[myrmoteras]

@slint can you add the files of this bat upload NOT to the bats_project a new bat to the biosyslit, coviho and the globalbioticinteractions

[slint]

Regarding communities, yes we can add both the biosyslit and coviho communities.

For the GloBi community though, it's Jorrit who is curating it, in which case he will have to accept all the record inclusion requests (we can also do this on our side quickly for all of them if he agrees).

[myrmoteras]

@slint @Donat Agosti please feel free to automatically submit the docs to the globalbioticinteractions zenodo community. We can auto-accept if needed.

[myrmoteras]

via @slint

Regarding Lycophron, I agree we can proceed with pushing to production. There were though a couple of clean-up tasks still (there are some HTML tags in titles, missing DOIs, etc.), that we can’t do on our side without the domain knowledge. We can revive the GitHub issue and ping the right people. Afterwards Manuel can close it up and make the run (and also automatically add it to the GloBi community as well).

@flsimoes can you tell @juwingert to look into this issue?

[flsimoes]

• https://docs.google.com/spreadsheets/d/19T3S6kKJyVJpe7lgd5GKYKLfZXMCTZhgo6d0N8EhfIk/edit

I do not have access to the spreadsheet in question

[juwingert]

• https://docs.google.com/spreadsheets/d/19T3S6kKJyVJpe7lgd5GKYKLfZXMCTZhgo6d0N8EhfIk/edit

I do not have access to the spreadsheet in question

Me neither.

[slint]

I'm sorry, can you try again here: https://docs.google.com/spreadsheets/d/19T3S6kKJyVJpe7lgd5GKYKLfZXMCTZhgo6d0N8EhfIk/edit?usp=sharing

[myrmoteras]

@slint is it just cleaning up the column A with the article title

[juwingert]

@slint @myrmoteras I changed the i tag to em in column A and added some missing DOIs, is that it?

[slint]

• Regarding the HTML tags, they need to be completely removed from the titles, since they're not actually supported, and are rendered as regular text on the record page
• For the missing DOIs, if there is no known corresponding DOI, then they don't have to be filled-in and we will register a Zenodo DOI for them

[flsimoes]

• Regarding the HTML tags, they need to be completely removed from the titles, since they're not actually supported, and are rendered as regular text on the record page

Ah ok, I don't recall why, but when Marcus started development on Lycophron he asked us to fill the HTML tags.

[juwingert]

• Regarding the HTML tags, they need to be completely removed from the titles, since they're not actually supported, and are rendered as regular text on the record page

   OK, I removed them.

• For the missing DOIs, if there is no known corresponding DOI, then they don't have to be filled-in and we will register a Zenodo DOI for them

   I completed two or three DOI's, only the ones I undoubtedly had.

[slint]

Ah ok, I don't recall why, but when Marcus started development on Lycophron he asked us to fill the HTML tags.

For the description column that is good, since we render the HTML.

---

Thanks a lot for having a look and fixing the entries @juwingert and @flsimoes, we are ready to upload it on production now. The team is a bit swamped this week, but I think we can have it up by the end of next week. I'll respond here, with links to the records.

[myrmoteras]

@slint very good. Let us know. We are having a second part of the bat publications for this Covid-related bat corroosting project by next week, so we can prepare them for a next upload

Also, please make sure that the upload is to BLR and covihio community, and that we have to admin rights, becuase we eventually will process those files to move them also into biodivPMC at SIBiLS.

tx

[myrmoteras]

For the description column that is good, since we render the HTML.

@slint the reason the emphasis are in the text is because these are taxonomic names. May be later this could be used to format taxonomic names in the title of the deposit?

[myrmoteras]

@slint can we move this now into production, so we have this done? tx

[slint]

@myrmoteras, @alejandromumo has uploaded the records up in production! We've managed to narrow them down using this search query, in case you want to share.

Two things missing:

• We need to accept them to the GloBi community (will soon run a script to do so automatically)
• One of the records from the Google Sheet was already uploaded in the past on BLR (I guess via the usual workflow, since it has lots of links): https://zenodo.org/record/4588432 (https://doi.org/10.1038/s41598-020-64159-1)

[myrmoteras]

@alejandromumo @slint thanks for this. Is there a way to get this upload selected and or viewed, that is, give me all the articles of this batch upload? Or would we need to add a special keyword or similar to it?

This is question that will be raised tomorrow afternoon when I will show it to my covid colleagues...

[myrmoteras]

@slint I understand, that we also could add keywords, or custom keywords. Is this right?

[myrmoteras]

another question is, do we have a list of CSV or similar in which we have the Zenodo ID added to the bibliographic records?

[slint]

Is there a way to get this upload selected and or viewed, that is, give me all the articles of this batch upload? Or would we need to add a special keyword or similar to it?

Right now, the search query I shared above was basically "open access articles from 2023-08-01 that are in BLR, COVIHO, and GloBi", which narrows it down to the exact set. But I agree, that's not a very trivial differentiating factor. That's why originally we proposed placing these either in a new Zenodo community (as if they were a specific "collection" of sorts), or maybe marking them with a keyword (or even better if there was a DarwinCore custom field that could describe them all).

@slint I understand, that we also could add keywords, or custom keywords. Is this right?

Yes, all of the Zenodo metadata fields are supported in the input CSV.

another question is, do we have a list of CSV or similar in which we have the Zenodo ID added to the bibliographic records?

I think @alejandromumo keeps the Zenodo ID in the Lycophron database file, so we could make an export.

[alejandromumo]

@myrmoteras @flsimoes @slint

Find below the sheet with the results of the run in prod: link

It is currently private so I will accept your access requests as soon as you send them.

[myrmoteras]

@alejandromumo can we share the link with the COVID task force group to show them? Or may be just make it open for viewing?

[alejandromumo]

@alejandromumo can we share the link with the COVID task force group to show them? Or may be just make it open for viewing?

if it makes it easier to share, I can make it public for view. Is there any particular data they might be interested in? Otherwise, since we'd be making it public, we can show just some public fields in the sheet (e.g. DOI and zenodo URL).

[myrmoteras]

I would just make it public as is for viewing. It is interesting for this group to look at the structure and granularity.

[myrmoteras]

@alejandromumo would it be possible to get the Zenodo deposition ID linked to the original XLS file, which includes the bibibliographic data?

[myrmoteras]

@alejandromumo @slint

we are running a publisher workshop in Carouge on Sept 14/15, where Lars will participate. Is it possible that we could have a version of Lycorphron there, including the write back of the Zenodo id into the original source file, so we can refer to it?

I would bring this to the attention of the audience that authors or publishers could just upload all the articles to BLR so they have all their articles

1. with a DOI if not available
2. ready for further processing
3. processed through TB, either directly through the webhook or via TB and thus have also the data there in FAIR. 3 is essentially the next step of TNA in BiCIKL

Thanks