Check for missing references on build

[hexylena]

https://training.galaxyproject.org/training-material/topics/metagenomics/tutorials/nanopore-16S-metagenomics/tutorial.html one was found there, search 'missing reference'

[mariipia10]

@hexylena Hi Helena , I've got the URL to reference but i'm not sure how i can edit an already existing tutorial that isn't mine. How can i work to do so?

[bebatut]

@mariipia10 I recommend you to check the following tutorial: https://training.galaxyproject.org/training-material/topics/contributing/tutorials/gitpod/tutorial.html. It has a step on "Editing Training Materials on GitPod" and then how to push the changes to GitHub and make a contribution

[hexylena]

Hi @mariipia10 I think I've found a way to detect the missing references, but I'd love help fixing the existing broken ones!

[GTN/scholar] Could not render Langmead_2012 from topics/variant-analysis/tutorials/sars-cov-2-variant-discovery/tutorial.md
[GTN/scholar] Could not render Men2017 }). Hyperactive oncogenic pathway  signatures can be useful in predicting sensitivity to therapeutic agents ({% cite Bild2005 from topics/statistics/tutorials/aberrant_pi3k_pathway_analysis/tutorial.md
[GTN/scholar] Could not render beghini2021integrating from topics/metagenomics/tutorials/metatranscriptomics-short/tutorial.md
[GTN/scholar] Could not render beghini2021integrating from topics/metagenomics/tutorials/metatranscriptomics/tutorial.md
[GTN/scholar] Could not render Wood_2019 from topics/metagenomics/tutorials/nanopore-16S-metagenomics/tutorial.md
[GTN/scholar] Could not render  from topics/computational-chemistry/tutorials/htmd-analysis/tutorial.md
[GTN/scholar] Could not render Vigdis2020 from topics/climate/tutorials/fates/tutorial.md
[GTN/scholar] Could not render Cakir2020  {% cite Moreno2020.04.08.032698 from topics/transcriptomics/tutorials/scrna-seq-basic-pipeline/tutorial_ES.md

That's the output from my tool, so you can see e.g. Langmead_2012 was used in a {% cite Langmead_2012 %} block in topics/variant-analysis/tutorials/sars-cov-2-variant-discovery/tutorial.md, but it could not be found in the topics/variant-analysis/tutorials/sars-cov-2-variant-discovery/tutorial.bib. Maybe it's due to a typo, maybe the reference is just missing completely.

But if you open a PR fixing these we'll get them merged!

[hexylena]

No pressure to fix them all, any improvement is good :)

[mariipia10]

Perfect thanks to both once again! Just got done with the gitpod tutorial and gitpod is up and running.

@hexylena I'll check those once i'm a bit more fluid on markdown, it's a pretty cool and (seems) a fairly easy language. I haven't used it before but I'm excited to learn it as I go 😁.

I have a question, I saw that there are two ways to reference a URL, one using a format like {% cite Hermans2016 %} which gets the reference from the .bib file (looks like the tidiest option) and another one using the markdown format like Duck Duck Go (i used the backlash or else github wouldn't show it as i wanted it to, haha)

Hence, when i use the markdown way of reference everything is working but when I'm citing with the first option I get this error ([citation hidden; run ‘make serve-full’ to show]) I ran the command in the terminal, refreshed the page and nothing happened.

Am i doing something incorrectly? Also, which would be the correct/preferred format for a URL reference? PS: I think the mistake is that its citing the .bib reference with an underscore (around line 267) and it shouldn't have it. Would love to do the PR but I'd like to see that it works on my end first before doing so :D

[bebatut]

I have a question, I saw that there are two ways to reference a URL, one using a format like {% cite Hermans2016 %} which gets the reference from the .bib file (looks like the tidiest option) and another one using the markdown format like Duck Duck Go (i used the backlash or else github wouldn't show it as i wanted it to, haha)

The best way to add a link/URL is the second [Duck Duck Go](https://duckduckgo.com) (Protip: use ` around something you do not want GitHub to interpret

([citation hidden; run ‘make serve-full’ to show]) I ran the command in the terminal, refreshed the page and nothing happened.

The make serve-full command takes long time to update the website every time you save changes in a file. You may need to wait until ...done in <sth> appears in the terminal

PS: I think the mistake is that its citing the .bib reference with an underscore (around line 267) and it shouldn't have it.

It seems like you caught the error for this reference

[hexylena]

You may need to wait until ...done in appears in the terminal

and yesterday we made a big change to the GTN that should make that significantly faster, please be sure you're using the latest version of the GTN's main branch which has all the performance improvements! it will make finding them easier as well (the list I shared https://github.com/galaxyproject/training-material/issues/2788#issuecomment-940792632 has the list of currently broken references that are output as part of the new build process)

[mariipia10]

Sorry i didn't get back, week has ben a bit hectic with exams coming up!

Just did the pull-request on the error I caught and it just got merged and approved by @bgruening

Gitpod wouldn't comply so I'm currently trying to set up a local version of the training materials, with google and Björn's help. I followed a couple of tutorials and got a galaxy instance up and running but still fighting with some dependencies to correctly run training-materials.

@hexylena I think I got the hang of it, next i'll follow your list of missing references.

I think I've found a way to detect the missing references, but I'd love help fixing the existing broken ones!

I'm curious as to how you detected the missing references, did you somehow run each slide with the shell to catch them?

[hexylena]

I'm curious as to how you detected the missing references, did you somehow run each slide with the shell to catch them?

When I rewrote the cite plugin the other day as a custom GTN plugin, I had it print some lines if it couldn't parse the reference. That's the output I wrote above

[GTN/scholar] Could not render Langmead_2012 from topics/variant-analysis/tutorials/sars-cov-2-variant-discovery/tutorial.md
[GTN/scholar] Could not render Men2017 }). Hyperactive oncogenic pathway  signatures can be useful in predicting sensitivity to therapeutic agents ({% cite Bild2005 from topics/statistics/tutorials/aberrant_pi3k_pathway_analysis/tutorial.md
[GTN/scholar] Could not render beghini2021integrating from topics/metagenomics/tutorials/metatranscriptomics-short/tutorial.md
[GTN/scholar] Could not render beghini2021integrating from topics/metagenomics/tutorials/metatranscriptomics/tutorial.md
[GTN/scholar] Could not render Wood_2019 from topics/metagenomics/tutorials/nanopore-16S-metagenomics/tutorial.md
[GTN/scholar] Could not render  from topics/computational-chemistry/tutorials/htmd-analysis/tutorial.md
[GTN/scholar] Could not render Vigdis2020 from topics/climate/tutorials/fates/tutorial.md
[GTN/scholar] Could not render Cakir2020  {% cite Moreno2020.04.08.032698 from topics/transcriptomics/tutorials/scrna-seq-basic-pipeline/tutorial_ES.md

and which shows up in the GTN build output. Thanks for fixing a few of them! Looks like we still need a few more and then we can enable the script I wrote to ensure they're there (bin/check-citations.rb)

[GTN/scholar] Could not render beghini2021integrating from topics/metagenomics/tutorials/metatranscriptomics-short/tutorial.md
[GTN/scholar] Could not render beghini2021integrating from topics/metagenomics/tutorials/metatranscriptomics/tutorial.md
[GTN/scholar] Could not render  from topics/computational-chemistry/tutorials/htmd-analysis/tutorial.md
[GTN/scholar] Could not render Cakir2020  {% cite Moreno2020.04.08.032698 from topics/transcriptomics/tutorials/scrna-seq-basic-pipeline/tutorial_ES.md

[mariipia10]

When` I rewrote the cite plugin the other day as a custom GTN plugin, I had it print some lines if it couldn't parse the reference. That's the output I wrote above

Awesome very clear, thanks for explaining!

Yep I'm still missing a few haven't quite caught what's wrong with them yet, the other ones had almost all the same pattern. Not giving up 'til the list is complete :P

[hexylena]

Awesome @mariipia10, good luck!

[mariipia10]

Hi, just PR'rd remaining conflicts for references,! @hexylena there's one that in your output that is an empty cite

[GTN/scholar] Could not render from topics/computational-chemistry/tutorials/htmd-analysis/tutorial.md

I went straight to the tutorial , checked one by one and they all seem to be just up and running. The only empty cite is one that is commented, could that be the issue when running your script?

I don't know if it's commented bc of a specific reason say sintax for e.g. or something else thus why I didn't remove it Line 145 has <!-- ({% cite %}) #-->

If removing it is fine, I'll gladly do so!

[hexylena]

Yes! That looks like the issue. Yes,.please remove it.

El mié., 20 de octubre de 2021 21:27, mariipia10 @.***> escribió:

Hi, just PR'rd remaining conflicts for references,! @hexylena https://github.com/hexylena there's one that in your output that is an empty cite

[GTN/scholar] Could not render from topics/computational-chemistry/tutorials/htmd-analysis/tutorial.md

I went straight to the tutorial https://training.galaxyproject.org/training-material/topics/computational-chemistry/tutorials/htmd-analysis/tutorial.html, checked one by one and they all seem to be just up and running. The only empty cite is one that is commented, could that be the issue when running your script?

I don't know if it's commented bc of a specific reason say sintax for e.g. or something else thus why I didn't remove it Line 145 has

If removing it is fine, I'll gladly do so!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/galaxyproject/training-material/issues/2788#issuecomment-947970768, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADP7O7PO6WFXR2AP4L6XCDUH4J2RANCNFSM5ETFN7BA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[mariipia10]

Great just did it, it's in the same PR!

[hexylena]

That's it! It's done!!! amazing, they're all fixed, thank you @mariipia10 !

[mariipia10]

Great, thanks for the help! Happy to see the issue is done and closed :)