North-Wyke-Farm-Platform / NWFP-WEB-000

To serve as a placeholder for all ideas regarding the new site for the NWFP - there might be some code there eventually
MIT License
0 stars 0 forks source link

Model: publications #14

Open nathcast opened 1 month ago

nathcast commented 1 month ago

A table to list publications - usually references. I have already quite a lot of scripts for that methods and they are a good start

The references are held in a Endnote reference software:

nathcast commented 1 month ago

Hi: I have put the output style in OneDrive - Rothamsted Research\Documents - NWFP Website Project\General\bibliography\styles

I hope you manage to integrate that. We have a field called GRID in our biblio refering to some references - you will not need that one

Nath

soilsci commented 4 weeks ago

Thanks for the output style. I have managed to find a way of importing Endnote bibliography into R using a package called RefManageR. r which imports BibTex style output but I have edited that to fit in with the EraTabDelimited2021 output style output. I am still experimenting with how to transfer seamlessly into into a database. It would be useful to have the e-RAdoc schema so I can see how the database is set up. I usually use SQlite but am familiarising myself with DBeaver.

nathcast commented 4 weeks ago

The table for the papers in eradoc is: We won't need GRID in nwfp

CREATE TABLE eraPapers ( PaperID int(11) NOT NULL AUTO_INCREMENT, RefType tinytext, Authors text, Year int(11) DEFAULT '0', Title text, Journal text, Volume text, Issue text, Pages text, Date text, Keywords text, Comment text, DOI tinytext, GRID tinytext, URL mediumtext, PRIMARY KEY (PaperID), FULLTEXT KEY Authors (Authors,Title,Journal), FULLTEXT KEY Keywords (Keywords), FULLTEXT KEY DOI (DOI), FULLTEXT KEY URL (URL) ) ENGINE=MyISAM AUTO_INCREMENT=3011 DEFAULT CHARSET=latin1 COMMENT='List papers imported from Endnote';

soilsci commented 4 weeks ago

Ah OK - there is only one table then. I thought that there might be separate tables for Author, Year, Title etc. that would need writing to. That makes it little more simpler! Thanks Nathalie

nathcast commented 4 weeks ago

So basically, in EN, all the managment of the references. The EN output a CSV that is then imported in the db. This is straight forward. Why so many steps? In exel: I check that there is only one line per record because if a Carriage return has been included in the csv file, it creates havoc. So in in the CSV, I can spot these mistakes that are then corrected in EN and export again. Then in the XL, I also correct some problem to do with 2 papers that for some reason I have not been able to correct in EN, that won't be a problem in your library, I also check that the formating of the DOI or URLs are correct as sometimes there are some discrepancies. I also remove the links to For some reason, XL saves as a UTF that is not understood by MySQL, so I have a quick step in Notepad++ to add the table hearder (could be done in xls) and reconvert as UTF so that the caracters are properly done So , EN outputs a CSV and if your R code validates the rows, and imports in msql with the proper encoding, that would be great. I could show you how it is done if you want, and then you can see if you need R to replace the workflow.

Thanks,