search

SANBIBiodiversityforLife / speciesstatus

0 stars 0 forks source link

Investigate butterfly data extract from Silvia #98

Open rukayaj opened 7 years ago

rukayaj commented 7 years ago

NewAssessDB_Dewidine example.xlsx

Attached is an example of what our assessment data extract looks like - the final export will look like this. Some of the fields aren't applicable for this assessment.

We will also have separate csv files for the various tables (e.g. threats, bibliography, veg types, etc.), as per the plant example sent to us by Lize a long time ago. These are not yet ready.

Dewidine commented 6 years ago

@rukayaj @DomitillaRaimondo We need to urgently figure this out, the butterflies will be ready soon and all the data needs to be uploaded. I will check with them, whether they have any of the csv's that we sent them, ready for export. This spreadsheet is different from the SIS exports, so a new import script might be needed here... So time needs to be spent on understandings this task.

Dewidine commented 6 years ago

@rukayaj NewAssessDB_andre_absolute final.xlsx Update of export

rukayaj commented 6 years ago

Ok it's possible to import this apart from the references which is going to be a headache. This is an example of one of their references: "Mecenero S., Ball J.B., Edge D.A., Hamer M.L., Henning G.A., Kruger M., Pringle E.L., Terblanche R.F., Williams M.C.: , 2013; Conservation Assessment of Butterflies of South Africa, Lesotho and Swaziland: Red List and Atlas" What I really need is the bibtex of these OR for the reference to be in the same format as SIS, where each part of the reference is listed separately. Who do they have doing the development work? Perhaps might be worth showing them https://github.com/AusterweilLab/bibtex-js-apa and similar http://okfnlabs.org/bibjson/ which can get stored + searched in any database which has jsonb fields. I really feel like this is the way to go when it comes to storing bibliographic data.

Just want to check before I start writing this script (which will probably take me a couple of days) that they aren't going to have it in SIS export format any time soon? If so then might be worthwhile us waiting for that...

Dewidine commented 6 years ago

@rukayaj I'll send your suggestion for their references to their guy and cc you in, if he has any further technical questions, he can communicate with you. Maybe we need a Skype, but lets see. Just to confirm, the import from the spreadsheet is fine, its only the references that will need a different method? I'll keep you in the loop.

rukayaj commented 6 years ago

Yeah the import from the spreadsheet is fine - by fine i mean a few hours to write an import script to handle it. If we had their SIS extract instead I could import it immediately like I did with the Sparids. If they're not going to get that sorted though I had better get started writing that script.

Dewidine commented 6 years ago

@rukayaj This is an update of a national assessment so we need to apply versioning, we nee to discuss this urgently, Tilla will try and get some of your time, while you are in Pretoria to discuss it.

rukayaj commented 6 years ago

@Dewidine I didn't see Tilla so you and I will have to chat about this in cape town. When are you free? I have broken the reports on opus.sanbi.org and Ferozah needs them fixed asap, so need to work on that today. I'm hoping I will be done by the end of the day if I concentrate solidly on them.

rukayaj commented 6 years ago

Ok there's a bit of confusion with Silvia having a problem with conservation actions, but apart from that I can just wait for their SIS extract