Bulk-upload of names

[reupost]

Administrators should be able to bulk-upload non-assessed names (as per issue #60). They would need to ensure beforehand (i.e. manually) that the species names do not 'clash' with existing names on the site, since I assume these need to be unique.

The format of the upload could comprise a delimited text file with each line structured something like: species group | family | species | non-assessed status

where "species group" = beetle, bird etc. and "non-assessed status" is either "Not assessed - assumed not sensitive" or "Not assessed - assume sensitive until formal assessment".

I am hoping your existing code to load the 2010 list could be amended to support this.

Once uploaded, the administrator needs to be able to reassign the ownership of these new content items to the appropriate expert.

[rukayaj]

So there's two options. Option 1: we use my code. In order to use my code a CSV of the correct format needs to be in the /files folder on the server, and once that's in place there's a menu link in the administrative menu which needs to be clicked. So I'm not really sure if that's good enough for administrators to use, there isn't a proper UI or anything. But perhaps it's ok if this functionality is just required for a little bit at the start of the project, what do you think? Keeping in mind too that this site is temporary and will get ditched for something else soon.

No, names don't need to be unique on the site. It wouldn't work if they were because some animals have the same names as plants I believe?

Admins assigning ownership of new content items - if I end up doing the bulk uploads I can set the user ID as I upload them using my code. So that's ok.

Option 2: Alternatively I'll have to install a module like https://www.drupal.org/node/2574707 or https://www.drupal.org/project/feeds or (perhaps most promisingly) https://www.drupal.org/project/contentimport. If I end up having to go down this route it will take a little while to get right.

[reupost]

I think its fine. I can prepare the CSV file and pass that over to you to put onto the server. By 'admin' I really meant you or I at this point!

[rukayaj]

Ok so I should remove the past justification field right? As that's no longer going to get populated. It was never shown in the edit form to begin with, so it's just a behind the scenes kind of change.

[reupost]

Yes that sounds good.

On 15 Nov 2017 08:01, "Rukaya" notifications@github.com wrote:

Ok so I should remove the past justification field right? As that's no longer going to get populated. It was never shown in the edit form to begin with, so it's just a behind the scenes kind of change.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/rukayaj/ssp/issues/65#issuecomment-344514759, or mute the thread https://github.com/notifications/unsubscribe-auth/AEJ8TAxFpEiiQp0pDTb5AMjjsX1rA_txks5s2ppsgaJpZM4Qb_gl .

[rukayaj]

Great I think I've done this one.

In order to do a bulk import, one needs to log into the server and navigate to the NSSL folder in IIS, and then under /web/sites/default/files edit or replace the species_for_import.csv. Example of the CSV + headers:

Group,Family,Species,Non-assessed status,Author UID plant,Apocynaceae,Adenium swazicum Stapf,Not sensitive,10 plant,Asphodelaceae,Aloe bowiea Schult. & J.H.Schult.,Not sensitive,52 Scorpions,Scorpionidae,Opistophthalmus wahlbergii,Sensitive,52 Butterflies,Lycaenidae,Lepidochrysops lotana,Sensitive,52 Butterflies,Hesperiidae,Kedestes barberae subsp. bunta,Not sensitive,10

Author UID is visible in the URL of the relevant person's profile, you can just mouse over the person's name under http://nssl.sanbi.org.za/admin/people.

Have a test and see if it works?

[rukayaj]

Oh dear, no, something a bit wrong there. Let me have another look.

[reupost]

Before I get in and check, it might be best if the CSV is tab delimited. Some plant species have commas embedded in their names, e.g.: Oscularia cremnophila Van Jaarsv., Desmet & A.E.van Wyk Ceratonia oreothauma Hillc., Lewis & Verdc. subsp. somalensis Hillc., Lewis & Verdc. Euphorbia enopla Boiss. var. viridis A.C.White, R.A.Dyer & B.Sloane

[rukayaj]

With the commas in the names thing, shouldn't be a problem if you save the csv in excel? It wraps them in quotes automatically and when I tried with the examples you gave just now it seems to work.

Oh once the file is in the right place you trigger the import by opening up the drupal site as an admin and going Configuration > Development > Run SSP Import module. Maybe test now?

So I remember what I've done here: when I imported a species and set it as sensitive it id work, but it gets immediately overwritten by the script which runs every time a node saves (and resets the sensitivity status). So now what I do is check and see if a node is being saved for the first time AND has one of these 'preliminary' statuses. So people should still be able to add manually as usual and whenever one of the preliminary sensitive species gets edited the checking sensitivity script will overwrite them on save as usual.

[rukayaj]

Ok one problem might be if you want the preliminary sensitive/non sensitive status to persist even after a user has edited + saved the node again.

So imagine the following

• species A is bulk imported and is given a status of sensitive (preliminary prior to an assessment)
• A user clicks edit on species A
• The user hits save without having made any changes
• The script which runs every time a node is saved will kick in and assign the status as the ordinary not sensitive

Could be problematic and confusing?

[rukayaj]

Ok the thing that flags species as sensitive in 2010 has broken now. Maybe don't test because I'm trying to figure out what's wrong there.

[reupost]

Thanks, that was a good thing to consider.

Is it possible to make it so that if the node has one of the 'not assessed' statuses, and is being saved without anything selected on the various criteria (i.e. its just got a group, a family and a taxon entered, and whatever the default values are for the other fields), that they original 'not assessed' status persists?

[rukayaj]

Ok i fixed the other thing now. Errrm yes I suppose so, that would be possible. Should it look at the justification or just the radio button options?

[reupost]

Its getting pretty late on your side so I'm guessing we won't sort all this out today. I'm in a conference tomorrow and Fri, so I don't know how much time I'll get for testing. Maybe tomorrow evening I can have a look.

Selwyn is hoping to get the work packages out to the taxon experts today, so would it be possible to stabilise the site without the bulk-upload stuff working for now? If you can get it sorted then great, but otherwise I'd rather the site is 'up' and we hold back on this tranche of functionality until we can block out a few hours (or however long you think it will take) next week, say.

What do you think?

[reupost]

Just the radio buttons should be fine, I think.

[rukayaj]

The bulk upload stuff does seem to be working now. The only thing is after something's edited after being bulk uploaded it changes the status, so I haven't written the logic to check the radio buttons yet.

But I think it makes sense to get the taxon experts on it asap because I'm 100% sure that as soon as they start properly trying to use the site they're gonna find issues and things they want changing. You know how people always do.

And by the way I'm at a workshop on Monday - Wednesday next week, so I won't be able to do any work on it there.

[reupost]

Ok, i agree. Selwyn is issuing the work packages today so lets see if we get any feedback.

Thanks for all the work so far!

Enjoy the workshop. I'll try not to mail you distracting comments during the first half of the week 🎢

On 16 Nov 2017 07:03, "Rukaya" notifications@github.com wrote:

The bulk upload stuff does seem to be working now. The only thing is after something's edited after being bulk uploaded it changes the status, so I haven't written the logic to check the radio buttons yet.

But I think it makes sense to get the taxon experts on it asap because I'm 100% sure that as soon as they start properly trying to use the site they're gonna find issues and things they want changing. You know how people always do.

And by the way I'm at a workshop on Monday - Wednesday next week, so I won't be able to do any work on it there.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/rukayaj/ssp/issues/65#issuecomment-344834147, or mute the thread https://github.com/notifications/unsubscribe-auth/AEJ8TASsiVLNMfZBlGJGIvnW69jQ8wBSks5s294vgaJpZM4Qb_gl .

[rukayaj]

Ok this one we're waiting to see if we get feedback still right?

[reupost]

Would you mind loading the attached so I can test? Thanks test_bulk_20171127.zip

[rukayaj]

Oh I've uploaded it and run the import, not sure if you wanted to run the import so you can see how it works? You can run it again by going Configuration > Development > Run ssp module and it should just make 2 duplicates. I just uploaded + renamed your file to http://nssl.sanbi.org.za/sites/default/files/species_for_import.csv, and the script just loads anything in that CSV.

[reupost]

Just checking what happens if these are saved and it looks like they go to 'not sensitive', even if nothing is changed. I guess the logic to check the radio buttons is still missing?

[rukayaj]

Yes I haven't added that in. Should I do that? I thought we were going to wait for people to test it to see if it's necessary?

[reupost]

If you've got time maybe add it now (just in case there's no time later, and this functionality feels a little incomplete without it). But if you're busy then it should be fine to tackle when we actually get some names to bulk-load.

[rukayaj]

I'm busy with something else at the mo, if it's alright I'll leave it till we actually get names to bulk upload.

[reupost]

Yes that's fine - there might be some birds in the next couple of weeks but let's see what turns up.