Pantrap data entry - additional plant names

[DavidRoy]

@andrewvanbreda we need to be able to enter plant names to the pan trap entry form, for species not on the UKSI list. The immediate example is: Carduus acanthoides https://pollinator-monitoring.net/enter-pan-trap-results-fixed-square-survey

Is the best way to handle this in the short term to create a second species list for 'additional plants'. Can the form then use both lists for data entry?

Alternatively, we could create a new list for SPRING/EUPoMS that takes the UKSI list and then we add to it?

[DavidRoy]

Centaurea stoebe also needs adding

[andrewvanbreda]

Hi @DavidRoy As these are additional species, but not the main species list, I think we need to create a second grid which points to a new list. Otherwise it will look odd as these additional species will appear above the existing box (even though they might not be intended to be an important/dominant part of the page).

This would be easy to do if it was initially setup like that, however we have the complication that we have to take into account displaying existing data. I will need to test that and get back to you. Can you send me the list of new species please, and any info like Common Names you want to use to. Also preferably include the keys so the Warehouse system can identify it is the same species in different lists on the Warehouse.

[andrewvanbreda]

Hi @DavidRoy I have tested this in practice on my machine now. There are two possibilities

1. This can work as two grids, but there is a problem where the old occurrences will not show when editing existing data. If a database hack is to be avoided, the way to deal with this would be to leave the current Pan Trap form on the system, but not in the menu, so that editing rows can be routed to that form still. Then create a new Pan Trap form with the new species list differences, this latter form would appear in the menu system.

2. The other possibility is that if it is a small fixed list of species, it could be made to be just part of the existing species grid. These would appear as a fixed list. The only problem with that (as mentioned in my previous comment) is that we normally do that for commonly used/important species, so might be odd if the species are not to be used very often are always appearing in the list in that way. Having said, if we use a free text for the new species list, if there are only one or two species it might make searching problematic unless you know what is available.

I other approach which I don't think can work in this case is to have a Child Species list of the UK Master List. But I don't think it is practical to add all the flowering plant UK Master List species into a child list.

Anyway, let me know your thoughts when you are able, in particular what size you think the list might grow to, and how often those species will be used.

[DavidRoy]

@andrewvanbreda sorry for the delay in replying to this one. I think we should go with a new 'European' list, starting with a clone of the UKSI list for taxon_group 'flowering plants'

The editing can point back to the old form as you suggest.

Two additions for the new list are: Carduus acanthoides - https://en.wikipedia.org/wiki/Carduus_acanthoides Centaurea stoebe - https://en.wikipedia.org/wiki/Centaurea_stoebe

There will be other additions needed and common names in other languages in due course. I will raise new issues for these

[andrewvanbreda]

Hi @DavidRoy,

OK noted. As some thought will be required, I won't comment further at this precise second as on another task, but will get back to you.

[andrewvanbreda]

Hi @johnvanbreda Are you able to give me your opinion on the best approach to this before I spend significant time on something that might be wrong approach. We need a a list of the "flowering plants" taxon group from the UK Master List, then we will add some new species too. Am I correct in thinking it would be necessary to generate this with SQL? or perhaps I could add a new option to the Warehouse? (where there is a new option where Child Species Lists can have their species created from an entire taxon group instead of 1 at a time?), if you think that would be useful generally?

[johnvanbreda]

@andrewvanbreda is this for a form where you enter a list of species into an initially empty grid? If the species in the additional list are also in the taxon group "flowering plants", then you can probably do something like this (not tested):

[species]
@extraParams=<!--{
  "taxon_list_id": [15,x],
  "taxon_group_id": 89
}-->

replacing x with your additional species list's ID. Then you only need 1 grid.

[andrewvanbreda]

@johnvanbreda Yes it is for an empty grid. Thanks for this idea. I will confirm that I can get this options sturcture to work when I experiment with it, as have not seen this combination of options in this way before. I presume they are flowering plants because it is a grid that uses flower attributes for all the data in the columns. I will check your suggestion first, then confirm with David. What you are saying there would certainly make it easier and on first galnce looks ideal.

[DavidRoy]

@andrewvanbreda that sounds promising. The additional list is for flowing plants

[andrewvanbreda]

Hi @johnvanbreda I have experimented with this, I believe it can work if one thing can be fixed.

Firstly to note, I had to add a @lookupListId( or continue to have a species list selected in drop-down) option to stop the page moaning this option is needed, even though in reality the option seems to gete completely overriden by extraParams.

So the main problem I observed is this. The taxon_list_id needs to be a single integer. It seems to not work with an array. It doesn't complain, it just interprets this as return nothing. Do you think this would be a hard fix if I tried, or do you have alternate suggestion?

Also I would be interested to know why the extraParams have the "" comment style syntax around the options. I initially assumed these needed removing, but can see actually they are required for it to work.

[DavidRoy]

@andrewvanbreda is this option working? We have additional plants that need to be added

[andrewvanbreda]

Hi @DavidRoy Awaiting feedback from John on above comment I made. So the option is not working yet. I will ask John directly tomorrow if this is urgent.

[DavidRoy]

Please do, it's holding up data entry

[johnvanbreda]

@andrewvanbreda having experimented the parameter needs to be a string containing the array of integers, not an actual array, so it formats in the request URL correctly:

[species]
@extraParams=<!--{
  "taxon_list_id": "[15,x]",
  "taxon_group_id": 89
}-->

BTW the JSON is parsed using a stricter parser, so it expects "" around the property names. In JS objects you can skip the double quotes as the parser is less strict.

I think your observation about the lookupListId being required, but then overwritten, is as I'd expect.

[andrewvanbreda]

@johnvanbreda Thanks for the info.

[andrewvanbreda]

Hi @DavidRoy This is now working.

The only extra items are the moment are Carduus acanthoides And Centaurea stoebe

You can either add extra species to the following species list http://warehouse1.indicia.org.uk/index.php/taxon_list/edit/278 or instruct me on what needs adding.

If you have any concerns about particular issues, you can add Priority High tag in Github, and I was look at ASAP. Currently working on the adding the verification info to a species grid as requested by Jordy.

[LisaCabiddu]

Hi everyone, I'm Lisa Cabiddu from CREA, Italy. I'm working on a simple list of plant for SPRING sites in Italy. The list is only scientific name in Excel, is it suitable in this format?

Or have I to divide in the section like in SPRING site (common name, scientific name and floral unit)?

For the moment I have about 300 species, but I think they will increase.

many thanks

[DavidRoy]

@andrewvanbreda please advise

[andrewvanbreda]

Hi @LisaCabiddu @DavidRoy

Yes this is fine, I can import just the scientific name, it just means when you search for the species, that is the only name you can search by.

Excel is fine, I can convert it into the required format.

You can either try to attach the species to this Github issue by dragging and dropping onto a post, or the file can be emailed to. I am not sure if Github supports Excel files, so you might need to email it.

support@avb-it.co.uk

The only thing to note is that other users who are not Italian can select the new species also as they are not limited by country, but this shouldn't matter as they won't want to use them anyway.

Hope that helps

Andy

[LisaCabiddu]

Hi @andrewvanbreda, thank you for the replay.

I try to attach file here, tell me if it's working fine.

I think most of the species are usefull for other Mediterranean countries.

Additional plant species for SPRING - Italy.xlsx

[andrewvanbreda]

Hi @LisaCabiddu,

Yes this looks like the right kind of thing. I will let you know once done, or if I have further questions.

Andy

[LisaCabiddu]

Great, surely I'll have more plants to add. I make another file with all the addictional plant.

Lisa

[andrewvanbreda]

Hi @LisaCabiddu @DavidRoy

I have imported these species and they are live.

Only one slight problem I wish had realised before the import, that actually quite a few of the species were already appearing.

This means when you do a search, the duplicates will appear twice.

I guess in practice this shouldn't matter too much (unless you think otherwise), with so many variations, often species searches on websites return more than one match, although I guess it would be preferable not to have the duplicates.

Having said that, it may of taken a fair amount of effort to get rid of them, which may or may not have been worth it.

[JimBacon]

Do duplicate taxa have the same taxon_meaning_id and point to a single preferred_taxa_taxon_list_id?

If not, is that going to make reporting against these taxa much harder?

[DavidRoy]

@andrewvanbreda we need to remove the duplicates. Can you do this before the new taxon are used for data entry

[andrewvanbreda]

@DavidRoy @JimBacon No they won't have the same taxon_meaning_id. I can't remove them before data entry. The reporting would be ok if the new ones included external key, but they don't. Don't panic. What i can do tomorrow is write an update statement to remove duplicates not in use (which will hopefully be all). If any have been used, these occurrences can be reported, then remove those remaining ones. So it is undoable. It probably isn't much more difficult than it would of been to remove the duplicates in the first place. I will note this in future though that these additional lists are not necessarily just additional.

[andrewvanbreda]

Hi @BirenRathod,

Could you try running these please, hopefully give me better idea of what is going on.

Thanks

select count(*) from indicia.taxa_taxon_lists ttl join indicia.taxa t on t.id = ttl.taxon_id and t.deleted=false join indicia.taxa t2 on t2.taxon = t.taxon and t2.taxon_group_id = 89 and t2.deleted=false join indicia.taxa_taxon_lists ttl2 on ttl2.taxon_id = t2.id AND ttl2.taxon_list_id = 15 and ttl2.deleted=false where ttl.taxon_list_id = 278 AND ttl.deleted=false;

select count(*) from indicia.taxa_taxon_lists ttl join indicia.taxa t on t.id = ttl.taxon_id and t.deleted=false join indicia.taxa t2 on t2.taxon = t.taxon and t2.taxon_group_id = 89 and t2.deleted=false join indicia.taxa_taxon_lists ttl2 on ttl2.taxon_id = t2.id AND ttl2.taxon_list_id = 15 and ttl2.deleted=false join indicia.occurrences o on o.taxa_taxon_list_id = ttl.id and o.deleted=false where ttl.taxon_list_id = 278 AND ttl.deleted=false;

select count(*) from indicia.taxa_taxon_lists ttl join indicia.taxa t on t.id = ttl.taxon_id and t.deleted=false where ttl.taxon_list_id = 278 AND ttl.deleted=false AND ttl.id NOT in (select ttl2.id from indicia.taxa_taxon_lists ttl2 join indicia.taxa t2 on t2.id = ttl2.taxon_id and t2.taxon_group_id = 89 and t2.deleted=false where ttl2.taxon_list_id = 15 and ttl2.deleted=false);

[BirenRathod]

@andrewvanbreda

Here are the results in the same order as you posted queries above. 437 0 415

[andrewvanbreda]

Hi @BirenRathod

Ok that is great thanks.

Actually I will email you some further statements otherwise this thread is just going to be full of SQL.

[andrewvanbreda]

@DavidRoy @JimBacon @LisaCabiddu @BirenRathod This situation has now been corrected, and the list now only includes the non duplicates. No-one has used the duplicate entries. Be aware that some entries such as Bellardia trixago have been removed as a duplicate even though they won't appear in a search currently. This is because in the UK Master List that is a non-preferred name, and the form is currently setup for preferred or common name searching. The preferred name for that species is Bartsia trixago. We can change that if needed, at the moment I think it is designed to mirror UKPoMS functionality, let me know. Thanks Biren for running the tests and final SQL.

[iverkaik]

Hello,

According to some volunteers I have few plants (see list below) that needs to be added to the list.

Thank you, iraima

Plants missing according to volunteers from the Atlantic-Mediterranean Region:

Carduus defloratus Peucedanum cervaria Seseli montanum Teucrium montanum Thymus praecox Vincetoxicum hirundinaria Tolpis umbellata Petrosedum sediforme Daphne gnidium Fumana ericoides Blackstonia acuminata Helycrisum Teucrium dunense Bistorta officinalis Cervaria rivini

[andrewvanbreda]

Hi @iverkaik OK, I will deal with this and let you know.

[andrewvanbreda]

Hi @iverkaik Done and live.

[iverkaik]

Thank you @andrewvanbreda :)

[LisaCabiddu]

Hi everyone!

I have a short list of new plants to add to the list:

Clinopodium nepeta Dianthus barbisii Prospero autumnalis Drimia marittima Helminthotheca (all genera) Boerhavia coccinea

Thank you very much

Lisa

[andrewvanbreda]

Hi @LisaCabiddu I have added these new entries which are now available

Clinopodium nepeta Dianthus barbisii Prospero autumnalis Drimia marittima Helminthotheca sp. Boerhavia coccinea

[LisaCabiddu]

Thank you @andrewvanbreda

[iverkaik]

Hi,

There is a volunteer who is experiencing some problems to complete the flower list. The have tried a couple of times and is stops as they go to the third record. Here the screenshot:

Any suggestion what is happening? They have tried it several times in different days.

Thank you, iraima

[DavidRoy]

@andrewvanbreda can you investigate please

[andrewvanbreda]

@DavidRoy Yes I will look today, I have no idea what would cause this off the top of my head.

[andrewvanbreda]

Hi @iverkaik @DavidRoy I cannot reproduce this. Although am noticing the site is not showing in English there when I am not aware of an translation work being carried out on pollinator-monitoring.net, are those trenaslations being done by Windows/Browser as I don't think the site has a language selector? Does it behave the same way in English? I am wondering if some of the html code the site relies on to work is having words translated and it is confusing it........that is a very wild guess though....although if it was that I wouldn't expect the second row to work.

[BirenRathod]

@andrewvanbreda you're right, it needs to enable the language module first and facilitate Drupal to integrate according to the browser's default language.

[iverkaik]

thank you @andrewvanbreda @BirenRathod, I passed the information and will let you know if worked!

[iverkaik]

Yes! it worked good in English, thank you :)

[andrewvanbreda]

@iverkaik That is great :) Thanks for letting me know.

[iverkaik]

Hello @andrewvanbreda,

According to some volunteers of the Mediterranean-Atlantic Region there are some specimens missing in the lists of bees and plants.

The bumblebee: Bombus xanthopus

And the next plants: Asparagus aphyllus Carlina hispanica Cistus crispus Dorycnopsis gerardi Gomphocarpus sp. Leontodon taraxacoides Phytolacca heterotepala Silene colorata Arisarum simorrhinum Raphanus raphanistrum Knautia integrifolia

Can these be added? Thank you, iraima

[andrewvanbreda]

Hi @DavidRoy @iverkaik I added the plant names, with exception of Raphanus raphanistrum which is already present for selection on the form.

I didn't add the bee yet, as I wasn't sure about it. Isn't it a synonym of Bombus terrestris? Bombus terrestris is present, although currently the form is not setup to show synonym names, nor is the synonym entered on the species list either. So am not sure whether to add it as a synonym (if I am right about that) then change the way the form displays names, or whether you want it as a separate entry.....although then there would be no indication they are same species.

[DavidRoy]

@iverkaik can you advise?

[iverkaik]

Hi @andrewvanbreda This report is from the volunteer in Corsica and Bombus xanthopus is considered an endemic species.

Here below information copied from a recent article https://www.researchgate.net/publication/369973774_Bombus_terrestris_terrestris_Linnaeus_1758_and_hybrids_with_the_endemic_Bombus_xanthopus_spotted_on_Capraia_Island_Tuscan_Archipelago_Italy_some_conservation_management_implications [accessed Aug 30 2023]

"Bombus xanthopus Kriechbaumer, 1870, present in Corsica, Elbaand Capraia Islands and considered as a subspecies of B. terrestris until 2015 (Rasmont et al., 2008), has been elevated to the status of endemic Corsican species dueto its molecular and eco-chemical features (Lecocq et al.2015; Rasmont et al., 2021)."

thank you, iraima

[DavidRoy]

Thanks. @andrewvanbreda please add as a full species to enable data entry on this basis