Warehouse ID creation and potential issues - Brazil

[JimChiazzese1]

@BirenRathod @kazlauskis @kitenetter @CCarvell @andrewvanbreda

This maybe a question for Andy Van Breda - Please can you include him, I cannot find him on GitHub. Who is responsible for adding new species/pollinator groups to the warehouse.

Brazil's Fit Count App will have 4 new species to add to their Fit Count, however there are two other pollinator groups that also may be new entries in the warehouse.

"bumblebee sensu lato", to include both bumblebees (genus Bombus) and carpenter bees (genus Xylocopa), since both are recognized by the common name of "bumblebees" in Brazil. ("bumblebees" from UK comprise only Bombus species). Should this be a new warehouse entry?

Brazil want to use a single entry of 'flies' to represent 'hoverflies' and 'other flies', as they thought it would be difficult for users to discriminate between these two groups. Should this be considered a new warehouse entry, or just use 'other flies' and replace the text accordingly in the translation only.

Thanks Jim

[kitenetter]

As far as I'm aware, both the UK and Cyprus implementations of the app use the same species list. In the warehouse this list is called "Insect groups (PoMS)" and has ID 221.

I think @kazlauskis will have to let us know what is needed to enable a different list for Brazil. I guess the options are:

1. Set up a completely new species list in the warehouse that the app uses when set to Brazil.
2. Extend the existing species list so that it contains all taxa for all countries, and introduce filtering within the app so that the appropriate taxa are displayed for the country chosen.

[JimChiazzese1]

Thanks. (2) Is what we're doing currently, I believe - and the filtering within the app works well. For me personally, it's more understanding the implications for how adding new entries to the species list might impact the data usability if a country is adding a new species that's more to manage local user expectations, than representing an actual new species.

[andrewvanbreda]

Hi @JimChiazzese1 @kazlauskis @kitenetter

Sorry for the slightly slow response.

As it is Karolis that is dealing with the coding on the app, it is probably best if comments first on how he might like to proceed. Then I will comment on whether I think that will work with the website, or if I can foresee problems.

However, here are some thoughts on if the existing list were to be extended...

• It might get messy with all the filtering. The website uses the list too, and I think the new SPRING project too. On the websites it is not hard to filter, but it could be messy with so many different sites, apps, languages using that list and they all have to filter out Brazilian entries.

• The existing list is a series of common names already, and the suggestion is to have new entires with the same common name. So that needs consideration.

• The new "Flies" entry would probably be best if it were a parent of the existing 'hoverflies', 'other flies' entries. We can't just move those without fiddling with the database.

If it were to be a new list, the system may need to know the equivalent entries in the two lists have the same meaning if the data is ever to be compared for reporting.

[DavidRoy]

Is it possible to use taxon attributes to tag with countries? We do something similar for the ebms butterfly species list

[kazlauskis]

Hi all, from the app perspective we can go with both options.

At the moment, we use a single long list (option 2) with custom filtering in the app. This helps to share a species across multiple countries (keeps the same warehouse ID and taxon attributes between countries). But like Andy says it can get messy in reports and websites which might need to implement some sort of per-country filtering.

We could also go with individual lists for countries (option 1). I would slightly have to tweak the app to accept different species warehouse IDs for different countries but that's about it.

I am happy with both options. If you see this project growing as per-country then option 1 might be better, otherwise if you want to keep it more centralised then option 2.

[DavidRoy]

I would favour a single list with taxon attributes as there is a lot of overlap in lists per country

[CCarvell]

Thanks all for your inputs. I am happy to go with @DavidRoy 's recommendation (option 2) of a single list and use lessons learned from the ebms. But its important to remember that with FIT Counts we are in most cases talking about groups of species (ie. insect groups) rather than single species. I guess we also need to design a system that is resilient and transparent for others eg. if a different developer or web designer needed to take on part of this within SPRING or one of the S American countries (I am not suggesting that will happen, just covering possibilities). Having a single list also helps streamline choices of groups across countries where possible, which I think is desirable to maximise benefits of this as a global survey activity.

[andrewvanbreda]

Hi @BirenRathod @kazlauskis @kitenetter @CCarvell,

My thoughts are....

1. Presumably there is existing code that can filter a website based on an attribute. I have not seen that. Otherwise that will need to be developed. This is a nice idea though in theory, it could mean the website/app would just automatically show the correct species, and pick up new changes for a country automatically without the developer touching it.

I do have a thought though. What would the attribute actually store? If it stored the ID of the countries it could be used by, then if there lots of countries, were would could end up with a mess of IDs associated with each taxon.

Perhaps a similar kind of idea to Plant Portal uses to group locations. So a list of several taxa could be grouped together and given a name, and that group can be assigned to different counties. This might avoid the need for each taxon to be assigned to individual countries. (So the taxon is assigned a grouping and that group is assigned to countries (noting, there is nothing stoping this group only being 1 long, or a taxon being assigned to more than 1 group))

Perhaps I am getting a little ahead of myself with detail here.

2. Note previously what I said about the flies entry.

3. What Claire is saying there about potential confusion with the names is correct. For instance there would be more than one entry in Brazil saying "bumblebees" under the new model. Probably the most elegant way to deal with this is to have a common name entry for all these groups for each country defined in the Warehouse. Then the actual taxon name would be the description you want to give that group, but it would never actually be shown on any of the sites. So for instance, the name in the Warehouse could be changed to "bumblebees (genus Bombus)" but then this is never used on the websites themselves, which would use the defined common name for each country.

[CCarvell]

@JimChiazzese1 are you happy that we have just about resolved this in order to give a clear answer to our SURPASS partners in the meeting tomorrow? @andrewvanbreda 's point 3 above might not quite match my interpretation. There should only be one entry for Brazil that includes bumblebees. For Brazil, we need to have the group "bumblebees (genus Bombus)" turned off, and their suggested new group "bumblebee sensu lato", to include both bumblebees (genus Bombus) and carpenter bees (genus Xylocopa), turned on. We should discuss the merits of the precise "common name" that is used here, since I personally am not convinced about the term "sensu lato" for this audience (how about "bumblebees and carpenter bees"?), but Sheina and co. are best to advise on that.

[andrewvanbreda]

@CCarvell Ok that is fine if the intention is to have both entries under the one name, I did indeed misunderstand the intention with that. (What I said was true though if it were to be two separate entries).

[JimChiazzese1]

@andrewvanbreda @kazlauskis I think now that we're settled on the Brazil use of Flies and bumblebees, it's now a case of getting their species into the warehouseID so we can create the beta. There are 3 new habitat types, 16 flower types and 7 insect species groups. Please can you let me know what's needed from my/our side to move forward. Thanks Jim

[kazlauskis]

You should add the new options to the warehouse now but I am not sure who is responsible for this. From my perspective I just need the new warehouse IDs of these options in the spreadsheet

[DavidRoy]

@JimBacon can you do the warehouse setup please

[JimBacon]

Among the new 'insects' are Bats and Hummingbirds. When I add them I have to assign them to a taxonomic group. There are existing taxonomic groups beginning 'Pollinators - ...' but none of the existing groups would be correct. Shall I create 'Pollinators - vertebrates' and put both in that group or would you prefer some other grouping?

[DavidRoy]

Good question. I think 'Pollinators - vertebrates' is the right option. In reality, I don't think the taxon groups are very important for a small list such as this.

[JimBacon]

This is done and the spreadsheet has been updated @kazlauskis

[andrewvanbreda]

@JimBacon @kazlauskis @kitenetter @DavidRoy Sorry busy, so not following this thread in detail, but just poked my nose in. I see the following are appearing suddenly on UKPoMS Live, is that a correct situation?

Bumblebees sensu lato Don't know Flies Hummingbirds Orchid bees Stingless bees

[kitenetter]

Thanks for flagging this @andrewvanbreda - these new terms should definitely NOT be appearing on UK PoMS.

Does this mean we've got to do some extra filtering?

[JimBacon]

Perhaps if I temporarily disable them for data entry they will disappear until the website has been fixed. I'll just give that a try.

[DavidRoy]

Thanks @JimBacon and sorry we forgot this complication!

[JimBacon]

The species grid didn't seem to respect the 'Allow data entry' flag so I edited the configuration of the recording form by adding the following. I'll have a quick browse to see if there are other pages besides https://ukpoms.org.uk/enter-fit-count which have been affected.

[JimBacon]

Have to say, I'm not clear why the habitat and flower species lists have not acquired a Brazilian influence.

[kitenetter]

The other one I would expect to be affected is https://ukpoms.org.uk/1km-fit-count but it looks like that is already displaying the correct list.

[kitenetter]

I think @andrewvanbreda has been adding suitable filtering for the other elements, and we need to ensure that none of this is going to conflict with his approaches to filtering elsewhere.

[BirenRathod]

@kitenetter Did you check after clearing cookies or with nocache page?

[andrewvanbreda]

@JimBacon @kitenetter @BirenRathod The termlists have code in them to always disallow any new terms that were not originally to be part of the UK site. It does this with jQuery, and this was implemented after Karolis said there were going to be some app only terms

1km was showing the Brazilian species, I have added Jim's fix to that page.

The Welsh version of the Public FIT Count page I don't think needs the change as it is designed to only show Welsh.

[kitenetter]

I can't see any additional actions required - closing.