Summary stats not accurate

[MeganLowe]

Hi, there is something a bit odd going on with the summary statistics displayed on the website. So far I've noticed and have had reports of inaccurate numbers for the number of samples on the 'My Sites' and 'All Sites' pages, and the number and quantity of species on the annual summary pages. Here are a couple of examples:

WCBS-BC square TF8039 has been surveyed 11 times (confirmed by recorder and on the data downloads). On 'All Sites' it says it has 15 samples, on the recorders 'My Sites' page it says 14 samples. Why are all of these numbers different, and where is it getting '15' from if there are only 11 samples that have been submitted?

WCBS-BC square J0197. On the annual summary page for 2019 the species count shown on the summary table does not match the data download. The May records are completely wrong on the summary table (i.e. 27 small heath and 8 species) as on the data download only 3 species were seen. As a slight aside the recorder actually claims that they didn't even visit in May 2019, but I'm hoping that there's some human error in that claim rather than the system creating it's own records!

I feel that this is a relatively urgent issue as some (?) users are not being shown accurate information. Normal users can't check the data downloads so I've had a couple of panicky recorders recently who think they have done something wrong and can't work out how to fix it. Thanks, Megan

[DavidRoy]

@MeganLowe can you give the url for the pages you are seeing this problem on

[Gary-van-Breda]

For TF8039 pt1: There is a bug in the All sites report, which is counting deleted walks as well.

[Gary-van-Breda]

TF8039 Pt 2: It looks like there are 3 duplicated walks in 2016 (could be seen on the Annual summary page, Raw data). These appear to be part of a bulk upload of data carried out on 1st Dec 2017, and have no counts associated with them. They had not been "attached" to Chris, so did no appear on their "My walks". I will manually delete these 3, but I'll have to run through all the walks uploaded as part of that process to see if there are any other similar issues.

[Gary-van-Breda]

TF8039 Pt 2: It looks like a substantial amount (if not all) of this download have added duplicated walks.

J0197: It looks like data changes on 15 May 2020 have added some messed up data: there appears to be section level samples from a UKBMS Transect walk bolted into the WCBS-BC walk. The May walk was done by "Scullion, Alex", not "Conway, Rachael". Both are allocated to J0197.

[Gary-van-Breda]

@BirenRathod or @johnvanbreda: can you copy the updated all_sites.xml to the appropriate reports directory on Warehouse1? Thanks.

[BirenRathod]

@Gary-van-Breda I have pulled this report on live warehouse now.

[Gary-van-Breda]

@MeganLowe : TF8039 pt1 - counts on the "My Sites" and "All Sites" pages now match (13 as of just now). Other issues still "work in progress".

[Gary-van-Breda]

@MeganLowe : TF8039 Pt 2: of the 1044ish WCBS-BC walks that were loaded on the 1st Dec 2017 for the Year 2016, 539 appear to be duplicates of data input manually. What do you want to do with this data? (of note that the other square mentioned, J0197, has 2 duplicated walks from this data load).

[Gary-van-Breda]

@MeganLowe : data for J0197 sorted.

[MeganLowe]

Hi @Gary-van-Breda, please can the ‘admin’ duplicates loaded on 1st Dec 2017 be removed – i.e. where there are duplicates, keep the samples connected to a user account and remove the sample loaded by admin? Thanks, Megan

[Gary-van-Breda]

Admin duplicates removed for this upload. cache tables updated, and summary builder (Annual summary) triggered to recalculate its data. @MeganLowe : is there anything else you need done for this issue?

UKBMS Issue 238.sql.txt

[IanMiddlebrook]

Thanks @Gary-van-Breda

@MeganLowe has a new role at BC now and is no longer involved with UKBMS, so you can tag myself and/or @RachaelConway for any other outstanding issues that she raised.

Best wishes, Ian