Disappearing Wood Whites

[IanMiddlebrook]

Report from recorder cazmowe: I walked the Bury Ditches transect yesterday, and went online today to record my sightings. I just realised that I'd entered it on the wrong day, so went back into the site to correct it. In so doing, I noticed that I had somehow missed entering the Wood Whites I had seen....or so I thought. So I put the data in and checked that the date was now showing correctly as the 3rd (it was) and went back into the listings to make sure the new Wood White entries were there...gone! I have since tried twice more, and every time it takes the data, but when you come out of that page and then go back in to it, the Wood White info is gone. Everything else is still there, just no WWs! What's going on? Help, please.

So the Wood Whites are showing up on the Annual Summary page and are listed in the downloads (3 times), but when you look at the walk data entry page they are missing: http://www.ukbms.org/mydata/input-data?sample_id=2398119

[IanMiddlebrook]

Bill Downey is having similar problems with Wood Whites recorded at Chiddingfold West & South.

Seems to be a fundamental problem.

Could this please be assigned to someone?

Thanks,

Ian

[Gary-van-Breda]

The taxon for wood white was updated in Oct 2016, which changed its meaning id. The taxon cache has not been updated, and still refers to the old meaning id. This means there is a mismatch between the meaning ids used to identify each row (which come from the cache - wrong) and the data itself (which comes from the occurrences - right). There is a similar issue with the cache_occurrences_functional for this taxon, but that may come out in the wash when the taxon cache is fixed.

This feels similar to the Mother Shipton issue, but is different.

[Gary-van-Breda]

Email from John:

Hi Gary There is nothing in the immediate pipeline that will fix this. John

I propose to modify the cache_builder taxa_taxon_lists and taxon_searchterms get_missing_items_querys to include a check on whether the updated_on field is after the cache date, and to include an interface for admins on the warehouse to reset the variables that switch off the get_missing_items_query once the initial data build is complete.

This will take approx 1 day.

[DavidRoy]

Please go ahead and fix as this is a critical bug. I'll contacting you by email to discuss details of funding this support time

[DavidRoy]

@IanMiddlebrook - please advice the recorders that the data is being captured but is missing from some of the reports that are built from a summary table. Hopefully this will prevent duplicate entries.

The code to build the summary table will be fixed/updated in due course

[IanMiddlebrook]

Have done

[cid:image001.jpg@01D2CFBD.477F0140]

Ian Middlebrook Butterfly Monitoring Co-ordinator

Email: imiddlebrook@butterfly-conservation.org Direct line: 01929 406032

Butterfly Conservation, Manor Yard, East Lulworth, Wareham, Dorset BH20 5QP Tel: 01929 400209

Company limited by guarantee, registered in England (2206468). Charity registered in England and Wales (254937) and in Scotland (SCO39268).

www.butterfly-conservation.orghttp://www.butterfly-conservation.org/

[cid:image002.png@01D2CFBD.477F0140]https://www.facebook.com/savebutterflies/

[cid:image003.jpg@01D2CFBD.477F0140]https://twitter.com/savebutterflies

Follow me on Twitter @UKBMSLive

The views and comments expressed in this email do not necessarily express the views of Butterfly Conservation. Our promise to you: We will never swap, sell or rent your details to anyone and we will always follow the strict code of conduct set out by the Fundraising Regulator.

From: DavidRoy [mailto:notifications@github.com] Sent: 18 May 2017 09:56 To: BiologicalRecordsCentre/UKBMS-online Cc: Ian Middlebrook; Mention Subject: Re: [BiologicalRecordsCentre/UKBMS-online] Disappearing Wood Whites (#32)

@IanMiddlebrookhttps://github.com/ianmiddlebrook - please advice the recorders that the data is being captured but is missing from some of the reports that are built from a summary table. Hopefully this will prevent duplicate entries.

The code to build the summary table will be fixed/updated in due course

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/BiologicalRecordsCentre/UKBMS-online/issues/32#issuecomment-302343548, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AXjjkU4a0yTUXboAf47CwZcxzc50-ah8ks5r7AeVgaJpZM4NRzVh.

---

This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com

---

[Gary-van-Breda]

I've dropped some code into the warehouse, ready for review by John, as he wanted to see it due to areas it touches on.

[IanMiddlebrook]

Did this critical bug ever get progressed? I guess not as the problem still seems to be there.

Message just received from one of our branch co-ordinators:

"To let you know that the Wood Whites are still not displaying in the Data Entry Screens on UKBMS. Although what I think are incorrect numbers are still showing in the Annual Summary. I reported this through in April/May and there is still the same problem with the second brood."

[Gary-van-Breda]

Just some information to track for the future.

The data entry page uses uncache taxon information when looking up existing data, but uses the cached taxon data when generating the grid. Thus the taxon meaning ids are out of sync as the cache has not been updated. These is code in progress to allow the taxon cache to update automatically: see warehouse pull request 250. I have "kicked" this to get it moving again. The summary builder (annual summary) uses the cache to generate its summary fields, so these are in sync with the report on the annual summary page, which also uses the cache to build its grid. When a change occurs in the cached taxa, this is automatically picked up by the summary builder, and the data rebuilt so that they match.

For Wood White: taxon TTL ID = 238039; preferred TTL ID = 238038; old meaning id (in cache and summary_occurrences) = 127651; new meaning id = 393298

When warehouse pull request 250 is deployed we'll need to instigate a full catch-up for the cache builder in order to sweep up the missed taxon changes.

[Gary-van-Breda]

Cache Builder sweep up carried out last night. Wood White appearing in both DE and AS screens. The initial Bury Ditches transect mentioned above has 4 WW occurrences per section (different occurrences, same value), so is giving the wrong figure in the AS. These will need to be removed. Note as well there seems to be 2 transects on Bury Ditches on 04-05-2017, walked by different people, which is complicating the AS figures further.

[IanMiddlebrook]

Thanks Gary, I've tidied up the data for Bury Ditches, and a few other duplicates entries in Surrey. However, for some sites (eg. three Chiddingfold Forest sites) the Wood Whites now appear in Data Entry and Annual Summary tables, but have disappeared from the Raw Data tables. In contrast, the Wood Whites at Bury Ditches appear in all three locations?

[Gary-van-Breda]

I've just rechecked these, and I can see the wood whites for the 3 chiddingfold forest sites in the raw data as well. I wonder if this was a caching issue?

[IanMiddlebrook]

Yes, they all look OK now, it may well have been a caching issue. I'll close this one.

Thanks