Retrieval of Data

[IanMiddlebrook]

Hi @DavidRoy

Not sure who can deal with this one.

A Branch Co-ordinator has managed to delete all the data for a long-running transect site. He was adjusting the transect route by splitting it into sections (it previously only had one section). In the process, he actually deleted the existing route/section, and this has deleted all the data associated with that site (as they were all on section 1). The walks are still there, but no counts.

Is it possible to retrieve these data and assign them to the new 'Section 1' for that site?

The site involved is Sun Lane Nature Reserve (site_id=3356).

Thanks, Ian

[Gary-van-Breda]

@IanMiddlebrook : Can you confirm whether or not there was count data on the old data when this issue occurred?

It looks like there was a data upload task which uploaded 361 walks for location_id 3356 from 2003 to 2015, done in 2020 and 2021. I can find no evidence in the warehouse of any section subsamples (or occurrences on them) uploaded at the same time as the walks, prior to the section being deleted - all counts and subsection samples have been created (uploaded?) after this issue happened. (I have checked for any section level samples falling within the transect, allowing a buffer around it, whether deleted or not).

There are at the time of writing some 671 subsamples created on walks on location 3356. It looks like only 1 of these (sample id 14196231, child of walk 14196230) has been disassociated from a section - which implies it was attached to the deleted section 1. The creation timing etc match up with the date this mix up with section 1 happened (2021-05-24). However both 14196231 and its walk have been deleted, and there is no other walk on that day (2021-04-18) to bolt it into.

If we can confirm there was no count data on the imported walks at the time, then I think there is nothing more to do on this issue.

[IanMiddlebrook]

Hi @Gary-van-Breda There would/should have been 1989 occurrences/records for the period 2003-2015 uploaded for this transect alongside the 361 walks/samples for that period. All the records were associated with section 1, which is why they disappeared when that section was deleted. I know there were some problems with these historic uploads, but I can't think that no records for this particular site would have been uploaded over that period.

The attached file shows all those records from the master UKBMS Oracle database. Sun.csv

[Gary-van-Breda]

@IanMiddlebrook The trouble is that there are no records in the warehouse at all associated with the counts/sections upload for this period on this site, not even deleted ones, except for 6 random walks (10157407, 2003-04-23; 10154500, 2003-05-15; 10154515, 2003-10-16; 10347011, 2004-03-31; 11467276, 2010-04-17; 13707658, 2014-05-17) that have section level samples, but no occurrences. It looks to me like someone has gone in and edited these samples from the front end and saved it, which then creates the sub-samples (4 of the six where done on 2021-11-19). There are also no "orphaned" samples with Sun Lane or its Code (or the associated sections) in the location name, which would happen if the upload took place but did not link in properly.

I don't think the counts upload has taken place at all for this site - is there any way we could check if it did?

[IanMiddlebrook]

Hi @Gary-van-Breda, it's quite possible that the counts were never uploaded for this site, though I'm not aware of any other transect where all historic walks have been uploaded without a single record.

I'm not directly involved with the upload of historic data, which is still incomplete and, to my knowledge, has not progressed for a couple of years. @DavidRoy would be better placed to answer this.

My recollection at the time (over two years ago) was that the co-ordinator had seen the historic data online before he played around with the route, which is why it was raised as an issue.

I suspect the specific problems with Sun Lane will just need to be addressed along with the other outstanding historic data, rather than individually.

Thanks, Ian

[Gary-van-Breda]

@IanMiddlebrook : In which case, I think I'll bounce this to @DavidRoy to check the status of the upload.

[DavidRoy]

I'll look into this. The upload of historic data was never properly resourced so I know there are gaps