Check we have full coverage of Scotland including Islands

[quantoidb]

Check that we have full coverage of Scotland including Islands up to Shetland. Follow up with JNCC if not.

[geojamesc]

plotting the extents of metadata from the images in pgadmin I got the following which seems to indicate a lack of images for the far north east of scotland and shetland. Will double-check that I`ve not missed anything and check against the list of datasets that I think JNCC sent indicating which images they supplied

[spatial-scales]

@StephenRESAS would you be able to chase JNCC as to why there is no ARD for the areas in the figure above and get them to provide some ASAP? We will need these data to proceed with the project.

[geojamesc]

Yep. I'm looking into it. From checking my inbox, Daniel Colson sent an email to all of us on 14th May indicating that some data was still being processed. Checking on the JNCC web space today there are files with 15th May timestamps. So I`m assuming that this is data that has now been processed and we just need to pull down to our local store on the VM the files that have been made available since we first did the pull down of the data. I will check though this week and contact Daniel for clarification etc.

[geojamesc]

I pulled down from the JNCC webspace the 2018 data that is now available to my local drive. Just comparing the sizes (G=Gb) of the month sub-folders in the 2018 data pulled down from the JNCC webspace, there is a lot more data now than there was a month ago.

sub-folder downloaded-mid-may downloaded-yesterday 01 67G 135G 02 57G 106G 03 96G 145G 04 38G 68G 05 38G 77G 06 19G 48G 07 29G 48G 08 19G 48G 09 29G 48G

So presumably this means temporal resolution has been increased.

Again though when I checked the extents of the data, the geographic coverages are as before, i.e. no images for NE Scotland.

[quantoidb]

Chrissy requested data for the remainder of Scotland from JNCC. Heard back that The Borders was the only data JNCC was expecting to provide. Stephen discussing further with his counterpart.

[quantoidb]

Kostas, While Richard is on vacation 5-12 July, Please provide regular updates on what JNCC is providing to us. Thanks!

[geojamesc]

Based on the S1 radar images that have been through the RFI masking script and where RFI artefacts have been removed using the regions digitised and provided as shapefiles, S1 data extents of the masked S1 image data seem to be like so which seems much more comprehensive:

[quantoidb]

Looking good 👍

On Wed, 10 Jul 2019 at 15:33, geojamesc notifications@github.com wrote:

Based on the S1 radar images that have been through the RFI masking script and where RFI artefacts have been removed using the regions digitised and provided as shapefiles, S1 data extents of the masked S1 image data seem to be like so which seems much more comprehensive:

[image: data_extents_100719] https://user-images.githubusercontent.com/11922702/60977717-db4fc180-a327-11e9-9950-10611839108c.png

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/cropmapteam/Scotland-crop-map/issues/27?email_source=notifications&email_token=ACOMQKRABTFM5VX7YL72QWLP6XXLVA5CNFSM4HREWSC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZTVARQ#issuecomment-510087238, or mute the thread https://github.com/notifications/unsubscribe-auth/ACOMQKWYH73W435ZVIXA2C3P6XXLVANCNFSM4HREWSCQ .

-- Beata

[geojamesc]

Having now pulled down all data from jncc, done RFI-masking as required and uploaded ALL to the epcc VM and then build the master image list, for the dates we are concerned coverage is as follows:

so South Uist and Barra we now have data for.

[geojamesc]

I initially missed Richard`s comment in the Sentinel1_TotalProcessedScenes_20190712.xlsx spreadsheet that there are 2 extra orbit 23 scenes for 20180320 that were pulled down from the JNCC webstore but are not for use in crop map but for another non-crop map project. These are the only scenes that give coverage for the extreme west of Scotland i.e. S Uist/Barra etc.

I have now removed these images from the image metadata csv that controls for which images zonal statistics are calculated for.

The count of the untouched images pulled down from JNCC + the images TeamScotCropMap have done RFI masking on (as described by the image metadata csv) equals the expected image count in Sentinel1_TotalProcessedScenes_20190712.xlsx. The final coverage of the data looks like this (i.e. there is in fact no data for South Uist and Barra after all).