search

sgrieve / gdbm

MIT License
0 stars 0 forks source link

V2 revisions #1

Closed sgrieve closed 3 years ago

sgrieve commented 4 years ago

Following extensive discussions and a round of peer review there are a number of changes we wish to make to our channel extraction to ensure that what we are extracting are meaningful channels/valleys that we can study. These changes are summarised below:

1. Switch from the 30m srtm data to the 90m data

See this email from Hansjorg:

Hi,

nice to hear from you. The hacking affected several sites across Europe. We were offline for 2 weeks too.

I remember a few papers which compared JAXA SRTM(versions) and Aster, but I can't exactly remember the titles/journals etc. I will have to search a bit again. (below links that came up, but they do not include the papers I remember)

What I meant with "well known" was that the SRTM(90m) we have and what e.g. Lehner used for his analysis has already been heavily processed and improved (I think 4 versions only on CGIAR) over decades.  The newly released 30m product is on the other hand at it's initial stage (maybe quite improved by USGS and US-DOD but we don;t know that) 

Recently, quite a few DEM comparision studies (which I haven't seen before) have been published. These may help to evaluate the quality comparison question independently.

https://esurf.copernicus.org/articles/5/211/2017/

https://www.frontiersin.org/articles/10.3389/feart.2019.00141/full

https://www.tandfonline.com/doi/full/10.1080/08120091003677553

https://www.mdpi.com/2072-4292/9/3/262/htm

https://www.tandfonline.com/doi/full/10.1080/01431161.2019.1677968?scroll=top&needAccess=true

https://www.researchgate.net/publication/307530141_VERTICAL_ACCURACY_ASSESSMENT_OF_30-M_RESOLUTION_ALOS_ASTER_AND_SRTM_GLOBAL_DEMS_OVER_NORTHEASTERN_MINDANAO_PHILIPPINES

Note, what surprised me in the Esurf manuscript is that the supposedly best product, 

TanDEM-X 12m resolution performs so poorly, although the technique is the same as SRTM but 20 years newer and without any temporal constraints for acquisition. This is definitely a result of poor post processing.

Cheers,

hj

P.S. (I included so many links also for me as a reference)

2. Clip out lakes

We can do a better job of terminating channels that drain into lakes by clipping our climate zone polygons by using a global lake dataset

3. Increase size of tiles

If we change to the 90m data, we can increase the size of our tiles, to try and remove tiling artefacts that may be present in our data

4. Intersect pit filled areas

To identify anomalously straight sections of channel that are impacted by our pit filling algorithm, we can flag channels that intersect significantly with pit filled areas in our DEMs. This will give us a starting point to see how significant this issue is, and then we can exclude the worst affected channels from future analysis.

5. Remove straight sections

We can test the extracted channel network for anomalously straight sections as another method to remove potentially problematic channels.

6. Test for local concavity

This test may run earlier then step 5 as it will likely be embedded in the initial processing and extraction code rather than in the post processing.

The idea is that we can test pixels orthogonal to the recorded flow direction, in the immediate neighbourhood (and possibly beyond) to see if the channel pixel is surrounded by topography which would constrain its flow. This should identify spurious channels flowing across flat areas of unconstrained topography.

sgrieve commented 4 years ago

As I work on each of these steps, I will create a new issue to track progress, leading to a PR.

sgrieve commented 4 years ago

The lake clipping work has been completed and is documented in #2. Decided no need for a PR here as I was mainly creating new files rather than touching existing stuff.

sgrieve commented 4 years ago

Discussions with Michael have resulted in us agreeing to press on with the 30 meter data for now, knowing that we can swap for the 90 meter data later if needed.

sgrieve commented 3 years ago

Intersections of pit filled areas are now logged - see issue #3 for some detail.

We also now quantify anomalous straightness by tracking streaks of constant flow direction within a single channel. We can filter these based on the proportion of the channel which is straight, relative to the total number of pixels in the channel.

We have decided not to process the data for local concavity - it is a massive can of worms. And so we have now completed all of the changes listed above and are ready to start analysing some data! 🚀