hobuinc / usgs-lidar

AWS Entwine Point Tiles USGS LiDAR Public Dataset GitHub repo
https://registry.opendata.aws/usgs-lidar/
137 stars 14 forks source link

Reason for EPT data missing entirely? - MO_FEMANRCS_2020_D20 #71

Open bkgelder opened 4 months ago

bkgelder commented 4 months ago

Hello, I was looking for this data in EPT format but was not able to find it in the WESM database: https://rockyweb.usgs.gov/vdelivery/Datasets/Staged/Elevation/LPC/Projects/MO_FEMANRCS_2020_D20/

I'm sure there is a reason some 3DEP data makes it in to WESM and some doesn't but I haven't been able to find that. All I can find is this vague reference in the WESM data dictionary: 'A new WESM is written daily as more elevation projects are processed and published'

from: https://www.usgs.gov/ngp-standards-and-specifications/wesm-data-dictionary

This data has been published for over a year now and I'm guessing that was enough time for 'processing'. I'm pretty sure I can create my own local EPT but won't if it's going to be served soon. Thanks!

jstoker74 commented 4 months ago

Hi, I am seeing these work units in the WESM for this project: image

I am also seeing the EPTs for these 3 workunits: https://s3-us-west-2.amazonaws.com/usgs-lidar-public/MO_FEMANRCS_1_2020/ept.json

https://s3-us-west-2.amazonaws.com/usgs-lidar-public/MO_FEMANRCS_2_2020/ept.json

https://s3-us-west-2.amazonaws.com/usgs-lidar-public/MO_FEMANRCS_3_2020/ept.json

Are there other workunits in this project that you are looking for?

Hope that helps, Jason

@.*** Jason M. Stoker, PhD. Physical Scientist at USGS National Geospatial Program

Phone 970-226-9227 Mobile 605-496-3513 Web www.usgs.gov/3DEPhttp://www.usgs.gov/3DEP Email @.**@.>

[Title: LinkedIn - Description: image of LinkedIn icon]https://www.linkedin.com/in/jasonmstoker/

From: bkgelder @.> Sent: Tuesday, April 30, 2024 9:17 AM To: hobuinc/usgs-lidar @.> Cc: Subscribed @.***> Subject: [EXTERNAL] [hobuinc/usgs-lidar] Reason for EPT data missing entirely? - MO_FEMANRCS_2020_D20 (Issue #71)

This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding.

Hello, I was looking for this data in EPT format but was not able to find it in the WESM database: https://rockyweb.usgs.gov/vdelivery/Datasets/Staged/Elevation/LPC/Projects/MO_FEMANRCS_2020_D20/

I'm sure there is a reason some 3DEP data makes it in to WESM and some doesn't but I haven't been able to find that. All I can find is this vague reference in the WESM data dictionary: 'A new WESM is written daily as more elevation projects are processed and published'

from: https://www.usgs.gov/ngp-standards-and-specifications/wesm-data-dictionary

This data has been published for over a year now and I'm guessing that was enough time for 'processing'. I'm pretty sure I can create my own local EPT but won't if it's going to be served soon. Thanks!

- Reply to this email directly, view it on GitHubhttps://github.com/hobuinc/usgs-lidar/issues/71, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGKT33OCB7N4U5R6F2BKRZLY76YYDAVCNFSM6AAAAABHAPM5R2VHI2DSMVQWIX3LMV43ASLTON2WKOZSGI3TCNZZGA4TMNA. You are receiving this because you are subscribed to this thread.Message ID: @.**@.>>

bkgelder commented 4 months ago

Thank you - exactly what I was looking for. It's not showing up in my local copy of WESM and I had checked on usgs.entwine.io before I messaged and must have missed it there as it doesn't show in the area I am looking for; it shows an older dataset. The area I am looking for is also covered by the 2011/2016 Salt River collection and that shows up but the more recent FEMANRCS 2020 data does not show up there, just elsewhere. Shouldn't the USGS Entwine just show most recent collections?

jstoker74 commented 4 months ago

I'll have to defer to @hobu as to how the workunits are represented on the usgs.entwine.io page. Since you shared the rockyweb link I was able to just scroll down in the list and find them, I didn't use the map. It does look like a messy non-contiguous acquisition collection though, so I definitely can understand why it was hard to see where things are, especially with several older collections over the same areas.

hobu commented 4 months ago

I had checked on usgs.entwine.io

https://github.com/hobuinc/usgs-lidar/tree/master/action/usgs_boundary computes the boundaries nightly by listing the bucket and running PDAL to extract a boundary. Check the boundaries action to see if something got dropped for whatever reason.