search

hobuinc / usgs-lidar

AWS Entwine Point Tiles USGS LiDAR Public Dataset GitHub repo
https://registry.opendata.aws/usgs-lidar/
133 stars 14 forks source link

entire state txt download for uget? #73

Open duranduran127 opened 3 weeks ago

duranduran127 commented 3 weeks ago

@hobu is it possible to use the aws resources to get all of the links to download lidar for an entire state? I have tried to do this through the data download but it is limited to 5000 items per txt file.

keythread commented 3 weeks ago

@duranduran127 It is possible (though not advisable) to use our TNMAccess API to programmatically obtain links for all the available raw lidar (not Entwined) for an entire state with a call like this:

https://tnmaccess.nationalmap.gov/api/v1/products?datasets=Lidar Point Cloud (LPC)&offset=0&max=5&polyCode=22&polyType=state&

where polyCode is a state code found by calling this web service:

https://www.sciencebase.gov/catalog/extents/get?q=Louisiana&extentType="U.S. State"offset=0&fields=name,shortName,extentType,webMercatorBounds,code,abbreviation&

One would need to paginate through the records by calling with different 'offset=' values. I think max=1000 is the maximum number of records returned at a time.

The API will return the download URLs from a non-AWS web server called rockyweb. This is the API the Downloader uses to query for all the National Map product download URLs.

See: TNMAccess API

Unless you are interested in a smaller state I wouldn't recommend this approach due to the volume of files, but it is possible.

duranduran127 commented 1 week ago

@keythread Thanks for the information!