USGS Harvest Sources Not Harvesting

[thejuliekramer]

These harvest sources confirmed as active by USGS staff are not harvesting properly and errors are not occurring in production. I've documented individual errors to debug below.

source	error
national-elevation-dataset-ned-1-arc-second-collection	Could not harvest WAF link http://localhost: HTTPConnectionPool(host='localhost', port=80)
usgs-national-geologic-map-database	Validation Error - name or id missing value
usgs-ned-1-3-arc-second-contours	Could not harvest WAF link http://localhost: HTTPConnectionPool(host='localhost', port=80)
national-elevation-dataset-ned-alaska-2-arc-second-collection	Validation Error - name or id missing value
usgs-national-watershed-boundary-dataset-wbd-downloadable-data-collection	Validation Error - name or id missing value
5-meter-alaska-digital-elevation-models-dems-usgs-national-map	Transformation to ISO Failed
alaska-orthorectified-radar-intensity-image-usgs-national-map	Transformation to ISO Failed
usgs-national-boundary-dataset-nbd-downloadable-data-collection	Transformation to ISO Failed
usgs-land-cover-woodland-downloadable-data-collection	Transformation to ISO Failed
fsa-10-1-naip-imagery-collection	Validation Error - name or id missing value
usgs-lidar-point-cloud-las-harvest-source	443 timeout when importing harvest source
usgs-ned-original-product-resolution-opr-downloadable-data-collection	443 timeout when importing harvest source
usgs-us-topo-maps	443 timeout when importing harvest source

Acceptance Critera

The following harvest sources are importing properly:

• [x] national-elevation-dataset-ned-1-arc-second-collection
• [x] usgs-national-geologic-map-database
• [x] usgs-ned-1-3-arc-second-contours
• [x] national-elevation-dataset-ned-alaska-2-arc-second-collection
• [x] usgs-national-watershed-boundary-dataset-wbd-downloadable-data-collection
• [x] 5-meter-alaska-digital-elevation-models-dems-usgs-national-map
• [x] alaska-orthorectified-radar-intensity-image-usgs-national-map
• [x] usgs-national-boundary-dataset-nbd-downloadable-data-collection
• [x] usgs-land-cover-woodland-downloadable-data-collection
• [x] fsa-10-1-naip-imagery-collection
• [x] usgs-lidar-point-cloud-las-harvest-source
• [x] usgs-combined-vector-downloadable-data-collection
• [x] usgs-ned-original-product-resolution-opr-downloadable-data-collection
• [x] national-elevation-dataset-ned-1-9-arc-second-collection
• [x] national-elevation-dataset-ned-1-3-arc-second-collection
• [x] usgs-us-topo-maps

[thejuliekramer]

related to #510

[thejuliekramer]

@hkdctol

national-elevation-dataset-ned-alaska-2-arc-second-collection has its collection_metadata_url in production as localhost - we need this to be updated - can you ask your contact about this one?

[hkdctol]

@thejuliekramer Have reached out to USGS lead on this issue and copied you on email.

[thejuliekramer]

Updated above with additional harvest sources from the USGS spreadsheet Todo items: 1. Need to update the above in the main progress spreadsheet. 2. Need to check if sources that are failing are failing in production 3. Ask REI team if there is anything we can do about timeouts

[hkdctol]

@thejuliekramer did you see in email the Alaska source - USGS says they fixed the URL

[thejuliekramer]

Moving to QA - and updating the above still not working in the main harvest source progress spreadsheet

[hkdctol]

This ticket can be moved to done, as the only sources left are the ones to be added manually