Test SWOT 2.0 collections with L2ss-py service

podaac / hitide

High-level Tool for Interactive Data Extraction

Apache License 2.0

2 stars 0 forks source link

Test SWOT 2.0 collections with L2ss-py service #51

Open jamesfwood opened 6 months ago

jamesfwood commented 6 months ago

Test with l2ss-py service:

SWOT_L2_LR_SSH_BASIC_2.0 (Test subset and merge/concise in HITIDE-UI) SWOT_L2_LR_SSH_EXPERT_2.0 (Test subset and merge/concise in HITIDE-UI) SWOT_L2_LR_SSH_WINDWAVE_2.0 (Test subset and merge/concise in HITIDE-UI)

SWOT_L2_NALT_GDR_GDR_2.0 (Test with EDSC, or directly through harmony or data-subscriber) SWOT_L2_NALT_GDR_SGDR_2.0 (Test with EDSC, or directly through harmony or data-subscriber) SWOT_L2_NALT_GDR_SSHA_2.0 (Test with EDSC, or directly through harmony or data-subscriber)

SWOT_L2_NALT_IGDR_GDR_2.0 (Test with EDSC, or directly through harmony or data-subscriber) SWOT_L2_NALT_IGDR_SGDR_2.0 (Test with EDSC, or directly through harmony or data-subscriber) SWOT_L2_NALT_IGDR_SSHA_2.0 (Test with EDSC, or directly through harmony or data-subscriber)

SWOT_L2_NALT_OGDR_GDR_2.0 (Test with EDSC, or directly through harmony or data-subscriber) SWOT_L2_NALT_OGDR_SSHA_2.0 (Test with EDSC, or directly through harmony or data-subscriber)

SWOT_L2_RAD_GDR_2.0 (Test with EDSC, or directly through harmony or data-subscriber) SWOT_L2_RAD_IGDR_2.0 (Test with EDSC, or directly through harmony or data-subscriber) SWOT_L2_RAD_OGDR_2.0 (Test with EDSC, or directly through harmony or data-subscriber)

jamesfwood commented 4 months ago

@davidcolemanjpl please test these collections in HiTIDE UI or directly with harmony (test both l2ss-py and concise).

Test in UAT only

Also some collections are getting renamed. Refer to this for the new names: https://wiki.jpl.nasa.gov/pages/viewpage.action?pageId=889614615

Thanks!

davidcolemanjpl commented 4 months ago

Note: SIDE Observation

SWOT_L2 collection are not currently visible in EDSC (UAT):

SWOT_L2_NALT_GDR_GDR_2.0 - Unable to locate collection in EDSC - 0 Matching Collections SWOT_L2_RAD_GDR_2.0 - Unable to locate collection in EDSC - 0 Matching Collections

SWOT_L2_NALT_GDRSGDR1.0** = C1256783374-POCLOUD - (Unable to currently locate collection in EDSC) SWOT_L2_NALT_GDRSSHA1.0** = C1256783378-POCLOUD - (Unable to currently locate collection in EDSC)

jamesfwood commented 4 months ago

@davidcolemanjpl yeah some of them might not be ready. Can you put results with the ones that actually do work?

Good idea to test in EDSC. Thanks!

davidcolemanjpl commented 4 months ago

HiTIDE UI (UAT) v4.17.0-RC.6 l2ss-py:v2.8.0rc3 concise:v0.7.0rc7

Test with l2ss-py service: SWOT_L2_LR_SSH_BASIC_2.0 (Test subset and merge/concise in HITIDE-UI) - PASS SWOT_L2_LR_SSH_EXPERT_2.0 (Test subset and merge/concise in HITIDE-UI) - PASS SWOT_L2_LR_SSH_WINDWAVE_2.0 (Test subset and merge/concise in HITIDE-UI) - PASS

davidcolemanjpl commented 4 months ago

L2SS-py v2.8.0rc3 PO.DAAC data subscriber v1.15.0

SWOT_L2_NALT_GDR_GDR_2.0 (Test directly through harmony or data-subscriber) - Unable to currently locate collection in EDSC - 0 Matching Collections (C1256783373-POCLOUD) podaac-data-subscriber - FAIL - Error- AttributeError: 'Namespace' object has no attribute 'search_cycles' podaac-data-downloader - FAIL - Error- Exception: ('Bad Request', 'Error: No matching granules found.') Harmony - L2SS-Py request - OK/PASS - Harmony JobID: 0230e7dd-3b71-44e0-9c90-45f1fd4afd47 _SWOT_L2_NALT_GDR_GDR1.0 = (C1261072650-POCLOUD) podaac-data-subscriber - FAIL - Error- AttributeError: 'Namespace' object has no attribute 'search_cycles' podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony - L2SS-Py request - "Error: No matching granules found."

SWOT_L2_NALT_GDR_SGDR_2.0 (Test directly through harmony or data-subscriber) - (C1261072651-POCLOUD) podaac-data-subscriber - FAIL - Error- AttributeError: 'Namespace' object has no attribute podaac-data-downloader - OK/PASS Harmony - L2SS-Py request - OK/PASS - Harmony JobID: 92231768-5120-423b-9dc0-cf26ad937cb8 _SWOT_L2_NALT_GDR_SGDR1.0 (C1256783374-POCLOUD) - (Unable to locate collection in EDSC (UAT) podaac-data-subscriber - FAIL - AttributeError: 'Namespace' object has no attribute 'search_cycles' podaac-data-downloader - 'Error: No matching granules found.' Harmony - L2SS-Py request - "Error: No matching granules found."

SWOT_L2_NALT_GDR_SSHA_2.0 (Test directly through harmony or data-subscriber) - (C1261072652-POCLOUD) podaac-data-subscriber - - FAIL - AttributeError: 'Namespace' object has no attribute 'search_cycles' podaac-data-downloader - OK/PASS Harmony - L2SS-Py request - OK/PASS - Harmony JobID: afa33a85-1782-46d1-a8ce-c950d5b7bdf9 _SWOT_L2_NALT_GDR_SSHA1.0 (C1256783378-POCLOUD) (Unable to locate collection in EDSC) podaac-data-subscriber - FAIL - AttributeError: 'Namespace' object has no attribute 'search_cycles' podaac-data-downloader - 'Error: No matching granules found.' Harmony - L2SS-Py request - "Error: No matching granules found."

SWOT_L2_NALT_IGDR_GDR_2.0 (Test directly through harmony or data-subscriber) (C1261072654-POCLOUD) podaac-data-subscriber - FAIL - AttributeError: 'Namespace' object has no attribute 'search_cycles' podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony - L2SS-Py request - "Error: No matching granules found."

SWOT_L2_NALT_IGDR_SGDR_2.0 (Test directly through harmony or data-subscriber) (C1261072655-POCLOUD) podaac-data-subscriber - FAIL - AttributeError: 'Namespace' object has no attribute 'search_cycles' podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony - L2SS-Py request - "Error: No matching granules found."

SWOT_L2_NALT_IGDR_SSHA_2.0 (C1261072656-POCLOUD) podaac-data-subscriber - FAIL - AttributeError: 'Namespace' object has no attribute 'search_cycles' (see PODAAC-6119 bug ticket) podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony Request - "Error: No matching granules found."

SWOT_L2_NALT_OGDR_GDR_2.0 (C1261072658-POCLOUD) podaac-data-subscriber - FAIL - AttributeError: 'Namespace' object has no attribute 'search_cycles' (see PODAAC-6119 bug ticket) podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony Request - "Error: No matching granules found."

SWOT_L2_NALT_OGDR_SSHA_2.0 (C1261072659-POCLOUD) podaac-data-subscriber - FAIL - AttributeError: 'Namespace' object has no attribute 'search_cycles' (PODAAC-6119 bug ticket) podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony Request - "Error: No matching granules found."

SWOT_L2_RAD_GDR_1.0 (C1261072660-POCLOUD) podaac-data-subscriber - FAIL - Error- AttributeError: 'Namespace' object has no attribute 'search_cycles' podaac-data-downloader - FAIL - Error- Exception: ('Bad Request', 'Error: No matching granules found.') Harmony - L2SS-Py request - OK/PASS - Harmony jobID: 3e9b1c7d-0918-4d03-b181-acbbf2ca6e53 SWOT_L2_RAD_GDR_2.0 (C1262095061-POCLOUD) - Unable to currently locate collection in EDSC - 0 Matching Collections podaac-data-subscriber - FAIL - Error- AttributeError: 'Namespace' object has no attribute 'search_cycles' podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony - request - "Error: No matching granules found."

SWOT_L2_RAD_IGDR_2.0 (C1261072661-POCLOUD) podaac-data-subscriber - FAIL - Error- AttributeError: 'Namespace' object has no attribute 'search_cycles' podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony - request - "Error: No matching granules found."

SWOT_L2_RAD_OGDR_2.0 (C1261072662-POCLOUD) podaac-data-subscriber - FAIL - Error- AttributeError: 'Namespace' object has no attribute 'search_cycles' (PODAAC-6119) podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony - request - OK/PASS - Harmony JobID: ffc39daf-ca67-4323-964a-8740510dfb43 or 0fafb14f-0c3a-4e57-a9d0-20d23080b3f4

SWOT_L2_LR_SSH_BASIC_2.0 (C1261072645-POCLOUD) podaac-data-subscriber - FAIL - Error- AttributeError: 'Namespace' object has no attribute 'search_cycles' (PODAAC-6119) podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony - request - FAIL - L2SS-py step does not succeed in Harmony UAT (see https://harmony.uat.earthdata.nasa.gov/jobs/ad7a887f-16a2-4a65-93e1-aaa43d0bb04d)

SWOT_L2_LR_SSH_EXPERT_2.0 (C1261072646-POCLOUD) podaac-data-subscriber - FAIL - Error- AttributeError: 'Namespace' object has no attribute 'search_cycles' (PODAAC-6119) podaac-data-downloader - FAIL - downloader currently hangs at "Waiting for Harmony subsetting job to complete" - (downloader took a while to process, completed w/ errors) [2024-02-28 16:19:13,292] {podaac_access.py:928} ERROR - Error processing harmony subsetting request: The job failed with 363 errors. See the errors field for more details [2024-02-28 16:19:13,293] {podaac_access.py:929} ERROR - Removing job id [89720a12-2019-4f0b-a9d9-1e2c8083b007] from harmony statefile ./data/.harmony Harmony - APIrequest - FAIL - L2SS-py step does not succeed in Harmony UAT (see JobStatus: https://harmony.uat.earthdata.nasa.gov/jobs/4098c9f3-7057-4c56-ac35-1a006b38e7b9)

SWOT_L2_LR_SSH_WINDWAVE_2.0 (C1261072648-POCLOUD) podaac-data-subscriber - FAIL - Error- AttributeError: 'Namespace' object has no attribute 'search_cycles' (PODAAC-6119) podaac-data-downloader - FAIL - downloader tool currently hangs at [2024-02-28 08:43:24,192] "Waiting for Harmony subsetting job to complete" - (downloader took a while to process, completed w/ errors) [2024-02-28 11:34:21,766] {podaac_access.py:928} ERROR - Error processing harmony subsetting request: The job failed with 370 errors. See the errors field for more details [2024-02-28 11:34:21,766] {podaac_access.py:929} ERROR - Removing job id [297337ad-27f5-4d93-9087-24d4d4a96591] from harmony statefile ./data/.harmony Harmony - APIrequest - FAIL - L2SS-py step does not succeed in Harmony UAT (see JobStatus: https://harmony.uat.earthdata.nasa.gov/jobs/4098c9f3-7057-4c56-ac35-1a006b38e7b9)

jamesfwood commented 4 months ago

@davidcolemanjpl there was some renaming of collections and stuff when you did that last test so the results may be bad. It should be ironed out now.
Can you do these tests again in both UAT and OPS? Everything should work in OPS now. Thanks!

davidcolemanjpl commented 4 months ago

Re-TEST: L2SS-py v2.9.0rc1 PO.DAAC data subscriber v1.15.0a3

_SWOT_L2_LR_SSH_BASIC2.0 (C1261072645-POCLOUD) podaac-data-subscriber - FAIL - Error- AttributeError: 'Namespace' object has no attribute 'search_cycles' (PODAAC-6119) NOTE: podaac-data-subscriber tool currently seems to work for the SWOT_L2_LR_SSH_BASIC_2.0 collection when the --subset flag NOT used podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony - request - FAIL - L2SS-py step does not succeed in Harmony UAT (see Harmony Job Status https://harmony.uat.earthdata.nasa.gov/jobs/c3138bac-8463-4bad-a88b-4f242ca0c4fa)

related data subscriber issue: https://github.com/podaac/data-subscriber/pull/162

UPDATE(030824): L2SS-py v2.9.0rc1 / 2.9.0rc3 PO.DAAC data subscriber v1.15.1a1

SWOT_L2_LR_SSH_BASIC_2.0 (C1261072645-POCLOUD) podaac-data-subscriber - FAIL - "IndexError: list index out of range" (update:031124: Error- HTTPError: 413 Client Error now occurs) podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony - request - FAIL - L2SS-py step did not succeed in Harmony UAT (see https://harmony.uat.earthdata.nasa.gov/jobs/a69c3ac9-6329-46bb-abb5-da962c7d4626):

https://harmony.uat.earthdata.nasa.gov/C1261072645-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2023-01-01T18:00:00%2B00:00":"2024-12-21T07:00:00%2B00:00")&skipPreview=true

note: Harmony - request works okay, L2SS-Py step(s) succeeds in Harmony UAT when the subset=lon info removed from API request: https://harmony.uat.earthdata.nasa.gov/C1261072645-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=time("2023-01-01T18:00:00%2B00:00":"2024-12-21T07:00:00%2B00:00")&skipPreview=true (see Job Status: https://harmony.uat.earthdata.nasa.gov/jobs/dd0b02ec-2c85-435c-83ae-59515a01e7e1)

SWOT_L2_LR_SSH_EXPERT_2.0 (C1261072646-POCLOUD) podaac-data-subscriber - FAIL - Error- HTTPError: 413 Client Error: Request Entity Too Large for url (see PODAAC-6119) podaac-data-downloader - FAIL - downloader currently hangs at "Waiting for Harmony subsetting job to complete" - podaac-data-downloader tool does NOT execute as expected Harmony - request - FAIL - L2SS-py step does not succeed in Harmony UAT (see JobStatus: https://harmony.uat.earthdata.nasa.gov/jobs/45cf61cf-3f24-47e9-8f3f-16d5a2d1ca78)

note: the Harmony - request still FAILs - the L2SS-Py step(s) failed in Harmony UAT when the subset=lon (and/or subset=lat) info removed from request: https://harmony.uat.earthdata.nasa.gov/C1261072646-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=time("2023-01-01T18:00:00%2B00:00":"2024-12-21T07:00:00%2B00:00")&skipPreview=true

(See Job Statuses: https://harmony.uat.earthdata.nasa.gov/jobs/3cc72a0f-244e-43e8-a9eb-9e80df90726c or https://harmony.uat.earthdata.nasa.gov/jobs/1baf824d-8d11-4830-8cec-72a4b2cd92a5)

SWOT_L2_LR_SSH_WINDWAVE_2.0 (C1261072648-POCLOUD) Harmony - APIrequest - FAIL - L2SS-py step does not succeed in Harmony UAT (see JobStatus: https://harmony.uat.earthdata.nasa.gov/jobs/4098c9f3-7057-4c56-ac35-1a006b38e7b9) note: Harmony - request still FAILs - the L2SS-Py step(s) failed in Harmony UAT when the subset=lat/lon info removed from API request: (See recent Harmony Job statuses: https://harmony.uat.earthdata.nasa.gov/jobs/7157bc0c-09fd-4717-a1b6-0407a8dc82ff or https://harmony.uat.earthdata.nasa.gov/jobs/38c3b232-0c0e-48e6-83c5-0b031541f167)

davidcolemanjpl commented 3 months ago

L2SS-py v2.9.0rc3 PO.DAAC data subscriber v1.15.1a2 (https://github.com/podaac/data-subscriber/pull/165) Harmony UAT v0.0.999 Related JIRA (PODAAC-6119)

SWOT_L2_LR_SSH_BASIC_2.0 (C1261072645-POCLOUD) podaac-data-subscriber -PASS - podaac-data-subscriber tool now executes as expected podaac-data-downloader - PASS - podaac-data-downloader tool executes as expected

Harmony (UAT) - request - FAIL - L2SS-py step did not succeed in Harmony UAT (see Harmony Job status: https://harmony.uat.earthdata.nasa.gov/jobs/8f28329f-9207-4388-b38f-45c582efe11a)

note: Harmony - request works okay (PASSED), the L2SS-Py steps were successful in Harmony UAT when the subset=lon info was removed from API request: https://harmony.uat.earthdata.nasa.gov/C1261072645-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=time("2023-01-01T18:00:00%2B00:00":"2024-12-21T07:00:00%2B00:00")&skipPreview=true (see Harmony Job status : https://harmony.uat.earthdata.nasa.gov/jobs/e4472c82-c4af-412e-bb54-fcd91b354f12)

SWOT_L2_LR_SSH_EXPERT_2.0 (C1261072646-POCLOUD) podaac-data-subscriber - FAIL - subscriber currently hangs at "Waiting for Harmony subsetting job to complete" podaac-data-downloader - FAIL - downloader currently hangs at "Waiting for Harmony subsetting job to complete" - podaac-data-downloader tool does NOT execute as expected

Harmony - request - FAIL - L2SS-py steps do not succeed in Harmony UAT (see JobStatus: Job status: https://harmony.uat.earthdata.nasa.gov/jobs/67da5d3d-8459-425f-87a1-857507d8db3f) note: Harmony - request did NOT work okay for SWOT_L2_LR_SSH_EXPERT_2.0, the L2SS-Py steps were NOT successful in Harmony UAT when the subset=lon info was removed from request

request: Failed: https://harmony.uat.earthdata.nasa.gov/C1261072646-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931%3A35.83342)&subset=lon(-125.57812%3A-124.73438)&subset=time(%222020-01-01T18%3A00%3A00%2B00%3A00%22%3A%222024-12-21T07%3A00%3A00%2B00%3A00%22)&skipPreview=true

see Harmony Job status: https://harmony.uat.earthdata.nasa.gov/jobs/67da5d3d-8459-425f-87a1-857507d8db3f

SWOT_L2_LR_SSH_WINDWAVE_2.0 (C1261072648-POCLOUD): podaac-data-subscriber - FAIL - subscriber currently hangs at "INFO - Waiting for Harmony subsetting job to complete..." podaac-data-downloader - FAIL - downloader currently hangs at "INFO - Waiting for Harmony subsetting job to complete..."

Harmony-request - FAIL - L2SS-py step does not succeed in Harmony UAT as expected :
request: https://harmony.uat.earthdata.nasa.gov/C1261072648-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931%3A35.83342)&subset=lon(-125.57812%3A-124.73438)&subset=time(%222020-01-01T18%3A00%3A00%2B00%3A00%22%3A%222024-12-21T07%3A00%3A00%2B00%3A00%22)&skipPreview=true

(see recent Harmony Job Statuses: https://harmony.uat.earthdata.nasa.gov/jobs/61488a2a-8a2a-4e4f-b8bb-7c41533a8070 or https://harmony.uat.earthdata.nasa.gov/jobs/1e30e025-213d-4e04-b34e-b5a0a8c7c907 )

davidcolemanjpl commented 3 months ago

L2SS-py v2.9.0rc3 PO.DAAC Data Subscriber v1.15.1a2

SWOT_L2_NALT_GDR_GDR_2.0 - (C1261072650-POCLOUD) podaac-data-subscriber - OK/PASS - podaac-data-subscriber tool executes as expected podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony - L2SS-Py request - OK/PASS - see JobStatus: https://harmony.uat.earthdata.nasa.gov/jobs/a2d7f441-7225-4fdb-b2bb-a439365345d4

SWOT_L2_NALT_GDR_SGDR_2.0 - (C1261072651-POCLOUD) podaac-data-subscriber - OK/PASS - podaac-data-subscriber tool executes as expected podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony - L2SS-Py request - OK/PASS - see JobStatus: https://harmony.uat.earthdata.nasa.gov/jobs/58c4223b-95e3-4e9e-8b45-b31f494a0cd3

SWOT_L2_NALT_GDR_SSHA_2.0 - (C1261072652-POCLOUD) podaac-data-subscriber - OK/PASS - podaac-data-subscriber tool executes as expected podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony - L2SS-Py request - OK/PASS - see Harmony JobID: cdbc670f-95c9-49bc-a266-aaeb8a018eee or JobStatus: https://harmony.uat.earthdata.nasa.gov/jobs/cdbc670f-95c9-49bc-a266-aaeb8a018eee

SWOT_L2_NALT_IGDR_GDR_2.0 - (C1261072654-POCLOUD) podaac-data-subscriber - OK/PASS - podaac-data-subscriber now currently executes as expected podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony - request - FAIL -"Error: No matching granules found." Request: https://harmony.uat.earthdata.nasa.gov/C1261072654-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2020-01-01T18:00:00%2B00:00":"2024-12-21T07:00:00%2B00:00")&skipPreview=true

SWOT_L2_NALT_IGDR_SGDR_2.0 - (C1261072655-POCLOUD) podaac-data-subscriber - OK/PASS - podaac-data-subscriber now currently executes as expected podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony - request - FAIL - "Error: No matching granules found." Request: https://harmony.uat.earthdata.nasa.gov/C1261072655-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2020-01-01T18:00:00%2B00:00":"2024-12-21T07:00:00%2B00:00")&skipPreview=true

SWOT_L2_NALT_IGDR_SSHA_2.0 - (C1261072656-POCLOUD) podaac-data-subscriber - OK/PASS - podaac-data-subscriber now currently executes as expected podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony Request - FAIL -"Error: No matching granules found." Request: https://harmony.uat.earthdata.nasa.gov/C1261072656-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2020-01-01T18:00:00%2B00:00":"2024-12-21T07:00:00%2B00:00")&skipPreview=true

SWOT_L2_NALT_OGDR_GDR_2.0 (C1261072658-POCLOUD) podaac-data-subscriber - OK/PASS - podaac-data-subscriber tool executes as expected podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony Request - FAIL - "Error: No matching granules found." Request: https://harmony.uat.earthdata.nasa.gov/C1261072658-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2020-01-01T18:00:00%2B00:00":"2024-12-21T07:00:00%2B00:00")&skipPreview=true

SWOT_L2_NALT_OGDR_SSHA_2.0 (C1261072659-POCLOUD) podaac-data-subscriber - OK/PASS - podaac-data-subscriber tool executes as expected podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony Request - FAIL - "Error: No matching granules found." Request: https://harmony.uat.earthdata.nasa.gov/C1261072659-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2020-01-01T18:00:00%2B00:00":"2024-12-21T07:00:00%2B00:00")&skipPreview=true

SWOT_L2_RAD_GDR_2.0 (C1261072660-POCLOUD) podaac-data-subscriber -- OK/PASS - podaac-data-subscriber tool executes as expected podaac-data-downloader -- OK/PASS - podaac-data-downloader tool executes as expected Harmony - request - OK/PASS - Harmony jobIDs: d5e2862d-d061-46be-b133-2f09975de66d or 3017cd54-1369-4e79-9349-f05554938a90

SWOT_L2_RAD_IGDR_2.0 (C1261072661-POCLOUD) podaac-data-subscriber - OK/PASS - podaac-data-subscriber tool executes as expected podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony - request - FAIL - "Error: No matching granules found." Request: https://harmony.uat.earthdata.nasa.gov/C1261072661-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2020-01-01T18:00:00%2B00:00":"2024-12-21T07:00:00%2B00:00")&skipPreview=true

SWOT_L2_RAD_OGDR_2.0 (C1261072662-POCLOUD) podaac-data-subscriber - OK/PASS - podaac-data-subscriber tool executes as expected podaac-data-downloader - OK/PASS - podaac-data-downloader tool executes as expected Harmony - request - OK/PASS - Harmony JobID: 1679fd13-cba6-401f-98a6-5a751d53082e or 2a3be7de-3eb9-4f3b-8aae-74bac655d173

jamesfwood commented 3 months ago

Hi @davidcolemanjpl Looks like you made good progress here. Nice!
So the ones failing still, did you verify there are granules in your subset bbox? Maybe you can increase the bbox size to see if it still fails. Did you also test this in OPS?

It could be helpful to see all this in a table. Like have each collection listed and for each environment tested have a cell for each data subscriber, data-downloader, harmony, hitide, l2ss, concise, thumbnails, footprints, info, etc so we can see exactly which still need work.

davidcolemanjpl commented 3 months ago

@jamesfwood, FYI... Please let me know if you are able to reproduce the encountered L2SS-py failures in UAT/OPS (using the respective Harmony requests)

L2SS-Py v2.9.0 (OPS) PO.DAAC Data Subscriber v1.15.1a2 Harmony (OPS) v0.0.999 CONCISE v0.8.0 EDSC v24.1.3-2

SWOT_L2_LR_SSH_BASIC_2.0 - C2799465428-POCLOUD: podaac-data-subscriber -PASS - podaac-data-subscriber tool now executes as expected podaac-data-downloader - PASS - podaac-data-downloader tool executes as expected Harmony (OPS) - request - FAIL - all L2SS-py steps did not succeed in Harmony OPS Request: https://harmony.earthdata.nasa.gov/C2799465428-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2023-01-01T18:00:00%2B00:00":"2024-12-21T07:00:00%2B00:00")&skipPreview=true "WorkItem failed: Could not get result item file size" see Harmony Job status:https://harmony.earthdata.nasa.gov/jobs/b11a0aa8-51c9-46c9-bcbb-9466fbc687f8 - (26 granules)

note: L2SS-py PASSED when using single granule in Request: https://harmony.earthdata.nasa.gov/C2799465428-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&maxResults=1000000&skipPreview=true&subset=lat(-90%3A90)&subset=lon(-180%3A180)&granuleName=SWOT_L2_LR_SSH_Basic_006_545_20231121T234622_20231122T003750_PIC0_01_swot&granuleId=G2808365105-POCLOUD (see Job status: https://harmony.earthdata.nasa.gov/jobs/2a5e830d-fc2e-499d-b443-92c75834c62a)

L2SS-py/CONCISE PASSED when using following Request (3 granules): https://harmony.earthdata.nasa.gov/C2799465428-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&maxResults=1000000&skipPreview=true&subset=lat(-90%3A90)&subset=lon(-180%3A180)&granuleName=SWOT_L2_LR_SSH_Basic_006_547_20231122T012915_20231122T022044_PIC0_01_swot&granuleName=SWOT_L2_LR_SSH_Basic_006_546_20231122T003749_20231122T012917_PIC0_01_swot&granuleName=SWOT_L2_LR_SSH_Basic_006_545_20231121T234622_20231122T003750_PIC0_01_swot&granuleId=G2808365200-POCLOUD&granuleId=G2808365138-POCLOUD&granuleId=G2808365105-POCLOUD&concatenate=true (see Job status:https://harmony.earthdata.nasa.gov/jobs/f0fdc2ec-f397-455b-856f-f87f1bd73d96)

SWOT_L2_LR_SSH_EXPERT_2.0 - C2799465497-POCLOUD: podaac-data-subscriber - FAIL - subscriber currently hangs at "Waiting for Harmony subsetting job to complete" - "WorkItem failed: podaac/l2ss-py:2.9.0: Service request failed with an unknown error" (https://harmony.earthdata.nasa.gov/jobs/205c7950-829e-44ff-ab7f-e4d397253e95) (423 granules) podaac-data-downloader - FAIL - downloader currently hangs at "Waiting for Harmony subsetting job to complete" - podaac-data-downloader tool does NOT execute as expected - "WorkItem failed: podaac/l2ss-py:2.9.0: Service request failed with an unknown error" (https://harmony.earthdata.nasa.gov/jobs/61a6a6e8-6236-4bc0-a61d-8312d715ffb3) (423 granules) Harmony - request - FAIL - L2SS-py steps do not succeed in Harmony OPS - (8 granules) request: Failed: "WorkItem failed: podaac/l2ss-py:2.9.0: Service request failed with an unknown error" https://harmony.earthdata.nasa.gov/C2799465497-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2023-01-01T18:00:00%2B00:00":"2023-12-21T07:00:00%2B00:00")&skipPreview=true

(see Harmony Job status: https://harmony.earthdata.nasa.gov/jobs/20870153-193a-4fab-83d6-8077bb0853f2)

note: L2SS-py PASSED when using following Request (4 granules)): https://harmony.earthdata.nasa.gov/C2799465497-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&maxResults=1000000&skipPreview=true&subset=lat(-90%3A90)&subset=lon(-180%3A180)&granuleName=SWOT_L2_LR_SSH_Expert_012_340_20240318T203012_20240318T212057_PIC0_01_swot&granuleName=SWOT_L2_LR_SSH_Expert_012_341_20240318T212236_20240318T221307_PIC0_01_swot&granuleName=SWOT_L2_LR_SSH_Expert_012_342_20240318T221306_20240318T230434_PIC0_01_swot&granuleName=SWOT_L2_LR_SSH_Expert_012_343_20240318T230432_20240318T235601_PIC0_01_swot&granuleId=G2903718276-POCLOUD&granuleId=G2903718299-POCLOUD&granuleId=G2903718323-POCLOUD&granuleId=G2903718344-POCLOUD&concatenate=true (see Job status: : https://harmony.earthdata.nasa.gov/jobs/36172acb-aebd-4b78-a1f5-080eb9849a20)

SWOT_L2_LR_SSH_WINDWAVE_2.0 - C2799465507-POCLOUD: podaac-data-subscriber - FAIL - subscriber currently hangs at "INFO - Waiting for Harmony subsetting job to complete..." (58 granules) see Job Status: https://harmony.earthdata.nasa.gov/jobs/0627aebc-e81f-4403-b4a2-01e86714de11 podaac-data-downloader - FAIL - downloader currently hangs at "INFO - Waiting for Harmony subsetting job to complete..." see Job Status https://harmony.earthdata.nasa.gov/jobs/fd5502b0-3f32-46da-9933-f54992e6f340

Harmony-request - FAIL - L2SS-py step does not succeed in Harmony OPS as expected : Request: (17 granules) https://harmony.earthdata.nasa.gov/C2799465507-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2024-03-01T18:00:00%2B00:00":"2024-03-21T07:00:00%2B00:00")&skipPreview=true (see recent Harmony Job Status: https://harmony.earthdata.nasa.gov/jobs/1567ecef-3900-4429-863f-969ee879ca94)

note: L2SS-py/concise PASSED when following Request: (3 granules): https://harmony.earthdata.nasa.gov/C2799465507-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&maxResults=1000000&skipPreview=true&subset=lat(-90%3A90)&subset=lon(-180%3A180)&granuleName=SWOT_L2_LR_SSH_WindWave_012_341_20240318T212236_20240318T221307_PIC0_01_swot&granuleName=SWOT_L2_LR_SSH_WindWave_012_342_20240318T221306_20240318T230434_PIC0_01_swot&granuleName=SWOT_L2_LR_SSH_WindWave_012_343_20240318T230432_20240318T235601_PIC0_01_swot&granuleId=G2903718306-POCLOUD&granuleId=G2903718317-POCLOUD&granuleId=G2903718347-POCLOUD&concatenate=true

(see Job status: https://harmony.earthdata.nasa.gov/jobs/593aa4af-c66a-4adb-a3ba-e145cbaf5a55)

L2SS-Py v2.9.0 - OPS

_Re-test in the Harmony OPS env, SWOT collections that previously failed L2SS-py in Harmony UAT ( SWOT_L2NALT and RAD): PASS

SWOT_L2_NALT_IGDR_GDR_2.0 - (C2799465526-POCLOUD) Harmony - Request - PASS (12 granules) https://harmony.earthdata.nasa.gov/C2799465526-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2023-01-01T18:00:00%2B00:00":"2024-12-21T07:00:00%2B00:00")&skipPreview=true (see Harmony Job status : https://harmony.earthdata.nasa.gov/jobs/ebe20daa-6164-4f44-824e-02fe2fe094d2)
SWOT_L2_NALT_IGDR_SGDR_2.0 - (C2799465529-POCLOUD) Harmony - Request - PASS (12 granules ) https://harmony.earthdata.nasa.gov/C2799465529-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2023-01-01T18:00:00%2B00:00":"2024-05-21T07:00:00%2B00:00")&skipPreview=true (see Harmony OPS Job status : https://harmony.earthdata.nasa.gov/jobs/60630ad2-8c80-4c1d-862d-d3d7e314e362)
SWOT_L2_NALT_IGDR_SSHA_2.0 - (C2799465538-POCLOUD) Harmony Request - PASS (12 granules ) https://harmony.earthdata.nasa.gov/C2799465538-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2023-01-01T18:00:00%2B00:00":"2024-12-21T07:00:00%2B00:00")&skipPreview=true (see Harmony OPS Job status : https://harmony.earthdata.nasa.gov/jobs/db478098-c0c8-4308-b931-82ea3aaa403b)
SWOT_L2_NALT_OGDR_GDR_2.0 (C2799465542-POCLOUD) Harmony-Request - PASS (see Harmony OPS Job status : https://harmony.earthdata.nasa.gov/jobs/d327247c-b3c6-4145-ac3d-730260ea06d4 )
SWOT_L2_NALT_OGDR_SSHA_2.0 (C2799465544-POCLOUD) Harmony-Request - PASS (see Harmony OPS Job status: https://harmony.earthdata.nasa.gov/jobs/77fa285e-4542-4501-9ba5-8e18bc330964)
SWOT_L2_RAD_IGDR_2.0 (C2799438351-POCLOUD) Harmony - Request -PASS (9 granules) https://harmony.earthdata.nasa.gov/C2799438351-POCLOUD[/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2023-01-01T18:00:00%2B00:00":"2024-12-21T07:00:00%2B00:00")&skipPreview=true (see Harmony OPS jobID: fa217beb-a9db-4b5f-a1ff-db4954655c99)

Re-Test Harmony L2SS-py in OPS env: (_SWOT_L2_LR_SSHBASIC, _SSHExpert, and _SSHWindwave): L2SS-Py v2.9.0 - OPS (3/23/24)

SWOT_L2_LR_SSH_BASIC_2.0 - (C2799465428-POCLOUD): Harmony - request - FAIL - "WorkItem failed: Could not get result item file size" Request: (67 granules) https://harmony.earthdata.nasa.gov/C2799465428-POCLOUD[/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2023-01-01T18:00:00%2B00:00":"2024-12-21T07:00:00%2B00:00")&skipPreview=true (see failures in Harmony OPS jobStatus: https://harmony.earthdata.nasa.gov/jobs/df91021c-6116-44c0-a3f3-8591440764b9)
SWOT_L2_LR_SSH_EXPERT_2.0 - (C2799465497-POCLOUD) Harmony - request - FAIL - "WorkItem failed: podaac/l2ss-py:2.9.0: Service request failed with an unknown error" Request: (67 granules) https://harmony.earthdata.nasa.gov/C2799465497-POCLOUD[/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2023-01-01T18:00:00%2B00:00":"2024-12-21T07:00:00%2B00:00")&skipPreview=true (see Harmony OPS jobStatus:https://harmony.earthdata.nasa.gov/jobs/ef236df1-0b49-4902-ba9c-a32065e51536)
SWOT_L2_LR_SSH_WINDWAVE_2.0 - (C2799465507-POCLOUD) Request: - FAIL - (67 granules) - "WorkItem failed: podaac/l2ss-py:2.9.0: Service request failed with an unknown error" https://harmony.earthdata.nasa.gov/C2799465507-POCLOUD[/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2023-03-01T18:00:00%2B00:00":"2024-12-21T07:00:00%2B00:00")&skipPreview=true (see Harmony OPS jobID: 10ad5df1-2627-40af-88c2-fbc5baca7070)

jamesfwood commented 2 months ago

Hi @davidcolemanjpl. We updated l2ss-py in UAT so you could try some of your failed UAT tests again. Thanks!

davidcolemanjpl commented 2 months ago

L2SS-py v2.10.0rc4 - UAT (L2SS-py:2.9.0 OPS) Postman v10.24.16 (10.24.16) podaac-data-subscriber v1.15.2 Harmony UAT v 0.0.1015 / Harmony OPS v0.0.1011 EDSC UAT v24.1.6-6

When re-testing SWOT_L2 collections that previously failed tests in Harmony UAT env, user received error results as before:

E.g.,

SWOT_L2_NALT_IGDR_GDR_2.0 - (C1261072654-POCLOUD) Harmony - request - FAIL -"Error: No matching granules found." Request: https://harmony.uat.earthdata.nasa.gov/C1261072654-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2020-01-01T18:00:00%2B00:00":"2024-12-21T07:00:00%2B00:00")&skipPreview=true

note: when user attempted to remove the time parameter from the CMR request, also received error : { "code": "harmony.RequestValidationError", "description": "Error: No matching granules found." }

Also, tried request: FAIL - https://harmony.uat.earthdata.nasa.gov/C1261072654-POCLOUDogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931%3A35.83342)&subset=time(%222020-01-01T18%3A00%3A00%2B00%3A00%22%3A%222024-12-21T07%3A00%3A00%2B00%3A00%22)&skipPreview=true

{ "code": "harmony.NotFoundError", "description": "Error: The requested page was not found." }

SWOT_L2_NALT_IGDR_SGDR_2.0 - (C1261072655-POCLOUD) Harmony - request - FAIL - "Error: No matching granules found." Request: https://harmony.uat.earthdata.nasa.gov/C1261072655-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2020-01-01T18:00:00%2B00:00":"2024-12-21T07:00:00%2B00:00")&skipPreview=true { "code": "harmony.RequestValidationError", "description": "Error: No matching granules found." }

SWOT_L2_LR_SSH_WINDWAVE_2.0 (C1261072648-POCLOUD): podaac-data-subscriber - FAIL - subscriber currently hangs at "INFO - Waiting for Harmony subsetting job to complete..." podaac-data-downloader - FAIL - podaac-data-downloader -c SWOT_L2_LR_SSH_WINDWAVE_2.0 -d ./data --start-date 2023-05-01T00:00:00Z --end-date 2024-12-21T00:00:00Z --subset -b="120,-30,160,20" [2024-04-19 10:52:57,205] {subsetting.py:100} INFO - Resuming existing harmony job id... [2024-04-19 10:52:57,205] {subsetting.py:102} INFO - Waiting for Harmony subsetting job to complete... [2024-04-19 10:53:00,270] {podaac_access.py:921} ERROR - Error processing harmony subsetting request: The job failed with 436 errors. See the errors field for more details [2024-04-19 10:53:00,271] {podaac_access.py:922} ERROR - Removing job id [05c9bd4b-9d16-4efa-b8c2-9b22a5c13551] from harmony statefile ./data/.harmony [2024-04-19 10:53:01,380] {podaac_data_downloader.py:177} INFO - Success Count: 1 [2024-04-19 10:53:01,917] {podaac_data_downloader.py:186} INFO - END

Harmony-request - FAIL - L2SS-py step does not succeed in Harmony UAT as expected : harmony UAT request: https://harmony.uat.earthdata.nasa.gov/C1261072648-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931%3A35.83342)&subset=lon(-125.57812%3A-124.73438)&subset=time(%222020-01-01T18%3A00%3A00%2B00%3A00%22%3A%222024-12-21T07%3A00%3A00%2B00%3A00%22)&skipPreview=true (see Harmony UAT Job Status : https://harmony.uat.earthdata.nasa.gov/jobs/b81a5429-84d8-4b1a-8582-dfc081edabbf) NOTE: the Harmony UAT request succeeded when using following request: https://harmony.uat.earthdata.nasa.gov/C1261072648-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931%3A35.83342)&subset=time(%222020-01-01T18%3A00%3A00%2B00%3A00%22%3A%222024-12-21T07%3A00%3A00%2B00%3A00%22)&skipPreview=true

SWOT_L2_LR_SSH_EXPERT_2.0 (C1261072646-POCLOUD):

podaac-data-subscriber - FAIL - see Harmony job status: https://harmony.earthdata.nasa.gov/jobs/5f65f5db-96f7-4085-ad02-a817cd7d5794 podaac-data-downloader - FAIL - podaac-data-downloader tool does NOT execute as expected - see Harmony job status: https://harmony.earthdata.nasa.gov/jobs/1e4c4cc3-566c-4dcc-b6ea-b39ba512cddd Harmony - request - FAIL - L2SS-py step does not succeed in Harmony UAT - see Job Status: https://harmony.uat.earthdata.nasa.gov/jobs/c4a544f6-c682-4ffe-8ad7-f7fb3f781c69) Request (9 granules) : https://harmony.uat.earthdata.nasa.gov/C1261072646-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2023-04-18T18:00:00%2B00:00":"2024-04-20T07:00:00%2B00:00")&skipPreview=true

NOTE: the Harmony UAT Request succeeded when using following request (25 granules): https://harmony.uat.earthdata.nasa.gov/C1261072646-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931%3A35.83342)&subset=time(%222020-01-01T18%3A00%3A00%2B00%3A00%22%3A%222024-12-21T07%3A00%3A00%2B00%3A00%22)&skipPreview=true see job status: https://harmony.uat.earthdata.nasa.gov/jobs/2fcb07f7-8368-4f9a-95ad-43d8cf0d7747

davidcolemanjpl commented 2 months ago

UAT L2SS-py v2.10.0rc6 - UAT (L2SS-py:2.9.0 OPS) podaac-data-subscriber v1.15.2 Harmony UAT / OPS v0.0.1017 EDSC UAT v24.2.1-1

SWOT_L2_LR_SSH_BASIC_2.0 (C1261072645-POCLOUD) podaac-data-subscriber - Partial PASS ( see https://harmony.earthdata.nasa.gov/jobs/26de72fa-ad35-458f-a2d4-11cabde514ce) podaac-data-downloader - Partial PASS ( see https://harmony.earthdata.nasa.gov/jobs/0c5a2f0d-9538-47ba-a3cc-883265bf44ff) Harmony - request - FAIL - (see job status https://harmony.uat.earthdata.nasa.gov/jobs/a6a0eebd-a2db-4298-8cfa-73994f246be9)
SWOT_L2_LR_SSH_EXPERT_2.0 (C1261072646-POCLOUD) podaac-data-subscriber - FAIL - (see job status https://harmony.earthdata.nasa.gov/jobs/228aed79-54f9-4bb2-90d4-888208522ab2) podaac-data-downloader - FAIL Harmony - request - FAIL - L2SS-py step does not succeed in Harmony UAT (see JobStatus: https://harmony.uat.earthdata.nasa.gov/jobs/ef11f3ab-bba2-4722-aa5d-bda8c70052e2)
SWOT_L2_LR_SSH_WINDWAVE_2.0 (C1261072648-POCLOUD) podaac-data-subscriber - FAIL - (See job status: https://harmony.earthdata.nasa.gov/jobs/9a677e38-b710-4685-9cd8-7b77f4561335) podaac-data-downloader - FAIL Harmony - request - FAIL - (See job status: https://harmony.uat.earthdata.nasa.gov/jobs/7386e840-c4ad-403c-9fc8-4957c0c84fc5)
SWOT_L2_NALT_GDR_GDR_2.0 (C1261072650-POCLOUD) podaac-data-subscriber - PASS podaac-data-downloader - PASS Harmony - request - PASS
SWOT_L2_NALT_GDR_SGDR_2.0 ( C1261072651-POCLOUD) podaac-data-subscriber - PASS podaac-data-downloader - PASS Harmony - request - PASS
SWOT_L2_NALT_GDR_SSHA_2.0 (C1261072652-POCLOUD) podaac-data-subscriber - PASS - podaac-data-subscriber tool executes as expected podaac-data-downloader - PASS - podaac-data-downloader tool executes as expected Harmony - L2SS-Py request - PASS - (see Harmony Job status: https://harmony.uat.earthdata.nasa.gov/jobs/d16e6a5e-7ce3-4d25-90b1-c53e0c90a781)
SWOT_L2_NALT_IGDR_GDR_2.0 (C1261072654-POCLOUD) podaac-data-subscriber - PASS - (see harmony Job status: see job status : https://harmony.earthdata.nasa.gov/jobs/0c4b50e6-31ab-4191-abd1-3ebe9abd7ddb) podaac-data-downloader - PASS - podaac-data-downloader tool executes as expected Harmony - request - FAIL - "Error: No matching granules found."
SWOT_L2_NALT_IGDR_SGDR_2.0 - (C1261072655-POCLOUD) podaac-data-subscriber - PASS - podaac-data-subscriber currently executes as expected podaac-data-downloader - PASS - podaac-data-downloader tool executes as expected Harmony - request - FAIL - "Error: No matching granules found."
SWOT_L2_NALT_IGDR_SSHA_2.0 (C1261072656-POCLOUD) podaac-data-subscriber - PASS podaac-data-downloader - PASS Harmony Request - FAIL - "Error: No matching granules found."
SWOT_L2_NALT_OGDR_GDR_2.0 (C1261072658-POCLOUD) podaac-data-subscriber - PASS - podaac-data-subscriber tool executes as expected podaac-data-downloader - PASS - podaac-data-downloader tool executes as expected Harmony Request - FAIL - "Error: No matching granules found."
SWOT_L2_NALT_OGDR_SSHA_2.0 (C1261072659-POCLOUD) podaac-data-subscriber - PASS - podaac-data-subscriber tool executes as expected podaac-data-downloader - PASS - podaac-data-downloader tool executes as expected Harmony Request - FAIL - "Error: No matching granules found."
SWOT_L2_RAD_GDR_2.0 (C1261072660-POCLOUD) podaac-data-subscriber -- Partial PASS - Job completed with error (see job status https://harmony.earthdata.nasa.gov/jobs/36ecbf98-2bcc-4a32-839a-c6b811af446f) podaac-data-downloader -- Partial PASS - Partial PASS - Job completed with error ( see https://harmony.earthdata.nasa.gov/jobs/2de0e209-ee15-4a27-ae5e-789a78cccd48) Harmony - request - PASS - see Harmony job status: (https://harmony.uat.earthdata.nasa.gov/jobs/5ac596ac-be1e-4f7e-b5da-bb4c9ded808b)
SWOT_L2_RAD_IGDR_2.0 (C1261072661-POCLOUD) podaac-data-subscriber - PASS podaac-data-downloader - PASS - podaac-data-downloader tool executes as expected Harmony - request - FAIL - "Error: No matching granules found."
SWOT_L2_RAD_OGDR_2.0 (C1261072662-POCLOUD) podaac-data-subscriber - PASS podaac-data-downloader - PASS Harmony - request - PASS - see Harmony JobID: 2c27f0c6-3122-4677-99d5-78a7dc8bf831

https://cae-testrail.jpl.nasa.gov/testrail/index.php?/tests/view/8825100&group_by=cases:title&group_order=asc&group_id=-1

davidcolemanjpl commented 2 months ago

OPS HiTIDE (OPS) UI-v4.17.1 l2ss-py:2.9.0 (OPS) concise:0.8.0 Harmony OPS v0.0.1017 EDSC v24.1.3-prod-database-1 / v24.1.3-prod-database-x-2 / v24.1.6-6

SWOT_L2_LR_SSH_BASIC_2.0 (C2799465428-POCLOUD): podaac-data-subscriber - Partial PASS ( see https://harmony.earthdata.nasa.gov/jobs/26de72fa-ad35-458f-a2d4-11cabde514ce) podaac-data-downloader - Partial PASS ( see https://harmony.earthdata.nasa.gov/jobs/0c5a2f0d-9538-47ba-a3cc-883265bf44ff) Harmony - request - FAIL - (see job status https://harmony.earthdata.nasa.gov/jobs/f86c4604-0123-4535-a74c-28ea746a0a6c)
SWOT_L2_LR_SSH_EXPERT_2.0 (C2799465497-POCLOUD) podaac-data-subscriber - FAIL - (see job status https://harmony.earthdata.nasa.gov/jobs/cd67854f-ed3f-4651-8619-b7b7d8e1e8fd) podaac-data-downloader - FAIL (https://harmony.earthdata.nasa.gov/jobs/919fd607-a898-45f9-a356-f282cc4b0da2) Harmony - request - FAIL - L2SS-py step does not succeed in Harmony UAT (see JobStatus: https://harmony.earthdata.nasa.gov/jobs/82e25975-4bee-414d-8c98-0fc8ddff514b)
SWOT_L2_LR_SSH_WINDWAVE_2.0 (C2799465507-POCLOUD): podaac-data-subscriber - FAIL - (see https://harmony.earthdata.nasa.gov/jobs/218673ea-9add-4540-ad5a-7928eecda17c) podaac-data-downloader - FAIL - (see JobStatus: https://harmony.earthdata.nasa.gov/jobs/4fe71a6e-e084-4b60-a77b-def81124c05a) Harmony-request - FAIL - L2SS-py steps do not succeed in Harmony OPS as expected : (see Harmony job status: https://harmony.earthdata.nasa.gov/jobs/9c7027e8-4314-49c7-905b-39d3696fd46a)
SWOT_L2_NALT_GDR_GDR_2.0 (C2799465509-POCLOUD) podaac-data-subscriber - PASS podaac-data-downloader - PASS Harmony - request - PASS
SWOT_L2_NALT_GDR_SGDR_2.0 ( C2799465518-POCLOUD) podaac-data-subscriber - PASS podaac-data-downloader - PASS Harmony - request - PASS
SWOT_L2_NALT_GDR_SSHA_2.0 - (C2799465522-POCLOUD) podaac-data-subscriber - PASS - podaac-data-subscriber tool executes as expected podaac-data-downloader - PASS - podaac-data-downloader tool executes as expected Harmony - L2SS-Py request - PASS - see Harmony Job status: https://harmony.earthdata.nasa.gov/jobs/15cc4e59-ba05-457f-b10e-4615980898dc
SWOT_L2_NALT_IGDR_GDR_2.0 (C2799465526-POCLOUD) podaac-data-subscriber -PASS podaac-data-downloader - PASS - podaac-data-downloader tool executes as expected Harmony - request - PASS - see job status: https://harmony.earthdata.nasa.gov/jobs/3e636895-c0e2-4645-a9e2-54c4135e3279
SWOT_L2_NALT_IGDR_SGDR_2.0 (C2799465529-POCLOUD) podaac-data-subscriber - PASS - podaac-data-subscriber currently executes as expected podaac-data-downloader - PASS - podaac-data-downloader tool executes as expected Harmony - request - PASS
SWOT_L2_NALT_IGDR_SSHA_2.0 (C2799465538-POCLOUD) podaac-data-subscriber - PASS - podaac-data-subscriber currently executes as expected podaac-data-downloader - PASS - podaac-data-downloader tool executes as expected Harmony - request - PASS
SWOT_L2_NALT_OGDR_GDR_2.0 ( C2799465542-POCLOUD) podaac-data-subscriber - PASS - podaac-data-subscriber tool executes as expected podaac-data-downloader - PASS - podaac-data-downloader tool executes as expected Harmony Request - PASS
SWOT_L2_NALT_OGDR_SSHA_2.0 (C2799465544-POCLOUD)) podaac-data-subscriber - PASS - podaac-data-subscriber tool executes as expected podaac-data-downloader - PASS - podaac-data-downloader tool executes as expected Harmony Request - PASS ( see job status: https://harmony.earthdata.nasa.gov/jobs/7f530aa0-77e3-46a6-94a2-8f4213fa283d)
SWOT_L2_RAD_GDR_2.0 (C2799438350-POCLOUD) podaac-data-subscriber -- PASS
podaac-data-downloader -- PASS
Harmony - request - PASS - but NOTE: the request FAILs when CONCISE is enabled: see Harmony job status: https://harmony.earthdata.nasa.gov/jobs/a9f23ee8-7ecb-47fc-8d50-f433a4af658b
SWOT_L2_RAD_IGDR_2.0 (C2799438351-POCLOUD) podaac-data-subscriber - PASS podaac-data-downloader - PASS Harmony - request - PASS
SWOT_L2_RAD_OGDR_2.0 (C2799438353-POCLOUD) podaac-data-subscriber - PASS podaac-data-downloader - PASS Harmony - request - PASS - see Harmony JobID: 216eaa9c-f4c0-49ad-a37b-adb6432202db

sliu008 commented 1 month ago

I have ran some of these swot collections and some granules succeed and some fails, what i notice is that those that fails also have a bounding box. So most likely is that the data being subsetted isn't falling in the bounding box and creates an empty granule. Although it still subset the resulting granule seem to make harmony throw an error. Will talk to harmony to see why that is .

jamesfwood commented 1 month ago

Hi @davidcolemanjpl Can you please test these again? Also update the HiTIDE Doctor Testing table. We updated l2ss-py to 2.10 in OPS and UAT. Thanks!

davidcolemanjpl commented 1 month ago

OPS HiTIDE (OPS) UI-v4.17.1 l2ss-py v2.10.0 (OPS) concise v0.8.0 Harmony (OPS) v0.0.1036 EDSC (OPS) v24.2.2-6

SWOT_L2_LR_SSH_BASIC_2.0 (C2799465428-POCLOUD):

podaac-data-subscriber - PASS ( see job status https://harmony.earthdata.nasa.gov/jobs/6df1d4b1-5da6-4b3d-8904-b578e631939a) podaac-data-downloader - PASS ( see https://harmony.earthdata.nasa.gov/jobs/d883a3e2-dba3-45d7-ba42-3af499652e5a)

Harmony - API request - FAIL - (see job status https://harmony.earthdata.nasa.gov/jobs/c2f20760-20e9-46df-9288-1e59f480cb6d) - 10 of 43 granules failed subsetting

Try using API request: https://harmony.earthdata.nasa.gov/C2799465428-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2023-01-01T18:00:00%2B00:00":"2024-12-21T07:00:00%2B00:00")&skipPreview=true

(note: same results occur if/ when using a curl request and submitting to Harmony)

Harmony API job-completed with errors-1

UAT l2ss-py:2.10.0rc8 concise:0.9.0rc4 Harmony UI (UAT): v0.0.1043 EDSC - UAT - v24.2.3-2

SWOT_L2_LR_SSH_BASIC_2.0: (C1261072645-POCLOUD) podaac-data-subscriber - PASS ( see https://harmony.earthdata.nasa.gov/jobs/d9e913a5-a552-4b9b-95cb-6705c01bc032) podaac-data-downloader - PASS Harmony - API request - FAIL - (see job status https://harmony.uat.earthdata.nasa.gov/jobs/182a6929-fe0e-46cf-85ec-eb1cba2b8767) Request (3 granules): https://harmony.uat.earthdata.nasa.gov/C1261072645-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2023-11-15T18:00:00%2B00:00":"2023-11-18T07:00:00%2B00:00")&skipPreview=true

note: Harmony - API request works okay for SWOT_L2_LR_SSH_BASIC_2.0, L2SS-Py steps were successful in Harmony UAT when the subset=lon info removed from Harmony API request: see Harmony job status: https://harmony.uat.earthdata.nasa.gov/jobs/01629f5b-e9e4-4c52-a08f-152857ce5633

davidcolemanjpl commented 1 month ago

Harmony (OPS) - v 0.0.1043 Data subscriber - v1.15.2 L2SS-Py - v2.10.0 / concise v0.8.0

Harmony (UAT)- v0.0.1043 L2SS-Py v2.10.0rc8

OPS:

SWOT_L2_LR_SSH_EXPERT_2.0 (C2799465497-POCLOUD):

podaac-data-subscriber - PASS - (see https://harmony.earthdata.nasa.gov/jobs/89cce66d-b83f-404d-8067-827a35480170) podaac-data-downloader - PASS - (see job status https://harmony.earthdata.nasa.gov/jobs/adc60907-8d1f-40ff-a47d-1bfb017f56bf) - 7 of 7 granules passed L2SS-py steps Harmony-API request - Partial PASS - all L2SS-py steps do not succeed in Harmony OPS, 1 of 3 granules passed: Request: https://harmony.earthdata.nasa.gov/C2799465497-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time("2024-03-24T18:00:00%2B00:00":"2024-04-01T07:00:00%2B00:00")&skipPreview=true&concatenate=true (see Harmony job status:https://harmony.earthdata.nasa.gov/jobs/b9bd1a29-9b75-4356-9e34-0be37a01d0b4)

UAT:

SWOT_L2_LR_SSH_Expert (C1261072646-POCLOUD): Harmony - API request - FAIL - '9' of '9' granules Failed (See job status: https://harmony.uat.earthdata.nasa.gov/jobs/cdaf9426-8225-46f4-bdcc-4d8c00ac6c20)

OPS:

SWOT_L2_LR_SSH_WINDWAVE_2.0 (C2799465507-POCLOUD):

podaac-data-subscriber - PASS - (see https://harmony.earthdata.nasa.gov/jobs/0f91c36c-380c-4ac6-bf33-2916e337195a) podaac-data-downloader - PASS - (see job status https://harmony.earthdata.nasa.gov/jobs/92d4f269-cc19-44a7-9791-add63217dc4e) - 7 of 7 granules passed L2SS-py steps Harmony-API request - Partial PASS - all L2SS-py steps do not succeed in Harmony OPS as expected, 1 of 3 granules passed (see Harmony job status:https://harmony.earthdata.nasa.gov/jobs/70a1eee2-9662-4165-bd9c-12ad85e63155)

UAT:

SWOT_L2_LR_SSH_WINDWAVE_2.0 (C1261072648-POCLOUD)

Harmony - API request - FAIL ( 3 granules): Request: https://harmony.uat.earthdata.nasa.gov/C1261072645-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(34.84931:35.83342)&subset=lon(-125.57812:-124.73438)&subset=time(%222023-11-15T18:00:00%2B00:00%22:%222023-11-18T07:00:00%2B00:00%22)&skipPreview=true&concatenate=true (See job status:https://harmony.uat.earthdata.nasa.gov/jobs/5baf5fb2-209e-40c4-a728-1abb0dbed992)