crim-ca / weaver

Weaver: Workflow Execution Management Service (EMS); Application, Deployment and Execution Service (ADES); OGC API - Processes; WPS; CWL Application Package
https://pavics-weaver.readthedocs.io
Apache License 2.0
24 stars 6 forks source link

Substitute `parse_cwl_array_type` with `get_cwl_io_type` in `processes.wps_package.WpsPackage.make_inputs` #546

Closed francisPLT closed 1 year ago

francisPLT commented 1 year ago

This PR aims to fix a problem that was encountered while uploading a process to Weaver that featured an allowed values list in one of it's inputs:

inputs:
  command:
    type:
      type: enum
      symbols:
        - value-1
        - value-2
        - etc.

Substituting the parse_cwl_array_type function with the get_cwl_io_type function in the WpsPackage.make_inputs function fixes the issue completely and doesn't seem to have adverse effects in my live tests using a local Weaver instance.

However, the following tests are now failing and will need to be addressed before merging this PR:

FAILED tests/functional/test_builtin.py::BuiltinAppTest::test_jsonarray2netcdf_execute_async - AssertionError: Error job info:
FAILED tests/functional/test_builtin.py::BuiltinAppTest::test_jsonarray2netcdf_execute_async_output_by_reference_dontcare_response_document - AssertionError: Error job info:
FAILED tests/functional/test_builtin.py::BuiltinAppTest::test_jsonarray2netcdf_execute_async_output_by_reference_response_raw - AssertionError: Error job info:
FAILED tests/functional/test_builtin.py::BuiltinAppTest::test_jsonarray2netcdf_execute_async_output_by_value_response_raw - AssertionError: Error job info:
FAILED tests/functional/test_builtin.py::BuiltinAppTest::test_jsonarray2netcdf_execute_sync - AssertionError: Error: {"$schema":"https://schemas.opengis.net/ogcapi/processes/part1/1.0/openapi/schemas/statusInfo.yaml","jobID":"434e88ad-18e8-431b-95ad-9e170b3bd6fe","processID":"jsonarray2netcdf","providerID":...
FAILED tests/functional/test_cli.py::TestWeaverClient::test_execute_inputs_cwl_file_schema_auto_resolve_vault - AssertionError: Could not retrieve any output results from job.
FAILED tests/functional/test_cli.py::TestWeaverClient::test_execute_inputs_cwl_literal_schema_auto_resolve_vault - AssertionError: Could not retrieve any output results from job.
FAILED tests/functional/test_cli.py::TestWeaverClient::test_execute_inputs_ogc_mapping_file_schema_auto_resolve_vault - AssertionError: Could not retrieve any output results from job.
FAILED tests/functional/test_cli.py::TestWeaverClient::test_execute_inputs_ogc_mapping_literal_schema_auto_resolve_vault - AssertionError: Could not retrieve any output results from job.
FAILED tests/functional/test_cli.py::TestWeaverClient::test_execute_inputs_old_listing_file_schema_auto_resolve_vault - AssertionError: Could not retrieve any output results from job.
FAILED tests/functional/test_cli.py::TestWeaverClient::test_execute_inputs_old_listing_literal_schema_auto_resolve_vault - AssertionError: Could not retrieve any output results from job.
FAILED tests/functional/test_cli.py::TestWeaverClient::test_execute_inputs_representation_literal_schema_auto_resolve_vault - AssertionError: Could not retrieve any output results from job.
FAILED tests/functional/test_cli.py::TestWeaverClient::test_execute_manual_monitor_status_and_download_results - AssertionError: {
FAILED tests/functional/test_cli.py::TestWeaverCLI::test_deploy_docker_auth_username_password_valid - AssertionError: process returned with error code: -1
FAILED tests/functional/test_cli.py::TestWeaverCLI::test_execute_remote_input - AssertionError: process returned with error code: -1
FAILED tests/functional/test_docker_app.py::WpsPackageDockerAppTest::test_execute_rest_xml_post_resp_json - AssertionError: Error job info:
FAILED tests/functional/test_docker_app.py::WpsPackageDockerAppTest::test_execute_wps_kvp_get_resp_json - AssertionError: Error job info:
FAILED tests/functional/test_docker_app.py::WpsPackageDockerAppTest::test_execute_wps_kvp_get_resp_xml - AssertionError: Error job info:
FAILED tests/functional/test_docker_app.py::WpsPackageDockerAppTest::test_execute_wps_rest_resp_json - AssertionError: Error job info:
FAILED tests/functional/test_docker_app.py::WpsPackageDockerAppTest::test_execute_wps_xml_post_resp_json - AssertionError: Error job info:
FAILED tests/functional/test_docker_app.py::WpsPackageDockerAppTest::test_execute_wps_xml_post_resp_xml - AssertionError: Error job info:
FAILED tests/functional/test_workflow.py::WorkflowTestCase::test_workflow_directory_input_output_chaining - RuntimeError: --------------------------------------------------------------------------------
FAILED tests/functional/test_workflow.py::WorkflowTestCase::test_workflow_mixed_rest_builtin_wps1_docker_scatter_requirements - RuntimeError: --------------------------------------------------------------------------------
FAILED tests/functional/test_workflow.py::WorkflowTestCase::test_workflow_mixed_rest_builtin_wps1_docker_select_requirements - RuntimeError: --------------------------------------------------------------------------------
FAILED tests/functional/test_workflow.py::WorkflowTestCase::test_workflow_mixed_wps1_builtin_rest_docker_scatter_requirements - RuntimeError: --------------------------------------------------------------------------------
FAILED tests/functional/test_wps_package.py::WpsPackageAppTest::test_execute_file_type_io_format_references - AssertionError: Error job info:
FAILED tests/functional/test_wps_package.py::WpsPackageAppTest::test_execute_job_with_array_input - AssertionError: assert {'basename': ..._s3.txt', ...} == 'THIS IS A GE...E FOR S3 TEST'
FAILED tests/functional/test_wps_package.py::WpsPackageAppTest::test_execute_job_with_custom_file_name - AssertionError: Error job info:
FAILED tests/functional/test_wps_package.py::WpsPackageAppTest::test_execute_job_with_inline_input_values - AssertionError: assert {'basename': ...u5.json', ...} == {'VALUE': {'M..., 'UOM': 'M'}}
FAILED tests/functional/test_wps_package.py::WpsPackageAppTest::test_execute_with_browsable_directory - AssertionError: Error job info:
FAILED tests/functional/test_wps_package.py::WpsPackageAppTest::test_execute_with_bucket_directory - AssertionError: Error job info:
FAILED tests/functional/test_wps_package.py::WpsPackageAppTest::test_execute_with_json_listing_directory - AssertionError: Error job info:
fmigneault commented 1 year ago

@francisPLT Found the cause of the problem.

codecov[bot] commented 1 year ago

Codecov Report

Merging #546 (c3ab558) into master (de9fba2) will increase coverage by 0.00%. The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master     #546   +/-   ##
=======================================
  Coverage   84.46%   84.46%           
=======================================
  Files          79       79           
  Lines       17703    17704    +1     
  Branches     2685     2685           
=======================================
+ Hits        14952    14953    +1     
  Misses       2022     2022           
  Partials      729      729           
Files Changed Coverage Δ
weaver/processes/convert.py 84.19% <100.00%> (+<0.01%) :arrow_up:
weaver/processes/wps_package.py 84.05% <100.00%> (ø)