updates for CWL CUDA, Resources, NetworkAccess requirements

[fmigneault]

Changes

• Update documentation with examples for cwltool:CUDARequirement, ResourceRequirement and NetworkAccess.
• Improve schema definition of ResourceRequirement.
• Deprecate DockerGpuRequirement, with attempts to auto-convert it into corresponding DockerRequirement combined with cwltool:CUDARequirement definitions. If this conversion does not work transparently for the user, explicit CWL updates with those definitions should be made.
• Ensure that validation check finds exactly one provided CWL requirement or hint to represent the application type. In case of missing requirement, the reported error will contain a documentation link to guide the user in adjusting its Application Package accordingly.

Refernces

• Temporary documentation link used due to https://github.com/common-workflow-language/user_guide/issues/414
• Relates to common-workflow-language/cwl-v1.2#138
• Fixes https://crim-ca.atlassian.net/browse/DAC-437

TODO

• [x] tests for new _update_package_compatibility operation
• [x] tests for result ResourceRequirement schema validation
• [x] documentation of CWL-like hint/requirement for remote process

[codecov[bot]]

Codecov Report

Merging #506 (96c49c0) into master (1a92e1c) will increase coverage by 0.08%. The diff coverage is 97.50%.

@@            Coverage Diff             @@
##           master     #506      +/-   ##
==========================================
+ Coverage   83.74%   83.82%   +0.08%     
==========================================
  Files          78       78              
  Lines       16256    16349      +93     
  Branches     2509     2525      +16     
==========================================
+ Hits        13614    13705      +91     
- Misses       1945     1947       +2     
  Partials      697      697              

Impacted Files	Coverage Δ
weaver/processes/convert.py	83.41% <ø> (ø)
weaver/store/mongodb.py	76.39% <ø> (ø)
weaver/processes/wps_package.py	82.50% <94.33%> (+0.50%)	:arrow_up:
weaver/datatype.py	75.91% <100.00%> (+0.17%)	:arrow_up:
weaver/exceptions.py	92.03% <100.00%> (+0.07%)	:arrow_up:
weaver/processes/builtin/__init__.py	88.79% <100.00%> (ø)
weaver/processes/constants.py	100.00% <100.00%> (ø)
weaver/wps_restapi/swagger_definitions.py	99.92% <100.00%> (+<0.01%)	:arrow_up:
weaver/processes/utils.py	80.58% <0.00%> (ø)
... and 1 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[mr-c]

• Deprecate DockerGpuRequirement, with attempts to auto-convert it into corresponding DockerRequirement combined with cwltool:CUDARequirement definitions. If this conversion does not work transparently for the user, explicit CWL updates with those definitions should be made.

I'm glad to see this and I'll eventually add similar code to cwl-upgrader when CUDARequirement is part of a released version of CWL (likely v1.3).

What is the namespace for the DockerGpuRequirement extension?

[fmigneault]

@mr-c

What is the namespace for the DockerGpuRequirement extension?

There was no specific namespace employed. It was only used in hints and warnings were ignored. Unless someone copied the idea, it should only exist in Weaver.

[fmigneault]

Merging as is for the moment. The issue discussed in https://github.com/crim-ca/weaver/pull/506#pullrequestreview-1199302977 (allow CUDARequirement in requirements section) will be handled by subsequent PR #510.