Open daw3rd opened 1 month ago
revit13
commented
1 month ago It is added to the kfp black list in https://github.com/IBM/data-prep-kit/pull/684. Additionally, the kfp_ray/Makefile has been restored to enable local testing and to verify the workflow build in the CI/CD pipeline.
revit13
commented
2 weeks ago With https://github.com/IBM/data-prep-kit/pull/781 the pipeline still fails with:
03:46:04 INFO - Getting supported licenses from file test/license_select/sample_approved_licenses.json
03:46:04 ERROR - Error reading file test/license_select/sample_approved_licenses.json: [Errno 2] No such file or directory: 'test/license_select/sample_approved_licenses.json'
03:46:04 ERROR - Failed to read file: test/license_select/sample_approved_licenses.json due to [Errno 2] No such file or directory: 'test/license_select/sample_approved_licenses.json'
03:46:04 INFO - pipeline id runtime_pipeline_id
Search before asking
Component
Transforms/Other
What happened + What you expected to happen
With the recent PR to properly enabled kfp workflow tests (PR https://github.com/IBM/data-prep-kit/pull/672) on transforms, we found that repo_level_ordering was not passing (see log here). To get the PR to pass,
license_select/kfp_ray/Makefilewas renamed toMakefile.disable-cicd.Reproduction script
To reproduced
cd transforms/code/license_select/kfp_ray git mv Makefile.disable-cicd Makefile git commit -s -a -m "restore kfp_ray test for license_select" git push
Make a PR and watch the cicd
Anything else
Image: "quay.io/dataprep1/data-prep-kit/kfp-data-processing:latest" with ID "sha256:37d2e98f3b1b4f406f20254b3ae5e96a1081bd731917aa62848a95bcadda080c" not yet present on node "dataprep-worker2", loading... Image: "quay.io/dataprep1/data-prep-kit/kfp-data-processing:latest" with ID "sha256:37d2e98f3b1b4f406f20254b3ae5e96a1081bd731917aa62848a95bcadda080c" not yet present on node "dataprep-control-plane", loading... Image: "quay.io/dataprep1/data-prep-kit/kfp-data-processing:latest" with ID "sha256:37d2e98f3b1b4f406f20254b3ae5e96a1081bd731917aa62848a95bcadda080c" not yet present on node "dataprep-worker", loading... make[3]: Leaving directory '/home/runner/work/data-prep-kit/data-prep-kit/kfp/kfp_ray_components' . /home/runner/work/data-prep-kit/data-prep-kit/transforms/code/license_select/kfp_ray/../../../..//transforms/venv/bin/activate && python -m workflow_support.pipeline_utils.pipelines_tests_utils -c "sanity-test" -p /home/runner/work/data-prep-kit/data-prep-kit/transforms/code/license_select/kfp_ray/license_select_wf.yaml -e "http://localhost:8080/" 12:25:34 WARNING - Failure to get pipeline. Number of pipelines with name license_select_wf is 0 12:25:34 INFO - Pipeline uploaded 12:25:34 INFO - Pipeline run license_select_wf 2024_10_05_12_25_34 submitted 12:26:34 INFO - Got pipeline execution status Running 12:27:34 INFO - Got pipeline execution status Running 12:28:34 INFO - Got pipeline execution status Running 12:29:34 INFO - Got pipeline execution status Running 12:30:34 INFO - Got pipeline execution status Running 12:31:34 INFO - Got pipeline execution status Running 12:32:35 INFO - Got pipeline execution status Running 12:33:35 INFO - Got pipeline execution status Failed 12:33:35 WARNING - Pipeline license_select_wf failed with error None and status Failed make[2]: [/home/runner/work/data-prep-kit/data-prep-kit/transforms/code/license_select/kfp_ray/../../../..//transforms/.make.workflows:46: .workflows.test-pipeline] Error 1 make[1]: [Makefile:45: workflow-test] Error 2 make[2]: Leaving directory '/home/runner/work/data-prep-kit/data-prep-kit/transforms/code/license_select/kfp_ray' make[1]: Leaving directory '/home/runner/work/data-prep-kit/data-prep-kit/transforms/code/license_select/kfp_ray' make: *** [Makefile:56: workflow-test] Error 2 make: Leaving directory '/home/runner/work/data-prep-kit/data-prep-kit/transforms/code/license_select' Error: Process completed with exit code 2.
OS
Ubuntu
Python
3.10.x
Are you willing to submit a PR?