search

vmware / versatile-data-kit

One framework to develop, deploy and operate data workflows with Python and SQL.
Apache License 2.0
434 stars 57 forks source link

vdk_exit plugin receives wrong exit code #894

Open antoniivanov opened 2 years ago

antoniivanov commented 2 years ago

Describe the bug

vdk_exit is a called just before CLI exits with correct exit code by contract. But it does not work always

Steps To Reproduce

  1. vdk run empty-job
  2. Observer vdk_exit is called with code "0"
  3. but the data job fails with code 1

Expected behavior vdk_exit is invoked with correct status code (in this case 1)

Screenshots If applicable, add screenshots to help explain your problem.

Version (please complete the following information): OS: Mac Version: Versatile Data Kit (VDK) Version: 0.3.572339461 Build details: RELEASE_VERSION=0.3.572339461, BUILD_DATE=Fri Jun 24 14:57:27 UTC 2022, BUILD_MACHINE_INFO=Linux runner-opyxurxq-project-28359933-concurrent-0wcg7j 5.4.181-99.354.amzn2.x86_64 #1 SMP Wed Mar 2 18:50:46 UTC 2022 x86_64 GNU/Linux, GITLAB_CI_JOB_ID=2637062491, GIT_COMMIT_SHA=e8e2359b407acbef5982e190e38c9a9af51fa2b2, GIT_BRANCH=main Python version: 3.7.9 64bit (/Users/aivanov/.pyenv/versions/3.7.9/envs/sc-oss-vdk37/bin/python3)

Installed plugins: vdk-kerberos-auth (from package vdk-kerberos-auth, version 0.3.520417292) vdk-impala (from package vdk-impala, version 0.3.520417292) vdk-ingest-file (from package vdk-ingest-file, version 0.1.461192871) vdk-server (from package vdk-server, version 0.1.487944116) vdk-lineage (from package vdk-lineage, version 0.1.516951482) vdk-marquez (from package vdk-lineage, version 0.1.516951482) vdk-ingest-http (from package vdk-ingest-http, version 0.2.477708478) vdk-control-service-properties (from package vdk-plugin-control-cli, version 0.1.527956211) vdk-plugin-control-cli (from package vdk-plugin-control-cli, version 0.1.527956211) vdk-csv (from package vdk-csv, version 0.1.477708478)

Additional context Add any other context about the problem here. Consider attaching logs if there are any.

antoniivanov commented 1 year ago

Will try to reproduce