search

vmware / versatile-data-kit

One framework to develop, deploy and operate data workflows with Python and SQL.
Apache License 2.0
428 stars 57 forks source link

vdk-server: user errors result in missing frontend logs #2493

Open DeltaMichael opened 1 year ago

DeltaMichael commented 1 year ago

Describe the bug Data job runs resulting in user errors do not show logs in frontend.

Steps To Reproduce Steps to reproduce the behavior:

  1. vdk server -i
  2. Copy this data job https://github.com/DeltaMichael/vdk-dagster-comparison/tree/main/dagster-hello-in-vdk/hackernews-top-stories
  3. cd into job directory
  4. Change the file name to a file that does not exist https://github.com/DeltaMichael/vdk-dagster-comparison/blob/main/dagster-hello-in-vdk/hackernews-top-stories/20_hackernews_top_stories.py#L15
  5. Run vdk deploy -n hackernews-top-stories -t dagster -p .
  6. Go to http://localhost:8092/manage/data-jobs/dagster/hackernews-top-stories/details
  7. Open the logs for the failed run

Expected behavior There should be logs

Actual behavior

No logs show, just an empty json

Screenshots vdk-error

DeltaMichael commented 1 year ago

Do https://github.com/vmware/versatile-data-kit/issues/2042 before this one