This PR proposes a complete refactor of the soda_scan_execute task.
Using Soda Scan APIs seems not feasible, due to this issue.
To overcome the issue, the task has been rewritten to use Soda CLI and prefect-shell task.
⚠️ prefect_shell.shell_run_command and prefect_soda_core.soda_scan_execute are async methods, for which I wasn't able to write working tests.
Help on writing tests would be much appreciated!
Summary
This PR proposes a complete refactor of the
soda_scan_executetask. Using Soda Scan APIs seems not feasible, due to this issue. To overcome the issue, the task has been rewritten to use Soda CLI andprefect-shelltask.⚠️
prefect_shell.shell_run_commandandprefect_soda_core.soda_scan_executeare async methods, for which I wasn't able to write working tests. Help on writing tests would be much appreciated!