DalgoT4D / webapp

GNU Affero General Public License v3.0
5 stars 35 forks source link

Ylabs setup issues- Transform page selection #821

Closed nairabhishek73 closed 1 month ago

nairabhishek73 commented 2 months ago

Selection not showing up for Y labs.

Ensure this is not happening for all orgs on beta, demo, or production

nairabhishek73 commented 1 month ago

Will be looked into this week and try to solution for it. Related to prefect.

fatchat commented 1 month ago
  1. What can cause this to fail, list the steps (in django, in proxy, in prefect)
  2. How does one handle an error, what do we report to the user, how do we roll back our changes

for (2) we could just reset the entire dbt workspace. this would reset the screen, so the user should be shown a message, with a button to acknowledge they have seen it, and that button triggers the workspace reset

fatchat commented 1 month ago

step 1: post to transform/dbt_project/ - this creates the dbt project directory on disk

step 2: post to prefect/tasks/transform/ - this is the step which fails because of prefect

step 3: post to transform/dbt_project/sync_sources/

if step 2 fails then running step 1 again yields "the project already exists"

-- solution 1: roll back step 1 if step 2 fails

solution 2: let the flow continue if step 1 fails with "the project already exists"

fatchat commented 1 month ago

solution 2 doesn't work because the flow starts with the user selecting github-vs-ui, and we can't guarantee that they'll choose ui again