search

LewisDavies / upstream-prod

A dbt package for easily using production data in a development environment.
32 stars 4 forks source link

--no-partial-parse flag fails with this library enabled on branch #23

Open alittlesliceoftom opened 10 months ago

alittlesliceoftom commented 10 months ago

Hi @LewisDavies,

Firstly huge thanks on your work pulling this package together, it's a great tool and helps us with workflows!

A small bug I noticed whilst trying to get this working inside a CI workflow:

dbt --no-partial-parse compile -t dev_ci Fails, where: dbt compile -t dev_ci Works fine.

The error is like this: image

This actually happens on basically all the models (try with a dbt -d --no-partial-parse compile -t dev_ci to see a full error set.

In this case dev_ci is a target with upstream prod enabled.

Is this behaviour known? If so I could raise a small PR to the README to document it. Else, any ideas?!

alittlesliceoftom commented 10 months ago

P.s. dbt core - 1.3.0 LewisDavies/upstream_prod 0.5.1

Running on synapse (hence the laggardly dbt version :( )

alittlesliceoftom commented 10 months ago

Update, the same failure occurs when the argument is disabled, but on the CI pipe, i.e. I assume that the project hasn't compiled before.

LewisDavies commented 7 months ago

Hey @alittlesliceoftom, sorry for the big delay in responding. Appreciate the kind words and I'm glad this package is useful!

Someone else had the same error when using the dbt-unit-testing package; if you're using that package or custom tests, that's probably the cause. This was fixed in v0.6.2 but that version requires dbt v1.5 or later.

If you're still having this problem, I recommend forking the repo and applying the above changes to ref.sql to the v0.5.1 code. Unfortunately I don't have the time to try backporting this fix myself, but let me know if this doesn't work and I'll try to support where I can.

alittlesliceoftom commented 2 months ago

Hi @LewisDavies ,

I can confirm that a local edit of the ref function with those lines fixes up the issue for me... I'm not sure a backport is worthwhile as v1.4 which we're on is EOL - so clearly we should just upgrade! Thanks for looking into it!