Open fabientra opened 1 month ago
Thanks for reporting this @fabientra and identifying the originating PR 👑
I was able to reproduce this (see details in "Reprex" below).
A possible fix might be leaving msg
as-if here unless there were any secrets scrubbed:
def scrub_secrets(msg: str, secrets: List[str]) -> str:
scrubbed = str(msg)
for secret in secrets:
scrubbed = scrubbed.replace(secret, "*****")
return msg if str(msg) == scrubbed else scrubbed
i.e. just replace the return statement here with this:
return msg if str(msg) == scrubbed else scrubbed
Of course you'll probably still get an error during dbt retry
if you included a secret within your --vars
, but that's acceptable because:
1) we'd hope folks don't include secrets in --vars
, and
2) we explicitly don't want to write out secrets in plain text (which would be required for retry).
For completeness, we'd probably want to update the type hints as well:
def scrub_secrets(msg: Any, secrets: List[str]) -> Any:
Thank you for the fix @dbeatty10 :)
Is this a regression in a recent version of dbt-core?
Current Behavior
Our
dbt retry
using a variable from CLI of type INT are failing because during the retry dbt sees it as STR.I compared the
run_results.json
from dbt 1.8 torun_results.json
from dbt 1.7 and I can see that the one from 1.8 have all the vars as STR whereas from 1.7 they kept their original type.Expected/Previous Behavior
vars
inrun_results.json
should keep the same type as the one give through the CLISteps To Reproduce
dbt run
command with a variable of type intRelevant log output
No response
Environment
Which database adapter are you using with dbt?
redshift
Additional Context
I believe the regression is caused by this PR that applies the scrubbing function to ALL cli variables whereas this function cast its input to str systematically