search

sodadata / soda-sql

Soda SQL and Soda Spark have been deprecated and replaced by Soda Core. docs.soda.io/soda-core/overview.html
https://docs.soda.io/
Apache License 2.0
61 stars 16 forks source link

MarkUpSafe version conflict for BigQuery #210

Closed albinkjellin closed 2 years ago

albinkjellin commented 2 years ago

Describe the bug I'm experiencing an issue while using soda-sql on a PythonVirtualEnv on Airflow. From what I've seen, when the code does pip install soda-sql-bigquery>=2.1&&<3' there's a library called MarkUpSafe, which has updated to version 2.1.0 and according to their patch notes https://markupsafe.palletsprojects.com/en/2.1.x/changes/#version-2-1-0 there's no longe soft_unicode, which was previously deprecated and soda library is still using it.

To Reproduce Steps to reproduce the behavior:

  1. Create a new test in scan.yml
  2. Run soda scan ... 3 ...

Context Include your scan.yml or warehouse.yml when relevant

OS: Python Version: Soda SQL Version: Warehouse Type:

albinkjellin commented 2 years ago

As a workaround toworkaround, its possible to pass "MarkupSafe==2.0.1" as a requirement before "soda-sql-bigquery>=2.1&&<3".

mahdiqb commented 2 years ago

I think that the problem is the "Jinja2>=2.11.3, <3.0" dependency for soda-core, the 2.11.3 Jinja version simply has MarkupSafe>=0.23 in its requirements, which made it vulnerable to the breaking change.

The thing is that this is also used for the Soda dbt package (dbt has a strict dependency Jinja2==2.11.3), and so for Soda to update the Jinja dependency dbt needs to do the same first to avoid conflicts, but it seems that it'll be a complex operation based on this issue.

guillesd commented 2 years ago

Same here, this is quite a problem. The root issue is that markupsafe latest release (2.1.0) has removed soft_unicode