Support jsonschema 3.2.0

[kdeggelman]

My apologies for opening an issue without having more insight into the feasibility of updating the version of this package. I'm happy to take a look at the changelog of jsonschema to see if there are any breaking changes from 3.1.1 and 3.2.0.

For context, we'd like to use airflow-dbt to run dbt on Airflow using Google's Cloud Composer. Composer is pretty strict on making sure all package versions line up and something else in the Composer environment requires jsonschema 3.2.0.

[groodt]

I'm seeing this issue too.

[gudata]

The same is in on the AWS Managed Airflow.

[andrewhharmon]

We face a similar problem with library conflicts. Multiple libraries requiring jsonschema 3.2.0

[felippecaso]

Mentioning from dbt's Slack that the jsonschema version also limits integrating dbt2looker with dbt v0.21.0 within the same env

[4sushi]

Hello, same issue with Airflow 2.1.4 (on Google Cloud Composer v2) and Airflow 1.10.15 (on Google Cloud Composer v1). I tested with several version of dbt (1.0, 0.22, etc..).

hologram 0.0.14 has requirement jsonschema<3.2,>=3.0, but you have jsonschema 3.2.0

That means DBT do not work with Airflow on Google Composer natively (installation with Composer pypi packages). It's interesting that nobody mentions this on technical blogs, since it becomes a popular product. There are some workarounds to solve the problem, but still.

[praveer-k]

It will be great if this issue can be resolved. I am also facing the same problem.

[adamantike]

This has been fixed some months ago in #45. Can we get a new library release so we can benefit from that change?

[volkangurel]

+1, we would really like this fixed as well. Can we please get a release of the latest master branch?

[hkc-8010]

Hi Team,

This issue seems to be open for more than a year now. Can we please get this released asap?

We are not able to use dbt-snowflake with apache-airflow-2.3.0 due to following dependency conflict.

The conflict is caused by:
    apache-airflow 2.3.0+astro.1 depends on jsonschema<5.0 and >=3.2.0
    flask-appbuilder 3.4.5 depends on jsonschema<5 and >=3
    hologram 0.0.14 depends on jsonschema<3.2 and >=3.0

[jtcohen6]

Hi everyone, we've released hologram==0.15.0 which includes the loosened jsonschema pin. dbt-core==1.1.1 (currently available as RC2) will include compatibility with hologram>=0.0.14,<=0.0.15, and thereby jsonschema<4.0,>=3.0.

[smackesey]

Could you consider raising this to jsonschema<=5.0? Better yet, please consider dropping the upper bound pin altogether. Users can always fix an incompatibility with their own pin, but have no ability to fix an overstrict pin (which is what preemptive upper bound pins usually end up being).

[groodt]

Yes, would love some insight into why the upper pin exists.

[jtcohen6]

@smackesey @groodt We don't make changes in this codebase frequently, so I can't promise any immediate action here. I do see that jsonschema==4.0.0 was first released in September 2021.

Is this causing you immedaite issues when trying to install dbt-core in the same environment as another project?

I can share that we're thinking more about our policies & approaches to dependencies more broadly: https://github.com/dbt-labs/dbt-core/discussions/6495

In the meantime, I'm going to close this specific issue, since it's been long resolved.

[groodt]

Thanks for the response. Yes, this particular issue (i.e. jsonschema==3.2.0) has been resolved. I can raise a new issues for jsonschema>=4.

Is this causing you immedaite issues when trying to install dbt-core in the same environment as another project?

It does make the installation of dbt more painful for us, yes. Our environment also has jupyterlab and newer versions of jupyterlab_server require newer versions of jsonschema.

I took a look at your draft for dependency management. I have a few concerns with both options proposed (they're generally not following best-practices for python package dependency management). Is it best to comment there or wait for it to be open for comment?

[jtcohen6]

Is it best to comment there or wait for it to be open for comment?

As with any of our GitHub discussions, please consider it open for comment!