dbt-labs / hologram

A library for automatically generating Draft 7 JSON Schemas from Python dataclasses
MIT License
9 stars 13 forks source link

remove top pin for jsonschema #51

Closed bollwyvl closed 1 year ago

bollwyvl commented 1 year ago

Hi! Thanks for hologram!

This removes the top pin for jsonschema. Under test, this passes with ~90% coverage over on https://github.com/conda-forge/hologram-feedstock/pull/4 where this pin is becoming problematic for downstreams.

See also #50.

dnascimento commented 1 year ago

Hi! Could this be merged please? It causes issues with dbt-core

sandervandorsten commented 1 year ago

Also curious if this can be merged.

@dnascimento @bollwyvl did you guys find a workaround?

i'm currently trying to install dbt-core inside Amazon MWAA with apache-airflow==2.4.3 using python 3.10. I'm currently stuck on the hologram induced dependency conflict of jsonschema as well (need version jsonschema==4.17.0 based on airflows constraint file in my setup)

bollwyvl commented 1 year ago

find a workaround?

Yerp: as mentioned, on conda-forge, we patch the dependency, and are already shipping it. As mentioned, we run the full test suite with coverage, so I'm quite confident it's a "safe" patch.

When this repo finally makes a canonical release with the fix, the conda-forge build will break when the bot PR finds it, we'll remove the patch, and forget this happened.

sandervandorsten commented 1 year ago

hmm okay, that sounds good! Could you help me out how to use the conda-forge version in my installation? I've never worked with that.

because of my CI/CD requirements I need to specify my dependencies in the requirements.txt file. this looks like so

requirements.txt

--constraint https://raw.githubusercontent.com/apache/airflow/constraints-2.4.3/constraints-3.10.txt
... [other dependencies] 
dbt-core=1.4.1
... [other dependencies]

the dbt-core version uses hologram=0.0.15, which refers to the jsonschema version. should i then do something like this?

--constraint https://raw.githubusercontent.com/apache/airflow/constraints-2.4.3/constraints-3.10.txt
... [other dependencies] 
hologram==0.0.15 --index-url=[conda-forge/hologram]
dbt-core=1.4.1
... [other dependencies]
bollwyvl commented 1 year ago

pip (or whatever .whl-based tool you're using) is entirely unable to install conda-forge-built packages.

The reverse is not true, and tools like conda-lock can help bridge the gap between the two.

If that's not an option, you could be to take the patches applied on the conda-forge repo, and build your own wheels. If that is incompatible with your workflow, and relying on "stuff you find on pypi" as being "the truth," is more important than "it works," then maybe that's a problem with your workflow.

The owners of this repo know this is causing problems for end users: if you are a paying dbt customer, you could escalate to this with them, but otherwise, we (conda-forge) as a downstream have done all we can.

sandervandorsten commented 1 year ago

If that's not an option, you could be to take the patches applied on the conda-forge repo, and build your own wheels.

I was already thinking along that axis, it makes sense. We're using a private (AWS CodeArtifact) repository manager, that should work :)

The owners of this repo know this is causing problems for end users: if you are a paying dbt customer, you could escalate to this with them, but otherwise, we (conda-forge) as a downstream have done all we can.

I'll try to get in contact with dbt via our company's partnership, not sure yet if that provides any leverage but maybe that provides a way to escalate, good suggestion!

Thanks!

gshank commented 1 year ago

Already merged this change. Closing.