Closed bollwyvl closed 1 year ago
Hi! Could this be merged please? It causes issues with dbt-core
Also curious if this can be merged.
@dnascimento @bollwyvl did you guys find a workaround?
i'm currently trying to install dbt-core
inside Amazon MWAA with apache-airflow==2.4.3
using python 3.10. I'm currently stuck on the hologram induced dependency conflict of jsonschema as well (need version jsonschema==4.17.0
based on airflows constraint file in my setup)
find a workaround?
Yerp: as mentioned, on conda-forge
, we patch the dependency, and are already shipping it. As mentioned, we run the full test suite with coverage, so I'm quite confident it's a "safe" patch.
When this repo finally makes a canonical release with the fix, the conda-forge build will break when the bot PR finds it, we'll remove the patch, and forget this happened.
hmm okay, that sounds good! Could you help me out how to use the conda-forge version in my installation? I've never worked with that.
because of my CI/CD requirements I need to specify my dependencies in the requirements.txt
file. this looks like so
requirements.txt
--constraint https://raw.githubusercontent.com/apache/airflow/constraints-2.4.3/constraints-3.10.txt
... [other dependencies]
dbt-core=1.4.1
... [other dependencies]
the dbt-core version uses hologram=0.0.15
, which refers to the jsonschema
version.
should i then do something like this?
--constraint https://raw.githubusercontent.com/apache/airflow/constraints-2.4.3/constraints-3.10.txt
... [other dependencies]
hologram==0.0.15 --index-url=[conda-forge/hologram]
dbt-core=1.4.1
... [other dependencies]
pip
(or whatever .whl
-based tool you're using) is entirely unable to install conda-forge
-built packages.
The reverse is not true, and tools like conda-lock can help bridge the gap between the two.
If that's not an option, you could be to take the patches applied on the conda-forge repo, and build your own wheels. If that is incompatible with your workflow, and relying on "stuff you find on pypi" as being "the truth," is more important than "it works," then maybe that's a problem with your workflow.
The owners of this repo know this is causing problems for end users: if you are a paying dbt
customer, you could escalate to this with them, but otherwise, we (conda-forge) as a downstream have done all we can.
If that's not an option, you could be to take the patches applied on the conda-forge repo, and build your own wheels.
I was already thinking along that axis, it makes sense. We're using a private (AWS CodeArtifact) repository manager, that should work :)
The owners of this repo know this is causing problems for end users: if you are a paying
dbt
customer, you could escalate to this with them, but otherwise, we (conda-forge) as a downstream have done all we can.
I'll try to get in contact with dbt
via our company's partnership, not sure yet if that provides any leverage but maybe that provides a way to escalate, good suggestion!
Thanks!
Already merged this change. Closing.
Hi! Thanks for
hologram
!This removes the top pin for
jsonschema
. Under test, this passes with ~90% coverage over on https://github.com/conda-forge/hologram-feedstock/pull/4 where this pin is becoming problematic for downstreams.See also #50.