Open geoHeil opened 1 month ago
I could fix this by changing the package_name to: package_name: foo.foo
.
@geoHeil can you clarify what the file/package structure in the "foo" package in question when this happens? What's in the __init__.py
file?
In both 1.7.15 and 1.8.0 I can reproduce that error if the __init__.py
for that module is empty and doesn't explicitly import the definitions, but I haven't been successful yet at finding something there that would have regressed between 1.7.15 and 1.8.0 .
@geoHeil can you clarify what the file/package structure in the "foo" package in question when this happens? What's in the
__init__.py
file?
the __init__.py
file is exactly like the one shared above. I think the important thing is the folder structure
βββ foo
β βββ foo
β β βββ __init__.py
β β βββ __pycache__
β β βββ __init__.cpython-310.pyc
β β βββ __init__.cpython-311.pyc
β βββ foo_tests
β β βββ __init__.py
β β βββ test_assets.py
β βββ pyproject.toml
β βββ setup.cfg
βββ workspace_docker.yaml
βββ workspace.yaml
this is how it looks like
Thanks for sharing the folder structure - I'm still seeing identical results when I try with that folder structure on 1.7.15 and 1.8.0 though. Can you double-check that
load_from:
- python_package:
package_name: foo
location_name: foo
works for you with that folder structure on 1.7.15?
(What i'm seeing on both versions is what I would expect, which is that only foo.foo
works as a package name - since the __init__.py
with the code in it is twofoo
s down in the hierarchy)
interesting - this is different for me - it works in 1.7.15 only with foo - and fails for 1.8
Given that there is a simple fix - it is ok probably if we close this one as you cannot reproduce it
Hmmm that is very strange. No need to keep digging if you don't want, but if you do - this is in the same virtual environment for both dagster versions? Is foo pip installed in some way?
(and to double-check, you're running it in the same folder on both versions? i.e. its definitely pulling its instructions from the workspace.yaml and not the pyproject.toml?)
Hmmm that is very strange. No need to keep digging if you don't want, but if you do - this is in the same virtual environment for both dagster versions? Is foo pip installed in some way?
yes, foo is pip installed - except for the dagster version change the environment is the same
(and to double-check, you're running it in the same folder on both versions? i.e. its definitely pulling its instructions from the workspace.yaml and not the pyproject.toml?)
to my best knowledge yes - is there a quick log analysis I could do to help you confirm this?
If there's a [tool.dagster] entry in the pyproject.toml, you could remove it (that would be the only way it would be able to be pulling it from somewhere other than the workspace.yaml)
Is foo installed editably ("pip install -e") or normally (just "pip install")
(maybe you could share the contents of the pyproject.toml file / setup.cfg file if relevant?)
should we jump on a quick call? I can send a zoom link maybe this is simpler than the back-and-forth?
in any case:
[tool.dagster]
module_name = "foo"
setup.cfg
[metadata]
name = foo
is part of the toml
yes editable install
When changing the toml to:
[tool.dagster]
module_name = "foo.foo"
the issue still persists - and is only fixed when changing the workspace.yml
Canβt do a call at the moment but thanks for all the back and forth.
Last question for a bit - are you switching back and forth between 1.7.15 and 1.8.0 in the same venv? Or using a different venv for each version? I just want to be sure the Dagster version is literally the only difference, because Iβve seen weird things with different versions of pip handling editable installs differently, or if foo was pip installed in one venv but not another I think that could also explain this.
I freshly create the venv with the different versions
Dagster version
1.8.0
What's the issue?
The latest 1.7 (1.7.15) version of dagster can load something like this fine:
however, upgrading to 1.8.0 fails the import.
I am loading via the package workspace.yaml like:
Error is:
What did you expect to happen?
the project to successfully load
How to reproduce?
see the examples above
Deployment type
Local
Deployment details
No response
Additional information
When loading without the workspace.yaml file the code location loads just fine
Message from the maintainers
Impacted by this issue? Give it a π! We factor engagement into prioritization.