Open ccosborne opened 1 year ago
Yes, thanks for sending along the installation errors. It sheds light on this.
It looks like there are two separate issues. One has to do with the error in pyproject.toml
.
But what is more likely causing the error when importing config
is the problem with the python path.
Googling for this error reveals that the problem might be solved by removing the option --user
from the installation command:
https://github.com/NVIDIA/flownet2-pytorch/issues/183
This may be a problem with our installation instructions, as --user can conflict with virtualenv functionality.
Thanks for your help, Seb.
Removing --user
removed the PYTHONPATH
error but the pyproject.toml
error persists. The error message says I should list dependencies
and optional-dependencies
under dynamic
- do you know how to do that?
@ccosborne The issues with pyproject.toml
appear to be warnings, not errors. They are about parts of the build system that are soon to be (but not yet) deprecated.
Thanks for bringing that to our attention!
But it should not be blocking the execution of your Jupyter notebook.
Do you still have problems with import config
?
@sbenthall Unfortunately I get the same error message with import config
in the Jupyter notebook. Any clue why that might be?
Hello @ccosborne I see the issue. It's a problem with the configuration on our end. I'll work on this and let you know when I've pushed the change.
Ok, I've pushed a change and tests are passing. Can you please try the installation again?
Unfortunately I get the same warning message when running python3 setup.py develop
and the same error message when importing from bigbang.archive import Archive
in the notebook, even after deleting bigbang and cloning it again... :/
This is indeed an issue, and this pops up because the config
module isn't properly packaged. It's a sibling to the bigbang
module and there isn't any way to automatically make config
importable unless explicitly adding it the the python system paths.
The tests will pass because they are running in the root dir so they can do import config
without ever having this installed properly. This can be fixed with some adding the paths to sys.paths
but I would highly discourage that. The best way here would be to restructure the package along the lines of:
- bigbang/
- bigbang
- config
- tests
- bin
with from config.config import CONFIG
becoming something like from bigbang.config import CONFIG
.
Let me know if I am missing something here regarding the structure of bigbang (and thus the current structure).
@MridulS Thanks so much for this analysis!
Not a lot of thought was put into the location of the config file. I will implement your suggestions here.
One question though -- in another project I've been able to import sibling modules by adding them to the packages
list:
https://github.com/sbenthall/SHARKFin/blob/master/setup.cfg#L18
Is there something wrong with that approach?
Ultimately, we use the config
module just to keep track of a bunch of default file system paths and regular expressions.
We want to access them from Python, but store them in a YAML file.
Having them bundled with the bigbang
source code means that there would be no way for a 'user' who is just using BigBang in development mode to modify that file -- they would need to look around in their site-packages or something.
I wonder @MridulS how you would approach this from the perspective of the on-line dashboard, which I believe is built on the package available on PyPI and not a development installation. Is there a better way to expose these configuration options?
One question though -- in another project I've been able to import sibling modules by adding them to the packages list: https://github.com/sbenthall/SHARKFin/blob/master/setup.cfg#L18 Is there something wrong with that approach?
This does indeed work! But the issue here is that sharkfin
, simulate
and pnl_market
are essentially different "packages". They will work just fine in isolation but as soon as someone wants to use them as an external library it can mess things up, like simulate
is way common of a name and there is no way a person can import import sharkfin.simulate
. When I do pip install sharkfin
it shouldn't install simulate
or pnl_market
IMO (this is one of my annoyance with econ-ark too, the distribution name should match the actual package) This looks very close to namespace packages. I would rather just have the standard layout unless there is some proper need for this.
Ultimately, we use the config module just to keep track of a bunch of default file system paths and regular expressions. We want to access them from Python, but store them in a YAML file.
Having them bundled with the bigbang source code means that there would be no way for a 'user' who is just using BigBang in development mode to modify that file -- they would need to look around in their site-packages or something.
This can live under the bigbang
folder too and I think we can make the whole config bits better with using Pathlib
, it would lead to much cleaner code. The user can write their own YAML file and the initial load of bigbang can read that in (the user can provide an explicit path).
I wonder @MridulS how you would approach this from the perspective of the on-line dashboard, which I believe is built on the package available on PyPI and not a development installation. Is there a better way to expose these configuration options?
Currently bigbang isn't usuable to build a dashboard or as an external library, (atleast while using the CLBE tutorial). The pypi
version breaks (atleast the latest one at https://pypi.org/project/bigbang-py/) when I try to import Archive
. In a fresh env:
>>> from bigbang.archive import Archive
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/ms/mambaforge/envs/env-test-0/lib/python3.10/site-packages/bigbang/__init__.py", line 1, in <module>
from . import ingress
ImportError: cannot import name 'ingress' from partially initialized module 'bigbang' (most likely due to a circular import) (/Users/ms/mambaforge/envs/env-test-0/lib/python3.10/site-packages/bigbang/__init__.py)
We need to change the structure and make sure the imports are fixed :)
To get around for now I have moved around the files and fixed some imports locally. I can send the changes as a PR but it's pretty hackish and the config folder should probably change anyway.
Thanks so much @MridulS
This is all my fault -- I packaged BigBang for PyPI not long ago, but clearly made a mess of it.
Any PRs you make along these lines would be greatly appreciated. I defer to your guidance on these things.
@MridulS Fixed with #579 ?
Hi there,
I am running into problems during installation when I execute the
python3 setup.py develop --user
command. For context, my previous steps are: (1) clone bigbang repo to remote server, (2) create virtualenv, and (3) executepip3 install -r requirements.txt
. When I runpython3 setup.py develop --user
, I get the output pasted below. After this, I add the virtualenv as a jupyter kernel, open an example notebook (Git Collection), and when I try to import the packages I get:ModuleNotFoundError: No module named 'config'
. I assume this is linked to the problems with executingsetup.py.
Do you know how to troubleshoot this? Thanks!