search

kaitaiStructCompile / kaitaiStructCompile.py

Compilation of *.ksy into *.py from python
https://gitlab.com/kaitaiStructCompile.py/
The Unlicense
4 stars 2 forks source link

Add an alternative plugin discovery mechanism in a form of an env variable, listing entry points in `pkg_resources` format. #8

Open abitrolly opened 4 years ago

abitrolly commented 4 years ago

README.md example mentions importer, but it doesn't seem to be available.

>>> import kaitaiStructCompile
>>> import kaitaiStructCompile.importer
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'kaitaiStructCompile.importer'
KOLANICH commented 4 years ago

Hi.

You likely look for https://github.com/kaitaiStructCompile/kaitaiStructCompile.importer

I have splitted the stuff into multiple packages, for proper working you need most of them. Unfortunately the stuff has some problems with installation, since it tries to share the same dir in order not to junk site-packages, but pip has some problems with that. Try to install everything manually. Under manually I mean that you first build wheels, then istall them. Editable install won't work for this - yet another bug in pip.

BTW: do you know any good replacement for GitLab with Docker-based CI (the CI should load my images, and should not load own images first, like GitHub does, since loading each image takes long) and without shit like mandatory JS, reCAPTCHA and CloudFlare (sr.ht is not considered suitable because of ddvault's unpredictable behavior ).

abitrolly commented 4 years ago

I suspect these are bugs of setuptools and not pip. I never liked its entrypoints mechanism for the plugin system exactly for the reason described - that it doesn't allow to just drop the plugin in some dir and requires the dance with wheels, virtualenvs and pip install.

For fast CI I would just buy a VPS, or setup LXD build job on your own hardware.

KOLANICH commented 4 years ago

I suspect these are bugs of setuptools and not pip.

No, the wheels look fine. I have encountered problems some times when I installed all the wheels in single pip invocation (including the cases when pip auto-fetches the stuff and auto-builds it). But when each wheel is installed separately and all the wheels are installed into the same location (i.e. sudo pip install), it usually works fine.

I never liked its entrypoints mechanism for the plugin system exactly for the reason described - that it doesn't allow to just drop the plugin in some dir

Probably I should add an alternative plugin discovery mechanism in a form of an env variable, listing entry points in pkg_resources format. That can allow prototyping a backend without installing it.

and requires the dance with wheels, virtualenvs and pip install.

virtualenvs is just a cargo cult. If you need the stuff on CI and the installation is run every time the CI runs, just feel free to install it in main system, not venv, the CI container will be cleaned after pipeline execution anyway :) If you use a Docker-based CI, feel free to install into the image on the stage of building of 6he image. If you want kaitaiStructCompile .py in your main system, again feel free to just install it. I see no reason not to just install it into main system.

abitrolly commented 4 years ago

My point is.

...
Simple is better than complex.
Complex is better than complicated.
...

The plugin system based on wheel + setuptools + pip install + pkg_resources + CI/venv is complicated. Especially compared to simple file download as with compiler itself.

KOLANICH commented 4 years ago

As you wish. IMHO it is venvs that are ugly, complex, complicated and unneeded.

wheel + setuptools + pip install + pkg_resources

Are de-facto standard.

abitrolly commented 4 years ago

Are de-facto standard.

IMHO is missing.

KOLANICH commented 4 years ago

It is not "IMHO". They ARE de-facto standard.

abitrolly commented 4 years ago

It if was the standard, there would be at least draft PEP. Otherwise it is a just a bunch of crude hacks stacked on top of each other.

KOLANICH commented 4 years ago
  1. de-facto standard doesn't require any blessed docukents. What determines de-facto standard is de-facto adoption.
  2. Actually there are some peps, which are not drafts.

https://www.python.org/dev/peps/pep-0632/ https://packaging.python.org/specifications/entry-points/ https://www.python.org/dev/peps/pep-0427/ https://www.python.org/dev/peps/pep-0365/