python-poetry / poetry

Python packaging and dependency management made easy
https://python-poetry.org
MIT License
31.48k stars 2.27k forks source link

Add support for `scripts` #241

Closed ojii closed 3 years ago

ojii commented 6 years ago

It would be nice if poetry supported the scripts feature of setuptools (note: this is not the same as entry points).

I tried using a custom build script but it seems to be ignored by poetry when building wheels.

peterdeme commented 3 years ago

Guys, can you send a few upvotes to my PR? Hopefully it'll get some traction as it's been open since the end of June. https://github.com/python-poetry/poetry-core/pull/40

euberdeveloper commented 3 years ago

I think that you should add something similar to the Pipenv:

[scripts]
dev = "uvicorn main:app --workers 4 --port 3000 --reload"
start = "gunicorn --bind 0.0.0.0:$PORT -w 4 -k uvicorn.workers.UvicornWorker main:app"

Poetry has a scripts section but it is not as the one above. I think it is one of the few features where pipenv is a step forward. Also npm of nodejs allows that and I think that something like (of course I do not use npm with python):

{
   "scripts": {
     "test": "python test.py",
     "dev": "uvicorn main:app --workers 4 --port 3000 --reload",
     "test-run": "npm run test && npm run dev"
   }
}
ZelphirKaltstahl commented 3 years ago

I think adding something as half-cooked as scripts in package.json used by npm is a bad idea. A free software, battle tested tool for such things already exists. Its name is GNU Make. The world does not need another "scripts" incomplete solution. It is reinventing the wheel and probably will be only half of what GNU Make already offers and in a worse format, where everything is simple crammed into a string, where you use a lot of && and similar. All that just for running a few shell commands, plus the overhead of maintaining that stuff.

Poetry has got enough stuff to work on, that it does not need to include another half-cooked scripts solution.

euberdeveloper commented 3 years ago

I understand what you are saying and I know that make is a great software, but probably the greatest advantage of the "half-cooked" npm scripts and pipenv scripts over make is precisely the fact of being half-cooked.

If you need some os-indipendent command shortcuts, with make you would have to: 1) install it 2) learn make, which is an old (old means gold, but also a non-beautiful docs site and syntax) 3) create a Makefile 4) add two lines to that Makefile that use 1% of all make capabilities

In the other hand if you had these scripts shortcuts you just would add two lines to a file that already exists and you are done.

Make is a nice tool, but it would be just overkill using a powerful tool designed for building C in order to have two script shortcuts. The exact fact of being quick and simple is an advantage and the language python is the proof.

If this feature would be implemented with a low-priority (and it would not be such a big wheel to reinvent) I am sure lots of users would use it and find it comfortable, while the others could just keep using make.

euberdeveloper commented 3 years ago

I don't think python-poetry/poetry-core#40 is related to this issue

abn commented 3 years ago

@euberdeveloper as per the OP, this issue is talking about "scripts" similar to how setuptools does it. I think the issue(s) you are looking for are #591 #2496.

euberdeveloper commented 3 years ago

Ok, I read them and it was written that the owner of poetry won't implement it but there is a third party plugin that works pretty well, thank you

keattang commented 3 years ago

This is the only thing holding us back from rolling poetry out across our entire company. The posted solutions are workable but it's less than ideal to have to type run + <runner> + <cmd> when a lot of other tools let you run scripts directly.

To help mitigate this I've just created the poetry-exec-plugin, ready and waiting for poetry 1.2 to be released with its plugin system. It works just like npm or pipenv and can be installed with pip install poetry-exec-plugin. You then configure your scripts like this in your pyproject.toml:

[tool.poetry-exec-plugin.commands]
lint = "flake8 && black --check . && mypy ."
test = "pytest"

Then your scripts can be run with poetry exec <cmd> (e.g. poetry exec lint , poetry exec test).

Hopefully this is helpful to others 😄

miigotu commented 2 years ago

This is the only thing holding us back from rolling poetry out across our entire company. The posted solutions are workable but it's less than ideal to have to type run + <runner> + <cmd> when a lot of other tools let you run scripts directly.

To help mitigate this I've just created the poetry-exec-plugin, ready and waiting for poetry 1.2 to be released with its plugin system. It works just like npm or pipenv and can be installed with pip install poetry-exec-plugin. You then configure your scripts like this in your pyproject.toml:

[tool.poetry-exec-plugin.commands]
lint = "flake8 && black --check . && mypy ."
test = "pytest"

Then your scripts can be run with poetry exec <cmd> (e.g. poetry exec lint , poetry exec test).

Hopefully this is helpful to others 😄

https://pypi.org/project/poethepoet/

bluebrown commented 2 years ago

Why is this not built in? It's pretty much a must-have. Installing an extra plugin is strange.

purificant commented 2 years ago

Why is this not built in? Its pretty much a must have. Installing an extra plugin is strage.

From what I understand the team behind poetry has made a decision not to support this kind of user experience in the core codebase. Other similar teams in different communities / programming languages have made different decisions and this thread reflects that.

arielnmz commented 2 years ago

Re @hg-zt Still no way of making this work?

[tool.poetry.scripts]
# other entry points
my_shell_command = { path = "bin/my_shell_command.sh" }

I just want to package a .sh with my library

TBBle commented 2 years ago

Per the poetry-core ticket that closed this issue, the syntax is

[tool.poetry.scripts]
my_shell_command = {reference = "bin/my_shell_command.sh", type = "file"}

and it only works with poetry-core 1.1.0a5 or later (which will need to be specified in the build-system table in your pyproject.toml), and hence Poetry 1.2.0a1 or later, and poetry install won't set up the scripts as the feature is not yet implemented in the EditableBuilder in Poetry itself (See #2310 for the ticket tracking that).

But given the above caveats, it should package the desired scripts into the sdist and wheels, and then they will be installed by the installer (i.e. pip) as expected. You can see some examples in the poetry-core test suite.

shubb30 commented 2 years ago

Hi @TBBle,

Can you confirm the syntax is still correct?
I am using versions poetry==1.2.0a1 poetry-core==1.1.0a5 and when I run check, get the errors:


poetry check -vv

  Stack trace:

  8  .venv/lib/python3.8/site-packages/cleo/application.py:329 in run
       exit_code = self._run(io)

  7  .venv/lib/python3.8/site-packages/poetry/console/application.py:167 in _run
       return super()._run(io)

  6  .venv/lib/python3.8/site-packages/cleo/application.py:423 in _run
       exit_code = self._run_command(command, io)

  5  .venv/lib/python3.8/site-packages/cleo/application.py:465 in _run_command
       raise error

  4  .venv/lib/python3.8/site-packages/cleo/application.py:449 in _run_command
       exit_code = command.run(io)

  3  .venv/lib/python3.8/site-packages/cleo/commands/base_command.py:119 in run
       status_code = self.execute(io)

  2  .venv/lib/python3.8/site-packages/cleo/commands/command.py:83 in execute
       return self.handle()

  1  .venv/lib/python3.8/site-packages/poetry/console/commands/check.py:18 in handle
       check_result = Factory.validate(config, strict=True)

  KeyError

  'extras'

  at .venv/lib/python3.8/site-packages/poetry/core/factory.py:372 in validate
      368│                 for name, script in scripts.items():
      369│                     if not isinstance(script, dict):
      370│                         continue
      371│ 
    → 372│                     extras = script["extras"]
      373│                     for extra in extras:
      374│                         if extra not in config["extras"]:
      375│                             result["errors"].append(
      376│                                 'Script "{}" requires extra "{}" which is not defined.'.format(

To test, I changed the section to be

[tool.poetry.scripts]
my_script = {reference = "bin/my_script.py", type = "file", extras = []}

And then the check worked, and the package was correctly built, and included the script.

TBBle commented 2 years ago

That looks like a bug in poetry-core, so I'd report it separately.

I guess extras support was added later and accidentally broke the case of no extras by failing to default to []. (Edit: Nope, it's always been that way. Weird, must be a code-path that the test suite isn't actually testing) The line that failed probably should be

extras = script.get("extras", [])

and so should the use of "extras" a few lines down. Every other usage in this file of "extras" is protected like that.

That said, the example in the poetry-core test suite hasn't changed, so I'm not sure why it doesn't reproduce there. Either way, worth a new bug report since this ticket is closed and your issue won't get the attention it needs here.

Edit: Turns out there's already a fix in progress: https://github.com/python-poetry/poetry-core/pull/404, as this issue was reported at https://github.com/python-poetry/poetry/issues/4665 (including by you, I see now).

miigotu commented 2 years ago

Hi @TBBle,

Can you confirm the syntax is still correct?
I am using versions poetry==1.2.0a1 poetry-core==1.1.0a5 and when I run check, get the errors:


poetry check -vv

  Stack trace:

  8  .venv/lib/python3.8/site-packages/cleo/application.py:329 in run
       exit_code = self._run(io)

  7  .venv/lib/python3.8/site-packages/poetry/console/application.py:167 in _run
       return super()._run(io)

  6  .venv/lib/python3.8/site-packages/cleo/application.py:423 in _run
       exit_code = self._run_command(command, io)

  5  .venv/lib/python3.8/site-packages/cleo/application.py:465 in _run_command
       raise error

  4  .venv/lib/python3.8/site-packages/cleo/application.py:449 in _run_command
       exit_code = command.run(io)

  3  .venv/lib/python3.8/site-packages/cleo/commands/base_command.py:119 in run
       status_code = self.execute(io)

  2  .venv/lib/python3.8/site-packages/cleo/commands/command.py:83 in execute
       return self.handle()

  1  .venv/lib/python3.8/site-packages/poetry/console/commands/check.py:18 in handle
       check_result = Factory.validate(config, strict=True)

  KeyError

  'extras'

  at .venv/lib/python3.8/site-packages/poetry/core/factory.py:372 in validate
      368│                 for name, script in scripts.items():
      369│                     if not isinstance(script, dict):
      370│                         continue
      371│ 
    → 372│                     extras = script["extras"]
      373│                     for extra in extras:
      374│                         if extra not in config["extras"]:
      375│                             result["errors"].append(
      376│                                 'Script "{}" requires extra "{}" which is not defined.'.format(

To test, I changed the section to be

[tool.poetry.scripts]
my_script = {reference = "bin/my_script.py", type = "file", extras = []}

And then the check worked, and the package was correctly built, and included the script.

1.2.0a2 is behind the release version, I had to use the regular release rather than the alpha.

TBBle commented 2 years ago

What version do you mean by "regular release"? There's no non-pre-release version of 1.2 series Poetry (the latest is 1.2.0b2), and the current release versions of Poetry (1.1 series) do not have this feature. Poetry 1.1.x only supports what setuptools calls "console_scripts entry point", not what setuptools calls "scripts". (Both are different kinds of "scripts" in Poetry's terminology, and the latter is what this issue was requesting, and has now been implemented for Poetry 1.2).

rosaLux161 commented 1 year ago

Any updates here?

TBBle commented 1 year ago

See #2310 for where updates will appear, since this issue has been closed after the underlying support in poetry-core was delivered, so it works for package creation, but apparently not poetry install or poetry run of the package containing the script itself. https://github.com/python-poetry/poetry/issues/2310#issuecomment-1272582247 appears to be the most-recent status for the poetry side of this.

As noted in #2310, it's also missing from the docs still, although maybe that's because of the current non-working cases.

django-djack commented 1 year ago

Any updates here ?

django-djack commented 1 year ago

It's been almost 5 years lol

TBBle commented 1 year ago

Nope, no updates here, and there won't be any, as this issue has been closed, and the support for scripts was shipped in Poetry 1.2.0 (technically Poetry Core 1.1.0, which is the matching Poetry Core release) over a year ago. See https://github.com/python-poetry/poetry/issues/241#issuecomment-1459952704 for updates relating to scripts support in poetry install and poetry run, and (presumably) full documentation of this feature.

github-actions[bot] commented 7 months ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.