Canonical way for adding development dependencies to "setup.py" file "extras_require".

KelSolaar commented 5 years ago

[x] I have searched the issues of this repo and believe that this is not a duplicate.
[x] I have searched the documentation and believe that my question is not covered.

Issue

Hi,

I'm looking to move Colour to Poetry, so far so good except for a minor issue on Github Actions and Windows.

There is however something I might be overlooking but I did not find a good answer for. The context is that issuing pip install 'colour-science[development]' will install our development dependencies.

I obviously don't want to lose that capability and the only solution I found is rather unelegant: I need to have our development dependencies listed two times (well, three to be exact):

In the [tool.poetry.dependencies] section where they are marked as optional, then they can be used in the [tool.poetry.extras] section to populate accordingly the setup.py file generated with the poetry build command.
In [tool.poetry.dev-dependencies] so that poetry install installs everything we need for development.

Here is my current pyproject.toml file for reference:

[tool.poetry]
name = "colour"
version = "0.3.13"
description = "Colour Science for Python"
license = "BSD-3-Clause"
authors = [ "Colour Developers" ]
readme = 'README.rst'
repository = "https://github.com/colour-science/colour"
homepage = "https://www.colour-science.org/"
keywords = [
    "color",
    "color-science",
    "color-space",
    "color-spaces",
    "colorspace",
    "colorspaces",
    "colour",
    "colour-science",
    "colour-space",
    "colour-spaces",
    "colourspace",
    "colourspaces",
    "data",
    "dataset",
    "datasets",
    "python",
    "spectral-data",
    "spectral-dataset",
    "spectral-datasets"
]
classifiers=[
    "Development Status :: 3 - Alpha",
    "Environment :: Console",
    "Intended Audience :: Developers",
    "Intended Audience :: Science/Research",
    "License :: OSI Approved",
    "Natural Language :: English",
    "Operating System :: OS Independent",
    "Programming Language :: Python :: 2.7",
    "Programming Language :: Python :: 3",
    "Topic :: Scientific/Engineering",
    "Topic :: Software Development"
]

[tool.poetry.dependencies]
python = "~2.7 || ^3.5"
imageio = "*"
six = "*"
scipy = "*"

"backports.functools_lru_cache" = { version = "*", optional = true }
biblib-simple = { version = "*", optional = true }
coverage = { version = "*", optional = true }
flake8 = { version = "*", optional = true }
invoke = { version = "*", optional = true }
matplotlib = { version = "*", optional = true }
mock = { version = "*", optional = true }
networkx = { version = "*", optional = true }
nose = { version = "*", optional = true }
numpy = { version = "*", optional = true }
pandas = { version = "*", optional = true }
pygraphviz = { version = "*", optional = true }
restructuredtext-lint = { version = "*", optional = true }
sphinx = { version = "*", optional = true }
sphinx_rtd_theme = { version = "*", optional = true }
sphinxcontrib-bibtex = { version = "*", optional = true }
yapf = { version = "0.23", optional = true }

[tool.poetry.dev-dependencies]
biblib-simple = "*"
coverage = "*"
flake8 = "*"
invoke = "*"
nose = "*"
mock = "*"
restructuredtext-lint = "*"
sphinx = "*"
sphinx_rtd_theme = "*"
sphinxcontrib-bibtex = "*"
yapf = "0.23"

[tool.poetry.extras]
development = [
    "biblib-simple",
    "coverage",
    "flake8",
    "invoke",
    "nose",
    "mock",
    "restructuredtext-lint",
    "sphinx",
    "sphinx_rtd_theme",
    "sphinxcontrib-bibtex",
    "yapf"
]
graphviz = [ "pygraphviz" ]
optional = [ "networkx", "pandas" ]
plotting = [ "backports.functools_lru_cache", "matplotlib" ]
read-the-docs = [ "mock", "numpy", "sphinxcontrib-bibtex"]

[build-system]
requires = [ "poetry>=0.12" ]
build-backend = "poetry.masonry.api"

Please let me know if I missed something or if it is a current limitation.

Cheers,

Thomas

sdispater commented 5 years ago

You only have to specify them once in the tool.poetry.dependencies section as optional and add the development extra to tool.poetry.extras. No need to specify them in tool.poetry.dev-dependencies. Then you need to do poetry install --extras development

KelSolaar commented 5 years ago

Hi Seb,

Sure, but then poetry install does not install my development dependencies:

(colour-49B8_mty-py2.7) Kali:colour kelsolaar$ poetry install
Updating dependencies
Resolving dependencies... (11.2s)

Writing lock file

Package operations: 5 installs, 0 updates, 0 removals

  - Installing six (1.12.0)
  - Installing numpy (1.16.5)
  - Installing pillow (6.1.0)
  - Installing imageio (2.5.0)
  - Installing scipy (1.2.2)
  - Installing colour (0.3.13)

vs

(colour-49B8_mty-py2.7) Kali:colour kelsolaar$ poetry install
Updating dependencies
Resolving dependencies... (4.6s)

Writing lock file

Package operations: 63 installs, 0 updates, 0 removals

  - Installing pycparser (2.19)
  - Installing asn1crypto (0.24.0)
  - Installing cffi (1.12.3)
  - Installing enum34 (1.1.6)
  - Installing ipaddress (1.0.22)
  - Installing six (1.12.0)
  - Installing certifi (2019.9.11)
  - Installing chardet (3.0.4)
  - Installing cryptography (2.7)
  - Installing idna (2.8)
  - Installing latexcodec (1.0.7)
  - Installing markupsafe (1.1.1)
  - Installing pyparsing (2.4.2)
  - Installing pytz (2019.2)
  - Installing pyyaml (5.1.2)
  - Installing urllib3 (1.25.6)
  - Installing webencodings (0.5.1)
  - Installing alabaster (0.7.12)
  - Installing babel (2.7.0)
  - Installing bleach (3.1.0)
  - Installing configparser (4.0.2)
  - Installing docutils (0.15.2)
  - Installing imagesize (1.1.0)
  - Installing jinja2 (2.10.1)
  - Installing packaging (19.2)
  - Installing pybtex (0.22.2)
  - Installing pygments (2.4.2)
  - Installing pyopenssl (19.0.0)
  - Installing requests (2.22.0)
  - Installing snowballstemmer (1.9.1)
  - Installing sphinxcontrib-websupport (1.1.2)
  - Installing typing (3.7.4.1)
  - Installing coverage (4.5.4)
  - Installing docopt (0.6.2)
  - Installing entrypoints (0.3)
  - Installing funcsigs (1.0.2)
  - Installing functools32 (3.2.3-2)
  - Installing mccabe (0.6.1)
  - Installing numpy (1.16.5)
  - Installing oset (0.1.3)
  - Installing pillow (6.1.0)
  - Installing pkginfo (1.5.0.1)
  - Installing pybtex-docutils (0.2.1)
  - Installing pycodestyle (2.5.0)
  - Installing pyflakes (2.1.1)
  - Installing readme-renderer (24.0)
  - Installing requests-toolbelt (0.9.1)
  - Installing sphinx (1.8.5)
  - Installing tqdm (4.36.1)
  - Installing biblib-simple (0.1.1)
  - Installing coveralls (1.8.2)
  - Installing flake8 (3.7.8)
  - Installing imageio (2.5.0)
  - Installing invoke (1.3.0)
  - Installing mock (3.0.5)
  - Installing nose (1.3.7)
  - Installing restructuredtext-lint (1.3.0)
  - Installing scipy (1.2.2)
  - Installing sphinx-rtd-theme (0.4.3)
  - Installing sphinxcontrib-bibtex (0.4.2)
  - Installing toml (0.10.0)
  - Installing twine (1.15.0)
  - Installing yapf (0.23.0)
  - Installing colour (0.3.13)

I guess I would like to get the best of both worlds :)

Cheers,

Thomas

DanCardin commented 5 years ago

I've just run into the same thing, except I'm looking to have a [docs] extra, which ideally is installed normally as a dev dep in the general case, but is able to be included in the extras.

Could extras pull from both dependencies and dev-dependencies. Wherein, they're not included by default during a poetry install if they're in dependencies (a-la current behavior), but are included by default when they're dev-dependencies?

kelvin-tran commented 2 years ago

Seems that the solution is referenced here: https://github.com/python-poetry/poetry/issues/3180

Eg:

[tool.poetry.dev-dependencies]

moto = { extras=["s3", "sns", "sqs", "awslambda", "xray"], version = "^2.0.1" }

github-actions[bot] commented 9 months ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

python-poetry / poetry

Canonical way for adding development dependencies to "setup.py" file "extras_require". #1416

Issue