`hatching.metadata.utils.get_normalized_dependency` breaking extras with `_`

[slav0nic]

hatch does not work correctly if package extra has _ in name:

[project]
dependencies = [
  "sqlalchemy[postgresql_psycopg] == 2.0.6",
]

will be converted to sqlalchemy[postgresql-psycopg] == 2.0.6, as a result, synchronization of dependencies leads to WARNING: sqlalchemy 2.0.6 does not provide the extra 'postgresql-psycopg'

problem code: https://github.com/pypa/hatch/blob/63b692e3ac7c4156edbbf382b849eaa8707607eb/backend/src/hatchling/metadata/utils.py#L33-L34

pip works correct (safe_extra() )

[ofek]

https://hatch.pypa.io/latest/config/metadata/#allowing-ambiguous-features

[slav0nic]

didn't work for extras, but tnx for PEP

[ofek]

the option didn't work for you?

[slav0nic]

➤ cat pyproject.toml
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
dependencies = [
  ...
  "sqlalchemy[postgresql_psycopg] == 2.0.6",
]

[tool.hatch.metadata]
allow-ambiguous-features = true

➤ hatch dep show requirements
...
sqlalchemy[postgresql-psycopg]==2.0.6

➤ hatch run true
WARNING: sqlalchemy 2.0.6 does not provide the extra 'postgresql-psycopg'

[ofek]

Python metadata isn't dynamic, try again after https://hatch.pypa.io/latest/cli/reference/#hatch-env-prune

[slav0nic]

i did it before

➤ hatch env prune

➤ hatch run true
WARNING: sqlalchemy 2.0.6 does not provide the extra 'postgresql-psycopg'
WARNING: sqlalchemy 2.0.6 does not provide the extra 'postgresql-psycopg'

[slav0nic]

based on hatch src, allowing-ambiguous-features only checked in CoreMetadata.optional_dependencies_complex, but not in CoreMetadata.dependencies_complex

[DaniilAnichin]

Upon inspection, allowing-ambiguous-features only affects extra names defined for the project itself; Extras of dependencies, both optional & regular ones, are always normalized.

On one hand, an option to avoid such normalization sound good. But on the other, if a number of projects which do not abide by such rules is low, we might want to just encourage them to use normalized version themselves. @ofek you've had a long discussion with python community regarding this naming; Are there any statistics about how many projects (sqlalchemy included) have non-normalized extras? (quick analysis of bigquery-public-data.pypi.distribution_metadata shows 1663 projects with _ in extra name, at least 600+ of which have last release in 2023)

Should we approach sqlalchemy with PR to intpoduce normalized ones (maybe in addition to existing, to keep compatibility)? Or, considering a number of projects with non-normalized extra names, we may as well update allowing-ambiguous-features so that it will in fact affect extras of dependencies.

Thanks in advance