Fix `preferred-versions` format

amesgen commented 1 year ago

Cabal can not parse the preferred-versions format as written by foliage before this PR as the preceding package name is missing, so all deprecations were ignored before this PR. E.g. the mtl/preferred-versions file from the Hackage index looks like this:

mtl <2.1 || >2.1 && <2.3 || >2.3

Relevant links to the Cabal source code:

The contents of the preferred-versions files are parsed here. It is a line-wise format with simple comments, where lines are parsed via the Parsec instance for Dependency.
Sadly, for secure repos, Cabal uses parsePreferredVersions, which silently ignores parse errors. This could be reported upstream, especially as warnings are emitted for RepoLocalNoIndex.

I used this code to create this repo (src) for https://github.com/haskell/cabal/pull/8997; this can be used to verify that deprecations now work with this PR.

andreabedini commented 1 year ago

Thank you!

@michaelpj @yvan-sraka can you review (and merge)? It's public holiday on Monday.

@amesgen do we have already wrongly formatted preferred-versions on chap? If yes, this means we need to reset the index (😥) as the fix will change old entries.

amesgen commented 1 year ago

do we have already wrongly formatted preferred-versions on chap? If yes, this means we need to reset the index (disappointed_relieved) as the fix will change old entries.

No, CHaP does not yet have any deprecations, e.g. this returns no entries:

curl -sL https://input-output-hk.github.io/cardano-haskell-packages/01-index.tar.gz \
  | tar tzf - | rg preferred-versions

andreabedini commented 1 year ago

Note for future ourselves (and new visitors :wave:). A deprecated package is avoided if possible but it will still be choosen if it's the only solution.

Consider the following setup:

❯ cat cabal.project 
extra-packages: tagged

repository cabal-repo-override-example
  url: file:/home/andrea/cabal-repo-override-example-repo/_repo
  secure: True

active-repositories: hackage.haskell.org, cabal-repo-override-example:override

❯ tar tzf /home/andrea/cabal-repo-override-example-repo/_repo/01-index.tar.gz --full-time --utc | grep tagged
-rw-r--r-- foliage/foliage 2770 2023-06-02 18:05:25 tagged/0.8.6.1/tagged.cabal
-rw-r--r-- foliage/foliage  805 2023-06-02 18:05:25 tagged/0.8.6.1/package.json
-rw-r--r-- foliage/foliage   27 2023-06-02 18:37:52 tagged/preferred-versions

With this PR, foliage produces the correct preferred-version

❯ tar xzf /home/andrea/cabal-repo-override-example-repo/_repo/01-index.tar.gz -O tagged/preferred-versions
tagged <0.8.6.1 || >0.8.6.1

but then cabal-install picks tagged-0.8.6.1 anyway!

❯ cabal clean && cabal build tagged --dry-run -v3 | grep tagged
Warning: There are no packages or optional-packages in the project
targets: tagged
  tagged >=0 (user target)
  tagged <0.8.6.1 || >0.8.6.1
[__0] trying: tagged-0.8.6.1 (user goal)
[__1] trying: base-4.17.1.0/installed-4.17.1.0 (dependency of tagged)
[__5] trying: tagged:+deepseq
[__6] trying: deepseq-1.4.8.0/installed-1.4.8.0 (dependency of tagged +deepseq)
[__8] trying: tagged:+transformers
[__9] trying: transformers-0.5.6.2/installed-0.5.6.2 (dependency of tagged +transformers)
[_10] trying: template-haskell-2.19.0.0/installed-2.19.0.0 (dependency of tagged)
Component graph for tagged-0.8.6.1: component lib
...

This because cabal-repo-override-example:override has masked any other possible choice of tagged so tagged-0.8.6.1 has to be.

If we add tagged-0.8.6 to the repository, we can see the prefereces taking effect:

❯ tar tzf /home/andrea/cabal-repo-override-example-repo/_repo/01-index.tar.gz --full-time --utc | grep tagged
-rw-r--r-- foliage/foliage 2509 2023-06-02 17:05:25 tagged/0.8.6/tagged.cabal
-rw-r--r-- foliage/foliage  804 2023-06-02 17:05:25 tagged/0.8.6/package.json
-rw-r--r-- foliage/foliage 2770 2023-06-02 18:05:25 tagged/0.8.6.1/tagged.cabal
-rw-r--r-- foliage/foliage  805 2023-06-02 18:05:25 tagged/0.8.6.1/package.json
-rw-r--r-- foliage/foliage   27 2023-06-02 18:37:52 tagged/preferred-versions
❯ tar xzf /home/andrea/cabal-repo-override-example-repo/_repo/01-index.tar.gz -O tagged/preferred-versions
tagged <0.8.6.1 || >0.8.6.1

cabal-install is not picking the latest version of tagged anymore, because it's now deprecated.

❯ cabal clean && cabal build tagged --dry-run -v3 | grep tagged
Warning: There are no packages or optional-packages in the project
targets: tagged
  tagged >=0 (user target)
  tagged <0.8.6.1 || >0.8.6.1
[__0] trying: tagged-0.8.6 (user goal)
[__1] trying: base-4.17.1.0/installed-4.17.1.0 (dependency of tagged)
[__5] trying: tagged:+deepseq
[__6] trying: deepseq-1.4.8.0/installed-1.4.8.0 (dependency of tagged +deepseq)
[__8] trying: tagged:+transformers
[__9] trying: transformers-0.5.6.2/installed-0.5.6.2 (dependency of tagged +transformers)
[_10] trying: template-haskell-2.19.0.0/installed-2.19.0.0 (dependency of tagged)
Component graph for tagged-0.8.6: component lib
...

Setting index-state to just before the deprecation makes tagged-0.8.6.1 available again.

❯ cabal clean && cabal build tagged --dry-run -v3 --index-state='cabal-repo-override-example 2023-06-02T18:37:51Z' | grep tagged
...
targets: tagged
  tagged >=0 (user target)
[__0] trying: tagged-0.8.6.1 (user goal)
[__1] trying: base-4.17.1.0/installed-4.17.1.0 (dependency of tagged)
[__5] trying: tagged:+deepseq
[__6] trying: deepseq-1.4.8.0/installed-1.4.8.0 (dependency of tagged +deepseq)
[__8] trying: tagged:+transformers
[__9] trying: transformers-0.5.6.2/installed-0.5.6.2 (dependency of tagged +transformers)
[_10] trying: template-haskell-2.19.0.0/installed-2.19.0.0 (dependency of tagged)
Component graph for tagged-0.8.6.1: component lib
...

andreabedini commented 1 year ago

Thank you @amesgen <3

input-output-hk / foliage

Fix `preferred-versions` format #66