upload with lower pvp-bounds removes things in 1.6.0

mfine commented 7 years ago

General summary/comments

Uploading with pvp-bounds: lower is removing things from the cabal file in 1.6.0.

Steps to reproduce

Publish a package to hackage containing an executable with a dependency on the package's library with a stack.yaml containing pvp-bounds: lower.

Expected

I expected the bounds to be set on the dependencies for the executable.

Actual

There were missing fields:

Before:

library
  default-language:    Haskell2010
  hs-source-dirs:      src
  exposed-modules:     Development.Shakers
  ghc-options:         -Wall
  build-depends:       base >= 4.8 && < 5
                     , basic-prelude
                     , directory
                     , shake

executable shake-shakers
  default-language:    Haskell2010
  main-is:             Shakefile.hs
  ghc-options:         -threaded -rtsopts -with-rtsopts=-N -Wall
  build-depends:       base
                     , shakers

After:

library
    exposed-modules:
        Development.Shakers
    build-depends:
        base >=4.8 && <5,
        basic-prelude >=0.5.2,
        directory >=1.2.2.0,
        shake >=0.15.11
    default-language: Haskell2010
    hs-source-dirs: src
    ghc-options: -Wall

executable  shake-shakers
    main-is: Shakefile.hs

Missing default-language, ghc-options, and build-depends.

Here's the verbose output of sdist:

$ Version 1.6.0, Git revision 64794699803ff997dd9a1ff999c342c73d99bb20 (5322 commits) x86_64
Compiled with:
- Cabal-2.0.0.2
- Glob-0.9.0
- HUnit-1.6.0.0
- MonadRandom-0.5.1
- QuickCheck-2.10.1
- StateVar-1.1.0.4
- aeson-1.2.2.0
- aeson-compat-0.3.7.1
- annotated-wl-pprint-0.7.0
- ansi-terminal-0.6.3.1
- ansi-wl-pprint-0.6.8.1
- array-0.5.2.0
- asn1-encoding-0.9.5
- asn1-parse-0.9.4
- asn1-types-0.3.2
- async-2.1.1.1
- attoparsec-0.13.2.0
- attoparsec-iso8601-1.0.0.0
- auto-update-0.1.4
- base-4.10.0.0
- base-compat-0.9.3
- base-orphans-0.6
- base-prelude-1.2.0.1
- base16-bytestring-0.1.1.6
- base64-bytestring-1.0.0.1
- basement-0.0.3
- bifunctors-5.4.2
- binary-0.8.5.1
- bindings-uname-0.1
- bitarray-0.0.1.1
- blaze-builder-0.4.0.2
- blaze-html-0.9.0.1
- blaze-markup-0.8.0.0
- byteable-0.1.1
- bytestring-0.10.8.2
- call-stack-0.1.0
- case-insensitive-1.2.0.10
- cereal-0.5.4.0
- clock-0.7.2
- comonad-5.0.2
- conduit-1.2.12.1
- conduit-extra-1.1.17
- connection-0.2.8
- containers-0.5.10.2
- contravariant-1.4
- cookie-0.4.2.1
- cryptohash-0.11.9
- cryptohash-sha256-0.11.100.1
- cryptonite-0.24
- cryptonite-conduit-0.2.0
- data-default-class-0.1.2.0
- deepseq-1.4.3.0
- digest-0.0.1.2
- directory-1.3.0.2
- distributive-0.5.3
- dlist-0.8.0.3
- easy-file-0.2.1
- echo-0.1.3
- ed25519-0.0.5.0
- either-4.4.1.1
- exceptions-0.8.3
- extra-1.6
- fail-4.9.0.0
- fast-logger-2.4.10
- file-embed-0.0.10.1
- filelock-0.1.1.2
- filepath-1.4.1.2
- foundation-0.0.16
- free-4.12.4
- fsnotify-0.2.1.1
- generic-deriving-1.11.2
- ghc-boot-th-8.2.1
- ghc-prim-0.5.1.0
- gitrev-1.3.1
- hackage-security-0.5.2.2
- hashable-1.2.6.1
- hastache-0.6.1
- hfsevents-0.1.6
- hourglass-0.2.10
- hpack-0.19.3
- hpc-0.6.0.3
- hspec-2.4.4
- hspec-core-2.4.4
- hspec-discover-2.4.4
- hspec-expectations-0.8.2
- hspec-smallcheck-0.4.2
- http-api-data-0.3.7.1
- http-client-0.5.7.0
- http-client-tls-0.3.5.1
- http-conduit-2.2.3.2
- http-types-0.9.1
- ieee754-0.8.0
- integer-gmp-1.0.1.0
- integer-logarithms-1.0.2
- lifted-base-0.2.3.11
- logict-0.6.0.2
- memory-0.14.8
- microlens-0.4.8.1
- microlens-mtl-0.1.11.0
- microlens-th-0.4.1.1
- mime-types-0.1.0.7
- mintty-0.1.1
- mmorph-1.1.0
- monad-control-1.0.2.2
- monad-logger-0.3.25.1
- monad-loops-0.4.3
- mono-traversable-1.0.2.1
- mtl-2.2.1
- neat-interpolation-0.3.2.1
- network-2.6.3.2
- network-uri-2.6.1.0
- old-locale-1.0.0.7
- old-time-1.1.0.3
- open-browser-0.2.1.0
- optparse-applicative-0.14.0.0
- optparse-simple-0.0.3
- parsec-3.1.11
- path-0.6.1
- path-io-1.3.3
- path-pieces-0.2.1
- pem-0.2.2
- persistent-2.7.0
- persistent-sqlite-2.6.2
- persistent-template-2.5.3
- pid1-0.1.2.0
- prelude-extras-0.4.0.3
- pretty-1.1.3.3
- primitive-0.6.2.0
- process-1.6.1.0
- profunctors-5.2.1
- project-template-0.2.0
- quickcheck-io-0.2.0
- random-1.1
- regex-applicative-0.3.3
- regex-applicative-text-0.1.0.1
- resource-pool-0.2.3.2
- resourcet-1.1.9
- retry-0.7.4.2
- rts-1.0
- safe-0.3.15
- scientific-0.3.5.2
- semigroupoids-5.2.1
- semigroups-0.18.3
- setenv-0.1.1.3
- silently-1.2.5
- smallcheck-1.1.2
- socks-0.5.5
- split-0.2.3.2
- stm-2.4.4.1
- stm-chans-3.0.0.4
- store-0.4.3.2
- store-core-0.4.1
- streaming-commons-0.1.18
- syb-0.7
- tagged-0.8.5
- tar-0.5.0.3
- template-haskell-2.12.0.0
- temporary-1.2.1.1
- text-1.2.2.2
- text-metrics-0.3.0
- tf-random-0.5
- th-abstraction-0.2.6.0
- th-expand-syns-0.4.3.0
- th-lift-0.7.7
- th-lift-instances-0.1.11
- th-orphans-0.13.4
- th-reify-many-0.1.8
- th-utilities-0.2.0.1
- time-1.8.0.2
- time-locale-compat-0.1.1.3
- tls-1.4.0
- transformers-0.5.2.0
- transformers-base-0.4.4
- transformers-compat-0.5.1.4
- unicode-transforms-0.3.3
- unix-2.7.2.2
- unix-compat-0.4.3.1
- unix-time-0.3.7
- unliftio-0.1.1.0
- unliftio-core-0.1.0.0
- unordered-containers-0.2.8.0
- uri-bytestring-0.3.0.1
- uuid-types-1.0.3
- vector-0.12.0.1
- vector-algorithms-0.7.0.1
- void-0.7.2
- x509-1.7.2
- x509-store-1.6.5
- x509-system-1.6.6
- x509-validation-1.6.9
- yaml-0.8.23.3
- zip-archive-0.3.1.1
- zlib-0.6.1.2

Warning: this is an unsupported build that may have been built with different
versions of dependencies and GHC than the officially release binaries, and
therefore may not behave identically.  If you encounter problems, please try
the latest official build by running 'stack upgrade --force-download'.

2017-11-03 17:41:02.425345: [info] Getting project config file from STACK_YAML environment
@(Stack/Config.hs:875:13)
2017-11-03 17:41:02.425905: [debug] Loading project config file stack-lts-6.yaml
@(Stack/Config.hs:910:13)
2017-11-03 17:41:02.427351: [debug] Decoding build plan from: /Users/mark/.stack/build-plan/lts-6.35.yaml
@(Stack/Snapshot.hs:151:5)
2017-11-03 17:41:03.151444: [debug] Using standard GHC build
@(Stack/Setup.hs:613:9)
2017-11-03 17:41:03.151734: [debug] Asking GHC for its version
@(Stack/Setup/Installed.hs:98:13)
2017-11-03 17:41:03.152091: [debug] Getting Cabal package version
@(Stack/GhcPkg.hs:185:5)
2017-11-03 17:41:03.152199: [debug] Run process: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc --numeric-version
@(System/Process/Log.hs:37:3)
2017-11-03 17:41:03.152284: [debug] Getting global package database location
@(Stack/GhcPkg.hs:46:5)
2017-11-03 17:41:03.153220: [debug] Run process: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version
@(System/Process/Log.hs:37:3)
2017-11-03 17:41:03.153457: [debug] Run process: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Log.hs:37:3)
2017-11-03 17:41:03.188515: [debug] Process finished in [92m35ms[0m: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version
@(System/Process/Log.hs:44:3)
2017-11-03 17:41:03.190784: [debug] Process finished in [92m36ms[0m: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Log.hs:44:3)
2017-11-03 17:41:03.275509: [debug] Process finished in [92m123ms[0m: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc --numeric-version
@(System/Process/Log.hs:44:3)
2017-11-03 17:41:03.275650: [debug] GHC version is: ghc-7.10.3
@(Stack/Setup/Installed.hs:102:13)
2017-11-03 17:41:03.275736: [debug] Resolving package entries
@(Stack/Setup.hs:252:5)
2017-11-03 17:41:03.275859: [debug] Trying to decode /Users/mark/.stack/loaded-snapshot-cache/x86_64-osx/ghc-7.10.3/lts-6.35.cache
@(Data/Store/VersionTagged.hs:66:5)
2017-11-03 17:41:03.293739: [debug] Success decoding /Users/mark/.stack/loaded-snapshot-cache/x86_64-osx/ghc-7.10.3/lts-6.35.cache
@(Data/Store/VersionTagged.hs:70:13)
2017-11-03 17:41:03.294133: [debug] Starting to execute command inside EnvConfig
@(Stack/Runners.hs:170:18)
2017-11-03 17:41:03.296437: [info] Getting file list for /Users/mark/repos/shakers/
@(Stack/SDist.hs:117:5)
2017-11-03 17:41:03.296786: [debug] Parsing the targets
@(Stack/Build/Target.hs:473:3)
2017-11-03 17:41:03.320389: [debug] Start: getPackageFiles /Users/mark/repos/shakers/shakers.cabal
@(Stack/PrettyPrint.hs:134:16)
2017-11-03 17:41:03.321466: [debug] Finished in 0ms: getPackageFiles /Users/mark/repos/shakers/shakers.cabal
@(Stack/PrettyPrint.hs:134:16)
2017-11-03 17:41:03.322573: [debug] Getting global package database location
@(Stack/GhcPkg.hs:46:5)
2017-11-03 17:41:03.322712: [debug] Run process: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Log.hs:37:3)
2017-11-03 17:41:03.352515: [debug] Process finished in [92m29ms[0m: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Log.hs:44:3)
2017-11-03 17:41:03.353267: [debug] Run process: /Users/mark/.stack/setup-exe-cache/x86_64-osx/Cabal-simple_mPHDZzAJ_1.22.5.0_ghc-7.10.3 --builddir=.stack-work/dist/x86_64-osx/Cabal-1.22.5.0 sdist --list-sources /private/var/folders/wc/40h798sj7ln5gn8r6nmvqq880000gn/T/stack-sdist92658/source-files-list
@(System/Process/Log.hs:37:3)
2017-11-03 17:41:03.371695: [debug] Process finished in [92m18ms[0m: /Users/mark/.stack/setup-exe-cache/x86_64-osx/Cabal-simple_mPHDZzAJ_1.22.5.0_ghc-7.10.3 --builddir=.stack-work/dist/x86_64-osx/Cabal-1.22.5.0 sdist --list-sources /private/var/folders/wc/40h798sj7ln5gn8r6nmvqq880000gn/T/stack-sdist92658/source-files-list
@(System/Process/Log.hs:44:3)
2017-11-03 17:41:03.372790: [info] Building sdist tarball for /Users/mark/repos/shakers/
@(Stack/SDist.hs:119:5)
2017-11-03 17:41:03.373799: [debug] Parsing the targets
@(Stack/Build/Target.hs:473:3)
2017-11-03 17:41:03.397798: [debug] Start: getPackageFiles /Users/mark/repos/shakers/shakers.cabal
@(Stack/PrettyPrint.hs:134:16)
2017-11-03 17:41:03.398810: [debug] Finished in 0ms: getPackageFiles /Users/mark/repos/shakers/shakers.cabal
@(Stack/PrettyPrint.hs:134:16)
2017-11-03 17:41:03.399359: [debug] Finding out which packages are already installed
@(Stack/Build/Installed.hs:60:5)
2017-11-03 17:41:03.399485: [debug] Run process: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --global --no-user-package-db dump --expand-pkgroot
@(System/Process/Log.hs:37:3)
2017-11-03 17:41:03.431660: [debug] Process finished in [92m32ms[0m: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --global --no-user-package-db dump --expand-pkgroot
@(System/Process/Log.hs:44:3)
2017-11-03 17:41:03.432066: [debug] Ignoring package haskeline due to wanting version 0.7.4.0 instead of 0.7.2.1
@(Stack/Build/Installed.hs:190:5)
2017-11-03 17:41:03.432136: [debug] Ignoring package terminfo due to wanting version 0.4.1.0 instead of 0.4.0.1
@(Stack/Build/Installed.hs:190:5)
2017-11-03 17:41:03.432209: [debug] Ignoring package Cabal due to wanting version 1.22.8.0 instead of 1.22.5.0
@(Stack/Build/Installed.hs:190:5)
2017-11-03 17:41:03.432355: [debug] Run process: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --user --no-user-package-db --package-db /Users/mark/.stack/snapshots/x86_64-osx/lts-6.35/7.10.3/pkgdb dump --expand-pkgroot
@(System/Process/Log.hs:37:3)
2017-11-03 17:41:03.484011: [debug] Process finished in [92m51ms[0m: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --user --no-user-package-db --package-db /Users/mark/.stack/snapshots/x86_64-osx/lts-6.35/7.10.3/pkgdb dump --expand-pkgroot
@(System/Process/Log.hs:44:3)
2017-11-03 17:41:03.484600: [debug] Run process: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --user --no-user-package-db --package-db /Users/mark/repos/shakers/.stack-work/install/x86_64-osx/lts-6.35/7.10.3/pkgdb dump --expand-pkgroot
@(System/Process/Log.hs:37:3)
2017-11-03 17:41:03.511058: [debug] Process finished in [92m26ms[0m: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --user --no-user-package-db --package-db /Users/mark/repos/shakers/.stack-work/install/x86_64-osx/lts-6.35/7.10.3/pkgdb dump --expand-pkgroot
@(System/Process/Log.hs:44:3)
2017-11-03 17:41:03.517912: [info] Checking package 'shakers' for common mistakes
@(Stack/SDist.hs:359:5)
2017-11-03 17:41:03.518231: [warn] Package check reported the following warnings:
Packages using 'cabal-version: >= 1.10' must specify the 'default-language' field for each component (e.g. Haskell98 or Haskell2010). If a component uses different languages in different modules then list the other ones in the 'other-languages' field.
@(Stack/SDist.hs:370:9)
2017-11-03 17:41:03.519097: [info] Wrote sdist tarball to /Users/mark/repos/shakers/.stack-work/dist/x86_64-osx/Cabal-1.22.5.0/shakers-0.0.35.tar.gz
@(Stack/PrettyPrint.hs:63:22)

Stack version

See above ^^^.

Method of installation

Installed from HEAD.

mgsloan commented 7 years ago

Looks like this is an upstream bug in Cabal, I have opened https://github.com/haskell/cabal/issues/4863 with a repro. This version of stack now depends on Cabal 2.0 rather than Cabal 1.24. One of the big changes that has made the release long to allow for fixes.

Since stack-1.6 will be released imminently, I don't think there will be time for this to be fixed and a new version of it released. So, in order to make this clearer in the future, I have pushed a change which adds the following warning:

2017-11-03_733x214_scrot

mfine commented 7 years ago

Thanks for digging into this @mgsloan!!!

mgsloan commented 7 years ago

@mfine No problem!

Based on the discussion in https://github.com/haskell/cabal/issues/4863 , it sounds like this is fixed in development versions of Cabal. However, there are various experimental features that might change, and they are migrating to a new parser. It may still be worth a shot to try out:

Using the version of Cabal right after the fix was committed. The fix is in a quite large change, so unfortunately I doubt it would be straightforward to backport atop latest stable.
As suggested by @phadej , be sure to reject cabal-version: 2.1 and later in cabal files.

Since there's a fix in Cabal development version, I've updated the output:

Warning: Bug detected in Cabal library. ((parse . render . parse) === id) does not hold for the cabal file
         at /home/mgsloan/fpco/stack/stack.cabal

         This seems to be fixed in development versions of Cabal, but at time of writing, the fix is not in any released versions.

         Please see this GitHub issue for status: https://github.com/commercialhaskell/stack/issues/3549

         If the issue is closed as resolved, then you may be able to fix this by upgrading to a newer version of stack
         via stack upgrade for latest stable version or stack upgrade --git for the latest development version.

         If the issue is fixed, but updating doesn't solve the problem, please check if there are similar open issues,
         and if not, report a new issue to the stack issue tracker, at
         https://github.com/commercialhaskell/stack/issues/new

         If the issue is not fixed, feel free to leave a comment on it indicating that you would like it to be fixed.

psibi commented 7 years ago

I don't think this is a Cabal 2.0 bug specifically. I can reproduce this with Cabal-1.24.2.0 itself. Any workarounds for upload to hackage ?

mgsloan commented 7 years ago

@psibi I believe if you modify stack.yaml in stack repo to use Cabal from git, this should work. Might need to fix build errors if APIs changed. May be helpful to publish a branch with this change, others might benefit from it.

snoyberg commented 6 years ago

Just to update this issue with conclusions from elsewhere:

This is an upstream bug
This bug only affects user's of the pvp-bounds feature
- There was a separate issue temporarily on master which would affect other users, but that's been resolved
We're not going to fix this for Stack 1.6, as it would involve custom patching the Cabal code base, which is too risky

I think the takeaway here is simply that pvp-bounds will be unusable in Stack 1.6, and will continue to be unusable until a patched version of Cabal is officially released.

mfine commented 6 years ago

This looks resolved!

With 1.6.5:

Building sdist tarball for /Users/mark/repos/shakers/

Warning: Bug detected in Cabal library. ((parse . render . parse) === id) does not hold for the cabal file at /Users/mark/repos/shakers/shakers.cabal

         This seems to be fixed in development versions of Cabal, but at time of writing, the fix is not in any released versions.

         Please see this GitHub issue for status: https://github.com/commercialhaskell/stack/issues/3549

         If the issue is closed as resolved, then you may be able to fix this by upgrading to a newer version of stack via stack upgrade for latest stable version or stack upgrade --git for the latest
         development version.

         If the issue is fixed, but updating doesn't solve the problem, please check if there are similar open issues, and if not, report a new issue to the stack issue tracker, at
         https://github.com/commercialhaskell/stack/issues/new

         If the issue is not fixed, feel free to leave a comment on it indicating that you would like it to be fixed.

Checking package 'shakers' for common mistakes
Package check reported the following warnings:
Packages using 'cabal-version: >= 1.10' must specify the 'default-language' field for each component (e.g. Haskell98 or Haskell2010). If a component uses different languages in different modules then list the other ones in the 'other-languages' field.

With 1.7.0.1:

Building sdist tarball for /Users/mark/repos/shakers/
Checking package 'shakers' for common mistakes
Package check reported the following warnings:
Packages relying on Cabal 1.12 or later should specify a version range of the form 'cabal-version: x.y'. Use 'cabal-version: 1.22'.

The cabal stanzas for library and executable look fixed too:

With 1.6.5:

library
    exposed-modules:
        Development.Shakers
    build-depends:
        base >=4.8 && <5,
        basic-prelude >=0.7.0,
        directory >=1.3.0.2,
        lifted-base >=0.2.3.12,
        shake >=0.16.3
    default-language: Haskell2010
    hs-source-dirs: src
    ghc-options: -Wall

executable  shake-shakers
    main-is: Shakefile.hs

With 1.7.0.1:

library
    exposed-modules:
        Development.Shakers
    hs-source-dirs: src
    default-language: Haskell2010
    ghc-options: -Wall
    build-depends:
        base >=4.8 && <5,
        basic-prelude >=0.7.0,
        directory >=1.3.0.2,
        lifted-base >=0.2.3.12,
        shake >=0.16.3

executable shake-shakers
    main-is: Shakefile.hs
    default-language: Haskell2010
    ghc-options: -threaded -rtsopts -with-rtsopts=-N -Wall
    build-depends:
        base >=4.10.1.0,
        shakers -any

Thanks! Feel free to close if you think this is also resolved.

snoyberg commented 6 years ago

Awesome, thanks for checking on this!

commercialhaskell / stack