search

IntersectMBO / cardano-node

The core component that is used to participate in a Cardano decentralised blockchain.
https://cardano.org
Apache License 2.0
3.06k stars 720 forks source link

[BUG] - Build System: flag -systemd is not being respected by cabal-install #1200

Closed mark-stopka closed 4 years ago

mark-stopka commented 4 years ago

Exernal

Summary While there is clearly an attempt to remove systemd dependency in cardano-node, when trying to use it, cabal fails to resolve dependencies, see error below...

cabal: Could not resolve dependencies:
[__0] trying: lobemo-scribe-systemd-0.1.0.0 (user goal)
[__1] next goal: libsystemd-journal (dependency of lobemo-scribe-systemd)
[__1] rejecting: libsystemd-journal-1.4.4 (conflict: pkg-config package
libsystemd==209 || >209, not found in the pkg-config database)
[__1] rejecting: libsystemd-journal-1.4.3, libsystemd-journal-1.4.2,
libsystemd-journal-1.4.1, libsystemd-journal-1.4.0, libsystemd-journal-1.3.4,
libsystemd-journal-1.3.3, libsystemd-journal-1.3.1, libsystemd-journal-1.3.0,
libsystemd-journal-1.2.0, libsystemd-journal-1.1.0, libsystemd-journal-1.0.0
(constraint from project config TODO requires >=1.4.4)
[__1] fail (backjumping, conflict set: libsystemd-journal,
lobemo-scribe-systemd)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: libsystemd-journal,
lobemo-scribe-systemd

I am building on CentOS, using cabal-install-3.2.0.0 using following commands:

RUN cabal install cardano-cli \
      --installdir=/usr/local/bin \
      --install-method=copy \
      -f-systemd && \
    cabal install cardano-node \
      --install-method=copy \
      --installdir=/usr/local/bin \
      -f-systemd

I have also tried --flags="-systemd" notation with same results as per relevant cabal documentation. You can use following Dockerfile to reproduce the issue:

FROM perlur/centos-haskell-ghcup AS builder

ENV CABAL_VERSION="3.2.0.0"
ENV GHC_VERSION="8.6.5"

ENV CARDANO_NODE_VERSION="1.13.0"

RUN yum update -y && \
    yum install -y openssl-devel zlib-devel
RUN ghcup install-cabal ${CABAL_VERSION}
RUN ghcup install ${GHC_VERSION} && \
         ghcup set ${GHC_VERSION}

WORKDIR /usr/src
RUN git clone --recurse-submodules https://github.com/input-output-hk/cardano-node
WORKDIR cardano-node
RUN git fetch --tags
RUN git checkout ${CARDANO_NODE_VERSION}

# Ugly workaround for vty-5.25.1
RUN ln -s /usr/lib64/libtinfo.so.6 /usr/lib64/libtinfo.so
RUN cabal update
RUN cabal install cardano-cli \
      --installdir=/usr/local/bin \
      --install-method=copy \
      -f-systemd && \
    cabal install cardano-node \
      --install-method=copy \
      --installdir=/usr/local/bin \
      -f-systemd

FROM perlur/centos-base

MAINTAINER "Mark Stopka <mark.stopka@perlur.cloud>"

ENV SERVICE_NAME "cardano-node"

RUN yum update -y && \
    yum clean all && \
    dnf clean all && \
    rm -rf /var/cache/yum && \
    rm -rf /var/cache/dnf

EXPOSE 3000

COPY --from=builder /usr/local/bin/cardano-node /usr/local/bin/
COPY --from=builder /usr/local/bin/cardano-cli /usr/local/bin/
COPY usr/local/lib/* /usr/local/lib/
COPY usr/local/bin/* /usr/local/bin/

Expected behavior cabal-install will not require systemd-devel package to resolve dependencies and build.

System info (please complete the following information):

Screenshots and attachments

Additional context Add any other context about the problem here.

mark-stopka commented 4 years ago

Also see https://github.com/input-output-hk/cardano-node/pull/911/ for reference.

mark-stopka commented 4 years ago

Issue is also reproducible when running as follows:

(haskell-stack)[root@d0b84545610f /usr/src/cardano-node]$ cabal build all -f-systemd
HEAD is now at 4a457f4 Merge pull request #119 from input-output-hk/kderme/aeson-instances
HEAD is now at 2547ad1 Merge pull request #65 from dcoutts/dcoutts/drop-vrf-p256-openssl
HEAD is now at 3671fe88 Merge pull request #1517 from input-output-hk/nc/roll-vrf
HEAD is now at e0257be Merge #112
HEAD is now at 601bb43 Merge #361
HEAD is now at 43a036c Replace Universum with Cardano.Prelude
HEAD is now at 26d35ad Add SeedGoblin instance for `Maybe a`
HEAD is now at 71df15f Merge #563
HEAD is now at 16bca0814 Merge #2220
HEAD is now at 1a75bdf Update pipelines machines
HEAD is now at 42a8319 Merge pull request #223 from Jimbo4350/jordan/update-pprint
Warning: Requested index-state2020-04-01T00:00:00Z is newer than
'hackage.haskell.org'! Falling back to older state (2020-03-31T22:47:49Z).
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] trying: lobemo-scribe-systemd-0.1.0.0 (user goal)
[__1] next goal: libsystemd-journal (dependency of lobemo-scribe-systemd)
[__1] rejecting: libsystemd-journal-1.4.4 (conflict: pkg-config package
libsystemd==209 || >209, not found in the pkg-config database)
[__1] rejecting: libsystemd-journal-1.4.3, libsystemd-journal-1.4.2,
libsystemd-journal-1.4.1, libsystemd-journal-1.4.0, libsystemd-journal-1.3.4,
libsystemd-journal-1.3.3, libsystemd-journal-1.3.1, libsystemd-journal-1.3.0,
libsystemd-journal-1.2.0, libsystemd-journal-1.1.0, libsystemd-journal-1.0.0
(constraint from project config TODO requires >=1.4.4)
[__1] fail (backjumping, conflict set: libsystemd-journal,
lobemo-scribe-systemd)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: libsystemd-journal,
lobemo-scribe-systemd

(haskell-stack)[root@d0b84545610f /usr/src/cardano-node]$

or

RUN cabal build cardano-config \
      -f-systemd
RUN cabal install cardano-cli \
      --installdir=/usr/local/bin \
      --install-method=copy \
      -f-systemd
RUN cabal install cardano-node \
      --install-method=copy \
      --installdir=/usr/local/bin \
      -f-systemd
dcoutts commented 4 years ago

Try in your cabal.project.local

package cardano-config
  flag: -systemd
mark-stopka commented 4 years ago

Sadly no joy with either for 1.14.2 version of cardano-node...

[mark_stopka@docker-host-01 1.14.2]$ cat cabal.project.local
package cardano-crypto-praos
  flags: -external-libsodium-vrf
package cardano-config
  flag: -systemd

or

[mark_stopka@docker-host-01 1.14.2]$ cat cabal.project.local
package cardano-crypto-praos
  flags: -external-libsodium-vrf
package cardano-config
  flags: -systemd

Error is still the same:

Step 16/37 : RUN cabal install cardano-cli       --installdir=/usr/local/bin       --install-method=copy       -f -systemd       -f -external-libsodium-vrf
 ---> Running in 599333fd344f
cabal: Could not resolve dependencies:
[__0] trying: lobemo-scribe-systemd-0.1.0.0 (user goal)
[__1] next goal: libsystemd-journal (dependency of lobemo-scribe-systemd)
[__1] rejecting: libsystemd-journal-1.4.4 (conflict: pkg-config package
libsystemd==209 || >209, not found in the pkg-config database)
[__1] rejecting: libsystemd-journal-1.4.3, libsystemd-journal-1.4.2,
libsystemd-journal-1.4.1, libsystemd-journal-1.4.0, libsystemd-journal-1.3.4,
libsystemd-journal-1.3.3, libsystemd-journal-1.3.1, libsystemd-journal-1.3.0,
libsystemd-journal-1.2.0, libsystemd-journal-1.1.0, libsystemd-journal-1.0.0
(constraint from project config TODO requires >=1.4.4)
[__1] fail (backjumping, conflict set: libsystemd-journal,
lobemo-scribe-systemd)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: libsystemd-journal,
lobemo-scribe-systemd

The command '/bin/sh -c cabal install cardano-cli       --installdir=/usr/local/bin       --install-method=copy       -f -systemd       -f -external-libsodium-vrf' returned a non-zero code: 1
mark-stopka commented 4 years ago

I confirm this issue is still present in 97b3e95.

Step 16/38 : RUN cabal build cardano-cli cardano-node       -f -systemd       -f -external-libsodium-vrf
 ---> Running in 5f2dcca85e26
HEAD is now at 5e0b8bc Merge #131
HEAD is now at 2547ad1 Merge pull request #65 from dcoutts/dcoutts/drop-vrf-p256-openssl
HEAD is now at 12b13f39 Refactor nonce transitions in the exec spec.
HEAD is now at 316c854 Merge #113
HEAD is now at 5257fc8 Add Jimbo4350 to CODEOWNERS (#367)
HEAD is now at 43a036c Replace Universum with Cardano.Prelude
HEAD is now at 26d35ad Add SeedGoblin instance for `Maybe a`
HEAD is now at efa4b5e Merge #568
HEAD is now at 1361bb928 Merge #2321
HEAD is now at 1a75bdf Update pipelines machines
HEAD is now at 42a8319 Merge pull request #223 from Jimbo4350/jordan/update-pprint
Warning: Requested index-state2020-05-15T00:00:00Z is newer than
'hackage.haskell.org'! Falling back to older state (2020-05-14T23:39:58Z).
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] trying: lobemo-scribe-systemd-0.1.0.0 (user goal)
[__1] next goal: libsystemd-journal (dependency of lobemo-scribe-systemd)
[__1] rejecting: libsystemd-journal-1.4.4 (conflict: pkg-config package
libsystemd==209 || >209, not found in the pkg-config database)
[__1] rejecting: libsystemd-journal-1.4.3, libsystemd-journal-1.4.2,
libsystemd-journal-1.4.1, libsystemd-journal-1.4.0, libsystemd-journal-1.3.4,
libsystemd-journal-1.3.3, libsystemd-journal-1.3.1, libsystemd-journal-1.3.0,
libsystemd-journal-1.2.0, libsystemd-journal-1.1.0, libsystemd-journal-1.0.0
(constraint from project config TODO requires >=1.4.4)
[__1] fail (backjumping, conflict set: libsystemd-journal,
lobemo-scribe-systemd)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: libsystemd-journal,
lobemo-scribe-systemd
mark-stopka commented 4 years ago

Can reproduce with 1.16.0...

Config file path source is default config file.
Config file /root/.cabal/config not found.
Writing default configuration to /root/.cabal/config
Cloning into '/usr/src/cardano-node/dist-newstyle/src/cardano-b_-ef6f6555066e7b95'...
HEAD is now at 7d795c3 Merge pull request #144 from input-output-hk/dcoutts/hashing-api
Cloning into '/usr/src/cardano-node/dist-newstyle/src/cardano-c_-ff3e781b30c95867'...
HEAD is now at 2547ad1 Merge pull request #65 from dcoutts/dcoutts/drop-vrf-p256-openssl
Cloning into '/usr/src/cardano-node/dist-newstyle/src/cardano-l_-be5e6979fa855cac'...
HEAD is now at 550a70b9 Merge pull request #1672 from input-output-hk/nc/flag-assert
Cloning into '/usr/src/cardano-node/dist-newstyle/src/cardano-p_-29c69bdfb71ad002'...
HEAD is now at 316c854 Merge #113
Cloning into '/usr/src/cardano-node/dist-newstyle/src/cardano-s_-2fdcc826cd28bde1'...
HEAD is now at 43a036c Replace Universum with Cardano.Prelude
Cloning into '/usr/src/cardano-node/dist-newstyle/src/goblins-8f13f67cff0daf8c'...
HEAD is now at 26d35ad Add SeedGoblin instance for `Maybe a`
Cloning into '/usr/src/cardano-node/dist-newstyle/src/iohk-moni_-9bf5314d46ca988d'...
HEAD is now at efa4b5e Merge #568
Cloning into '/usr/src/cardano-node/dist-newstyle/src/ouroboros_-e7dffa0d85e2839'...
HEAD is now at 17d5199fd Merge #2417
Cloning into '/usr/src/cardano-node/dist-newstyle/src/http-clie_-2b133d3370a8857e'...
HEAD is now at 1a75bdf Update pipelines machines
Cloning into '/usr/src/cardano-node/dist-newstyle/src/cborg-6928f26268cb21b1'...
HEAD is now at 42a8319 Merge pull request #223 from Jimbo4350/jordan/update-pprint
Downloading the latest package list from hackage.haskell.org
Removing intermediate container bf290157da4e
 ---> 6e083b484b62
Step 16/37 : RUN cabal install cardano-cli       --installdir=/usr/local/bin       --install-method=copy       -f -systemd       -f -external-libsodium-vrf
 ---> Running in 206a3dfad696
cabal: Could not resolve dependencies:
[__0] trying: lobemo-scribe-systemd-0.1.0.0 (user goal)
[__1] next goal: libsystemd-journal (dependency of lobemo-scribe-systemd)
[__1] rejecting: libsystemd-journal-1.4.4 (conflict: pkg-config package
libsystemd==209 || >209, not found in the pkg-config database)
[__1] rejecting: libsystemd-journal-1.4.3, libsystemd-journal-1.4.2,
libsystemd-journal-1.4.1, libsystemd-journal-1.4.0, libsystemd-journal-1.3.4,
libsystemd-journal-1.3.3, libsystemd-journal-1.3.1, libsystemd-journal-1.3.0,
libsystemd-journal-1.2.0, libsystemd-journal-1.1.0, libsystemd-journal-1.0.0
(constraint from project config TODO requires >=1.4.4)
[__1] fail (backjumping, conflict set: libsystemd-journal,
lobemo-scribe-systemd)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: libsystemd-journal,
lobemo-scribe-systemd

The command '/bin/sh -c cabal install cardano-cli       --installdir=/usr/local/bin       --install-method=copy       -f -systemd       -f -external-libsodium-vrf' returned a non-zero code: 1

To be honest, I am puzzled by how the solver actually works anyway... I was experimenting with compiling using ghc-8.10.1 yesterday, and I had to manually remove Win32-network source-repository-package package section from cabal.project entirely to get around my first round of dependency solver issues :) I of course crashed and burned later on other packages, but IMO the solver on unix platforms should not even evaluate the requirements and constraints of before-mentioned package...

hasufell commented 4 years ago

This is due to https://github.com/haskell/cabal/issues/5444

Anything in source-repository-package is not optional.

vix-io commented 4 years ago

hi @mark-stopka , @hasufell can you still reproduce this?

hasufell commented 4 years ago

@vix-io yes, see https://github.com/input-output-hk/cardano-node/pull/1775

mark-stopka commented 4 years ago

@mimi1vx, is this something you would be comfortable to fix? Drop me a mail if so, we can agree details.

hasufell commented 4 years ago

@mark-stopka this is already fixed in my PR.

mark-stopka commented 4 years ago

Oh, sorry, my bad, I am now verifying with cherry-pick commit b143dd14c6474d2dba6707057ff465604cbebc6b, will let you know...

mark-stopka commented 4 years ago

@hasufell, still same issue 

cabal: Could not resolve dependencies:
[__0] trying: lobemo-scribe-systemd-0.1.0.0 (user goal)
[__1] next goal: libsystemd-journal (dependency of lobemo-scribe-systemd)
[__1] rejecting: libsystemd-journal-1.4.4 (conflict: pkg-config package
libsystemd==209 || >209, not found in the pkg-config database)
[__1] rejecting: libsystemd-journal-1.4.3, libsystemd-journal-1.4.2,
libsystemd-journal-1.4.1, libsystemd-journal-1.4.0, libsystemd-journal-1.3.4,
libsystemd-journal-1.3.3, libsystemd-journal-1.3.1, libsystemd-journal-1.3.0,
libsystemd-journal-1.2.0, libsystemd-journal-1.1.0, libsystemd-journal-1.0.0
(constraint from project config TODO requires >=1.4.4)
[__1] fail (backjumping, conflict set: libsystemd-journal,
lobemo-scribe-systemd)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: libsystemd-journal,
lobemo-scribe-systemd

my build process below

FROM perlur/centos-haskell-ghcup AS builder

ENV CABAL_VERSION="3.2.0.0"
ENV GHC_VERSION="8.6.5"

ENV CARDANO_NODE_VERSION="1.20.0"

RUN yum update -y && \
    yum install -y libsodium-devel openssl-devel zlib-devel
RUN ghcup install cabal ${CABAL_VERSION}
RUN ghcup install ghc ${GHC_VERSION} && \
    ghcup set ghc ${GHC_VERSION}

WORKDIR /usr/src
RUN git clone --recurse-submodules https://github.com/input-output-hk/cardano-node
WORKDIR /usr/src/cardano-node
RUN git remote add hasufell https://github.com/hasufell/cardano-node.git
RUN git fetch --all
RUN git checkout ${CARDANO_NODE_VERSION}
RUN git cherry-pick -n b143dd14c6474d2dba6707057ff465604cbebc6b
COPY cabal.project.local ./

# Ugly workaround for vty-5.25.1
RUN ln -s /usr/lib64/libtinfo.so.6 /usr/lib64/libtinfo.so
RUN cabal update
RUN cabal install cardano-cli \
      --installdir=/usr/local/bin \
      --install-method=copy \
      -f -systemd \
      -f -external-libsodium-vrf \
      --minimize-conflict-set
RUN cabal install cardano-node \
      --installdir=/usr/local/bin \
      --install-method=copy \
      -f -systemd \
      -f -external-libsodium-vrf
hasufell commented 4 years ago

@mark-stopka as described in the PR, there is a script that generates new project files due to a cabal bug. You have to:

mark-stopka commented 4 years ago

@mark-stopka this is already fixed in my PR.

Yes, I noticed, I am testing that now, thanks for the info! And sorry for not noticing sooner.

mark-stopka commented 4 years ago

I confirm https://github.com/input-output-hk/cardano-node/pull/1775 fixes this issue.

ghost commented 4 years ago

I would still note that this bug is NOT fixed with the latest 1.21.1, the file needed to be manually patched otherwise:

HEAD is now at a6ce8fed Merge pull request #1884 from input-output-hk/ts-slow-mode
HEAD is now at d4bb653 Merge #588
HEAD is now at 9e498e096 Merge #2645
Warning: Requested index-state2020-07-15T00:00:00Z is newer than
'hackage.haskell.org'! Falling back to older state (2020-07-14T22:50:55Z).
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] trying: lobemo-scribe-systemd-0.1.0.0 (user goal)
[__1] next goal: libsystemd-journal (dependency of lobemo-scribe-systemd)
[__1] rejecting: libsystemd-journal-1.4.4 (conflict: pkg-config package
libsystemd==209 || >209, not found in the pkg-config database)
[__1] rejecting: libsystemd-journal-1.4.3, libsystemd-journal-1.4.2,
libsystemd-journal-1.4.1, libsystemd-journal-1.4.0, libsystemd-journal-1.3.4,
libsystemd-journal-1.3.3, libsystemd-journal-1.3.1, libsystemd-journal-1.3.0,
libsystemd-journal-1.2.0, libsystemd-journal-1.1.0, libsystemd-journal-1.0.0
(constraint from project config TODO requires >=1.4.4)
[__1] fail (backjumping, conflict set: libsystemd-journal,
lobemo-scribe-systemd)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: libsystemd-journal,
lobemo-scribe-systemd

The proper answer is:

  scripts/gen-cabal-nosystemd.sh
  cabal build --project-file=cabal.nosystemd.project cardano-cli cardano-node

At least take your 2 seconds and provide a proper answer not some untested rant. Could we have a script checking NOT if systemd is installed because sadly removing the main systemd package can leave systemd residues in your system but checking IF systemd is the current init system and if it's not then it will by default disable systemd at compilation?

I don't want to have this dance at the next version.

mark-stopka commented 4 years ago

This is how it is documented in the relevant PR, that you need to use the script that generates custom Cabal project file...

-- Sent from mobile Best regards / S pozdravem, BSc. Mark Stopka, BBA Managing Partner @ PERLUR Group

mobile: +420 704 373 561 website: www.perlur.cloud

On Sun, Oct 11, 2020, 09:24 feraldruid22 notifications@github.com wrote:

I would still note that this bug is NOT fixed with the latest 1.21.1, the file needed to be manually patched otherwise:

HEAD is now at a6ce8fed Merge pull request #1884 from input-output-hk/ts-slow-mode HEAD is now at d4bb653 Merge #588 HEAD is now at 9e498e096 Merge #2645 Warning: Requested index-state2020-07-15T00:00:00Z is newer than 'hackage.haskell.org'! Falling back to older state (2020-07-14T22:50:55Z). Resolving dependencies... cabal: Could not resolve dependencies: [0] trying: lobemo-scribe-systemd-0.1.0.0 (user goal) [1] next goal: libsystemd-journal (dependency of lobemo-scribe-systemd) [1] rejecting: libsystemd-journal-1.4.4 (conflict: pkg-config package libsystemd==209 || >209, not found in the pkg-config database) [1] rejecting: libsystemd-journal-1.4.3, libsystemd-journal-1.4.2, libsystemd-journal-1.4.1, libsystemd-journal-1.4.0, libsystemd-journal-1.3.4, libsystemd-journal-1.3.3, libsystemd-journal-1.3.1, libsystemd-journal-1.3.0, libsystemd-journal-1.2.0, libsystemd-journal-1.1.0, libsystemd-journal-1.0.0 (constraint from project config TODO requires >=1.4.4) [__1] fail (backjumping, conflict set: libsystemd-journal, lobemo-scribe-systemd) After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: libsystemd-journal, lobemo-scribe-systemd

The proper answer is:

scripts/gen-cabal-nosystemd.sh cabal build --project-file=cabal.nosystemd.project cardano-cli cardano-node

At least take your 2 seconds and provide a proper answer not some untested rant. Could we have a script checking NOT if systemd is installed because sadly removing the main systemd package can leave systemd residues in your system but checking IF systemd is the current init system and if it's not then it will by default disable systemd at compilation?

I don't want to have this dance at the next version.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/input-output-hk/cardano-node/issues/1200#issuecomment-706662953, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEKMP3GIXSZG6OLQPXHVYETSKFMUPANCNFSM4NXDYNDQ .

hasufell commented 4 years ago

At least take your 2 seconds and provide a proper answer not some untested rant. Could we have a script checking NOT if systemd is installed because sadly removing the main systemd package can leave systemd residues in your system but checking IF systemd is the current init system and if it's not then it will by default disable systemd at compilation?

I'm not entirely sure what you're asking. The script works, does it not? Are you planning to build the node in an automated fashion across different systems that may or may not have systemd?

You could do something like:

if pkg-config --exists libsystemd ; then
   cabal build cardano-cli cardano-node
else
  scripts/gen-cabal-nosystemd.sh
  cabal build --project-file=cabal.nosystemd.project cardano-cli cardano-node
fi
mark-stopka commented 4 years ago

@hasufell, exactly, he can easily create own wrapper script...

I am, as a requester closing this issue, it works perfectly for 1.21.1 and 1.21.2...

MartinSchere commented 3 years ago

Thanks a lot @hasufell! I can confirm it fixes it for an alpine-based linux image