search

JCSDA / spack-stack

Creative Commons Zero v1.0 Universal
27 stars 48 forks source link

[INSTALL]: GitHub CLI #1105

Open TerrenceMcGuinness-NOAA opened 6 months ago

TerrenceMcGuinness-NOAA commented 6 months ago

Package name

gh

Package version/tag

v2.28.0

Build options

None

Installation timeframe

Only needed on all supported RDHPCS systems. Already in lmod on WCOSS2.

Other information

No response

WCOSS2

WCOSS2: General questions

No response

WCOSS2: Installation and testing

No response

WCOSS2: Technical & security review list

WCOSS2: Additional comments

No response

mathomp4 commented 5 months ago

gh v2.49.2 is now in spack mainline. I'm planning a PR for gh 2.50.0 now.

climbfuji commented 3 months ago

global-workflow needs version 2.28 or something close

eap commented 3 months ago

@TerrenceMcGuinness-NOAA - I'm a little confused on this issue. GitHub CLI 2.28 is a bit over 2 years old and is missing quite a few features and bug-fixes in the current release. Can global-workflow be updated to use a more recent version of the CLI?

aerorahul commented 3 months ago

@climbfuji The latest version of gh-cli is fine in ss-1.8.0

climbfuji commented 3 months ago

Thanks @aerorahul! @eap do you want to go ahead and add a recent version of gh cli?

eap commented 2 months ago

Adding an update to track status: the gh CLI requires golang which breaks intel platforms. This has been pushed to 1.9 pending fixes for go

mathomp4 commented 2 months ago

Adding an update to track status: the gh CLI requires golang which breaks intel platforms. This has been pushed to 1.9 pending fixes for go

Query: How does golang break intel? I mean...I've built go programs on Intel chips before.

climbfuji commented 2 months ago

go doesn't build with anything else than gcc in spack, and on many systems go isn't available as an external package. We now have the functionality in place to tell spack(-stack) to always build go with gcc, regardless of what the principal compiler for the environment is. But this was too close for release/1.8.0 to be implemented and tested.

mathomp4 commented 2 months ago

Ohhh. Intel the compiler! I've been thinking about chips too long. Makes more sense now. :)

eap commented 2 weeks ago

I'm assigning this to Steven G who should have a chance to look at it next sprint. If this is a straightforward fix as expected I'd expect this to be in for 1.9. But I'll hedge a little here and wait for Steve's assessment of points once he's had time to look at it in depth.

stiggy87 commented 2 weeks ago

I took a look into this, and it looks like updating the spack.yaml works for building it.

 [...]
  packages:
    all:
      prefer: ['%gcc', '%intel']
      providers:
        gh: [go, go-bootstrap]

Which could be added by: 

spack config add "packages:all:prefer:['%gcc','%intel']"
spack config add "packages:all:providers:gh:[go,go-bootstrap]"

The spack spec:

# spack spec gh
Input spec
--------------------------------
 -   gh

Concretized
--------------------------------
[+]  gh@2.50.0%gcc@11.4.0 build_system=generic arch=linux-rocky8-icelake
[+]      ^gcc-runtime@11.4.0%gcc@11.4.0 build_system=generic arch=linux-rocky8-icelake
[e]      ^glibc@2.28%gcc@11.4.0 build_system=autotools arch=linux-rocky8-icelake
[+]      ^go@1.22.2%gcc@11.4.0 build_system=generic arch=linux-rocky8-icelake
[e]          ^git@2.34.1%gcc@11.4.0+man+nls+perl+subtree~svn~tcltk build_system=autotools arch=linux-rocky8-icelake
[+]          ^go-bootstrap@1.20.6%gcc@11.4.0 build_system=generic arch=linux-rocky8-icelake

I tested this a few times with an intel environment of spack-stack-1.8. What would be the best way to add this to the global-work?

climbfuji commented 1 week ago

I am confused by the line providers: gh: ... is this supposed to be providers: go: ...? There is no virtual gh package. If it is go, then this part is ok to add to configs/common/packages.yaml.

There can only be one preferred compiler. I guess the other missing piece for you probably is to add a specific requirement to always build go or go-bootstrap with gcc. See existing use cases in configs/common/packages_*.yaml. This way, you won't need gcc listed as a preferred compiler.

stiggy87 commented 1 week ago

I am confused by the line providers: gh: ... is this supposed to be providers: go: ...? There is no virtual gh package. If it is go, then this part is ok to add to configs/common/packages.yaml.

I was following spack documentation on a path to make it work, and this is what I gathered from that

I forgot to mention that I did a spack add gh.

I was reading spack documentation on how to prefer specific compilers for packages and this is what I came up with that worked.

I'll try out specifying go and go-bootstrap to use gcc as you mention.

climbfuji commented 1 week ago

Providers are virtual packages that can satisfy dependencies for the virtual package with several backends.

$ spack providers
Virtual packages:
    D      blas  daal      flame       fuse  go-or-gccgo-bootstrap  ipp   lapack       libglx     lua-lang        mkl  mysql-client  pbs        qmake      sycl  unwind    xxd      zlib-api
    armci  c     elf       fortran     gl    golang                 java  libc         libifcore  luajit          mpe  onedal        pil        rpc        szip  uuid      yacc
    awk    cxx   fftw-api  fortran-rt  glu   iconv                  jpeg  libgfortran  libllvm    mariadb-client  mpi  opencl        pkgconfig  scalapack  tbb   wasi-sdk  ziglang

There is no virtual package gh. Can you share the link that told you to specify gh as a virtual package?

climbfuji commented 1 week ago

Or, in other words, the line gh: [go, go-bootstrap] would mean that "either go or go-bootstrap can be used to provide and satisfy the requirements for gh" - but neither go nor go-bootstrap provide the gh cli!

stiggy87 commented 1 week ago

You’re right! I misread the documentation from Spack on it. How it worked for me I don’t know.

So we can safely ignore my previous comment. I’ll update local package.yml to require gcc and make sure it works that way.

On Nov 15, 2024, at 10:09 AM, Dom Heinzeller @.***> wrote:

Providers are virtual packages that can satisfy dependencies for the virtual package with several backends.

$ spack providers Virtual packages: D blas daal flame fuse go-or-gccgo-bootstrap ipp lapack libglx lua-lang mkl mysql-client pbs qmake sycl unwind xxd zlib-api armci c elf fortran gl golang java libc libifcore luajit mpe onedal pil rpc szip uuid yacc awk cxx fftw-api fortran-rt glu iconv jpeg libgfortran libllvm mariadb-client mpi opencl pkgconfig scalapack tbb wasi-sdk ziglang There is no virtual package gh. Can you share the link that told you to specify gh as a virtual package?

— Reply to this email directly, view it on GitHub https://github.com/JCSDA/spack-stack/issues/1105#issuecomment-2479458002, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEJ6FNQBHJP7NOOTPOTR3D2AYTCZAVCNFSM6AAAAABHJPLUXWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINZZGQ2TQMBQGI. You are receiving this because you were assigned.