Raku / tap-harness6

A TAP harness for Raku
Artistic License 2.0
8 stars 15 forks source link

Getting an error in Windows during CI tests on Github that Tap won't install #61

Open sdondley opened 1 year ago

sdondley commented 1 year ago

Please see: https://github.com/sdondley/Directory/actions/runs/3972369620/jobs/6810208097

Error:

Run zef install --/test App::Prove6
[4](https://github.com/sdondley/Directory/actions/runs/3972369620/jobs/6810208097#step:5:5)
===> Searching for: App::Prove6
[5](https://github.com/sdondley/Directory/actions/runs/3972369620/jobs/6810208097#step:5:6)

[6](https://github.com/sdondley/Directory/actions/runs/3972369620/jobs/6810208097#step:5:7)
===> Searching for missing dependencies: Getopt::Long:ver<0.3.0+>, Path::Finder:ver<0.4.4+>, Pod::Usage, TAP:ver<0.3.4+>, sigpipe
[7](https://github.com/sdondley/Directory/actions/runs/3972369620/jobs/6810208097#step:5:8)
===> Installing: Getopt::Long:ver<0.4.1>
[8](https://github.com/sdondley/Directory/actions/runs/3972369620/jobs/6810208097#step:5:9)
===> Installing: Path::Finder:ver<0.4.5>
[9](https://github.com/sdondley/Directory/actions/runs/3972369620/jobs/6810208097#step:5:10)
===> Installing: Pod::Usage:ver<0.0.1>:auth<zef:leont>
[10](https://github.com/sdondley/Directory/actions/runs/3972369620/jobs/6810208097#step:5:11)
===> Installing: TAP:ver<0.3.13>
[11](https://github.com/sdondley/Directory/actions/runs/3972369620/jobs/6810208097#step:5:12)
===> Install [FAIL] for TAP:ver<0.3.13>: 
[12](https://github.com/sdondley/Directory/actions/runs/3972369620/jobs/6810208097#step:5:13)
Error: Process completed with exit code 1.
sdondley commented 1 year ago

I tested on a Wndows machine and TAP:ver<0.3.13> installed just fine so not sure what the issue is with github.

Leont commented 1 year ago

Yeah, this has been reported to me before and no-one can reproduce it outside of github-ci. It's very mysterious, and I suspect the issue isn't really related to TAP anyway (but that's just a guess), it's just the first to trip over whatever's wrong.

sdondley commented 1 year ago

Whoa, wtf? I modified the yaml file from zef install --/test App::Prove6 to zef install --verbose --/test App::Prove6 to see if I could get more info and now the test passes. Not sure if that's a fluke or what.

Logs are here:

https://github.com/sdondley/Directory/actions/runs/3976471886/jobs/6816975623

jaguart commented 1 year ago

It's a heisenbug - comes and goes - I think ugexe got closest to pinning it down to something in precompile in 2022.12. He points to the install dying here: https://github.com/rakudo/rakudo/blob/5f7ff013deb8fcab3e3599a50e2a77444dd0582b/src/core.c/CompUnit/PrecompilationRepository.pm6#L461 - but without a proper message.

He thinks perhaps: https://irclogs.raku.org/raku/2023-01-04.html#01:34

jeffythedragonslayer commented 1 year ago

I think "neutrinobug" is a better name for an intermittent problem - like neutrino oscillations.

jaguart commented 1 year ago

or even Schrodinger's Kittens

sdondley commented 1 year ago

OK, it looks iike simply re-running the failed job for windows separately can correct the problem on GitHub.

  1. Go to the workflows page on github for your distribution
  2. Click on last failed workflow
  3. Rerun the failing windows by itself by clicking the refresh icon next to the failed workflow for windows
atroxaper commented 1 year ago

@sdondley It is not works for me...

sdondley commented 1 year ago

@atroxaper yeah, I discovered it can sometimes take several attempts. Sometimes you get lucky and it works the very first time.

ugexe commented 1 year ago

I've had a github action for zef fail 15+ times in a row; it has never passed. At some point of consistent failure the action passing is only a coincidence and we have to accept there is a legitimate bug

jaguart commented 1 year ago

I've switched from 'latest' to '2022.07' for Windows tests, and TAP install is solid / never fails.

sdondley commented 1 year ago

I downgraded to 2022.07 for all OS's:

jobs:
  raku:
    strategy:
      matrix:
        os:
          - ubuntu-latest
          - macos-latest
          - windows-latest
        raku-version:
          - '2022.07'

How do I set it just for windows?

UPDATE: Figured it out. At least, this seems to work as I intend:

name: test

on:
  push:
    branches:
      - '*'
    tags-ignore:
      - '*'
  pull_request:

jobs:
  raku_nix:
    strategy:
      matrix:
        os:
          - ubuntu-latest
          - macos-latest
        raku-version:
          - 'latest'
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v3
      - uses: Raku/setup-raku@v1
        with:
          raku-version: ${{ matrix.raku-version }}
      - name: Install Dependencies
        run: zef install --/test --test-depends --deps-only .
      - name: Install App::Prove6
        run: zef install --verbose --/test App::Prove6
      - name: Run Tests
        run: raku -I.,lib t/01-basic.rakutest

  raku_win:
    strategy:
      matrix:
        os:
          - windows-latest
        raku-version:
          - '2022.07'
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v3
      - uses: Raku/setup-raku@v1
        with:
          raku-version: ${{ matrix.raku-version }}
      - name: Install Dependencies
        run: zef install --/test --test-depends --deps-only .
      - name: Install App::Prove6
        run: zef install --verbose --/test App::Prove6
      - name: Run Tests
        run: raku -I.,lib t/01-basic.rakutest
ugexe commented 1 year ago

https://github.com/ugexe/zef/actions/runs/4002030615/jobs/6878017528#step:9:21 is an example of zef failing its own test, including the --ll-exception output

sdondley commented 1 year ago

I'm also seeing similar strangness when running 2022.07 on windows latest for a different module.

Attempt #1 fails: https://github.com/sdondley/Distribution-Extension-Updater/actions/runs/4008470899/attempts/1 Attempt #2 succeeds: https://github.com/sdondley/Distribution-Extension-Updater/actions/runs/4008470899

I changed nothing between attempt #1 and attempt #2.

2colours commented 1 year ago

Is this https://github.com/rakudo/rakudo/issues/5168 ?

JJ commented 1 year ago

Is #62 related?

tbrowder commented 1 year ago

I just thought of my similar problem running xtest for docs. Could it be a subtle problem with non-linear execution?

UPDATE

I just changed, for Windows only, a test for one of my failing Windows modules to Rakudo version '2022.07' and it's now passing!

JJ commented 1 year ago

So the workaround is to fix the Windows version to 2022.07, right? No hint to what could be the actual error in Windows?

pmqs commented 1 year ago

Just triggered this issue on a windows github build https://github.com/pmqs/Archive-SimpleZip/actions/runs/5103285816/attempts/1

...
===> Searching for missing dependencies: Getopt::Long:ver<0.3.0+>, Path::Finder:ver<0.4.4+>, Pod::Usage, TAP:ver<0.3.4+>, sigpipe
===> Installing: Getopt::Long:ver<0.4.2>
===> Installing: Path::Finder:ver<0.4.5>
===> Installing: Pod::Usage:ver<0.0.1>:auth<zef:leont>
===> Installing: TAP:ver<0.3.14>
===> Install [FAIL] for TAP:ver<0.3.14>: 
Error: Process completed with exit code 1.

Rerunning the job made that problem go away.

coke commented 11 months ago

Discussion on IRC suggested this might be related to long path issues on windows; please try running the tests inside of a git clone of the repo starting from a very long working path (> 300 chars)

pmqs commented 10 months ago

Discussion on IRC suggested this might be related to long path issues on windows; please try running the tests inside of a git clone of the repo starting from a very long working path (> 300 chars)

I think Windows has a max path length of 256 chars

Yep, just tried & it complained

JJ commented 9 months ago

So I guess this issue is still open? But not specific to this repo?

2colours commented 9 months ago

I'm not sure it's only that error. Honestly, I never liked TAP and can't seem to find any good reason to go back to it for any CI but one thing is sure: not all repos fail on the path issue, in fact, most of them don't. I don't know, could be that TAP itself does, it couldn't really be solved so far, from what I understand.