ChristopherHX / github-act-runner

act as self-hosted runner
MIT License
272 stars 14 forks source link

Panic on Linux/arm64 when job fails #151

Closed KitsuneAlex closed 1 year ago

KitsuneAlex commented 1 year ago

I built the runner from main and ran into a panic when a job fails. Here's some hard- and software information:

The stacktrace is as follows:

panic: runtime error: invalid memory address or nil pointer dereference                                                                           
[signal SIGSEGV: segmentation violation code=0x1 addr=0xd0 pc=0x612e10]                                                                           

goroutine 111 [running]:                                                 
github.com/ChristopherHX/github-act-runner/actionsdotnetactcompat.(*ghaFormatter).Format(0x40003fe7e0, 0x40000e9ea0)                              
        /home/fux/Desktop/github-act-runner/actionsdotnetactcompat/act_worker.go:103 +0x420                                                       
github.com/nektos/act/pkg/runner.(*maskedFormatter).Format(0x4000534960, 0x303790?)                                                               
        /home/fux/go/pkg/mod/github.com/christopher!h!x/act@v0.2.23-0.20230807182133-72d04de87e9c/pkg/runner/logger.go:175 +0x4c                  
github.com/sirupsen/logrus.(*Entry).write(0x40000e9ea0)                  
        /home/fux/go/pkg/mod/github.com/sirupsen/logrus@v1.9.0/entry.go:289 +0xd8                                                                 
github.com/sirupsen/logrus.(*Entry).log(0x4000125030, 0x4, {0x40000fe620, 0x67})                                                                  
        /home/fux/go/pkg/mod/github.com/sirupsen/logrus@v1.9.0/entry.go:252 +0x468                                                                
github.com/sirupsen/logrus.(*Entry).Log(0x4000125030, 0x4, {0x4000317cf8?, 0x1?, 0x1?})                                                           
        /home/fux/go/pkg/mod/github.com/sirupsen/logrus@v1.9.0/entry.go:304 +0x60
github.com/sirupsen/logrus.(*Entry).Logf(0x4000125030, 0x4, {0x7d98fc?, 0x4000089800?}, {0x4000317d58?, 0x0?, 0x4000017d78?})                     
        /home/fux/go/pkg/mod/github.com/sirupsen/logrus@v1.9.0/entry.go:349 +0x88                                                                 
github.com/sirupsen/logrus.(*Entry).Infof(...)                                                                                                    
        /home/fux/go/pkg/mod/github.com/sirupsen/logrus@v1.9.0/entry.go:362                                                                       
github.com/nektos/act/pkg/runner.useStepLogger.func1.1({0x40000fe460?, 0x67?})                                                                    
        /home/fux/go/pkg/mod/github.com/christopher!h!x/act@v0.2.23-0.20230807182133-72d04de87e9c/pkg/runner/job_executor.go:181 +0x70            
github.com/nektos/act/pkg/common.(*lineWriter).handleLine(...)           
        /home/fux/go/pkg/mod/github.com/christopher!h!x/act@v0.2.23-0.20230807182133-72d04de87e9c/pkg/common/line_writer.go:45                    
github.com/nektos/act/pkg/common.(*lineWriter).Write(0x400041c880, {0x40004c2000?, 0x925d90?, 0x8000?})                                           
        /home/fux/go/pkg/mod/github.com/christopher!h!x/act@v0.2.23-0.20230807182133-72d04de87e9c/pkg/common/line_writer.go:31 +0x1a8             
github.com/nektos/act/pkg/container.(*ptyWriter).Write(0x40003fce70, {0x40004c2000, 0x7ec, 0x8000})                                               
        /home/fux/go/pkg/mod/github.com/christopher!h!x/act@v0.2.23-0.20230807182133-72d04de87e9c/pkg/container/host_environment.go:174 +0x164    
io.copyBuffer({0x92c620, 0x40003fce70}, {0x92cfe0, 0x4000010220}, {0x0, 0x0, 0x0})                                                                
        /snap/go/current/src/io/io.go:429 +0x1a8                         
io.Copy(...)                                                             
        /snap/go/current/src/io/io.go:386                                                                                                         
github.com/nektos/act/pkg/container.copyPtyOutput({0x92c620?, 0x40003fce70?}, {0x92cfe0?, 0x4000010220?}, 0x400043ec00?)                          
        /home/fux/go/pkg/mod/github.com/christopher!h!x/act@v0.2.23-0.20230807182133-72d04de87e9c/pkg/container/host_environment.go:238 +0x64     
created by github.com/nektos/act/pkg/container.(*HostEnvironment).exec   
        /home/fux/go/pkg/mod/github.com/christopher!h!x/act@v0.2.23-0.20230807182133-72d04de87e9c/pkg/container/host_environment.go:302 +0x4dc

Thanks in advance for taking some time to look at this issue :)

ChristopherHX commented 1 year ago

ran into a panic when a job fails.

How does your job yaml look like? Just to help me to understand how to reproduce this scenario.

I don't know what happens in your case after a null check like in #152, it probably has more consequences for logging after this fault.

KitsuneAlex commented 1 year ago

Thanks for the super quick reply! <3 My workflow for this particular instance looks as follows (i'm using a lot of reusable workflows):

name: Clang
on:
  workflow_call:
    inputs:
      name: 
        required: true
        type: string
      platform_name:
        required: true
        type: string
      std_version:
        required: true
        type: string
      version:
        required: true
        type: string
      cmake_args:
        required: false
        type: string
        default: ""
      ctest_args:
        required: false
        type: string
        default: ""
      build_type:
        required: false
        type: string
        default: "Debug"
env:
  BUILD_TYPE: ${{inputs.build_type}}
  CC: "clang-${{inputs.version}}"
  CXX: "clang++-${{inputs.version}}"
jobs:
  build:
    name: ${{inputs.name}}
    runs-on: ["arm64"]
    steps:
      - uses: actions/checkout@v3
      - name: Initialize Submodules
        run: git submodule update --init
      - name: Configure CMake (${{inputs.platform_name}})
        run: >-
          cmake -B ${{github.workspace}}/build-${{inputs.platform_name}}-clang-cpp${{inputs.std_version}}
          -DCMAKE_CXX_FLAGS=-stdlib=libc++
          -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}}
          -DGTEST_USE_OWN_TR1_TUPLE=1
          -DCMAKE_CXX_STANDARD=${{inputs.std_version}}
          ${{inputs.cmake_args}}
      - name: Build (${{inputs.platform_name}})
        run: >-
          cmake --build ${{github.workspace}}/build-${{inputs.platform_name}}-clang-cpp${{inputs.std_version}}
          --config ${{env.BUILD_TYPE}}
          --clean-first
      - name: Test (${{inputs.platform_name}})
        working-directory: ${{github.workspace}}/build-${{inputs.platform_name}}-clang-cpp${{inputs.std_version}}
        run: ctest -C ${{env.BUILD_TYPE}} ${{inputs.ctest_args}}
ChristopherHX commented 1 year ago

Your workflow isn't causing anything odd here, sounds like a temporary logger inconsitency.

KitsuneAlex commented 1 year ago

Your workflow isn't causing anything odd here, sounds like a temporary logger inconsitency.

Interesting. It happened multiple times to me tho, always in the same scenario; when a build fails. That is really odd.. I was thinking about the possibility of tmux causing issues, but that seems a bit far fetched.

ChristopherHX commented 1 year ago

To be honest always it failed for me due to missing repository content for cmake it never paniced for me, so I'm clueless how exactly this happens.

If you have a public repository with the code / you would invite me read access, I can take a deeper lock into this in September.

Eventually it would be helpful to dump all logging fields in json to the disk, to understand why the logger fall through to the end of all displayed sections. The problem is to recreate the section based log model, logging everything in plain text would be simple and fool proof.