GitHub Actions Runner: 'Object reference not set to an instance of an object' Error"

akaviyaPRH commented 4 months ago

Describe the bug The GitHub Actions runner application repeatedly encounters an error stating "An error occurred: Object reference not set to an instance of an object." This issue persists even after restarting the service. There are no clear indicators in the system logs that point to the cause of this error. This is the first time this error has been encountered.

To Reproduce Steps to reproduce the behavior:

Start the GitHub Actions runner service on the machine.
Observe the runner's log output.
The error "An error occurred: Object reference not set to an instance of an object" appears shortly after the service starts.

Expected behavior The GitHub Actions runner should start without encountering a null reference exception and be ready to execute workflows without errors.

Runner Version and Platform

Version of your runner: 2.317.0

OS of the machine running the runner: Linux (ubuntu)

What's not working?

The runner fails to operate normally due to a recurring error message: "An error occurred: Object reference not set to an instance of an object." This prevents the runner from executing any workflows.

● actions.runner.puk.UKLENOAPIPI001.service - GitHub Actions Runner (puk.UKLENOAPIPI001) Loaded: loaded (/etc/systemd/system/actions.runner.puk.UKLENOAPIPI001.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2024-07-17 08:48:58 UTC; 13min ago Main PID: 2129042 (runsvc.sh) Tasks: 8 (limit: 19044) Memory: 8.7M CPU: 1min 39.466s CGroup: /system.slice/actions.runner.puk.UKLENOAPIPI001.service ├─2129042 /bin/bash /home/infraeno/actions-runner/runsvc.sh └─2129044 ./externals/node16/bin/node ./bin/RunnerService.js

Jul 17 08:56:09 UKLENOAPIPI001 runsvc.sh[2129044]: Runner listener exited with error code 2 Jul 17 08:56:09 UKLENOAPIPI001 runsvc.sh[2129044]: Runner listener exit with retryable error, re-launch runner in 5 seconds. Jul 17 08:56:14 UKLENOAPIPI001 runsvc.sh[2129044]: Starting Runner listener with startup type: service Jul 17 08:56:14 UKLENOAPIPI001 runsvc.sh[2129044]: Started listener process, pid: 2130576 Jul 17 08:56:15 UKLENOAPIPI001 runsvc.sh[2129044]: √ Connected to GitHub Jul 17 08:56:16 UKLENOAPIPI001 runsvc.sh[2129044]: Current runner version: '2.317.0' Jul 17 08:56:16 UKLENOAPIPI001 runsvc.sh[2129044]: 2024-07-17 08:56:16Z: Listening for Jobs Jul 17 09:02:35 UKLENOAPIPI001 runsvc.sh[2129044]: An error occurred: Object reference not set to an instance of an object. Jul 17 09:02:35 UKLENOAPIPI001 runsvc.sh[2129044]: Runner listener exited with error code 2 Jul 17 09:02:35 UKLENOAPIPI001 runsvc.sh[2129044]: Runner listener exit with retryable error, re-launch runner in 5 seconds.

Job Log Output

Not applicable, as the error occurs at the service startup, before any job execution.

Runner and Worker's Diagnostic Logs

Diagnostic logs from the _diag folder might provide more insight, but specific log entries related to the error have not been provided. Users experiencing this issue are encouraged to check their runner's _diag folder for logs prefixed with Runner_ and Worker_ for more details and include relevant excerpts when reporting the issue.

s-bernhardt commented 4 months ago

same issues here, _diag files show problems like:

[2024-07-17 13:16:52Z INFO GitHubActionsService] AAD Correlation ID for this token request: Unknown
[2024-07-17 13:16:53Z INFO JobDispatcher] Shutting down JobDispatcher. Make sure all WorkerDispatcher has finished.
[2024-07-17 13:16:53Z INFO Runner] Deleting Runner Session...
[2024-07-17 13:16:53Z ERR  Terminal] WRITE ERROR: An error occurred: Object reference not set to an instance of an object.
[2024-07-17 13:16:53Z ERR  Listener] System.NullReferenceException: Object reference not set to an instance of an object.
   at GitHub.Runner.Listener.JobDispatcher.Run(AgentJobRequestMessage jobRequestMessage, Boolean runOnce)
   at GitHub.Runner.Listener.Runner.RunAsync(RunnerSettings settings, Boolean runOnce)
   at GitHub.Runner.Listener.Runner.RunAsync(RunnerSettings settings, Boolean runOnce)
   at GitHub.Runner.Listener.Runner.RunAsync(RunnerSettings settings, Boolean runOnce)
   at GitHub.Runner.Listener.Runner.ExecuteCommand(CommandSettings command)
   at GitHub.Runner.Listener.Program.MainAsync(IHostContext context, String[] args)

TingluoHuang commented 4 months ago

Can someone send the full runner diag log to me? tingluohuang@github.com Thanks a lot. 🙇

MarkusZoppelt commented 4 months ago

This happened to us as well, same day as we migrated our GitHub Organization to a GitHub Enterprise. Maybe this is what caused the issue for you too?

Unfortunately, the only solution we found that worked was to completely uninstall / remove the runners and re-add them (v2.317.0)

s-bernhardt commented 4 months ago

Other runners of my other repos are working fine. and they to the same job and sit on the same self-hosted runner instance

TingluoHuang commented 4 months ago

We rollback a feature flag that might cause this. The impact should be mitigated, we will continue investigate the root cause. Thanks, and sorry for the trouble.

s-bernhardt commented 4 months ago

thanks, runners succeeded approx. 46min ago.

davidsiu commented 4 months ago

We are having this same issue too and it is still happening. If it helps, we also just upgraded to Enterprise last night.

s-bernhardt commented 4 months ago

The failed action-runner was recreated by my colleague. This solved it most probably.

TingluoHuang commented 4 months ago

@davidsiu can you send me your runner diag log at tingluohuang@github.com? 🙇

TingluoHuang commented 4 months ago

@davidsiu sorry for the trouble, we applied a mitigation which should unblock your runner. we are working on a fix to hope to get it deployed within few hours. 🙇

lokesh755 commented 4 months ago

@davidsiu We have rolled out the fix.. Please let's know if you are still blocked on this issue.

davidsiu commented 4 months ago

Yes! Thank you @TingluoHuang and @lokesh755 ! Your fix worked for me. Much appreciated!

actions / runner