Using the bot with a private repo

laraPPr commented 5 months ago

At the UGent we are currently looking at letting the bot handle a private repo. The bot could handle the events from created prs, and commando in the private repo without any issues.

However the bot: build commando is not able to execute and errors out. This does not crash the event handler but the build job also never gets started. and there is also not anything posted to github that indicates that the ´event_handlerran into a problem when handling thebot: build` command.

When going into the Pyghee logs the error was very clear.

First error I ran into was the following:

[20231205-T15:00:06] Unexpected err=run_cmd(): Error running 'git clone https://github.com/laraPPr/software_layer_VSC /scratch/gent/461/vsc46128/EESSI/jobs/2023.12/pr_1/event_921c5b60-9376-11ee-912a-4abc2ba9feca/run_000/RHEL8_zen2-ib/install' in '/scratch/gent/461/vsc46128/EESSI/jobs/2023.12/pr_1/event_921c5b60-9376-11ee-912a-4abc2ba9feca/run_000/RHEL8_zen2-ib/install
           stdout ''
           stderr 'Cloning into '/scratch/gent/461/vsc46128/EESSI/jobs/2023.12/pr_1/event_921c5b60-9376-11ee-912a-4abc2ba9feca/run_000/RHEL8_zen2-ib/install'...
remote: Repository not found.
fatal: Authentication failed for 'https://github.com/laraPPr/software_layer_VSC/'
'
           exit code 128, type(err)=<class 'RuntimeError'>
[20231205-T15:00:06] WARNING: A crash occurred!
Traceback (most recent call last):
  File "/kyukon/scratch/gent/461/vsc46128/EESSI/venv_eessi_bot_login_p36/lib64/python3.6/site-packages/pyghee/lib.py", line 170, in process_event
    self.handle_event(event_info, log_file=log_file)
  File "/kyukon/scratch/gent/461/vsc46128/EESSI/venv_eessi_bot_login_p36/lib64/python3.6/site-packages/pyghee/lib.py", line 102, in handle_event
    handler(event_info, log_file=log_file)
  File "/kyukon/scratch/gent/461/vsc46128/EESSI/eessi-bot-software-layer/eessi_bot_event_handler.py", line 221, in handle_issue_comment_event
    update = self.handle_bot_command(event_info, cmd)
  File "/kyukon/scratch/gent/461/vsc46128/EESSI/eessi-bot-software-layer/eessi_bot_event_handler.py", line 398, in handle_bot_command
    return handler(event_info, bot_command)
  File "/kyukon/scratch/gent/461/vsc46128/EESSI/eessi-bot-software-layer/eessi_bot_event_handler.py", line 444, in handle_bot_command_build
    submitted_jobs = submit_build_jobs(pr, event_info, bot_command.action_filters)
  File "/kyukon/scratch/gent/461/vsc46128/EESSI/eessi-bot-software-layer/tasks/build.py", line 695, in submit_build_jobs
    jobs = prepare_jobs(pr, cfg, event_info, action_filter)
  File "/kyukon/scratch/gent/461/vsc46128/EESSI/eessi-bot-software-layer/tasks/build.py", line 455, in prepare_jobs
    download_pr(base_repo_name, base_branch_name, pr, job_dir)
  File "/kyukon/scratch/gent/461/vsc46128/EESSI/eessi-bot-software-layer/tasks/build.py", line 347, in download_pr
    clone_output, clone_error, clone_exit_code = run_cmd(git_clone_cmd, "Clone repo", arch_job_dir)
  File "/kyukon/scratch/gent/461/vsc46128/EESSI/eessi-bot-software-layer/tools/__init__.py", line 59, in run_cmd
    raise RuntimeError(error_msg)
RuntimeError: run_cmd(): Error running 'git clone https://github.com/laraPPr/software_layer_VSC /scratch/gent/461/vsc46128/EESSI/jobs/2023.12/pr_1/event_921c5b60-9376-11ee-912a-4abc2ba9feca/run_000/RHEL8_zen2-ib/install' in '/scratch/gent/461/vsc46128/EESSI/jobs/2023.12/pr_1/event_921c5b60-9376-11ee-912a-4abc2ba9feca/run_000/RHEL8_zen2-ib/install
           stdout ''
           stderr 'Cloning into '/scratch/gent/461/vsc46128/EESSI/jobs/2023.12/pr_1/event_921c5b60-9376-11ee-912a-4abc2ba9feca/run_000/RHEL8_zen2-ib/install'...
remote: Repository not found.
fatal: Authentication failed for 'https://github.com/laraPPr/software_layer_VSC/'
'
           exit code 128

This error I resolved by updating the adjusting the clone commando in tasks/build.py. This solution requires however that there is a valid private key linked to a GitHub account. Which is the case on my bot machine.

git_clone_cmd = ' '.join(['git clone', f'git@github.com:{repo_name}.git'    , arch_job_dir])

clone_output, clone_error, clone_exit_code = run_cmd(git_clone_cmd, "Clone repo", arch_job_dir)

After implementing the solution above I ran into a seccond error:

[20231205-T15:21:45] run_cmd(): Result for running 'curl -L https://github.com/laraPPr/software_layer_VSC/pull/1.diff > 1.diff' in '/scratch/gent/461/vsc46128/EESSI/jobs/2023.12/pr_1/event_9bd09c90-9379-11ee-969a-5621d402fe52/run_000/RHEL8_zen2-ib/install
           stdout ''
           stderr '  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100     9  100     9    0     0     50      0 --:--:-- --:--:-- --:--:--    50
'
           exit code 0
[20231205-T15:21:45] run_subprocess(): 'Apply patch' by running 'git apply 1.diff' in directory '/scratch/gent/461/vsc46128/EESSI/jobs/2023.12/pr_1/event_9bd09c90-9379-11ee-969a-5621d402fe52/run_000/RHEL8_zen2-ib/install'
[20231205-T15:21:45] run_cmd(): Error running 'git apply 1.diff' in '/scratch/gent/461/vsc46128/EESSI/jobs/2023.12/pr_1/event_9bd09c90-9379-11ee-969a-5621d402fe52/run_000/RHEL8_zen2-ib/install
           stdout ''
           stderr 'error: No valid patches in input (allow with "--allow-empty")
'
           exit code 128

A possible solution is adjusting the curl commando to the following:

 curl_cmd = f'curl -H "Accept: application/vnd.github.diff" \ 
    -H "Authorization: Bearer $GITHUB_TOKEN"  \ 
    -H "X-GitHub-Api-Version: 2022-11-28" \ 
    -L https://api.github.com/repos/{repo_name}/pulls/{pr.number} > {pr.number}.diff'

I However do not have a working solution yet. This solution requires the access to $GITHUB_TOKEN variable after the EESSIBotSoftwareLayer flask app is created. which seems to not be possible. I also cannot tell where the $GITHUB_TOKEN variable is used so I don't know if it can be a solution to create an .env file for the $GITHUB_TOKEN and $GITHUB_APP_SECRET_TOKEN variables. At the moment however I cannot figure out where these variables are used when creating the app. I only know that when trying to create app without the $GITHUB_TOKEN you get the following error.

ERROR: GitHub token is not available via $GITHUB_TOKEN!

And I'm struggling to figure out from where this error is coming.

bedroge commented 5 months ago

Haven't had time to really look into your issue, but regarding the $GITHUB_TOKEN: it's being used by PyGHee here https://github.com/boegel/PyGHee/blob/main/pyghee/lib.py#L72.

laraPPr commented 5 months ago

Haven't had time to really look into your issue, but regarding the $GITHUB_TOKEN: it's being used by PyGHee here https://github.com/boegel/PyGHee/blob/main/pyghee/lib.py#L72.

Thanks thats the code i was looking for. I’ll see if this helps me.

laraPPr commented 5 months ago

I found a solution you can check the changes that I made to the bot here:

https://github.com/laraPPr/eessi-bot-software-layer/pull/1/files

EESSI / eessi-bot-software-layer

Using the bot with a private repo #235