search

barthofu / tscord-docs

📃 Documentation for TSCord
https://barthofu.github.io/tscord-docs
2 stars 2 forks source link

About CI/CD #7

Open inBytes opened 2 months ago

inBytes commented 2 months ago

Hi @barthofu @Mr-Artemus !!

I've been struggling very hard with CI/CD last weeks in order to be able to automatically deploy my bot to my VPS when pushing to my Features branch at GitHub

There are two important issues I think you should include in your documentation about CI/CD: 1) First is about Actions permissions. 2) Second is about doing a pull using git pull https://github.com/<your_github_user>/<your_github_project>.git main

1) Actions Permissions

By default, for your GitHub project, in Settings > Actions > General > Actions permissions you'll have this option selected: Allow <your_github_user> actions and reusable workflows

This prevents you from using the deploy.yml action with this error: appleboy/ssh-action@master is not allowed to be used in <your_github_user>/<your_github_project>. Actions in this workflow must be: within a repository owned by <your_github_user>.

For being able to use external repositories inside your actions, you must set the Settings > Actions > General > Actions permissions to this: Allow all actions and reusable workflows :thumbsup:

2) Git pull from your VPS

If you try to do a pull from your VPS using git pull https://github.com/<your_github_user>/<your_github_project>.git main you'll get a prompt for inserting your GitHub user/password

But even if, in your deploy action, you insert your GitHub user/password in your HTTPS request like this: git pull https://${{ secrets.GITHUB_USERNAME }}:${{ secrets.GITHUB_PASSWORD }}@github.com/<your_github_user>/<your_github_project>.git main, you'll finally get this error:

remote: Support for password authentication was removed on August 13, 2021.
remote: Please see https://docs.github.com/get-started/getting-started-with-git/about-remote-repositories#cloning-with-https-urls for information on currently recommended modes of authentication.
fatal: Authentication failed for 'https://github.com/<your_github_user>/<your_github_project>.git/'

I'm not going to go into this subject in depth, but if you want to go through the HTTPS request path, instead of using your GitHub password, you must create a Personal access TOKEN

The other option (the one I chose !!) is using SSH against GitHub in order to do a pull from your VPS. Like this: git pull git@github.com:<your_github_user>/<your_github_project>.git Features :thumbsup:

That would be the two points I would add to your documentation

But... here is where the fun begins

This is because to make this work: git pull git@github.com:<your_github_user>/<your_github_project>.git Features I have had to go through HELL :japanese_goblin: because of the limitations of the different components of this integration. Here is the summary:

The limitations seem to be these:

My VPS Linux Setup

So, if someone wants to do the same CI/CD I did on my Almalinux v9 (maybe you'll have to change something if you are on another distro), you'll have to do this: NOTE: replace <my_user>, <my_user_password>, my@email.com, <my_VPS_IP> and <my_VPS_SSH_port> with your own data

1) Log in to your VPS as root 2) Run this instructions to create a user in which you'll run your bot:

cd /opt/
mkdir <my_user>
adduser <my_user>
passwd <my_user>
    # set the password: <my_user_password>
chown -R <my_user>:<my_user> <my_user>/
su - <my_user>
cd /opt/<my_user>/

3) Create an RSA Key with NO passphrase and add it to authorized_keys file:

ssh-keygen -t rsa -b 4096 -C "your@email.com"
    Generating public/private rsa key pair.
    Enter file in which to save the key (/home/<my_user>/.ssh/id_rsa): id_rsa
    Enter passphrase (empty for no passphrase):       # press ENTER for empty
    Enter same passphrase again:                      # press ENTER for empty
    Your identification has been saved in /home/<my_user>/.ssh/id_rsa
    Your public key has been saved in /home/<my_user>/.ssh/id_rsa.pub
    The key fingerprint is:
    SHA256:UTdAL/RandomAlphaNumericKeyMtRSF3DlfA my@email.com
    The key's randomart image is:
    +---[RSA 4096]---+
    |   .......  .   |
    |    .....  . .  |
    |   . ...... . . |
    |  . . ... .. .  |
    |   . .  . ...   |
    |  ... ... .. .  |
    | . .... . ...   |
    |...   ... ...   |
    |...... .....    |
    +----[SHA256]----+
cat /home/<my_user>/.ssh/id_rsa.pub >> /home/<my_user>/.ssh/authorized_keys

4) Add your PUBLIC key to GitHub:

This will allow you to communicate with GitHub using git from your VPS

5) On GitHub, go to the section Settings > Secrets and variables > Actions > Secrets and create the following entries:

6) On your VPS; edit _knownhosts file: Edit the file _/home//.ssh/knownhosts nano /home/<my_user>/.ssh/known_hosts

...and fill it with the next content:

github.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOMqqnkVzrm0SdG6UOoqKLsabgH5C9okWi0dh2l9GKJl
github.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCj7ndNxQowgcQnjshcLrqPEiiphnt+VTTvDP6mHBL9j1aNUkY4Ue1gvwnGLVlOhGeYrnZaMgRK6+PKCUXaDbC7qtbW8gIkhL7aGCsOr/C56SJMy/BCZfxd1nWzAOxSDPgVsmerOBYfNqltV9/hWCqBywINIR+5dIg6JTJ72pcEpEjcYgXkE2YEFXV1JHnsKgbLWNlhScqb2UmyRkQyytRLtL+38TGxkxCflmO+5Z8CSSNY7GidjMIZ7Q4zMjA2n1nGrlTDkzwDCsw+wqFPGQA179cnfGWOWRVruj16z6XyvxvjJwbz0wQZ75XK5tKSb7FNyeIEs4TT4jk+S4dhPeAUC5y+bDYirYgM4GC7uEnztnZyaVWQ7B381AK4Qdrwt51ZqExKbQpTUNn+EjqoTwvqNj4kqx5QUCI0ThS/YkOxJCXmPUWZbhjpCg56i+2aB6CmK2JGhn57K5mj0MNdBXA4/WnwH6XoPWJzK5Nyu2zB3nAZp+S5hpQs+p1vN1/wsjk=
github.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEmKSENjQEezOmxkZMy7opKgwFB9nkt5YRrYMjNuG5N87uRgg6CLrbo5wAdT/y6v0mKV0U2w0WZ2YB/++Tpockg=

7) Add the key to your SSH agent and clone the repository:

eval "$(ssh-agent -s)" # To be sure Agent is running
    Agent pid 480503
ssh-add /home/<my_user>/.ssh/id_rsa
git clone git@github.com:<your_github_user>/<your_github_project>.git

It should give you something similar to the following output:

Cloning into 'Project'...
remote: Enumerating objects: 530, done.
remote: Counting objects: 100% (530/530), done.
remote: Compressing objects: 100% (331/331), done.
remote: Total 530 (delta 235), reused 471 (delta 177), pack-reused 0

VERY IMPORTANT: if at some point something is not working as it should you can test what can be failing running this command: ssh -vT git@github.com You'll get a complete diagnosis about the SSH connection

8) Finally I changed the deploy.yml to this:

name: Deploy (SSH)

on:
  push:
    branches: [ Features ]

  # allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

jobs:

  #build:
  #  uses: ./.github/workflows/build.yml

  deploy:
    name: Deploy
    #needs: build
    runs-on: ubuntu-latest
    steps:
      - name: Executing remote SSH commands using Private Key
        uses: appleboy/ssh-action@master
        with:
          host: ${{ secrets.HOST }}
          port: ${{ secrets.PORT }}
          username: ${{ secrets.USERNAME }}
          key: ${{ secrets.SSH_PRIVATE_KEY }}
          script: |
            cd ${{ secrets.PROJECT_PATH }}
            git fetch origin Features
            git reset --hard FETCH_HEAD
            git pull git@github.com:<your_github_user>/<your_github_project>.git Features
            pm2 restart pm2.config.json

You can notice the following issues:

stephanvebrian commented 1 month ago

@inBytes

point no 2,

you can keep use passphrase for more stronger security, and when pulling you can utilize github token on your workflow

git pull https://${{ github.actor }}:${{ github.token }}@github.com/${{ github.repository }}.git main

above are default env var when you use github action

inBytes commented 1 month ago

Hi there @stephanvebrian !!

Thank you for your suggestion.

If you read the next Heading after 2) I swear to you that I tried using passphrase both with ed25519 key and rsa key with no success.

At least with SSH, that was my chosen path.

Using HTTPS (as far as I read) required me to use a token that has to be updated for a given period of time, that was not my ideal approach (I wanted to setup once and forget it).

stephanvebrian commented 1 month ago

yes @inBytes that was my setup also- not plannign to setup PAT token for each repository, but here is a magic part.

if you see this url 

https://${{ github.actor }}:${{ github.token }}@github.com/${{ github.repository }}.git

it used github.* secrets, which is a default variable in each repository (https://docs.github.com/en/actions/learn-github-actions/variables#default-environment-variables), so you dont have to setup any additional token except the ssh and passphrase that will be used for login into your machine

inBytes commented 1 month ago

Sorry for the delay in replying @stephanvebrian, I'm a father of two in addition to owning a business, and didn't have time for this side project.

I still don't get your point. In the link you are sharing with me (https://docs.github.com/en/actions/learn-github-actions/variables#default-environment-variables) I see a section with Default environment variables where I see:

But I don't see any "default" GITHUB_TOKEN. In fact, they don't mention the word "token" in all the text for that documentation page.

I don't want to bother you or waste your time, but if it's a simple thing, just tell me. Otherwise let it be, as I have it running the way I told you above.

Thank you anyway!!