search

deep-foundation / dev

Monorepository with gitmodules for easy development of the entire project in one place.
https://discord.gg/deep-foundation
The Unlicense
20 stars 18 forks source link

readme

deep-foundation

Gitpod Discord

Deep Architecture

IMG_1373

Gitpod usage

Start

Everything starts automatically. Just watch the terminal.

When all tasks are done, you can open Deep.Case App in browser: http://localhost:3007/ (ctrl/cmd + click by link)

It is also possible to open 3007 port manually, to do it open PORTS tab and select link for 3007 port.

image

Update

GitPod may fail to load prebuild, in that case it is recommended to use these sequence of actions in existing GitPod instance (use GitPod instance multiple times, remember GitPod still may fail):

  1. Press CTRL+C in Gitpod Task 2: bash terminal. Or stop active npm run gitpod-start command.
  2. Execute 
    npm run gitpod-update
  3. Press ↑ in Gitpod Task 2: bash terminal and restart npm run gitpod-start command.

Codespaces usage

Everything starts automatically.

When all tasks are done, you can open Deep.Case App in browser: http://localhost:3007/ (ctrl/cmd + click by link)

You may need to make port 3007 public.

image

When all tasks are done, you can open Deep.Case App using PORTS tab:

image

Local usage

Installation

Open

When all tasks are done, you can open Deep.Case App in browser: http://localhost:3007/ (ctrl/cmd + click by link)

Server usage

Preparation

Make sure you have virtual machine or server that is connected to the internet.

If you would like to use domain and SSL, make sure:

If you would like to use IP without SSL, make sure:

Note that access via IP is not secure (there is no way to issue SSL certificate to IP address), that means it can be only used for development purposes with no sensetive data.

Install docker

sudo apt update
sudo apt install -y git curl docker.io docker-compose

Add docker group (optional)

Sometimes this action is not done on docker's installation via apt/apt-get or you may need to add access for docker to another user.

sudo groupadd docker
sudo usermod -aG docker $(whoami)

Check docker

These commands must be able to run without sudo, if it is not the case restart computer or relogin as user.

docker run hello-world
docker rm $(docker ps -a -q --filter "ancestor=hello-world")
docker image rm hello-world

Continue only if docker run hello-world works without sudo and errors.

After docker installation it may be required to move files to another drive (for example on azure VMs).

Install nvm

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash
export NVM_DIR="$([ -z "${XDG_CONFIG_HOME-}" ] && printf %s "${HOME}/.nvm" || printf %s "${XDG_CONFIG_HOME}/nvm")"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"

Install node

nvm install 18 && nvm alias default 18 && nvm use default

Update npm

npm i -g npm@latest

Installation

Configure deep (to support domain with SSL)

export GQL_SSL="1"
export PROTOCOL="https"
export DEEPCASE_HOST="chatgpt.deep.foundation"
export DEEPLINKS_HOST="deeplinks.chatgpt.deep.foundation"

Install and configure nginx (to support domain with SSL)

mkdir nginx
cd nginx
wget https://raw.githubusercontent.com/deep-foundation/dev/main/configure-nginx.js
npm i commander
node configure-nginx.js --configurations "$DEEPCASE_HOST 3007" "$DEEPLINKS_HOST 3006" --certbot-email drakonard@gmail.com
cd ..
rm -rf nginx

This command can be used to update the certificate once it expires (every 3 months). It may be a good idea to add it as sh script to cron.

Configure deep (to support IP without SSL)

Do not execute this step if you selected steps for domain with SSL before.

export GQL_SSL="0"
export PROTOCOL="http"
export HOST_IP="185.105.118.59"
export DEEPCASE_HOST="$HOST_IP:3007"
export DEEPLINKS_HOST="$HOST_IP:3006"

Install deeplinks global command

npm rm --unsafe-perm -g @deep-foundation/deeplinks
npm install --unsafe-perm -g @deep-foundation/deeplinks@latest

Make your Deep instance configurations (aka call-options.json)

export HASURA_ADMIN_SECRET=$(node -e "console.log(require('crypto').randomBytes(24).toString('hex'));")
export POSTGRES_PASSWORD=$(node -e "console.log(require('crypto').randomBytes(24).toString('hex'));")
export MINIO_ACCESS_KEY=$(node -e "console.log(require('crypto').randomBytes(24).toString('hex'));")
export MINIO_SECRET_KEY=$(node -e "console.log(require('crypto').randomBytes(24).toString('hex'));")
tee call-options.json << JSON
{
  "operation": "run",
  "envs": {
    "DEEPLINKS_PUBLIC_URL": "${PROTOCOL}://$DEEPLINKS_HOST",
    "NEXT_PUBLIC_DEEPLINKS_URL": "${PROTOCOL}://$DEEPLINKS_HOST",
    "NEXT_PUBLIC_GQL_PATH": "$DEEPLINKS_HOST/gql",
    "NEXT_PUBLIC_GQL_SSL": "${GQL_SSL}",
    "NEXT_PUBLIC_DEEPLINKS_SERVER": "${PROTOCOL}://$DEEPCASE_HOST",
    "NEXT_PUBLIC_ENGINES_ROUTE": "0",
    "NEXT_PUBLIC_DISABLE_CONNECTOR": "1",
    "JWT_SECRET": "'{\"type\":\"HS256\",\"key\":\"$(node -e "console.log(require('crypto').randomBytes(50).toString('base64'));")\"}'",
    "DEEPLINKS_HASURA_STORAGE_URL": "http://host.docker.internal:8000/",
    "HASURA_GRAPHQL_ADMIN_SECRET": "$HASURA_ADMIN_SECRET",
    "MIGRATIONS_HASURA_SECRET": "$HASURA_ADMIN_SECRET",
    "DEEPLINKS_HASURA_SECRET": "$HASURA_ADMIN_SECRET",
    "POSTGRES_PASSWORD": "$POSTGRES_PASSWORD",
    "HASURA_GRAPHQL_DATABASE_URL": "postgres://postgres:$POSTGRES_PASSWORD@postgres:5432/postgres",
    "POSTGRES_MIGRATIONS_SOURCE": "postgres://postgres:$POSTGRES_PASSWORD@host.docker.internal:5432/postgres?sslmode=disable",
    "RESTORE_VOLUME_FROM_SNAPSHOT": "0",
    "MANUAL_MIGRATIONS": "1",
    "MINIO_ROOT_USER": "$MINIO_ACCESS_KEY",
    "MINIO_ROOT_PASSWORD": "$MINIO_SECRET_KEY",
    "S3_ACCESS_KEY": "$MINIO_ACCESS_KEY",
    "S3_SECRET_KEY": "$MINIO_SECRET_KEY"
  }
}
JSON

Check that no other deep instances installed at the moment

docker ps -a

On fresh/clean server this list should be empty. If you have any previously installed deep instances, make sure the data is backed up.

Install your own deep instance

export DEEPLINKS_CALL_OPTIONS=$(cat call-options.json)
export DEBUG="deeplinks:engine:*,deeplinks:migrations:*"
deeplinks

Check that deep instance is installed correctly

docker ps -a

Make sure that all containers have status "Up".

Restart

Entire docker

sudo systemctl stop docker
sudo systemctl start docker

Single docker container

docker restart deep-links

Restore/Update

Update deeplinks (optional)

npm rm --unsafe-perm -g @deep-foundation/deeplinks
npm install --unsafe-perm -g @deep-foundation/deeplinks@latest

Delete previous versions of docker images (optional)

Sometimes it is not require to delete previous versions of docker images, sometimes this is the only way to make update work.

docker stop deep-case deep-links
docker rm deep-case deep-links
docker image rm deepf/deeplinks:main
docker image rm deepf/deepcase:main
docker image pull deepf/deepcase:main
docker image pull deepf/deeplinks:main

Restore/update using installed deeplinks command

export DEEPLINKS_CALL_OPTIONS=$(cat call-options.json)
export DEBUG="deeplinks:engine:*,deeplinks:migrations:*"
deeplinks

Uninstall

Remove docker containers:

docker stop $(docker ps -a --format "{{.Names}}" | grep '^deep-.*$')
docker rm $(docker ps -a --format "{{.Names}}" | grep '^deep-.*$')

Remove unused volumes (may destroy you data, it is irreversible operation, this step is optional):

docker volume prune -af

Remove migrations state (should be executed only if you deleted volumes):

rm -f /tmp/deep/.migrate

Remove deeplinks global command:

npm rm --unsafe-perm -g @deep-foundation/deeplinks

Manual terminal methods

Docker control

Gitpod

Automatic

OR

Manual

Local

Automatic

Manual

Useful

Workspaces

If dependencies are updated in workspaces (deeplinks, deepcase, deepcase-app), we must use these specific commands to make GitPod and GitHub Actions to work correclty in these packages.

If package.json and package-lock.json was updated in submodule/workspace separately and committed, please make sure you also executed npm run sync (which also executes npm i in dev repository, and it also commits all changes in all submodules/workspaces), or to test it locally you may use npm run gitpod-update command.

Unsafe gulp methods

Diagnostics

Deep.Links

PostgreSQL

JS Docker Isolation Provider