search

HugoDataAnalyst / Psyduck

Apache License 2.0
5 stars 0 forks source link

readme

Psyduck

Welcome to PsyyyPsyPsyyyyyduckk - a multifaceted project designed to streamline data handling and analysis.

This 3-in-1 solution encompasses:

PsyyyPsyPsyyyyyduckk is built for performance and reliability, mirroring the unexpected power of Psyduck from the Pokémon universe. Just as Psyduck releases formidable psychic powers when its headaches peak, this project tackles the "headaches" of data analysis by efficiently processing and interpreting large volumes of information.

It's the perfect tool for anyone looking to burst through data-heavy challenges with speed and precision. Dive in and experience a new wave of data management and analysis!

Psyduck Flex

Special Thanks:

API paths:

Pokemon:

Quests:

Raids:

Invasions:

Pokemon TTH:

Metrics Pokemon:

Metrics Quests:

Metrics Raids:

Metrics Invasions:

Metrics Pokemon TTH:

Metrics/ is only for Prometheus/VictoriaMetrics types (plaintext).

Usage of APIs:

Currently I've setup dynamic caching TTL which means you don't need to ask for the API data more then the setup cache timers per hour/day/weekly/month.

Timers for each type:

Quest specific:

Quest also has a parameter called Scanned.

It's current logic works for scans based on normal quest timer (00-01 and 10-11):

You can change the procedures:

logic to add more cases or tighten its timers.

Requirements:

Python3 Libraries:

Installation:

python3.10 -m pip install -r requirements.txt

sudo apt install redis-server

Configuration:

Redis:

You can ignore these steps if you intend to use redis on the same server since the installation by default is protected.

Use bind to allow only designated ips example:

Disable protected-mode:

Properly set a very strong password for redis, because bruteforce can try 150k passwords per second:

Change the port if you want, default is 6379:

Config:

cd config/ && cp config.example.json config.json

Key considerations:

"redis":

"koji":

"logs for any section":

"receiver" Section:

"database" Section:

"celery" Section:

"api" Section:

Please fill in carefully your config.json

Database:

Take in mind you should allow for the creation of events/procedures in the Database.

cd SQL/ && python3.10 create_database_schema.py

Celery:

Run Celery the task executioner:

python3.10 celery_worker.py

Always run celery first to check for Database Migrations.

After applying them, you can now populate your total_pokestops table, which is used for stats in the quests, by doing the following:

Important Note: This is a standalone script that should be run once or whenever you want to update your total PokéStops numbers per area.

cd SQL && cp example_obtain_total_stops.py obtain_total_stops.py

Fill in the details for the databases access.

python3.10 obtain_total_stops.py

Webhook Receiver:

Run the start_webhookparser.py:

python3.10 start_webhookparser.py

API:

Run the API:

python3.10 start_api.py

Using PM2: -- Not working

npm install pm2@latest -g

Edit ecosystem.config.js as you prefer and run:

pm2 start ecosystem.config.js

Docker:

cd Psyduck

sudo cp example.database.Dockerfile database.Dockerfile

sudo cp example.celery.Dockerfile celery.Dockerfile

sudo cp example.webhook.Dockerfile webhook.Dockerfile

sudo cp example.api.Dockerfile api.Dockerfile

sudo cp example.docker-compose.yml docker-compose.yml

sudo docker-compose up

#Psyduck stack
services:
  psyduck-schema-setup:
    build:
      context: .
      dockerfile: database.Dockerfile
    depends_on:
      - psyduck-db

Remove this section from the docker-compose.yml after running it the first time.

sudo docker-compose up -d

Geofences in Koji:

Your Geofences cannot overlap each other, so kindly create a project and use it where they do not overlap.

When I made this as far as my knowledge goes, Koji did not properly support multi polygons and neither did Dragonite.