lorenzznerol
commented
3 years ago Installed PostgreSQL using the official docker guide of the postgresql website: https://hub.docker.com/_/postgres/ I got it to work in Adminer 4.8 in the browser window.
TODO:
- DONE for docker: Without plpython3 support up to now, but that should be possible on Linux after some tweaks, see https://stackoverflow.com/questions/64106929/postgresql-on-docker-how-to-install-and-run-python-dependencies-under-plpython or Add plpython3 Extension to Postgres/timescaledb Alpine Docker Image.
- DONE using psql instead of Adminer: I had problems importing the data to the table and skipping the first headerline when using Adminer, but that can be solved of course. We can also just prepare the CSVs without a header line. Or alternatively, the already available db can be prepared in pgAdmin 4 using the local installation, then the db can be dumped to a file and loaded in full to the postgres container. Uploading a dump: https://stackoverflow.com/questions/53019247/using-docker-in-adminer-i-do-not-see-imported-data.
- DONE using a TimescaleDB webserver, but that does not have plpython3u extension up to now: I tried accessing the server from another laptop, up to now without success, but should be possible in a network.
- DONE using a TimescaleDB webserver that allows 10 GB: max import size 128 MB but can be set higher, we should have 512 MB import limit and the db should have a limit of 1 GB at least.
docker pull postgres
docker run -it -e POSTGRES_PASSWORD=postgresql postgres postgres
docker run --name postgres1 -e POSTGRES_PASSWORD=postgres -d postgres
## https://github.com/docker-library/docs/blob/master/postgres/README.md
## Example stack.yml for postgres:
## Use postgres/example user/password credentials
#version: '3.1'
#
#services:
#
# db:
# image: postgres
# restart: always
# environment:
# POSTGRES_PASSWORD: example
#
# adminer:
# image: adminer
# restart: always
# ports:
# - 8080:8080
## Then open in a Browser
http://localhost:8080/Run
docker stack deploy -c stack.yml postgres(ordocker-compose -f stack.yml up), wait for it to initialize completely, and visit http://swarm-ip:8080, http://localhost:8080 [this is what I took], or http://host-ip:8080 (as appropriate).
You will see the Adminer opening an access to the db that is running from the container.
We see here that plpython3 is not included in the standard image of postgres:
No choice for skipping the header line.
The main problems are solved, though the plpython3u extension does not yet work on the webserver, but only on the docker container, while other problems of the docker container would be solved by the webserver. It can be expected that the webserver will finally support python3u as well, and then all of this is solved,
But Kafka is only needed for real-time access which is not this issue here, we do not need it for now, it is only needed in #19 which is pure vision, not realistic.
API is needed to feed the mobile app with instant data from a backend. This is on top of the normal data mining task that is supposed to prepare the clustering and ML as far as possible in the backend. Data to be sent is just the exact filtered zoom and area that is needed for the app visualization.