dfir-iris / iris-web

Collaborative Incident Response platform
GNU Lesser General Public License v3.0
999 stars 152 forks source link

[BUG] ERROR: function gen_random_uuid() does not exist at character 83 since version 2.4.0 #388

Open IOTech17 opened 6 months ago

IOTech17 commented 6 months ago

Describe the bug When deploying the stack I can see this error on both the db :

ERROR: function gen_random_uuid() does not exist at character 83

and in the app container :

sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedFunction) function gen_random_uuid() does not exist LINE 4: org_uuid UUID DEFAULT gen_random_uuid() NOT NULL,

To Reproduce Steps to reproduce the behavior: Followed the steps in the readme and also try deploying the image via portainer stack calling the env file via stack.env

I am using Fedora Core OS as docker host and to build the image

whikernel commented 6 months ago

Hi, thanks for reporting.

This is the same issue as #387. We're looking into it, please see https://github.com/dfir-iris/iris-web/issues/387#issuecomment-1900199038

whikernel commented 6 months ago

This should now be fixed in v.2.4.4.

IOTech17 commented 6 months ago

@whikernel

I tried with version 2.4.5, same error :

sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedFunction) function gen_random_uuid() does not exist LINE 4: org_uuid UUID DEFAULT gen_random_uuid() NOT NULL,

whikernel commented 6 months ago

@IOTech17 - You need to completely reset the IRIS docker containers with docker compose down --volumes, otherwise the DB will start initialized and won't go through the registration of the extension.
Can you confirm you have done such step?

IOTech17 commented 6 months ago

@whikernel

Yes this is exactly what I did

whikernel commented 6 months ago

I tried multiple times on fresh servers and I cannot reproduce the issue. The create_user.sh that is called at DB init should register the extension in the right iris_db DB.

CREATE USER ${POSTGRES_ADMIN_USER} WITH CREATEDB SUPERUSER PASSWORD '${POSTGRES_ADMIN_PASSWORD}';
\c iris_db;
CREATE EXTENSION IF NOT EXISTS pgcrypto CASCADE;
IOTech17 commented 6 months ago

Very interesting,

Could you try to build the image on a fedora core os instance?

I will try to build the image on another linux os

IOTech17 commented 6 months ago

@whikernel

I build the image on Solus OS then send the image to Dockerhub then downloaded the image via Portainer and applied the env via stack.env

Running iriswebapp ...
[2024-01-23 11:50:44 +0000] [7] [INFO] Starting gunicorn 20.1.0
[2024-01-23 11:50:44 +0000] [7] [INFO] Listening at: http://0.0.0.0:8000 (7)
[2024-01-23 11:50:44 +0000] [7] [INFO] Using worker: eventlet
[2024-01-23 11:50:44 +0000] [9] [INFO] Booting worker with pid: 9
2024-01-23 11:50:46 :: INFO :: post_init :: run_post_init :: IRIS v2.4.5
2024-01-23 11:50:46 :: INFO :: post_init :: run_post_init :: Running post initiation steps
2024-01-23 11:50:46 :: INFO :: post_init :: run_post_init :: Attempting to connect to the database...
2024-01-23 11:50:46 :: INFO :: post_init :: run_post_init :: Connecting to database, attempt 1/3
2024-01-23 11:50:46 :: INFO :: post_init :: run_post_init :: Creating all Iris tables
2024-01-23 11:50:46 :: ERROR :: views :: <module> :: Post init failed. IRIS not started
Traceback (most recent call last):
  File "/opt/venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1969, in _exec_single_context
    self.dialect.do_execute(
  File "/opt/venv/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 922, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.UndefinedFunction: function gen_random_uuid() does not exist
LINE 4:  org_uuid UUID DEFAULT gen_random_uuid() NOT NULL, 
                               ^
HINT:  No function matches the given name and argument types. You might need to add explicit type casts.
IOTech17 commented 6 months ago

@whikernel which os are you using to build the image? Wouldn't it be better to provide official image directly via dockerhub?

whikernel commented 6 months ago

@IOTech17 I tested on Ubuntu 24 and Debian 12, freshly installed and I have not this issue anymore.

We're not providing images directly on DockerHub mostly because we're lacking resources to do so. Any help with that is more than welcome! Ideally that should be done through the CI/CD once a release is done.

But that wouldn't solve your current issue because the addition of the PG extension is done through a bash script ran in the Postgres instance at boot time. Are you using our Postgres from the docker-compose, or your own Postgres?

IOTech17 commented 6 months ago

@whikernel

I am using the image build from the docker-compose build. I tried building the image from 2 different Linux OS (Solus OS and Fedora Core OS)

You can try to contact https://www.linuxserver.io/ to help you with the automation for the image ,they have automated the build of many docker image.

IOTech17 commented 6 months ago

@whikernel Same problem when building the image via a ubuntu wsl instance (ubuntu 22.04).

I then sent the image to dockerhub then download it via portainer and apply the env via stack.env

IOTech17 commented 6 months ago

@whikernel

Deploy the app only work on Debian/ubuntu host by directly calling docker-compose up from the folder. I tried using portainer and passing the env to the stack but I get the same error. Can you check if the image contains all the necessary files once created to start properly.

It would seems that something is missing within the image to be able to work properly via portainer or on other host os.

IOTech17 commented 6 months ago

What I do not understand is why the script is not being called despite the fact that I am using the image built via the command docker-compose build?

IOTech17 commented 6 months ago

@whikernel Do you know why this cannot be built on another other than Ubuntu and debian?

Have you contacted linuxserver.io to help you automate the build and publishing on new image

whikernel commented 6 months ago

Hi @IOTech17 You should be able to built the IRIS on other systems than Ubuntu and Debian. We didn't get the chance to test it out yet, but we will. We didn't contact linuxserver.io, we're just lacking time to do so. Publishing the image is something we meant to do for a while. We want to add it over CI/CD, and add testing in the middle. #395 is a step toward it.

As for the script, it's a good question why it's not calling it. There are visibly quite some issues for deploying IRIS with via portainer that we're going to look into.

IOTech17 commented 6 months ago

@whikernel

Let me know if you need help testing, I can help you with that.

genseirin commented 3 weeks ago

I had the same error.

First I went with a shell into the iriswebapp_db container and ran psql -c 'CREATE EXTENSION IF NOT EXISTS pgcrypto;' iris_db postgres.

I also had to change the POSTGRES_PASSWORD with the output of the recommended openssl rand -base64 64. Before I had a password with special characters !\,%]'>.

At some point I did docker volume rm to reset the installation, rebuilt several times, I don't remember in which order. Now all looks good.