Godojo install fails: psycopg2 missing?

[khalavak]

Hello, first time user here trying to install DefectDojo with godojo but get the following error:

Setting up Django for DefectDojo...(-----*----) 
##############################################################################
  ERROR: 2022/05/01 09:50:35 - Failed to run OS command cd /opt/dojo/django-DefectDojo && source ../bin/activate && python3 manage.py makemigrations dojo, error was: exit status 1
##############################################################################

...so something going wrong with Django makemigrations. The whole output can be seen below. When I then test this manually in the defectdojo virtualenv I get the following:

root@dojoserver:/opt/godojo# cd /opt/dojo/
root@dojoserver:/opt/dojo/django-DefectDojo# source bin/activate
(dojo) root@dojoserver:/opt/dojo/django-DefectDojo# python3 manage.py makemigrations
[01/May/2022 09:53:32] INFO [dojo.models:3653] enabling audit logging
[01/May/2022 09:53:33] DEBUG [dojo.forms:2089] templates: [('', '---'), ('issue-trackers/jira_full', 'jira_full'), ('issue-trackers/jira_limited', 'jira_limited')]
Traceback (most recent call last):
  File "manage.py", line 11, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/dojo/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/opt/dojo/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/dojo/lib/python3.8/site-packages/django/core/management/base.py", line 330, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/dojo/lib/python3.8/site-packages/django/core/management/base.py", line 368, in execute
    self.check()
  File "/opt/dojo/lib/python3.8/site-packages/django/core/management/base.py", line 392, in check
    all_issues = checks.run_checks(
  File "/opt/dojo/lib/python3.8/site-packages/django/core/checks/registry.py", line 70, in run_checks
    new_errors = check(app_configs=app_configs, databases=databases)
  File "/opt/dojo/lib/python3.8/site-packages/django/core/checks/urls.py", line 13, in check_url_config
    return check_resolver(resolver)
  File "/opt/dojo/lib/python3.8/site-packages/django/core/checks/urls.py", line 23, in check_resolver
    return check_method()
  File "/opt/dojo/lib/python3.8/site-packages/django/urls/resolvers.py", line 408, in check
    for pattern in self.url_patterns:
  File "/opt/dojo/lib/python3.8/site-packages/django/utils/functional.py", line 48, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/opt/dojo/lib/python3.8/site-packages/django/urls/resolvers.py", line 589, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/opt/dojo/lib/python3.8/site-packages/django/utils/functional.py", line 48, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/opt/dojo/lib/python3.8/site-packages/django/urls/resolvers.py", line 582, in urlconf_module
    return import_module(self.urlconf_name)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 848, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/dojo/django-DefectDojo/dojo/urls.py", line 34, in <module>
    from dojo.product.urls import urlpatterns as prod_urls
  File "/opt/dojo/django-DefectDojo/dojo/product/urls.py", line 3, in <module>
    from dojo.product import views
  File "/opt/dojo/django-DefectDojo/dojo/product/views.py", line 37, in <module>
    from django.contrib.postgres.aggregates import StringAgg
  File "/opt/dojo/lib/python3.8/site-packages/django/contrib/postgres/aggregates/__init__.py", line 1, in <module>
    from .general import *  # NOQA
  File "/opt/dojo/lib/python3.8/site-packages/django/contrib/postgres/aggregates/general.py", line 1, in <module>
    from django.contrib.postgres.fields import ArrayField
  File "/opt/dojo/lib/python3.8/site-packages/django/contrib/postgres/fields/__init__.py", line 1, in <module>
    from .array import *  # NOQA
  File "/opt/dojo/lib/python3.8/site-packages/django/contrib/postgres/fields/array.py", line 3, in <module>
    from django.contrib.postgres import lookups
  File "/opt/dojo/lib/python3.8/site-packages/django/contrib/postgres/lookups.py", line 4, in <module>
    from .search import SearchVector, SearchVectorExact, SearchVectorField
  File "/opt/dojo/lib/python3.8/site-packages/django/contrib/postgres/search.py", line 1, in <module>
    import psycopg2
ModuleNotFoundError: No module named 'psycopg2'
(dojo) root@dojoserver:/opt/dojo/django-DefectDojo#

..so seems like psycopg2 has not been installed. It is only present in the requirements.cya file, not in the requirements.txt file:

(dojo) root@dojoserver:/opt/dojo/django-DefectDojo# ls -la requirements.*
-rw-r--r-- 1 dojo-srv dojo-srv 1644 May  1 09:49 requirements.cya
-rw-r--r-- 1 dojo-srv dojo-srv 1651 May  1 09:49 requirements.txt
(dojo) root@dojoserver:/opt/dojo/django-DefectDojo# grep psycopg2 requirements.*
requirements.cya:psycopg2-binary==2.9.1
(dojo) root@dojoserver:/opt/dojo/django-DefectDojo#

The psycopg2 is present in the defectDojo git repository requirements file: https://github.com/DefectDojo/django-DefectDojo/blob/fef0d194ff985358be8e7ebbd4c13a02b9c5a539/requirements.txt#L34

...so what is going on here?

Complete godojo instsallation output:

(dojo) root@dojoserver:/opt/godojo# ./godojo
        ____       ____          __     ____          _      
       / __ \___  / __/__  _____/ /_   / __ \____    (_)___  
      / / / / _ \/ /_/ _ \/ ___/ __/  / / / / __ \  / / __ \ 
     / /_/ /  __/ __/  __/ /__/ /_   / /_/ / /_/ / / / /_/ / 
    /_____/\___/_/  \___/\___/\__/  /_____/\____/_/ /\____/  
                                               /___/         
    version  1.1.7

  Welcome to godojo, the official way to install DefectDojo on iron.
  For more information on how goDojo does an install, see:
  https://github.com/DefectDojo/godojo

==============================================================================
  Starting the dojo install at Sun May 1, 2022 10:00:46 UTC
==============================================================================

==============================================================================
  Determining OS for installation
==============================================================================

OS was determined to be Linux, Ubuntu:20.04
DefectDojo installation on this OS is supported, continuing

==============================================================================
  Bootstrapping the godojo installer
==============================================================================

Boostraping godojo installer complete

==============================================================================
  Checking for Python 3
==============================================================================

Python 3 found, install can continue

==============================================================================
  Downloading the source for DefectDojo
==============================================================================

Downloading the configured release of DefectDojo => version 2.9.1
Successfully downloaded and extracted the DefectDojo release file

==============================================================================
  Prompt set to false, non-interactive installation
==============================================================================

==============================================================================
  Installing OS packages needed for DefectDojo
==============================================================================

Installing OS packages complete

==============================================================================
  Installing database needed for DefectDojo
==============================================================================

Installing Database complete

==============================================================================
  Starting the database needed for DefectDojo
==============================================================================

Starting Database complete

==============================================================================
  Preparing the database needed for DefectDojo
==============================================================================

Validating DB connection

==============================================================================
  Preparing the OS for DefectDojo installation
==============================================================================

Preparing the OS complete

==============================================================================
  Creating settings.py for DefectDojo
==============================================================================

Creating settings.py for DefectDojo complete

==============================================================================
  Setting up Django for DefectDojo
==============================================================================

Setting up Django for DefectDojo...(----*-----) 
##############################################################################
  ERROR: 2022/05/01 10:01:45 - Failed to run OS command cd /opt/dojo/django-DefectDojo && source ../bin/activate && python3 manage.py makemigrations dojo, error was: exit status 1
##############################################################################

[shinstudio]

I'm running into this same issue too. My os is ubuntu 20.04.4 focal. the installer seems to pull the postgres dependencies and cause the issue even though I choose MySQL.

Also one thing I noticed was that the initial requirements.txt has psycopg2 in it. However, after the installer creates requirements.cya file, psycopg2 is gone from requirements.txt.

[mtesauro]

godojo originally tried to have a 'cleaner' install by selectively creating a requirements.txt based on the DB desired.

That's cause this and other issues and is in the process of being removed in a currently in process major refactor. That refactor is probably a week away from landing.

[giveen]

@mtesauro Is there any update on this?

[mtesauro]

Try the latest version 1.1.9 - it's working for for every install I've tried.