No longer installs

[giveen]

Due to upstream changes, and unless the Dev's do something about it, this will no longer install any version of DefectDojo.

I recently tried version 1.17 and 1.16 to install v2.4.1 plus many other version, all fail with missing python packages in the requirements.txt. Trying to install these by hand results in yet another error.

[Sujitha6496]

Are you facing the issue that the GoDojo executable file does not download all the required files to install DefectDojo?

[giveen]

Are you facing the issue that the GoDojo executable file does not download all the required files to install DefectDojo?

It doesnt include psycopg2-binary so the installation fails, as its not in the requirements.txt

[noloader]

@giveen,

I built the installer from the latest sources. I did not have trouble (except for some database install/configure hiccups).

Here is my requirements.txt. It includes psycopg2-binary version 2.9.1.

# cd /opt/dojo
# find . -name requirements.txt
./django-DefectDojo/requirements.txt

# cat django-DefectDojo/requirements.txt
# requirements.txt for DefectDojo using Python 3.x
asteval==0.9.25
bleach==4.1.0
celery==5.1.2
coverage==6.1.1
defusedxml==0.7.1
django_celery_results==2.2.0
django-auditlog==1.0a1
django-dbbackup==3.3.0
django-environ==0.8.1
django-filter==21.1
django-imagekit==4.0.2
django-multiselectfield==0.1.12
django-polymorphic==3.0.0
django-crispy-forms==1.13.0
django_extensions==3.1.3
django-slack==5.17.6
django-tagging==0.5.0
django-watson==1.5.5
django-prometheus==2.1.0
Django==3.1.13
djangorestframework==3.12.4
gunicorn==20.1.0
html2text==2020.1.16
humanize==3.12.0
jira==3.0.1
PyGithub==1.55
lxml==4.6.3
Markdown==3.3.4
mysqlclient==2.0.3
openpyxl==3.0.9
xlrd==1.2.0
Pillow==8.4.0  # required by django-imagekit
psycopg2-binary==2.9.1
cryptography==35.0.0
python-dateutil==2.8.2
pytz==2021.3
redis==3.5.3
requests==2.26.0
sqlalchemy==1.4.26  # Required by Celery broker transport
supervisor==4.2.2
urllib3==1.26.7
uWSGI==2.0.20
vobject==0.9.6.1
whitenoise==5.2.0
titlecase==2.3
social-auth-app-django==5.0.0
social-auth-core==4.1.0
Python-jose==3.3.0
gitpython==3.1.24
debugpy==1.5.1
python-gitlab==2.10.1
google-api-python-client==2.28.0
google-auth==2.3.2
google-auth-oauthlib==0.4.6
drf_yasg==1.20.0
cpe==1.2.1
packageurl-python==0.9.6
django-crum==0.7.9
JSON-log-formatter==0.4.0
django-split-settings==1.1.0
django-debug-toolbar==3.2.2
django-debug-toolbar-request-history==0.1.4
vcrpy==4.1.1
vcrpy-unittest==0.1.7
django-tagulous==1.3.0
PyJWT==2.3.0
cvss==2.3
django-fieldsignals==0.7.0
hyperlink==21.0.0
numpy==1.21.3
django-test-migrations==1.1.0
djangosaml2==1.3.4
drf-spectacular==0.20.2
django-ratelimit==3.0.1
argon2-cffi==21.1.0

[giveen]

@noloader Hi there, so I'm not a developer, I'm a Security Analyst. I have no idea how to compile Go programs, which is why binary installers are a huge help. I did tried to build from source and edit a few things but it just let to more issues with Go build that I do not understand.

[noloader]

@giveen,

I'm not a developer, I'm a Security Analyst.

Ah, mee too. We are trying to evaluate DefectDojo as a vulnerability management solution. We ruled them out about 6 months ago because of the problems. But our team wants to try again since they are Open Source, and we want to support them.

I did tried to build from source and edit a few things but it just let to more issues with Go build that I do not understand.

Yeah, I know what you mean. We could not build and install on Ubuntu 18 or 20. (Ubuntu is the only platform they officially support). The devs have a penchant for using bleeding edge components and the latest version of components, and it causes so many problems.

We added Fedora support so we could get something up-and-running. Fedora provides the latest version of components. Fedora's cadence seems to best match the requirements for DefectDojo.

Once you get something up and running, the next question is, how do we update all this stuff? We need a systemd unit to update the installation daily. I can't find anything related to updates supplied by the project. That's a nut we still need to crack.

Another problem is, the devs seem to lack the concept of supply chain management. They pull in so much unprovenanced and unaudited stuff I fear we will never be able to secure the installation. That's going to cause a lot of problems for us when we go to the Security Architect group and ask for permission to stand up a server in the company for testing.

Feel free to ping me at noloader, gmail account. I may be able to help with your build and installation. I can also give you some contacts for 10Security, which provides commercial support for DefectDojo.

And you might want to take a look at my fork at https://github.com/noloader/godojo . I'm fixing what I can as we work though the build and install .

[giveen]

@noloader I'll watch your repo and see how it goes. Yeah, until my server engineers figure out a solution to provide patch management to docker images and are willing to pay for it out of their budgets, we are saying no to docker, otherwise that would be the route I would be going right now. Seem's the dev's are completely focused on the docker image, but docker was designed to be a quick "test" system, not an always on virtual machine, so we aren't a fan of it.

[noloader]

@giveen,

I'll watch your repo and see how it goes.

Yeah, let me know how it goes.

Since I am a fork, I perform forced updates to keep the commit history flat. That means you can't do a git checkout && git pull. Instead, you will need to do a git fetch && git reset --soft origin.

And you may want to give my build/install script a try at https://github.com/noloader/godojo/blob/master/build-godojo.sh . The script is built for a PostgreSQL install.

[mtesauro]

Just an FYI to this issue thread.

I'm in the middle of a major refactor of godojo to include this library and am hoping to have it done within ~ 1 week.

What this means for you is two things:

(1) @noloader 's fork is going to be WAY off shortly (2) Adding additional OSes will be much more straight forward - that's why I'm doing the refactor

The original intent of creating godojo was to replace setup.bash which was ~2k lines of bash that was very brittle, didn't handle errors well, didn't log, and otherwise was a maintenance pain. Since several people have inquired about other OS support for godojo, I wanted to expand it from being .deb / Ubuntu specific - something that setup.bash never did.

I had planned to get that done over the summer but personal life issues derailed that completely. I believe I have most of that sorted out and am back to working on the refactor. I now have some spare time in my nights & weekends.

Cheers!

[noloader]

Thanks @mtesauro,

You may want to grab this file when you perform the installer update: pg_check_conn.cpp.

pg_check_conn is for PostgreSQL, and it actually works the way you expect by testing connectivity to the database using the supplied credentials. That is in contrast to PostgreSQL's pg_isready, which does not use the supplied credentials. So you get false positives back when using pg_isready.

pg_check_conn is public domain so it is free for anyone to use. I am happy to give it to godojo and assign the copyright or dual license it. Whatever work for the project works for me.

[mtesauro]

I'll take a look at pg_check_conn.cpp

I originally was using Go's built-in SQL support to check the connections but that was problematic especially supporting both MySQL and PostgreSQL. It was later changed to pg_isready in hopes that using a PosgreSQL supplied utility would make things better/more consistent.

PostgreSQL's pg_isready, which does not use the supplied credentials.

That feels like a bug to me. I'll have to look and see if anyone has filed a bug on that.

And, yeah, I'm still not happy with pg_isready as it's not performed as consistently as I'd expect it to.

[noloader]

I originally was using Go's built-in SQL support to check the connections but that was problematic especially supporting both MySQL and PostgreSQL.

Yeah, I was wondering about that. PostgreSQL's lack of CREATE IF NOT EXISTS means you are probably going to have to maintain different code for MySQL and PostgreSQL.

PostgreSQL's pg_isready, which does not use the supplied credentials.

That feels like a bug to me. I'll have to look and see if anyone has filed a bug on that.

Determine if a user and database are available on the pgsql-general mailing list.

[mtesauro]

Ahh, thanks for that link.

Yeah, that's definitely not what I thought it did based on the name. Good to know it's really only checking reach-ability.

[noloader]

@mtesauro,

I added a repo for pg_check_conn.cpp at https://github.com/noloader/pg_check_conn .

[mtesauro]

FWIW, I just had a successful install on Ubuntu 20.04 [1] - it looks like a recent update to pycurl has a dependency on libcurl4-openssl-dev

For a quick and dirty workaround, you can run this (Debian-based systems) to get past this problem:

sudo apt install libcurl4-openssl-dev

prior to running godojo so the library is present on the system.

I'm adding the new dependency to the refactor so it will be part of the next release.

[1] technical details on the install

• Ubuntu 20.04.5 LTS
• godojo 1.1.7
• PostgreSQL DB (non-existent prior to install) - basically a fresh VM
• DefectDojo v. 2.14.0