search

DefectDojo / django-DefectDojo

DevSecOps, ASPM, Vulnerability Management. All on one platform.
https://defectdojo.com
BSD 3-Clause "New" or "Revised" License
3.66k stars 1.53k forks source link

Unable to reproduce steps for setup from official documentation #1951

Closed AlexanderTyutin closed 4 years ago

AlexanderTyutin commented 4 years ago

Bug description Got the error During the installation according to the official documentation:

==============================================================================
  Creating dojo/settings/settings.py and .env file
==============================================================================

==============================================================================
  Installing DefectDojo Django application 
==============================================================================

/opt/dojo/bin/python3: No module named virtualenv

...

[5/5] Building fresh packages...
success Saved lockfile.
Done in 27.87s.
Traceback (most recent call last):
  File "manage.py", line 11, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/dojo/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/opt/dojo/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/dojo/lib/python3.6/site-packages/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/dojo/lib/python3.6/site-packages/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "/opt/dojo/lib/python3.6/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 188, in handle
    collected = self.collect()
  File "/opt/dojo/lib/python3.6/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 105, in collect
    for path, storage in finder.list(self.ignore_patterns):
  File "/opt/dojo/lib/python3.6/site-packages/django/contrib/staticfiles/finders.py", line 131, in list
    for path in utils.get_files(storage, ignore_patterns):
  File "/opt/dojo/lib/python3.6/site-packages/django/contrib/staticfiles/utils.py", line 39, in get_files
    yield from get_files(storage, ignore_patterns, dir)
  File "/opt/dojo/lib/python3.6/site-packages/django/contrib/staticfiles/utils.py", line 39, in get_files
    yield from get_files(storage, ignore_patterns, dir)
  File "/opt/dojo/lib/python3.6/site-packages/django/contrib/staticfiles/utils.py", line 23, in get_files
    directories, files = storage.listdir(location)
  File "/opt/dojo/lib/python3.6/site-packages/django/core/files/storage.py", line 316, in listdir
    if entry.is_dir():
OSError: [Errno 40] Too many levels of symbolic links: '/opt/dojo/django-DefectDojo/components/node_modules/@yarn_components/moment/meteor/moment.js'

Steps to reproduce Steps to reproduce the behavior:

  1. Clone repository
  2. Try to reproduce documentation steps and run setup.bash

Expected behavior System is up and running according to the documentation

Deployment method (select with an X)

Environment information

Sample scan files (optional) If applicable, add sample scan files to help reproduce your problem.

Screenshots (optional) As a result no static files are accessible image

Console logs (optional)

Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/dojo/django-DefectDojo, universal_newlines=False, shell=None, istream=<valid stream>)
[28/Feb/2020 18:04:18] "GET /login?next=/ HTTP/1.1" 200 11371
Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/dojo/django-DefectDojo, universal_newlines=False, shell=None, istream=<valid stream>)
Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/dojo/django-DefectDojo, universal_newlines=False, shell=None, istream=<valid stream>)
Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/dojo/django-DefectDojo, universal_newlines=False, shell=None, istream=<valid stream>)
[28/Feb/2020 18:04:19] "GET /static/bootstrap/dist/css/bootstrap.min.css HTTP/1.1" 404 9640
[28/Feb/2020 18:04:19] "GET /static/metisMenu/dist/metisMenu.min.css HTTP/1.1" 404 9640
[28/Feb/2020 18:04:19] "GET /static/startbootstrap-sb-admin-2/dist/css/sb-admin-2.css HTTP/1.1" 404 9640
Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/dojo/django-DefectDojo, universal_newlines=False, shell=None, istream=<valid stream>)
[28/Feb/2020 18:04:19] "GET /static/font-awesome/css/font-awesome.min.css HTTP/1.1" 404 9640
Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/dojo/django-DefectDojo, universal_newlines=False, shell=None, istream=<valid stream>)
Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/dojo/django-DefectDojo, universal_newlines=False, shell=None, istream=<valid stream>)
Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/dojo/django-DefectDojo, universal_newlines=False, shell=None, istream=<valid stream>)
[28/Feb/2020 18:04:19] "GET /static/jquery-ui/themes/flick/jquery-ui.min.css HTTP/1.1" 404 9640
[28/Feb/2020 18:04:19] "GET /static/chosen-bootstrap/chosen.bootstrap.min.css HTTP/1.1" 404 9640
[28/Feb/2020 18:04:19] "GET /static/fullcalendar/dist/fullcalendar.min.css HTTP/1.1" 404 9640

Additional context (optional) Add any other context about the problem here.

arkrwn commented 4 years ago

@AlexanderTyutin maybe this caused by the same problem with this https://github.com/DefectDojo/django-DefectDojo/issues/1931

czm41k commented 4 years ago

Having the same today with docker-compose build

czm41k commented 4 years ago

I have been managed this by editing Dockerfile.nginx to: RUN \ cd components && \ yarn && \ cd node_modules/\@yarn_components/moment/meteor && \ rm moment.js && \ ln -s ../moment.js moment.js && \ cd ../../../../.. && \ python3 manage.py collectstatic --noinput && \ true

PhilParisot commented 4 years ago

Same here with docker-compose build

czm41k commented 4 years ago

Same here with docker-compose build

Try out the solution above. This helped me with docker-compose build

jasonkalwa commented 4 years ago

I have the same issue - brand new install

devGregA commented 4 years ago

@AlexanderTyutin should be fixed now in both dev & master could you please confirm?

jasonkalwa commented 4 years ago

Nope. Just done another new installation, Ubuntu 18.04.

jasonkalwa commented 4 years ago

Starting development server at http://104.248.169.129:443/ Quit the server with CONTROL-C. /usr/local/lib/python3.6/dist-packages/whitenoise/base.py:116: UserWarning: No directory at: /opt/django-DefectDojo/static/ warnings.warn(u"No directory at: {}".format(root)) [22/Mar/2020 18:57:22] code 400, message Bad request syntax ('\x16\x03\x01\x02\x00\x01\x00\x01ü\x03\x03\x04®¾u@3\x13\x85´d>XWL9\x91\x82\x9dïÒlÇý}Fv\x8fO\x82Êe: :\x9fY\x0e¸÷ûñ\x8e\x88G\x99Ù\x9745I]§ò\x1fø\x9dì¹*}á¿æÎë\x00"\x1a\x1a\x13\x01\x13\x02\x13\x03À+À/À,À0̨̩À\x13À\x14\x00\x9c\x00\x9d\x00/\x005\x00') [22/Mar/2020 18:57:22] You're accessing the development server over HTTPS, but it only supports HTTP.

^Croot@Defects:/opt/django-DefectDojo# python3 manage.py runserver 104.248.169.129:80 Watching for file changes with StatReloader Performing system checks...

Popen(['git', 'version'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=None) Popen(['git', 'version'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=None) System check identified no issues (0 silenced). March 22, 2020 - 18:57:42 Django version 2.2.9, using settings 'dojo.settings.settings' Starting development server at http://104.248.169.129:80/ Quit the server with CONTROL-C. /usr/local/lib/python3.6/dist-packages/whitenoise/base.py:116: UserWarning: No directory at: /opt/django-DefectDojo/static/ warnings.warn(u"No directory at: {}".format(root))

^Croot@Defects:/opt/django-DefectDojo# python3 manage.py runserver 104.248.169.129:943 Watching for file changes with StatReloader Performing system checks...

Popen(['git', 'version'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=None) Popen(['git', 'version'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=None) System check identified no issues (0 silenced). March 22, 2020 - 18:59:30 Django version 2.2.9, using settings 'dojo.settings.settings' Starting development server at http://104.248.169.129:943/ Quit the server with CONTROL-C. /usr/local/lib/python3.6/dist-packages/whitenoise/base.py:116: UserWarning: No directory at: /opt/django-DefectDojo/static/ warnings.warn(u"No directory at: {}".format(root)) [22/Mar/2020 18:59:41] "GET / HTTP/1.1" 302 0 Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:41] "GET /login?next=/ HTTP/1.1" 200 11371 Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:41] "GET /static/bootstrap/dist/css/bootstrap.min.css HTTP/1.1" 404 9640 Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:42] "GET /static/startbootstrap-sb-admin-2/dist/css/sb-admin-2.css HTTP/1.1" 404 9640 Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:42] "GET /static/metisMenu/dist/metisMenu.min.css HTTP/1.1" 404 9640 Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:42] "GET /static/font-awesome/css/font-awesome.min.css HTTP/1.1" 404 9640 [22/Mar/2020 18:59:42] "GET /static/jquery-ui/themes/flick/jquery-ui.min.css HTTP/1.1" 404 9640 [22/Mar/2020 18:59:42] "GET /static/chosen-bootstrap/chosen.bootstrap.min.css HTTP/1.1" 404 9640 Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:42] "GET /static/fullcalendar/dist/fullcalendar.min.css HTTP/1.1" 404 9640

Exception happened during processing of request from ('77.97.234.6', 54907) Traceback (most recent call last): File "/usr/lib/python3.6/socketserver.py", line 654, in process_request_thread self.finish_request(request, client_address) File "/usr/lib/python3.6/socketserver.py", line 364, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/lib/python3.6/socketserver.py", line 724, in init self.handle() File "/usr/local/lib/python3.6/dist-packages/django/core/servers/basehttp.py", line 171, in handle self.handle_one_request() File "/usr/local/lib/python3.6/dist-packages/django/core/servers/basehttp.py", line 179, in handle_one_request self.raw_requestline = self.rfile.readline(65537) File "/usr/lib/python3.6/socket.py", line 586, in readinto return self._sock.recv_into(b) ConnectionResetError: [Errno 104] Connection reset by peer

Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:42] "GET /static/dojo/css/dojo.css HTTP/1.1" 404 9640 Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:42] "GET /static/dojo/css/datatables.min.css HTTP/1.1" 404 9640 Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:42] "GET /static/jquery/dist/jquery.js HTTP/1.1" 404 9640 [22/Mar/2020 18:59:42] "GET /static/jquery-ui/jquery-ui.min.js HTTP/1.1" 404 9640 Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:42] "GET /static/metisMenu/dist/metisMenu.min.js HTTP/1.1" 404 9640 [22/Mar/2020 18:59:42] "GET /static/bootstrap/dist/js/bootstrap.min.js HTTP/1.1" 404 9640 Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:42] "GET /static/moment/min/moment.min.js HTTP/1.1" 404 9640 [22/Mar/2020 18:59:42] "GET /static/startbootstrap-sb-admin-2/dist/js/sb-admin-2.js HTTP/1.1" 404 9640 Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:42] "GET /static/dojo/img/logo.png HTTP/1.1" 404 9640 Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:42] "GET /static/dojo/img/chop.png HTTP/1.1" 404 9640 Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:42] "GET /static/fullcalendar/dist/fullcalendar.min.js HTTP/1.1" 404 9640 [22/Mar/2020 18:59:42] "GET /static/metisMenu/dist/metisMenu.min.js HTTP/1.1" 404 9640 Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:42] "GET /static/jquery-cookie/jquery.cookie.js HTTP/1.1" 404 9640 [22/Mar/2020 18:59:42] "GET /static/dojo/js/datatables.min.js HTTP/1.1" 404 9640 Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:42] "GET /static/dojo/js/index.js HTTP/1.1" 404 9640 [22/Mar/2020 18:59:42] "GET /static/startbootstrap-sb-admin-2/dist/js/sb-admin-2.js HTTP/1.1" 404 9640 Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:42] "GET /static/moment/min/moment.min.js HTTP/1.1" 404 9640 Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:42] "GET /static/fullcalendar/dist/fullcalendar.min.js HTTP/1.1" 404 9640 Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:43] "GET /static/jquery-cookie/jquery.cookie.js HTTP/1.1" 404 9640 Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:43] "GET /static/dojo/js/datatables.min.js HTTP/1.1" 404 9640 Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:43] "GET /static/dojo/js/index.js HTTP/1.1" 404 9640

Exception happened during processing of request from ('77.97.234.6', 54931) Traceback (most recent call last): File "/usr/lib/python3.6/socketserver.py", line 654, in process_request_thread self.finish_request(request, client_address) File "/usr/lib/python3.6/socketserver.py", line 364, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/lib/python3.6/socketserver.py", line 724, in init self.handle() File "/usr/local/lib/python3.6/dist-packages/django/core/servers/basehttp.py", line 171, in handle self.handle_one_request() File "/usr/local/lib/python3.6/dist-packages/django/core/servers/basehttp.py", line 179, in handle_one_request self.raw_requestline = self.rfile.readline(65537) File "/usr/lib/python3.6/socket.py", line 586, in readinto return self._sock.recv_into(b) ConnectionResetError: [Errno 104] Connection reset by peer

Popen(['git', 'cat-file', '--batch-check'], cwd=/opt/django-DefectDojo, universal_newlines=False, shell=None, istream=) [22/Mar/2020 18:59:43] "GET /static/dojo/img/favicon.ico HTTP/1.1" 404 9640

00rex00 commented 4 years ago

some of the problems with setup.bash for ubuntu1804 are

1) the apt install nodejs installs an old version of 8 and the nodes setup runs for version 6, so if you remove the apt install of nodejs from line 214 of setup/scripts/os/linux.sh and

2) change last line of setup/scripts/common/config-vars.sh from NODE_URL="https://deb.nodesource.com/setup_6.x" to NODE_URL="https://deb.nodesource.com/setup_8.x"

your YARN section might complete and publish a /static folder

if YARN section still fails - tempoarily remove line 29 , "@yarn_components/moment": "moment/moment#^2.9.0", from components/package.json

moment is a date format library that breaks my Yarn and collectstatic https://momentjs.com/

jasonkalwa commented 4 years ago

Hi, thanks for that, that has fixed some of the issues. UI issue is still just text and no images/css etc so I moved the /static folder from /dojo into the application path root, there UI is improved but still clearly not right. Using fresh ubuntu 18.04.

adamtimmins commented 4 years ago

Still seeing this issue unfortunately :(

Edit:

I tried this out and the static folder does get published.

some of the problems with setup.bash for ubuntu1804 are

  1. the apt install nodejs installs an old version of 8 and the nodes setup runs for version 6, so if you remove the apt install of nodejs from line 214 of setup/scripts/os/linux.sh and
  2. change last line of setup/scripts/common/config-vars.sh from NODE_URL="https://deb.nodesource.com/setup_6.x" to NODE_URL="https://deb.nodesource.com/setup_8.x"

your YARN section might complete and publish a /static folder

if YARN section still fails - tempoarily remove line 29 , "@yarn_components/moment": "moment/moment#^2.9.0", from components/package.json

moment is a date format library that breaks my Yarn and collectstatic https://momentjs.com/

@00re00 do you know what functions exactly of the application I'm missing without having the moment installed?

Maffooch commented 4 years ago

As of release 1.6.0 this should be fixed in the master branch