linuxserver / docker-diskover

A Docker container for the Diskover space mapping application
GNU General Public License v3.0
76 stars 16 forks source link

Latest builds do not work with provided documentation #41

Closed Dids closed 3 years ago

Dids commented 3 years ago

linuxserver.io


Expected Behavior

The container should run with the provided documentation.

Current Behavior

Container/Diskover throws Python errors in an endless loop and never successfully starts.

Steps to Reproduce

Use the example docker-compose.yaml to create and launch Diskover and its dependencies.

Environment

OS: Ubuntu 18.04, macOS Big Sur CPU architecture: x86_64 How docker service was installed:

Command used to create docker container (run/create/compose/screenshot)

version: '2'
services:
  diskover:
    image: linuxserver/diskover
    container_name: diskover
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
      - REDIS_HOST=redis
      - REDIS_PORT=6379
      - ES_HOST=elasticsearch
      - ES_PORT=9200
      - ES_USER=elastic
      - ES_PASS=changeme
      - RUN_ON_START=true
      - USE_CRON=true
    volumes:
      - ./diskover_data/diskover/config:/config
      - ./:/data
    ports:
      - 18080:80
      - 19181:9181
      - 19999:9999
    mem_limit: 4096m
    restart: unless-stopped
    depends_on:
      - elasticsearch
      - redis
  elasticsearch:
    container_name: elasticsearch
    image: docker.elastic.co/elasticsearch/elasticsearch:5.6.9
    volumes:
      - ./diskover_data//elasticsearch/data:/usr/share/elasticsearch/data
    environment:
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms2048m -Xmx2048m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
  redis:
    container_name: redis
    image: redis:alpine
    volumes:
      - ./diskover_data/redis:/data

Docker logs

Attaching to diskover, elasticsearch, redis
diskover       | [s6-init] making user provided files available at /var/run/s6/etc...exited 0.
diskover       | [s6-init] ensuring user provided files have correct perms...exited 0.
diskover       | [fix-attrs.d] applying ownership & permissions fixes...
diskover       | [fix-attrs.d] done.
diskover       | [cont-init.d] executing container initialization scripts...
diskover       | [cont-init.d] 01-envfile: executing... 
diskover       | [cont-init.d] 01-envfile: exited 0.
diskover       | [cont-init.d] 10-adduser: executing... 
diskover       | 
diskover       | -------------------------------------
diskover       |           _         ()
diskover       |          | |  ___   _    __
diskover       |          | | / __| | |  /  \ 
diskover       |          | | \__ \ | | | () |
diskover       |          |_| |___/ |_|  \__/
diskover       | 
diskover       | 
diskover       | Brought to you by linuxserver.io
diskover       | -------------------------------------
diskover       | 
diskover       | To support LSIO projects visit:
diskover       | https://www.linuxserver.io/donate/
diskover       | -------------------------------------
diskover       | GID/UID
diskover       | -------------------------------------
diskover       | 
diskover       | User uid:    1000
diskover       | User gid:    1000
diskover       | -------------------------------------
diskover       | 
diskover       | [cont-init.d] 10-adduser: exited 0.
diskover       | [cont-init.d] 20-config: executing... 
diskover       | [cont-init.d] 20-config: exited 0.
diskover       | [cont-init.d] 30-keygen: executing... 
diskover       | generating self-signed keys in /config/keys, you can replace these with your own keys if required
diskover       | Generating a RSA private key
diskover       | .....+++++
diskover       | .............................................+++++
diskover       | writing new private key to '/config/keys/cert.key'
diskover       | -----
diskover       | [cont-init.d] 30-keygen: exited 0.
diskover       | [cont-init.d] 50-diskover-config: executing... 
diskover       | Initial run of dispatcher in progress
elasticsearch exited with code 137
diskover       | [cont-init.d] 50-diskover-config: exited 0.
diskover       | [cont-init.d] 60-diskover-web-config: executing... 
diskover       | [cont-init.d] 60-diskover-web-config: exited 0.
diskover       | [cont-init.d] 90-custom-folders: executing... 
diskover       | [cont-init.d] 90-custom-folders: exited 0.
diskover       | [cont-init.d] 99-custom-files: executing... 
diskover       | [custom-init] no custom files found exiting...
diskover       | [cont-init.d] 99-custom-files: exited 0.
diskover       | [cont-init.d] done.
diskover       | [services.d] starting services
diskover       | [services.d] done.
diskover       | 
diskover       | 
diskover       |    ___       ___       ___       ___       ___       ___       ___       ___
diskover       |   /\  \     /\  \     /\  \     /\__\     /\  \     /\__\     /\  \     /\  \
diskover       |  /::\  \   _\:\  \   /::\  \   /:/ _/_   /::\  \   /:/ _/_   /::\  \   /::\  \
diskover       | /:/\:\__\ /\/::\__\ /\:\:\__\ /::-"\__\ /:/\:\__\ |::L/\__\ /::\:\__\ /::\:\__\
diskover       | \:\/:/  / \::/\/__/ \:\:\/__/ \;:;-",-" \:\/:/  / |::::/  / \:\:\/  / \;:::/  /
diskover       |  \::/  /   \:\__\    \::/  /   |:|  |    \::/  /   L;;/__/   \:\/  /   |:\/__/
diskover       |   \/__/     \/__/     \/__/     \|__|     \/__/               \/__/     \|__|
diskover       |                                       v1.5.0.13
diskover       |                                       https://shirosaidev.github.io/diskover
diskover       |                                       Bringing light to the darkness.
diskover       |                                       Support diskover on Patreon or PayPal :)
diskover       | 
diskover       |     
diskover       | 2021-06-29 20:26:24,640 [INFO][diskover] Using config file: /app/diskover/diskover.cfg
diskover       | 2021-06-29 20:26:24,649 [INFO][diskover] Waiting for connection, listening on 0.0.0.0 port 9999 TCP (ctrl-c to shutdown)
diskover       | Traceback (most recent call last):
diskover       |   File "/usr/bin/rq-dashboard", line 11, in <module>
diskover       |     load_entry_point('rq-dashboard==0.6.1', 'console_scripts', 'rq-dashboard')()
diskover       |   File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 490, in load_entry_point
diskover       |     return get_distribution(dist).load_entry_point(group, name)
diskover       |   File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2862, in load_entry_point
diskover       |     return ep.load()
diskover       |   File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2462, in load
diskover       |     return self.resolve()
diskover       |   File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2468, in resolve
diskover       |     module = __import__(self.module_name, fromlist=['__name__'], level=0)
diskover       |   File "/usr/lib/python3.8/site-packages/rq_dashboard/cli.py", line 112, in <module>
diskover       |     def run(
diskover       |   File "/usr/lib/python3.8/site-packages/click/decorators.py", line 247, in decorator
diskover       |     _param_memo(f, OptionClass(param_decls, **option_attrs))
diskover       |   File "/usr/lib/python3.8/site-packages/click/core.py", line 2482, in __init__
diskover       |     super().__init__(param_decls, type=type, multiple=multiple, **attrs)
diskover       |   File "/usr/lib/python3.8/site-packages/click/core.py", line 2108, in __init__
diskover       |     raise ValueError(
diskover       | ValueError: 'default' must be a list when 'multiple' is true.

The Python error repeats over and over again, indefinitely. Note that I have also tried passing in the (required?) auth token, but the end result is the same.

github-actions[bot] commented 3 years ago

Thanks for opening your first issue here! Be sure to follow the bug or feature issue templates!

jrlaberge commented 3 years ago

I was also facing this issue. I managed to get it running by doing the following:

docker exec -it diskover bash
pip3 install 'click==7.1.2' --force-reinstall
exit
docker restart diskover

~Of course the long-term fix is adding click==7.1.2 to requirements.txt or correcting the data being passed to click.~

Actually upon a closer look, upgrading rq in the requirements.txt file to point to >=1.8.1

https://github.com/rq/rq/issues/1469

j0nnymoe commented 3 years ago

Unfortunately this container is getting depreciated soon. We had been meaning to do it for a while but other things had got in the way. The developer of diskover no long supports v1 and the V2 they're planning seems to be a commercial release (plus the version of ES it requires is also end of life).

jrlaberge commented 3 years ago

@j0nnymoe on their github repo they say

diskover v2 community will be returning to GitHub and will be fully open-source.

Would this be something linuxserver will offer? I only recently stumbled on diskover, and looking for a web based tool for monitoring disk usage

j0nnymoe commented 3 years ago

@jrlaberge Maybe, it depends if any of this comes to light, time will only tell.

Dids commented 3 years ago

I was also facing this issue. I managed to get it running by doing the following:

docker exec -it diskover bash
pip3 install 'click==7.1.2' --force-reinstall
exit
docker restart diskover

~Of course the long-term fix is adding click==7.1.2 to requirements.txt or correcting the data being passed to click.~

Actually upon a closer look, upgrading rq in the requirements.txt file to point to >=1.8.1

rq/rq#1469

Installing click==7.1.2 seems to have worked, although I don't yet have access to the web interface, but this could simply be from the initial delay (from what I hear anyway). Adding click==7.1.2 to requirements.txt is an even better fix.

I also tried upgrading rq to 1.8.1 (as well as 1.90), but this fails as the redis requirement doesn't match.

Even if this container is getting deprecated/archived, it would be great if this dependency issue could be fixed, as there's no telling when or if a new version of diskover (v2 or otherwise) will actually happen.

jrlaberge commented 3 years ago

@Dids verify that elasticsearch is running correctly.

Did you run the sysctl command to increase memory? Look at the logs for diskover to verify what's happening.

If you're on Windows:


wsl -d docker-desktop
sysctl -w vm.max_map_count=262144
Dids commented 3 years ago

I did and I have more than one elasticsearch container running on the same system, just using different ports.

I was able to get the diskover web interface to work now, and it seems to be able to connect to elasticsearch as it asks for the indices, but it doesn't seem to be able to see any files/folder. Not sure if this is simply due to how large the folder I mounted is, or if this is something else. Nothing in the logs either.

EDIT: I deleted the previously created elasticsearch index, restarted diskover and now it seems to be crawling again, so time will tell if it works this time around! :)

github-actions[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.