search

janis91 / ocr

Nextcloud OCR (optical character recoginition) processing for images with tesseract-js
GNU Affero General Public License v3.0
108 stars 17 forks source link

Docker: Error listen tcp 0.0.0.0:6379: bind: address already in use. #154

Closed WNYmathGuy closed 5 years ago

WNYmathGuy commented 6 years ago

Steps to reproduce

  1. Follow instructions on https://github.com/janis91/ocr/wiki/Installation-and-Configuration-(Docker-setup)
  2. Get to this section https://github.com/janis91/ocr/wiki/Installation-and-Configuration-(Docker-setup)#run-and-configure-the-docker-containers
  3. Fail while trying this docker run --name redis --network=isolated_ocr -p 6379:6379 -d myUserID/redis
  4. See error message docker: Error response from daemon: driver failed programming external connectivity on endpoint redis (giant-number): Error starting userland proxy: listen tcp 0.0.0.0:6379: bind: address already in use.

Expected behavior

A redis container should run to handle OCR stuff.

Actual behavior

It doesn't run and OCR is not functional.

Perhaps important details

I manage my server with Virtualmin. NextCloud runs in a subdomain of a virtual server. I also run a Moodle School on the same Ubuntu Server but on an entirely unrelated virtual server. I believe Moodle uses Redis too. I see redis-server and redis-server.service in my list of running Systemd services, both are running and restart at boot time.

I'm kind of an idiot, so I may have done things wrong while struggling to get this part of NextCloud to work. I searched for an error message OCR: 1 currently pending file in queue and hoped the post I read would help me solve the problem. Like I was reading this https://github.com/janis91/ocr/issues/35 and thought I should do this

Installation steps
sudo apt-get install python3-pip
sudo pip3 install --upgrade pip
sudo apt-get install libffi-dev
sudo pip3 install ocrmypdf
sudo apt-get install tesseract-ocr tesseract-ocr-deu
sudo apt-get install tesseract-ocr-spa
sudo apt-get install tesseract-ocr-por
sudo apt-get install tesseract-ocr-ndl
sudo apt-get install tesseract-ocr-ita
sudo apt-get install tesseract-ocr-fra
sudo apt-get install tesseract-ocr-eng
sudo apt-get install tesseract-ocr-deu-frak

because I didn't have any of that software installed on my server while I was looking for the reason I was having the infernal "OCR: 1 currently pending file in queue" error. Also, this recent time I noticed I didn't have the oc_ocr_status table in my database but used to have it. I tried disabling then removing the OCR app on my admin login, then reinstalling it and following the "Installation and Configuration (Docker setup)" instructions.

As an aside, I found the password selection/implementation part of https://github.com/janis91/ocr/wiki/Installation-and-Configuration-(Docker-setup)#build-the-docker-image very confusing. I may have guessed wrong, but I altered the password in ~/nextcloud/apps/ocr/redis/Dockerfile. from OCR to something else, then running docker build --build-arg PASSWORD="OCR" -t myUserID/redis . seemed to set my new password to something else during the install process.

But while finalizing https://github.com/janis91/ocr/wiki/Installation-and-Configuration-(Docker-setup)#configure-the-app no password would work without throwing a OCR: "Redis authentication error." error for NextCloud users.

Server configuration detail

Operating system: Linux 4.4.0-119-generic #143-Ubuntu SMP Mon Apr 2 16:08:24 UTC 2018 x86_64

Webserver: Apache/2.4.18 (cgi-fcgi)

Database: mysql 10.0.34 (actually MariaDB but whatever)

PHP version: 7.0.28-0ubuntu0.16.04.1 Modules loaded: Core, date, libxml, openssl, pcre, zlib, filter, hash, pcntl, Reflection, SPL, session, standard, cgi-fcgi, mysqlnd, PDO, xml, apcu, calendar, ctype, curl, dom, mbstring, fileinfo, ftp, gd, gettext, gmp, iconv, igbinary, imagick, imap, intl, json, ldap, exif, mcrypt, mysqli, pdo_mysql, pdo_pgsql, pdo_sqlite, pgsql, Phar, posix, pspell, readline, redis, shmop, SimpleXML, soap, sockets, sqlite3, sysvmsg, sysvsem, sysvshm, tokenizer, wddx, xmlreader, xmlrpc, xmlwriter, xsl, zip, Zend OPcache

Nextcloud version: 13.0.1 - 13.0.1.1

Updated from an older Nextcloud/ownCloud or fresh install:

Where did you install Nextcloud from: https://docs.nextcloud.com/server/13/admin_manual/installation/source_installation.html#ubuntu-installation-label

Signing status Array
List of activated apps ``` Enabled: - activity: 2.6.1 - admin_notifications: 1.0.1 - announcementcenter: 3.2.1 - apporder: 0.4.1 - audioplayer: 2.2.5 - bookmarks: 0.11.0 - bruteforcesettings: 1.0.3 - calendar: 1.6.1 - checksum: 0.3.5 - circles: 0.13.6 - comments: 1.3.0 - contacts: 2.1.3 - dashboard: 4.0.6 - dav: 1.4.6 - deck: 0.3.1 - drawio: 0.8.9 - federatedfilesharing: 1.3.1 - federation: 1.3.0 - files: 1.8.0 - files_downloadactivity: 1.2.0 - files_pdfviewer: 1.2.1 - files_reader: 1.2.3 - files_retention: 1.2.0 - files_sharing: 1.5.0 - files_texteditor: 2.5.1 - files_trashbin: 1.3.0 - files_versions: 1.6.0 - files_videoplayer: 1.2.0 - firstrunwizard: 2.2.1 - flowupload: 0.0.7 - gallery: 18.0.0 - gpxedit: 0.0.8 - gpxmotion: 0.0.6 - gpxpod: 2.2.2 - groupfolders: 1.2.0 - issuetemplate: 0.3.0 - logreader: 2.0.0 - lookup_server_connector: 1.1.0 - mail: 0.7.10 - metadata: 0.6.0 - news: 12.0.1 - nextcloud_announcements: 1.2.0 - notes: 2.3.2 - notifications: 2.1.2 - oauth2: 1.1.0 - ocr: 3.3.6 - ocsms: 1.12.2 - password_policy: 1.3.0 - phonetrack: 0.2.7 - polls: 0.8.1 - provisioning_api: 1.3.0 - quota_warning: 1.2.0 - radio: 0.6.1 - rainloop: 5.0.6 - ransomware_protection: 1.1.0 - richdocuments: 2.0.5 - serverinfo: 1.3.0 - sharebymail: 1.3.0 - socialsharing_diaspora: 1.0.2 - socialsharing_email: 1.0.3 - socialsharing_facebook: 1.0.2 - socialsharing_googleplus: 1.0.2 - socialsharing_twitter: 1.0.2 - spreed: 3.1.0 - survey_client: 1.1.0 - systemtags: 1.3.0 - tasks: 0.9.6 - telephoneprovider: 1.0.1 - theming: 1.4.1 - twofactor_backupcodes: 1.2.3 - updatenotification: 1.3.0 - weather: 1.5.1 - workflowengine: 1.3.0 Disabled: - admin_audit - encryption - files_external - orcid - user_external - user_ldap - user_saml - zenodo ```
Configuration (config/config.php) ``` { "instanceid": "***REMOVED SENSITIVE VALUE***", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "***REMOVED SENSITIVE VALUE***" ], "datadirectory": "***REMOVED SENSITIVE VALUE***", "overwrite.cli.url": "https:\/\/***REMOVED SENSITIVE VALUE***", "dbtype": "mysql", "version": "13.0.1.1", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbport": "", "dbtableprefix": "oc_", "mysql.utf8mb4": true, "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "installed": true, "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_smtpmode": "smtp", "mail_smtpauthtype": "LOGIN", "mail_domain": "***REMOVED SENSITIVE VALUE***", "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "mail_smtpport": "587", "mail_smtpauth": 1, "mail_smtpname": "***REMOVED SENSITIVE VALUE***", "mail_smtppassword": "***REMOVED SENSITIVE VALUE***", "maintenance": false, "memcache.local": "\\OC\\Memcache\\APCu", "theme": "", "updater.secret": "***REMOVED SENSITIVE VALUE***", "loglevel": 2 } ```

Are you using encryption: no, well not for OCR, but my server uses Let's Encrypt.

Happyfeet01 commented 6 years ago

The redis port is already in use. Look if have redis already running. 

netstat - tulpn | grep - v 6379
Error starting userland proxy: listen tcp 0.0.0.0:6379: bind: address already in use.
WNYmathGuy commented 6 years ago

@Happyfeet01 I'm not sure I can give you a good answer... netstat -tulpn | grep -v 6379 and netstat - tulpn | grep -v 6379 showed no redis or port 6379 but a redis server is still running on an OS level instead of in a docker level. e.g.

LinuxAdminID@hostmane:~$ sudo netstat -pna | grep $6379
tcp        0      0 127.0.0.1:6379          0.0.0.0:*               LISTEN      1705/redis-server 1
LinuxAdminID@hostmane:~$

BTW's as of this post I don't know what the flags -tulpn, -v, or -pna are for. ;)

janis91 commented 6 years ago

I think, like the output of the netstat command shows, the PORT 6379 is already in use by your local redis-server. Therefore you are not able to run the docker container on the same port (expose it on the same port). One possibility is to expose the redis container on a different port. for example 6380. Then you just have to adjust the REDIS PORT in the configuration of the app in nextcloud AND in the docker run command for the ocr worker. That should be basically it.

WNYmathGuy commented 6 years ago

@janis91 THANKS!!! It took me this long to get back around to reading & trying the advice. I looked around at Wikipedia Ports and saw Redis reserved 6379, but nobody claimed the next 10, so I figured taking 6380 was a real safe choice. I ran: docker run --name redis --network=isolated_ocr -p 6380:6379 -d myUserID/redis and got my container number without and error message. I assumed the next command needed this alteration -e "REDIS_PORT=6380". (side note: anything against sticking this --restart always into the two docker run commands?)

So at this point, I successfully completed the Installation and Configuration (Docker setup) recipie.

When trying to do an OCR operation on a business letter (that I scanned as an image to be stored in PDF format with "Simple Scan" on a Ubuntu 17.04 workstation), I see and infinitely long wait on this browser status message OCR: 1 currently pending file in queue. under the URL address bar. Looking inside the MariaDB table oc_ocr_jobs in database nextcloud shows me id='1', status='PENDING', type='1', error_log=NULL, replace='0'. The temp file ocr_8eiG2C exists in the domain's /tmp/ directory as ocr_8eiG2C.pdf, but it's last modification time stays at it's creation time.

Where have I gone wrong? (he asked knowing the infinite possibilities of follow-up questions)

Happyfeet01 commented 6 years ago

It seems to work. But maybe the same problem as #155

WNYmathGuy commented 6 years ago

Infinitely running "OCR: 1 currently pending file in queue." message.

Steps to reproduce

  1. select a file to try OCR on,
  2. activate OCR on the file,
  3. wait forever.

Expected behavior

The selected file should begin processing the OCR operation.

Actual behavior

It doesn't.

Server configuration detail

Operating system: Linux 4.4.0-127-generic #153-Ubuntu SMP Sat May 19 10:58:46 UTC 2018 x86_64

Webserver: Apache/2.4.18 (cgi-fcgi)

Database: mysql 10.0.34

PHP version: 7.0.30-0ubuntu0.16.04.1 Modules loaded: Core, date, libxml, openssl, pcre, zlib, filter, hash, pcntl, Reflection, SPL, session, standard, cgi-fcgi, mysqlnd, PDO, xml, apcu, bz2, calendar, ctype, curl, dom, mbstring, fileinfo, ftp, gd, gettext, gmp, iconv, igbinary, imagick, imap, intl, json, ldap, exif, mcrypt, mysqli, pdo_mysql, pdo_pgsql, pdo_sqlite, pgsql, Phar, posix, pspell, readline, redis, shmop, SimpleXML, soap, sockets, sqlite3, sysvmsg, sysvsem, sysvshm, tokenizer, wddx, xmlreader, xmlrpc, xmlwriter, xsl, zip, Zend OPcache

Nextcloud version: 13.0.2 - 13.0.2.1

Updated from an older Nextcloud/ownCloud or fresh install: Updated

Where did you install Nextcloud from: The NextCloud website using the Ubuntu instructions

Signing status Array
List of activated apps ``` Enabled: - activity: 2.6.1 - admin_notifications: 1.0.1 - announcementcenter: 3.2.1 - apporder: 0.4.1 - audioplayer: 2.3.0 - bookmarks: 0.11.0 - bruteforcesettings: 1.0.3 - calendar: 1.6.1 - checksum: 0.4.0 - circles: 0.13.6 - comments: 1.3.0 - contacts: 2.1.3 - dashboard: 4.0.6 - dav: 1.4.6 - deck: 0.3.1 - drawio: 0.8.9 - federatedfilesharing: 1.3.1 - federation: 1.3.0 - files: 1.8.0 - files_downloadactivity: 1.2.0 - files_pdfviewer: 1.2.1 - files_retention: 1.2.0 - files_sharing: 1.5.0 - files_texteditor: 2.5.1 - files_trashbin: 1.3.0 - files_versions: 1.6.0 - files_videoplayer: 1.2.0 - firstrunwizard: 2.2.1 - flowupload: 0.0.7 - gallery: 18.0.0 - gpxedit: 0.0.8 - gpxmotion: 0.0.6 - gpxpod: 2.2.2 - groupfolders: 1.2.0 - issuetemplate: 0.3.0 - logreader: 2.0.0 - lookup_server_connector: 1.1.0 - metadata: 0.6.0 - news: 12.0.1 - nextcloud_announcements: 1.2.0 - notes: 2.3.2 - notifications: 2.1.2 - oauth2: 1.1.0 - ocr: 3.3.6 - ocsms: 1.12.7 - password_policy: 1.3.0 - phonetrack: 0.2.7 - polls: 0.8.1 - provisioning_api: 1.3.0 - quota_warning: 1.2.0 - radio: 0.6.1 - rainloop: 5.1.0 - ransomware_protection: 1.1.0 - richdocuments: 2.0.8 - serverinfo: 1.3.0 - sharebymail: 1.3.0 - socialsharing_diaspora: 1.0.2 - socialsharing_email: 1.0.3 - socialsharing_facebook: 1.0.2 - socialsharing_googleplus: 1.0.2 - socialsharing_twitter: 1.0.2 - spreed: 3.2.1 - survey_client: 1.1.0 - systemtags: 1.3.0 - tasks: 0.9.6 - telephoneprovider: 1.0.1 - theming: 1.4.5 - twofactor_backupcodes: 1.2.3 - updatenotification: 1.3.0 - weather: 1.5.1 - workflowengine: 1.3.0 Disabled: - admin_audit - encryption - files_external - files_reader - orcid - user_external - user_ldap - user_saml - zenodo ```
Configuration (config/config.php) ``` { "instanceid": "***REMOVED SENSITIVE VALUE***", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "cloud.myexample.com" ], "datadirectory": "***REMOVED SENSITIVE VALUE***", "dbtype": "mysql", "version": "13.0.2.1", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbport": "", "dbtableprefix": "oc_", "mysql.utf8mb4": true, "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "installed": true, "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_smtpmode": "smtp", "mail_smtpauthtype": "LOGIN", "mail_domain": "***REMOVED SENSITIVE VALUE***", "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "mail_smtpport": "587", "mail_smtpauth": 1, "mail_smtpname": "***REMOVED SENSITIVE VALUE***", "mail_smtppassword": "***REMOVED SENSITIVE VALUE***", "maintenance": false, "memcache.local": "\\OC\\Memcache\\APCu", "theme": "", "updater.secret": "***REMOVED SENSITIVE VALUE***", "loglevel": 0, "default_language": "en", "enable_previews": true, "preview_max_x": 2048, "preview_max_y": 2048, "preview_max_scale_factor": 10, "preview_max_filesize_image": 50, "preview_libreoffice_path": "\/usr\/bin\/libreoffice", "preview_office_cl_parameters": " --headless --nologo --nofirststartwizard --invisible --norestore --convert-to pdf --outdir ", "enabledPreviewProviders": [ "OC\\Preview\\PNG", "OC\\Preview\\JPEG", "OC\\Preview\\GIF", "OC\\Preview\\BMP", "OC\\Preview\\XBitmap", "OC\\Preview\\SVG", "OC\\Preview\\TIFF", "OC\\Preview\\MP3", "OC\\Preview\\TXT", "OC\\Preview\\Font", "OC\\Preview\\MarkDown", "OC\\Preview\\Movie", "OC\\Preview\\MSOffice2003", "OC\\Preview\\MSOffice2007", "OC\\Preview\\MSOfficeDoc", "OC\\Preview\\OpenDocument", "OC\\Preview\\PDF", "OC\\Preview\\Postscript", "OC\\Preview\\StarOffice" ], "tempdirectory": "\/home\/ausername\/domains\/cloud.myexample.com\/tmp\/nextcloudtemp", "overwrite.cli.url": "https:\/\/cloud.myexample.com" } ```

Client configuration

Browser: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0

Operating system:

Logs

Nextcloud log [nextcloud.log](https://github.com/janis91/ocr/files/2052308/nextcloud.log)
WNYmathGuy commented 6 years ago

I just discovered the hovering, renewing, error message "OCR: 1 currently pending file in queue." had more in it that wasn't visible. When I triple-clicked it then immediately right-clicked the selected to search it on Duck Duck Go the search had the following full text:

OCR: 1 currently pending file in queue. <MY_SITE_NAME> "Nextcloud" server Files Search All files Recent Favorites Shared with you Shared with others Shared by link Tags Deleted files 21.5 GB of 60 GB used Settings Home OCR Scan New

I have no idea if this helps debug anything, but I'm back to giving up again with this OCR program.

yamenarahman commented 6 years ago

sudo service redis-server stop

janis91 commented 5 years ago

I won't support anything prior version 4, because the whole setup is just too much effort. I will close this issue now and hope that version 4 (which will be shipped in a few weeks probably) will fill your need. If not. please feel free to fork the repo and work on the old code base which will reside in the branch "not-maintained".

kinderasteroid commented 5 months ago

sudo service redis-server stop

Helpful, It worked