e-alfred / ocdownloader

ocDownloader - AGPL-licensed multi-protocol download manager for Nextcloud using ARIA2, youtube-dl and Curl (supports Youtube, BitTorrent, HTTP, FTP)
https://github.com/e-alfred/ocdownloader
GNU Affero General Public License v3.0
375 stars 85 forks source link

ocDownloader itself cannot recognize large torrent files #101

Closed huming2207 closed 1 year ago

huming2207 commented 6 years ago

I was planning to download some ancient Chinese textbooks (old stuff for 100+ years old, copyright-free for sure) from a private tracker. So far all good, except those torrent files are huge (3.5MB+) since there are a lot of books in that torrent (500GB+). I've found that when I uploaded the torrent file then create the task, the ocDownloader will show me "Returned GID is null ! Is Aria2c running as a daemon ?" but actually it will start to download that torrent later after a few minutes. Besides, the download task is not visible in the ocDownloader.

Steps to reproduce

  1. Pick a torrent file that larger than 3MB, and the download content is more than 500GB
  2. Upload it to ocDownloader
  3. Create the download task

Expected behaviour

Create the task successfully and download the books.

Actual behaviour

  1. Aria2 is running and downloading the task normally
  2. ocDownloader shows an error of "Returned GID is null ! Is Aria2c running as a daemon ?"
  3. The task is invisible on the ocDownloader GUI
  4. Other smaller tasks with a smaller size of book files and smaller torrents work just fine as expected.

Server configuration

Operating system: Ubuntu 18.04.1

Web server: Apache 2.4.34 (In the snap package)

Database: MySQL 5.7.22 (In the snap package)

PHP version: 7.1.22 (In the snap package)

Nextcloud version: 13.0.6 (In the snap package)

Updated from an older Nextcloud/ownCloud or fresh install: No

Where did you install Nextcloud from: Ubuntu snap package

Signing status:

Signing status ``` Login as admin user into your Nextcloud and access http://example.com/index.php/settings/integrity/failed paste the results here. ```
No errors have been found.

List of activated apps:

App list ``` If you have access to your command line run e.g.: sudo -u www-data php occ app:list from within your Nextcloud installation folder ```
Enabled:
  - activity: 2.6.1
  - comments: 1.3.0
  - dav: 1.4.7
  - federatedfilesharing: 1.3.1
  - federation: 1.3.0
  - files: 1.8.0
  - files_sharing: 1.5.0
  - files_texteditor: 2.5.1
  - files_trashbin: 1.3.0
  - files_versions: 1.6.0
  - files_videoplayer: 1.2.0
  - firstrunwizard: 2.2.1
  - gallery: 18.0.0
  - logreader: 2.0.0
  - lookup_server_connector: 1.1.0
  - nextcloud_announcements: 1.2.0
  - notifications: 2.1.2
  - oauth2: 1.1.1
  - ocdownloader: 1.5.6
  - password_policy: 1.3.0
  - provisioning_api: 1.3.0
  - serverinfo: 1.3.0
  - sharebymail: 1.3.0
  - survey_client: 1.1.0
  - systemtags: 1.3.0
  - theming: 1.4.5
  - twofactor_backupcodes: 1.2.3
  - workflowengine: 1.3.0
Disabled:
  - admin_audit
  - encryption
  - files_external
  - files_pdfviewer
  - user_external
  - user_ldap

Nextcloud configuration:

Config report ``` If you have access to your command line run e.g.: sudo -u www-data php occ config:list system from within your Nextcloud installation folder or Insert your config.php content here. Make sure to remove all sensitive content such as passwords. (e.g. database password, passwordsalt, secret, smtp password, …) ```
{
    "system": {
        "apps_paths": [
            {
                "path": "\/snap\/nextcloud\/current\/htdocs\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/snap\/nextcloud\/current\/nextcloud\/extra-apps",
                "url": "\/extra-apps",
                "writable": true
            }
        ],
        "supportedDatabases": [
            "mysql"
        ],
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "memcache.local": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0
        },
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            ***REMOVED SENSITIVE VALUE***
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "13.0.6.1",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true
    }
}

Are you using external storage, if yes which one (currently not supported by ocDownloader): Yes, I enabled the "Removeable media"function for Nextcloud snap package, i.e. mount a storage partition to /media and let the data store in that partition.

Are you using encryption (currently not supported by ocDownloader): No

Are you using an external user-backend, if yes which one: No

ocDownloader configuration:

Which downloader are you using: Aria2

Do you use a proxy: No

Which protocols do you allow: Bittorrent

Aria2c configuration (remove if not used):

disable-ipv6=false
max-concurrent-downloads=35
continue=true
max-connection-per-server=15
min-split-size=10M
split=20
max-overall-download-limit=10M
max-download-limit=10M
max-overall-upload-limit=0
max-upload-limit=0
dir=/var/downloads

ftp-pasv=true

input-file=/var/lib/aria2/aria2.session
save-session=/var/lib/aria2/aria2.session
save-session-interval=3
force-save=true
continue=true
disk-cache=512M

enable-dht=false
enable-dht6=false
bt-enable-lpd=false
enable-peer-exchange=false
#bt-max-peers=55
bt-require-crypto=false
bt-min-crypto-level=arc4
follow-torrent=true
listen-port=30443
peer-id-prefix=-TR2940-
user-agent=Transmission/2.94
bt-seed-unverified=true

enable-rpc=true
rpc-allow-origin-all=true
rpc-listen-all=false
rpc-listen-port=6800
Aria2c error log

I tried enable the log function before but there is nothing suspicous.

Client configuration

Browser: Chrome 69 Stable

Operating system: Windows 10 1809/Ubuntu 18.04.1

Logs

Web server error log

Web server error log ``` [Fri Oct 05 07:29:36.415414 2018] [mpm_event:notice] [pid 8762:tid 139704891189120] AH00489: Apache/2.4.34 (Unix) configured -- resuming normal operations [Fri Oct 05 07:29:36.416027 2018] [core:notice] [pid 8762:tid 139704891189120] AH00094: Command line: 'httpd -d /snap/nextcloud/8971 -D FOREGROUND' [Fri Oct 05 07:29:36.421762 2018] [unixd:alert] [pid 8765:tid 139704891189120] AH02155: getpwuid: couldn't determine user name from uid 4294967295, you probably need to modify the User directive [Fri Oct 05 07:29:36.422053 2018] [unixd:alert] [pid 8766:tid 139704891189120] AH02155: getpwuid: couldn't determine user name from uid 4294967295, you probably need to modify the User directive [Fri Oct 05 07:29:36.422053 2018] [unixd:alert] [pid 8764:tid 139704891189120] AH02155: getpwuid: couldn't determine user name from uid 4294967295, you probably need to modify the User directive [Fri Oct 05 07:33:02.893130 2018] [mpm_event:notice] [pid 8762:tid 139704891189120] AH00491: caught SIGTERM, shutting down [Fri Oct 05 07:33:08.054976 2018] [ssl:warn] [pid 9823:tid 139623598724992] AH01909: ***SENSITIVE_CONTENT***:443:0 server certificate does NOT include an ID which matches the server name [Fri Oct 05 07:33:08.087723 2018] [ssl:warn] [pid 9823:tid 139623598724992] AH01909: ***SENSITIVE_CONTENT***:443:0 server certificate does NOT include an ID which matches the server name [Fri Oct 05 07:33:08.090962 2018] [mpm_event:notice] [pid 9823:tid 139623598724992] AH00489: Apache/2.4.34 (Unix) OpenSSL/1.0.2g configured -- resuming normal operations [Fri Oct 05 07:33:08.091107 2018] [core:notice] [pid 9823:tid 139623598724992] AH00094: Command line: 'httpd -d /snap/nextcloud/8971 -D EnableHTTPS -D EnableHSTS -D FOREGROUND' [Fri Oct 05 07:33:08.093675 2018] [unixd:alert] [pid 9824:tid 139623598724992] AH02155: getpwuid: couldn't determine user name from uid 4294967295, you probably need to modify the User directive [Fri Oct 05 07:33:08.094044 2018] [unixd:alert] [pid 9826:tid 139623598724992] AH02155: getpwuid: couldn't determine user name from uid 4294967295, you probably need to modify the User directive [Fri Oct 05 07:33:08.094377 2018] [unixd:alert] [pid 9825:tid 139623598724992] AH02155: getpwuid: couldn't determine user name from uid 4294967295, you probably need to modify the User directive [Fri Oct 05 07:33:31.122428 2018] [unixd:alert] [pid 9995:tid 139623598724992] AH02155: getpwuid: couldn't determine user name from uid 4294967295, you probably need to modify the User directive [Fri Oct 05 07:35:57.576345 2018] [mpm_event:notice] [pid 9823:tid 139623598724992] AH00491: caught SIGTERM, shutting down [Fri Oct 05 07:38:06.592639 2018] [ssl:warn] [pid 11436:tid 140126051395456] AH01909: ***SENSITIVE_CONTENT***:443:0 server certificate does NOT include an ID which matches the server name [Fri Oct 05 07:38:06.620727 2018] [ssl:warn] [pid 11436:tid 140126051395456] AH01909: ***SENSITIVE_CONTENT***:443:0 server certificate does NOT include an ID which matches the server name [Fri Oct 05 07:38:06.624187 2018] [mpm_event:notice] [pid 11436:tid 140126051395456] AH00489: Apache/2.4.34 (Unix) OpenSSL/1.0.2g configured -- resuming normal operations [Fri Oct 05 07:38:06.624324 2018] [core:notice] [pid 11436:tid 140126051395456] AH00094: Command line: 'httpd -d /snap/nextcloud/8971 -D EnableHTTPS -D EnableHSTS -D FOREGROUND' [Fri Oct 05 07:38:06.626744 2018] [unixd:alert] [pid 11437:tid 140126051395456] AH02155: getpwuid: couldn't determine user name from uid 4294967295, you probably need to modify the User directive [Fri Oct 05 07:38:06.626886 2018] [unixd:alert] [pid 11439:tid 140126051395456] AH02155: getpwuid: couldn't determine user name from uid 4294967295, you probably need to modify the User directive [Fri Oct 05 07:38:06.627478 2018] [unixd:alert] [pid 11438:tid 140126051395456] AH02155: getpwuid: couldn't determine user name from uid 4294967295, you probably need to modify the User directive [Fri Oct 05 07:38:38.662982 2018] [unixd:alert] [pid 11554:tid 140126051395456] AH02155: getpwuid: couldn't determine user name from uid 4294967295, you probably need to modify the User directive [Tue Oct 09 07:41:34.296180 2018] [unixd:alert] [pid 13664:tid 140126051395456] AH02155: getpwuid: couldn't determine user name from uid 4294967295, you probably need to modify the User directive ```

Nextcloud log (data/nextcloud.log)

Nextcloud log Too long to upload, but there is something suspecious for you: ``` {"reqId":"2O3l0v6zntjHYmyDg7fi","level":3,"time":"2018-10-07T12:31:28+00:00","remoteAddr":"SENSITIVE_CONTENT","user":"SENSITIVE_CONTENT","app":"PHP","method":"POST","url":"\/index.php\/apps\/ocdownloader\/btdownloader\/add","message":"Division by zero at \/var\/snap\/nextcloud\/8971\/nextcloud\/extra-apps\/ocdownloader\/controller\/btdownloader.php#186","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/69.0.3497.100 Safari\/537.36","version":"13.0.6.1"} {"reqId":"Tykxr6uvShuR2rZliGcL","level":3,"time":"2018-10-07T12:31:59+00:00","remoteAddr":"SENSITIVE_CONTENT","user":"SENSITIVE_CONTENT","app":"PHP","method":"POST","url":"\/index.php\/apps\/ocdownloader\/btdownloader\/add","message":"Division by zero at \/var\/snap\/nextcloud\/8971\/nextcloud\/extra-apps\/ocdownloader\/controller\/btdownloader.php#186","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/69.0.3497.100 Safari\/537.36","version":"13.0.6.1"} {"reqId":"UYkEdUwbXbBfoETAYKDe","level":3,"time":"2018-10-07T12:32:22+00:00","remoteAddr":"SENSITIVE_CONTENT","user":"SENSITIVE_CONTENT","app":"PHP","method":"POST","url":"\/index.php\/apps\/ocdownloader\/btdownloader\/add","message":"Division by zero at \/var\/snap\/nextcloud\/8971\/nextcloud\/extra-apps\/ocdownloader\/controller\/btdownloader.php#186","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/69.0.3497.100 Safari\/537.36","version":"13.0.6.1"} ```

Browser log

Browser log ``` No log available ```
monsieurhannes commented 4 years ago

I also get the error in the nextcloud log: Division by zero at /data/nextcloud/apps/ocdownloader/controller/btdownloader.php#186