task remains on Pending

[ghost]

Describe the bug The processes cannot be executed, the status of the task remains on Pending.

To Reproduce Steps to reproduce the behavior:

1. Activate cloud_py_api app
2. activate MediaDC app
3. create rule to delete in Media DC app
4. wait
5. the status of the task remains on Pending

Desktop (please complete the following information):

• OS [Ubuntu]
• CPU [architecture x86]
• Nextcloud version [25]
• Database configuration [postgres]
• MediaDC version [0.3.2]
• cloud_py_api version [0.1.1]

Nextcloud AIO latest Stable

Additional context https://github.com/nextcloud/all-in-one/discussions/1701

[andrey18106]

@R7JANV1 Thanks for reporting, possibly duplicate of https://github.com/andrey18106/mediadc/issues/100.

Checkout wiki notice and try manual installation instead of pre-compiled binaries (seems like some issue with permission).

If you have any new info about that bug, feel free to write here.

[zainvegeta]

Facing the same issue with following log ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/tmp/onefile_7215_1673465229_464283/nc_py_api/occ.py", line 54, in php_call File "/tmp/onefile_7215_1673465229_464283/subprocess.py", line 528, in run subprocess.CalledProcessError: Command '['php', '/var/www/nextcloud/occ', '--no-warnings', 'config:system:get', 'dbname']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbname} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/tmp/onefile_7215_1673465229_464283/nc_py_api/occ.py", line 54, in php_call File "/tmp/onefile_7215_1673465229_464283/subprocess.py", line 528, in run subprocess.CalledProcessError: Command '['php', '/var/www/nextcloud/occ', '--no-warnings', 'config:system:get', 'dbuser']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbuser} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/tmp/onefile_7215_1673465229_464283/nc_py_api/occ.py", line 54, in php_call File "/tmp/onefile_7215_1673465229_464283/subprocess.py", line 528, in run subprocess.CalledProcessError: Command '['php', '/var/www/nextcloud/occ', '--no-warnings', 'config:system:get', 'dbpassword']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbpassword} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/tmp/onefile_7215_1673465229_464283/nc_py_api/occ.py", line 54, in php_call File "/tmp/onefile_7215_1673465229_464283/subprocess.py", line 528, in run subprocess.CalledProcessError: Command '['php', '/var/www/nextcloud/occ', '--no-warnings', 'config:system:get', 'dbhost']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbhost} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/tmp/onefile_7215_1673465229_464283/nc_py_api/occ.py", line 54, in php_call File "/tmp/onefile_7215_1673465229_464283/subprocess.py", line 528, in run subprocess.CalledProcessError: Command '['php', '/var/www/nextcloud/occ', '--no-warnings', 'config:system:get', 'dbtype']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbtype} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/tmp/onefile_7215_1673465229_464283/nc_py_api/occ.py", line 54, in php_call File "/tmp/onefile_7215_1673465229_464283/subprocess.py", line 528, in run subprocess.CalledProcessError: Command '['php', '/var/www/nextcloud/occ', '--no-warnings', 'config:system:get', 'datadirectory']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {datadirectory} value in NC config. INFO:nc_py_api.mediadc:bundle_info:bundle_info:Python: 3.9.16 (main, Dec 19 2022, 10:54:04) [GCC 10.2.1 20210130 (Red Hat 10.2.1-11)] INFO:nc_py_api.mediadc:bundle_info:bundle_info:nc_py_api: 0.0.8 INFO:nc_py_api.mediadc:bundle_info:bundle_info:mediadc: 0.3.0 INFO:nc_py_api.mediadc:bundle_info:bundle_info:pg8000: 1.29.4 INFO:nc_py_api.mediadc:bundle_info:bundle_info:pynacl: 1.5.0 INFO:nc_py_api.mediadc:bundle_info:bundle_info:cryptography: 38.0.4 INFO:nc_py_api.mediadc:bundle_info:bundle_info:pymysql: 1.0.2 INFO:nc_py_api.mediadc:bundle_info:bundle_info:pillow: 9.3.0 INFO:nc_py_api.mediadc:bundle_info:bundle_info:pi_heif: 0.9.0 INFO:nc_py_api.mediadc:bundle_info:bundle_info:numpy: 1.21.4 INFO:nc_py_api.mediadc:bundle_info:bundle_info:scipy: 1.7.2 INFO:nc_py_api.mediadc:bundle_info:bundle_info:pywavelets: 1.4.1

Tried manual installation instead of pre-compiled binaries still same error

[timonline]

I am seeing the same issue

root@c4df08b5fb48:/# sudo -H python3 -m pip install -r /config/www/nextcloud/apps/mediadc/requirements.txt 
Requirement already satisfied: pynacl in /usr/lib/python3.9/site-packages (from -r /config/www/nextcloud/apps/mediadc/requirements.txt (line 1)) (1.5.0)
Requirement already satisfied: numpy in /usr/lib/python3.9/site-packages (from -r /config/www/nextcloud/apps/mediadc/requirements.txt (line 2)) (1.24.1)
Requirement already satisfied: scipy in /usr/lib/python3.9/site-packages (from -r /config/www/nextcloud/apps/mediadc/requirements.txt (line 3)) (1.10.0)
Requirement already satisfied: pywavelets in /usr/lib/python3.9/site-packages (from -r /config/www/nextcloud/apps/mediadc/requirements.txt (line 4)) (1.4.1)
Requirement already satisfied: pillow in /usr/lib/python3.9/site-packages (from -r /config/www/nextcloud/apps/mediadc/requirements.txt (line 5)) (9.4.0)
Requirement already satisfied: hexhamming in /usr/lib/python3.9/site-packages (from -r /config/www/nextcloud/apps/mediadc/requirements.txt (line 6)) (2.2.3)
Requirement already satisfied: nc-py-api>=0.0.8 in /usr/lib/python3.9/site-packages (from -r /config/www/nextcloud/apps/mediadc/requirements.txt (line 7)) (0.0.9)
Requirement already satisfied: pi-heif>=0.9.0 in /usr/lib/python3.9/site-packages (from -r /config/www/nextcloud/apps/mediadc/requirements.txt (line 8)) (0.9.1)
Requirement already satisfied: cffi>=1.4.1 in /usr/lib/python3.9/site-packages (from pynacl->-r /config/www/nextcloud/apps/mediadc/requirements.txt (line 1)) (1.15.1)
Requirement already satisfied: pymysql in /usr/lib/python3.9/site-packages (from nc-py-api>=0.0.8->-r /config/www/nextcloud/apps/mediadc/requirements.txt (line 7)) (1.0.2)
Requirement already satisfied: cryptography in /usr/lib/python3.9/site-packages (from nc-py-api>=0.0.8->-r /config/www/nextcloud/apps/mediadc/requirements.txt (line 7)) (39.0.0)
Requirement already satisfied: pg8000 in /usr/lib/python3.9/site-packages (from nc-py-api>=0.0.8->-r /config/www/nextcloud/apps/mediadc/requirements.txt (line 7)) (1.29.4)
Requirement already satisfied: pycparser in /usr/lib/python3.9/site-packages (from cffi>=1.4.1->pynacl->-r /config/www/nextcloud/apps/mediadc/requirements.txt (line 1)) (2.21)
Requirement already satisfied: scramp>=1.4.3 in /usr/lib/python3.9/site-packages (from pg8000->nc-py-api>=0.0.8->-r /config/www/nextcloud/apps/mediadc/requirements.txt (line 7)) (1.4.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/lib/python3.9/site-packages (from pg8000->nc-py-api>=0.0.8->-r /config/www/nextcloud/apps/mediadc/requirements.txt (line 7)) (2.8.2)
Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil>=2.8.2->pg8000->nc-py-api>=0.0.8->-r /config/www/nextcloud/apps/mediadc/requirements.txt (line 7)) (1.16.0)
Requirement already satisfied: asn1crypto>=1.5.1 in /usr/lib/python3.9/site-packages (from scramp>=1.4.3->pg8000->nc-py-api>=0.0.8->-r /config/www/nextcloud/apps/mediadc/requirements.txt (line 7)) (1.5.1)

Yet the log shows this:

tail -f /config/www/nextcloud/data/appdata_oceni1or87cb/mediadc/logs/11-01-2023_21\:47\:44.log 
nohup: failed to run command '/config/www/nextcloud/data/appdata_oceni1or87cb/mediadc/binaries/main': No such file or directory

root@c4df08b5fb48:/config/www/nextcloud/apps/mediadc# ls -l /config/www/nextcloud/data/appdata_oceni1or87cb/mediadc/binaries/main
-rwxrwxrwx 1 abc abc 233389410 Jan 11 15:33 /config/www/nextcloud/data/appdata_oceni1or87cb/mediadc/binaries/main

[bigcat88]

@timonline does this log is from run with unchecked Use pre-compilbed Python binaries box in cloud_py_api?

As I see you installing python dependencies, I assume that you try running from source, am I right?

[timonline]

@timonline does this log is from run with unchecked Use pre-compilbed Python binaries box in cloud_py_api?

As I see you installing python dependencies, I assume that you try running from source, am I right?

Yes I’m using the requirement file from the source and yes the pre compiled python binaries is set

[andrey18106]

Yes I’m using the requirement file from the source and yes the pre compiled python binaries is set

Use pre-compiled Python binaries option must be disabled (unchecked) for usage from sources with manually installed Python packages.

[timonline]

Yes I’m using the requirement file from the source and yes the pre compiled python binaries is set

Use pre-compiled Python binaries option must be disabled (unchecked) for usage from sources with manually installed Python packages.

that did the trick, thank you. Now i am finally seeing the tasks progress instead of being stuck in pending

[bigcat88]

@timonline can you (or someone else with such problem) describe how we can reproduce this, what do specific your installations have which leads to situation where php cannot start binary file - looks like we miss something when rewriting code.

[andrey18106]

@timonline Could you check one thing please:

1. Edit two lines of cloud_py_api app file

change : to - in paths to logs on lines 116 and 121

L116: from $logFile = $cwd . 'logs/' . date('d-m-Y_H:i:s', time()) . '.log'; to $logFile = $cwd . 'logs/' . date('d-m-Y_H-i-s', time()) . '.log';

L121: from $logFile = $appDataDir . 'logs/' . date('d-m-Y_H:i:s', time()) . '.log'; to $logFile = $appDataDir . 'logs/' . date('d-m-Y_H-i-s', time()) . '.log';

2. Enable Use pre-compiled Python binaries in MediaDC Admin settings
3. Restart some task

[timonline]

@andrey18106 the naming format of the file changed but the error came back

root@c4df08b5fb48:/config/www/nextcloud/apps# ls -l /config/www/nextcloud/data/appdata_oceni1or87cb/mediadc/logs/
total 20
-rw-r--r-- 1 abc abc  128 Jan 11 15:34 11-01-2023_21:34:11.log
-rw-r--r-- 1 abc abc  128 Jan 11 15:47 11-01-2023_21:47:44.log
-rw-r--r-- 1 abc abc   49 Jan 11 16:26 11-01-2023_22:26:07.log
-rw-r--r-- 1 abc abc 3075 Jan 11 17:54 11-01-2023_22:27:11.log
-rw-r--r-- 1 abc abc  128 Jan 11 18:17 12-01-2023_00:17-27.log

root@c4df08b5fb48:/config/www/nextcloud/apps# cat /config/www/nextcloud/data/appdata_oceni1or87cb/mediadc/logs/12-01-2023_00:17-27.log
nohup: failed to run command '/config/www/nextcloud/data/appdata_oceni1or87cb/mediadc/binaries/main': No such file or directory

[andrey18106]

@timonline You missed to change one first :, it should be 12-01-2023_00-17-27.log, not 12-01-2023_00:17-27.log. Try again please.

[timonline]

@bigcat88

@timonline can you (or someone else with such problem) describe how we can reproduce this, what do specific your installations have which leads to situation where php cannot start binary file - looks like we miss something when rewriting code.

I mean i upgraded to nextcloud v25, install the cloud python api, installed mediadc, created a task, stayed pending. So i checked the log file and found the error.

Not sure there was anything special i did that reproduces the problem

EDIT: I even tried removing and reinstalling both multiple times

[timonline]

@andrey18106 don't see how i missed one.

root@c4df08b5fb48:/config/www/nextcloud/apps# grep "d-m-Y_H:i:s" ./cloud_py_api/lib/Service/PythonService.php

root@c4df08b5fb48:/config/www/nextcloud/apps# grep "d-m-Y_H:i-s" ./cloud_py_api/lib/Service/PythonService.php
                $logFile = $cwd . 'logs/' . date('d-m-Y_H:i-s', time()) . '.log';
                $logFile = $appDataDir . 'logs/' . date('d-m-Y_H:i-s', time()) . '.log';

[andrey18106]

@timonline in both date('d-m-Y_H:i-s', time()) colon : symbol to -, should be date('d-m-Y_H-i-s', time())

[bigcat88]

Can you describe what OS you are using, what type of Nextcloud installation you have, maybe some specific steps were followed to install php. I don't understand why it can't run the command when the binary is present and has execute permissions.

[timonline]

to either @andrey18106 or @bigcat88 I am happy to jump on discord or some sort of chat to troubleshoot the problem live. Would be happy to find the exact circumstances that leads to a broken install as it took me a whole day of messing with pip and installing the prerequisites to get it to work.

There were MULTIPLE linux packages missing that I had to install to get the pip install -r requirements.txt to work because it kept erroring

[timonline]

@andrey18106

@timonline in both date('d-m-Y_H:i-s', time()) colon : symbol to -, should be date('d-m-Y_H-i-s', time())

root@c4df08b5fb48:/config/www/nextcloud/apps# cat /config/www/nextcloud/data/appdata_oceni1or87cb/mediadc/logs/12-01-2023_00-31-43.log 
nohup: failed to run command '/config/www/nextcloud/data/appdata_oceni1or87cb/mediadc/binaries/main': No such file or directory

root@c4df08b5fb48:/config/www/nextcloud/apps# ls -ltr /config/www/nextcloud/data/appdata_oceni1or87cb/mediadc/logs/
total 28
-rw-r--r-- 1 abc abc  128 Jan 11 15:34 11-01-2023_21:34:11.log
-rw-r--r-- 1 abc abc  128 Jan 11 15:47 11-01-2023_21:47:44.log
-rw-r--r-- 1 abc abc   49 Jan 11 16:26 11-01-2023_22:26:07.log
-rw-r--r-- 1 abc abc 3075 Jan 11 17:54 11-01-2023_22:27:11.log
-rw-r--r-- 1 abc abc  128 Jan 11 18:17 12-01-2023_00:17-27.log
-rw-r--r-- 1 abc abc  128 Jan 11 18:31 12-01-2023_00-31-08.log
-rw-r--r-- 1 abc abc  128 Jan 11 18:31 12-01-2023_00-31-43.log

[timonline]

@bigcat88

Can you describe what OS you are using, what type of Nextcloud installation you have, maybe some specific steps were followed to install php. I don't understand why it can't run the command when the binary is present and has execute permissions.

Nextcloud is a docker container from https://github.com/linuxserver/docker-nextcloud running on UnRaid whose base is

baseimage-alpine-nginx:3.15

Inside the docker when i run uname, i get this:

root@c4df08b5fb48:/config/www/nextcloud/apps# uname -r
5.19.17-Unraid

[andrey18106]

@timonline ok, for your case, colon in logs paths not the issue, but in other cases it can be the reason. So, yea, it needs more investigation, discord is a good option.

[timonline]

@timonline ok, for your case, colon in logs paths not the issue, but in other cases it can be the reason. So, yea, it needs more investigation, discord is a good option.

I am at PhoenixCoder#4861 on discord

[ghost]

@timonline ok, for your case, colon in logs paths not the issue, but in other cases it can be the reason. So, yea, it needs more investigation, discord is a good option.

is there any progress on this?

[bigcat88]

With what guide I can install Nextcloud AIO to get the same configuration that you have? For all other reported cases we fixed this, it is the last case i think

[ghost]

With what guide I can install Nextcloud AIO to get the same configuration that you have? For all other reported cases we fixed this, it is the last case i think

# For x64 CPUs and without reverse proxy already in place:
sudo docker run \
--sig-proxy=false \
--name nextcloud-aio-mastercontainer \
--restart always \
--publish 80:80 \
--publish 8080:8080 \
--publish 8443:8443 \
--volume nextcloud_aio_mastercontainer:/mnt/docker-aio-config \
--volume /var/run/docker.sock:/var/run/docker.sock:ro \
nextcloud/all-in-one:latest

then activate cloud_py_api and madiadc

[teemue]

For me tasks seem to remain pending until i restart them

[ghost]

For me tasks seem to remain pending until i restart them

unfortunately not for me

[andrey18106]

@R7JANV1 Can you verify any errors in logs. Tasks run logs can be found here: /path/to/nextcloud/data/appdata_ncinstanceid/mediadc/logs/.

Have you tried usage from sources instead of pre-compiled binaries.

[uribenshu]

@andrey18106

I think the problem with linuxserver docker image is that the python and php binaries location is different than the default one in mediaDC settings: (I'm not sure !)

[bigcat88]

Update cloud_py_api, uninstall mediadc, after that remove /path/to/nextcloud/data/appdata_ncinstanceid/mediadc folder, and install last mediadc. Maybe it will work after that, need info about that.

[dakeeper]

any update for this? i still have the "pending" bug.