Pending

[Niemand112233]

I installed MediaDC on my NC27.0.0. Cloud-API is working.

However, I face the problem that MediaDC is stuck at "pending". it does not start doing it thing.

What's wrong?

[Niemand112233]

it is still pending!

[roydenyates]

Confirmed

PHP Version: 8.2.8

Nextcloud 27.0.1

[Niemand112233]

Still pending and no response from the developer....

[bigcat88]

Doesn't it work for everyone on the 27th cloud?

Still pending and no response from the developer....

developers are in developing unified system to allow easy writing apps like MediaDC, StableDiffusion and other Heavy Apps...

[DerKnerd]

@bigcat88 it is stuck for me in pending. I am using an Object Storage, might that be the isseu?

[bigcat88]

@bigcat88 it is stuck for me in pending. I am using an Object Storage, might that be the isseu?

probably it can be the issue, we did not test it with object storage, in theory it should work, but it is untested...

[roydenyates]

My instance does not use object storage and is eternally pending.

[berti34]

Still pending on my NC 27.0.2 / Debian 12 / PHP 8.2 / standard storage

[DerKnerd]

@bigcat88 I found the issue. During install the binaries are not downloaded properly and that causes the whole process to fail. After downloading the binaries by hand, adding them and then writing a simple shell script to execute it in my k8s pod it works. Here is the script:

#!/usr/bin/bash
export PHP_PATH=/usr/local/bin/php
export SERVER_ROOR=/var/www/html
export IS_SNAP_ENV=false
export USER_ID=imanuel.ulbricht
export LOGLEVEL=DEBUG
export CPA_LOGLEVEL=DEBUG

/data/appdata_INSTANCEID/mediadc/binaries/main -t 5

[dakeeper]

Doesn`t work for me....tons of errors and still pending. Will wait for a working updated version.

[vandman]

Hi, Nextcloud (docker): 27.1.2 Main storage: S3 (minio) For me too, the job is always pending. So the application is not working. Regards Pierre

[DerKnerd]

Try downloading the tar.gz from the releases page and extract it to the appdata folder under mediadc and try again.

[andrey18106]

Hi, Nextcloud (docker): 27.1.2 Main storage: S3 (minio) For me too, the job is always pending. So the application is not working. Regards Pierre

It'll not work with object storage (relies on local paths), we'll fix it as soon as possible.

[dakeeper]

Try downloading the tar.gz from the releases page and extract it to the appdata folder under mediadc and try again.

@DerKnerd can you explain this? where to download the binaries, and where to put them?

[vandman]

Hi, i put it on local storage:

root@f1993474a0cc:/var/www/html# ls -l data/appdata_xxxxx/mediadc
total 155668
drwxr-xr-x 2 www-data www-data     4096 Oct  8 15:01 PIL
-rw-r--r-- 1 www-data www-data    87793 Oct  8 15:01 _bisect.so
-rw-r--r-- 1 www-data www-data   330305 Oct  8 15:01 _blake2.so
-rw-r--r-- 1 www-data www-data   992321 Oct  8 15:01 _cffi_backend.so
-rw-r--r-- 1 www-data www-data   198201 Oct  8 15:01 _codecs_cn.so
-rw-r--r-- 1 www-data www-data   194305 Oct  8 15:01 _codecs_hk.so
-rw-r--r-- 1 www-data www-data   112345 Oct  8 15:01 _codecs_iso2022.so
-rw-r--r-- 1 www-data www-data   329329 Oct  8 15:01 _codecs_jp.so
-rw-r--r-- 1 www-data www-data   181817 Oct  8 15:01 _codecs_kr.so
-rw-r--r-- 1 www-data www-data   144953 Oct  8 15:01 _codecs_tw.so
-rw-r--r-- 1 www-data www-data    29849 Oct  8 15:01 _contextvars.so
-rw-r--r-- 1 www-data www-data    34201 Oct  8 15:01 _crypt.so
-rw-r--r-- 1 www-data www-data   163233 Oct  8 15:01 _csv.so
-rw-r--r-- 1 www-data www-data   717417 Oct  8 15:01 _ctypes.so
-rw-r--r-- 1 www-data www-data   676849 Oct  8 15:01 _datetime.so
-rw-r--r-- 1 www-data www-data  2279121 Oct  8 15:01 _decimal.so
-rw-r--r-- 1 www-data www-data   448393 Oct  8 15:01 _elementtree.so
-rw-r--r-- 1 www-data www-data  3506089 Oct  8 15:01 _hashlib.so
-rw-r--r-- 1 www-data www-data    79289 Oct  8 15:01 _heapq.so
-rw-r--r-- 1 www-data www-data   208745 Oct  8 15:01 _json.so
-rw-r--r-- 1 www-data www-data   213569 Oct  8 15:01 _multibytecodec.so
-rw-r--r-- 1 www-data www-data    80545 Oct  8 15:01 _multiprocessing.so
-rw-r--r-- 1 www-data www-data    34193 Oct  8 15:01 _opcode.so
-rw-r--r-- 1 www-data www-data   192113 Oct  8 15:01 _pi_heif.so
-rw-r--r-- 1 www-data www-data   618665 Oct  8 15:01 _pickle.so
-rw-r--r-- 1 www-data www-data    38521 Oct  8 15:01 _posixshmem.so
-rw-r--r-- 1 www-data www-data    85433 Oct  8 15:01 _posixsubprocess.so
-rw-r--r-- 1 www-data www-data    64001 Oct  8 15:01 _queue.so
-rw-r--r-- 1 www-data www-data    76393 Oct  8 15:01 _random.so
-rw-r--r-- 1 www-data www-data   104377 Oct  8 15:01 _sha512.so
-rw-r--r-- 1 www-data www-data   383169 Oct  8 15:01 _socket.so
-rw-r--r-- 1 www-data www-data  4625361 Oct  8 15:01 _ssl.so
-rw-r--r-- 1 www-data www-data    34081 Oct  8 15:01 _statistics.so
-rw-r--r-- 1 www-data www-data   262593 Oct  8 15:01 _struct.so
-rw-r--r-- 1 www-data www-data   303105 Oct  8 15:01 _tkinter.so
-rw-r--r-- 1 www-data www-data   251033 Oct  8 15:01 array.so
-rw-r--r-- 1 www-data www-data   236049 Oct  8 15:01 audioop.so
-rw-r--r-- 1 www-data www-data   117161 Oct  8 15:01 binascii.so
drwxr-xr-x 3 www-data www-data     4096 Oct  8 15:01 cryptography
-rw-r--r-- 1 www-data www-data    59409 Oct  8 15:01 fcntl.so
-rw-r--r-- 1 www-data www-data    63609 Oct  8 15:01 grp.so
-rw-r--r-- 1 www-data www-data   112113 Oct  8 15:01 hexhamming.so
-rw-r--r-- 1 www-data www-data  1352705 Oct  8 15:01 libX11.so.6
-rw-r--r-- 1 www-data www-data    22577 Oct  8 15:01 libXau-00ec42fe.so.6.0.0
-rw-r--r-- 1 www-data www-data    17513 Oct  8 15:01 libXau.so.6
-rw-r--r-- 1 www-data www-data  1292473 Oct  8 15:01 libde265-35dbe9a5.so
-rw-r--r-- 1 www-data www-data    34769 Oct  8 15:01 libffi.so.6
-rw-r--r-- 1 www-data www-data    94153 Oct  8 15:01 libgcc_s.so.1
-rw-r--r-- 1 www-data www-data  2723513 Oct  8 15:01 libgfortran-040039e1.so.5.0.0
-rw-r--r-- 1 www-data www-data  1161657 Oct  8 15:01 libheif-5c5e17bf.so.1.16.2
-rw-r--r-- 1 www-data www-data   699577 Oct  8 15:01 libjpeg-2c0fa17f.so.62.3.0
-rw-r--r-- 1 www-data www-data   508425 Oct  8 15:01 liblcms2-f8fefe53.so.2.0.14
-rw-r--r-- 1 www-data www-data   273849 Oct  8 15:01 liblzma-85b6360a.so.5.4.0
-rw-r--r-- 1 www-data www-data 33155937 Oct  8 15:01 libopenblas64_p-r0-742d56dc.3.20.so
-rw-r--r-- 1 www-data www-data 33087961 Oct  8 15:01 libopenblasp-r0-41284840.3.18.so
-rw-r--r-- 1 www-data www-data   580209 Oct  8 15:01 libopenjp2-fca9bf24.so.2.5.0
-rw-r--r-- 1 www-data www-data   251481 Oct  8 15:01 libquadmath-96973f99.so.0.0.0
-rw-r--r-- 1 www-data www-data  1792784 Oct  8 15:01 libtcl8.6.so
-rw-r--r-- 1 www-data www-data   697537 Oct  8 15:01 libtiff-ac0c3d92.so.6.0.0
-rw-r--r-- 1 www-data www-data  1450832 Oct  8 15:01 libtk8.6.so
-rw-r--r-- 1 www-data www-data   649689 Oct  8 15:01 libwebp-b246aa5a.so.7.1.5
-rw-r--r-- 1 www-data www-data    34073 Oct  8 15:01 libwebpdemux-ac83b303.so.2.0.11
-rw-r--r-- 1 www-data www-data    59089 Oct  8 15:01 libwebpmux-502c7428.so.3.0.10
-rw-r--r-- 1 www-data www-data   264201 Oct  8 15:01 libxcb-421a6fdb.so.1.1.0
-rw-r--r-- 1 www-data www-data   191705 Oct  8 15:01 libxcb.so.1
-rw-r--r-- 1 www-data www-data    96801 Oct  8 15:01 libz.so.1
-rwxr-xr-x 1 www-data www-data 56855112 Oct  8 15:01 main
-rw-r--r-- 1 www-data www-data   328137 Oct  8 15:01 math.so
-rw-r--r-- 1 www-data www-data   105865 Oct  8 15:01 mmap.so
drwxr-xr-x 2 www-data www-data     4096 Oct  8 15:01 nacl
drwxr-xr-x 6 www-data www-data     4096 Oct  8 15:01 numpy
-rw-r--r-- 1 www-data www-data  1102249 Oct  8 15:01 pyexpat.so
drwxr-xr-x 4 www-data www-data     4096 Oct  8 15:01 pywt
drwxr-xr-x 8 www-data www-data     4096 Oct  8 15:01 scipy
-rw-r--r-- 1 www-data www-data   130697 Oct  8 15:01 select.so
-rw-r--r-- 1 www-data www-data    71969 Oct  8 15:01 termios.so
-rw-r--r-- 1 www-data www-data  1244001 Oct  8 15:01 unicodedata.so
-rw-r--r-- 1 www-data www-data   224569 Oct  8 15:01 zlib.so

The thing is that because S3 is my primary Storage, Nextcloud expect all the files to be in the S3 strorage. It look like Nextcloud is not even reading local storage???

The log file for mediaDC is created, but never updated with logs. Look at the result in the database. In the S3 Storage, the file does not exist, probably normal if there is nothing in it. You can see that all the other appdata files are on the S3 (Storage 2 = object::store:amazon::nextcloud) The files that are not empty are in the S3.

SELECT * FROM `oc_filecache` WHERE `fileid` LIKE '%232759%' LIMIT 50 (1.124 s) Modifier

Modification | fileid | storage | path | path_hash | parent | name | mimetype | mimepart | size | mtime | storage_mtime | encrypted | unencrypted_size | etag | permissions | checksum -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- modifier | 1232759 | 2 | appdata_xxxxxxxxxxxx/preview/b/b/3/2/5/3/a/1027719/256-256-crop.jpg | 521831e8bcb801a37c216dafeaa3f75f | 1232757 | 256-256-crop.jpg | 5 | 3 | 7274 | 1684292524 | 1684292524 | 0 | 0 | 646443ac2f648 | 27 |   modifier | 2232759 | 2 | appdata_xxxxxxxxxxxx/preview/8/c/1/7/9/c/3/2232339/115-256.jpg | 8d8b00c9cd63c565cd007804e0b5d3e1 | 2232345 | 115-256.jpg | 5 | 3 | 5125 | 1694793662 | 1694793662 | 0 | 0 | 65047fbedb2fd | 27 |   modifier | 2327590 | 2 | appdata_xxxxxxxxxxxx/js/core/merged-template-prepend.js | a2bc93291a14257eb1d45ec2f2d1893b | 352364 | merged-template-prepend.js | 13 | 10 | 12238 | 1697604393 | 1697604393 | 0 | 0 | 652f6329a0638 | 27 |   modifier | 2327591 | 2 | appdata_xxxxxxxxxxxx/js/core/merged-template-prepend.js.deps | 0d965b22d41b3d5ae69a4bde4dc1d346 | 352364 | merged-template-prepend.js.deps | 14 | 10 | 246 | 1697604393 | 1697604393 | 0 | 0 | 652f6329d6a65 | 27 |   modifier | 2327592 | 2 | appdata_xxxxxxxxxxxx/js/core/merged-template-prepend.js.gzip | 0f33c79661e826134751daafb82d81f4 | 352364 | merged-template-prepend.js.gzip | 178 | 10 | 3098 | 1697604393 | 1697604393 | 0 | 0 | 652f6329f2650 | 27 |   modifier | 2327593 | 2 | appdata_xxxxxxxxxxxx/js/files/merged-index.js | b1b6fa511656aa2c1c5ccf71d3534bab | 352362 | merged-index.js | 13 | 10 | 423846 | 1697604486 | 1697604486 | 0 | 0 | 652f63865490f | 27 |   modifier | 2327594 | 2 | appdata_xxxxxxxxxxxx/js/files/merged-index.js.deps | deff269c65cb321af9330619f85a9342 | 352362 | merged-index.js.deps | 14 | 10 | 2030 | 1697604486 | 1697604486 | 0 | 0 | 652f63869358f | 27 |   modifier | 2327595 | 2 | appdata_xxxxxxxxxxxx/js/files/merged-index.js.gzip | 85edd5f3744c27a7b29d3abcbac32702 | 352362 | merged-index.js.gzip | 178 | 10 | 95975 | 1697604486 | 1697604486 | 0 | 0 | 652f6386c2fd0 | 27 |   modifier | 2327596 | 2 | appdata_xxxxxxxxxxxx/preview/a/7/2/1/e/c/1/84053/64-64-crop.jpg | 7634758639e260a84675ef495a4d3bc5 | 472338 | 64-64-crop.jpg | 5 | 3 | 1198 | 1697605223 | 1697605223 | 0 | 0 | 652f66678bb28 | 27 |   modifier | 2327597 | 2 | appdata_xxxxxxxxxxxx/preview/3/7/f/8/8/f/a/84049/64-64-crop.jpg | e3db56ff32326e274128bb55d7ade087 | 472313 | 64-64-crop.jpg | 5 | 3 | 1442 | 1697605223 | 1697605223 | 0 | 0 | 652f66678cfdd | 27 |   modifier | 2327598 | 2 | appdata_xxxxxxxxxxxx/preview/c/4/f/6/4/b/4/84050/64-64-crop.jpg | 76ef69b13c6fc84df857b373a64c075f | 472333 | 64-64-crop.jpg | 5 | 3 | 1260 | 1697605223 | 1697605223 | 0 | 0 | 652f666793e97 | 27 |   modifier | 2327599 | 2 | appdata_xxxxxxxxxxxx/mediadc/logs | aebd0ab6680e2d591e729075de335a49 | 2327249 | logs | 2 | 1 | 0 | 1697605372 | 1697605372 | 0 | 0 | 652f66fc25425 | 31 | NULL

SELECT * FROM `oc_storages` LIMIT 50 (0.001 s) Modifier

Modification | numeric_id | id | available | last_checked -- | -- | -- | -- | -- modifier | 1 | object::user:admin | 1 | NULL modifier | 2 | object::store:amazon::nextcloud | 1 | NULL modifier | 3 | object::user:user1 | 1 | NULL Regards Pierre

DerKnerd commented 9 months ago

@andrey18106 I use Minio too and it works without any issue.

@vandman Are you by any chance hosting your nextcloud instance on a shared hoster? I guess it won't work there. Most shared hosters block the exec calls this app relies on.

@dakeeper Yeah sure, here is the link: https://github.com/cloud-py-api/mediadc/releases/tag/v0.3.7 if you run Alpine use the musllinux tar.gz, if you use any glibc linux use the manylinux tar.gz. Then you need to extract the files in the mediadc directory in your appdata folder in your instances folder.

vandman commented 9 months ago

Hi, I am hosting everything on my server. The server run latest Debian and all stack are launched via docker compose. In oc_storages table, there is no reference to local storage, all are minio. regards Pierre

dakeeper commented 9 months ago

@dakeeper Yeah sure, here is the link: https://github.com/cloud-py-api/mediadc/releases/tag/v0.3.7 if you run Alpine use the musllinux tar.gz, if you use any glibc linux use the manylinux tar.gz. Then you need to extract the files in the mediadc directory in your appdata folder in your instances folder.

copyied the files.... Still "pending".... :(

DerKnerd commented 9 months ago

@vandman The app itself uses nextclouds apis to access the files so no need to check the nextcloud tables. But then I am not sure what the issue is.

@dakeeper Did you restart the job? I needed to restart the job to actually make it pick up the files. And could you post your path and directory content so I can compare it?

vandman commented 9 months ago

Hi, I think the issue is that I put manually the binaries in the local storage, not in the minio server. So nextcloud does not even see them as there is not local storage in the 'oc_storages' table.

I do not know how to send the binaries into minio through nextcloud. I can not just send them to minio because the file and there metadata need to be registered in the database.

Regards Pierre

dakeeper commented 9 months ago

Hi @DerKnerd

@dakeeper Did you restart the job? I needed to restart the job to actually make it pick up the files. And could you post your path and directory content so I can compare it?

I just restarted the jobs again, stillt pending. Then i checked the logs...i think theres something else wrong. it seems that the py api cant find the correct path?

ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, *params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '///vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'dbname']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbname} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '/**//vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'dbuser']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbuser} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '/**//vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'dbpassword']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbpassword} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '/**//vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'dbhost']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbhost} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '/**//vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'dbtype']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbtype} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '/**//vhosts/dake***.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'datadirectory']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {datadirectory} value in NC config. ERROR:nc_py_api.mediadc:main::Unable to parse config or connect to database. Does occ works?`

dakeeper commented 9 months ago

Hi @DerKnerd

@dakeeper Did you restart the job? I needed to restart the job to actually make it pick up the files. And could you post your path and directory content so I can compare it?

I just restarted the jobs again, stillt pending. Then i checked the logs...i think theres something else wrong. it seems that the py api cant find the correct path?

ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '///vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'dbname']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbname} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '///vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'dbuser']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbuser} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '///vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'dbpassword']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbpassword} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '///vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'dbhost']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbhost} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '///vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'dbtype']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbtype} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '///vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'datadirectory']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {datadirectory} value in NC config. ERROR:nc_py_api.mediadc:main::Unable to parse config or connect to database. Does occ works?`

Finally it`s running !

Seems it has something to do with apcu caching. I`m running Ubuntu 22.04.3 LTS with Plesk. PHP 8.1.2 I added apc.enable_cli=1 in the /etc/php/8.1/mods-available/apcu.ini Now it seems to do something. The logs now looks like this:

DEBUG:nc_py_api.mediadc:task:process_task:Processing task: id=11 DEBUG:nc_py_api.mediadc:task:analyze_and_lock:Processing new task. DEBUG:nc_py_api.mediadc:task:analyze_and_lock:Task locked. DEBUG:nc_py_api.mediadc:task:init_task_settings:Image hamming distance: 26 DEBUG:nc_py_api.mediadc:task:init_task_settings:Video hamming distance between 4 frames: 104 DEBUG:nc_py_api.mediadc:task:init_task_settings:Hashing algo: dhash DEBUG:nc_py_api.mediadc:task:start_background_thread:Starting background thread. DEBUG:nc_py_api.mediadc:images:process_images:calculating hash for image: fileid = 130390 DEBUG:nc_py_api.mediadc:images:hash_image_data:Exception during image processing: module 'PIL.Image' has no attribute 'ANTIALIAS' DEBUG:nc_py_api.mediadc:images:process_images:calculating hash for image: fileid = 130391 DEBUG:nc_py_api.mediadc:images:hash_image_data:Exception during image processing: etc...

DerKnerd commented 9 months ago

@vandman The app doesn't lookup in the table for anything. If you check the code it runs based on a self composed path. Binaries in minio wouldn't work. These tables are empty for me too. Even if you use minio, many apps store their configuration, logs and other helpers in the appdata folder.

DerKnerd commented 9 months ago

@dakeeper Perfect :)

dakeeper commented 9 months ago

@DerKnerd

Maybe theres another issue. The task completed without any issues but found 0 duplicates. This cant be correct.

I lowered the detection to 60% an will start another run. Before i start this i copy a image to my desktop, renamed it, and uploaded it to the same folder. So this is definately a Duplicate.

The run finished with 0 duplicates found. This ist definately wrong.

Maybe something to do with: DEBUG:nc_py_api.mediadc:images:hash_image_data:Exception during image processing: module 'PIL.Image' has no attribute 'ANTIALIAS'

Any clues?

vandman commented 9 months ago

Hi, I think the problem comes from this file: occ.py for whatever reason, _OCC_PATH is wrong. I had to change it even if I set SERVER_ROOT to the right path in my env. So for the time being, replace /var/www/nextcloud by /var/www/html. Regards Pierre

dakeeper commented 9 months ago

Hi, I think the problem comes from this file: occ.py for whatever reason, _OCC_PATH is wrong. I had to change it even if I set SERVER_ROOT to the right path in my env. So for the time being, replace /var/www/nextcloud by /var/www/html. Regards Pierre

@vandman In terms of wich error?

dakeeper commented 9 months ago

DEBUG:nc_py_api.mediadc:images:hash_image_data:Exception during image processing: module 'PIL.Image' has no attribute 'ANTIALIAS'

Hello again, found the error.

ANITALIAS was removed from Pillow 10.0.0 onwards and is replaced by LANCZOS. In /apps/mediadc/python/imagehash.py i changed Image.ANTIALIAS to Image.LANCZOS I tested it in the code for dhash and it worked.

Maybe we need an official update for this, but till then this seems to be a workaround quick and dirty.

Best Regards, Holger

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[DerKnerd]

@andrey18106 I use Minio too and it works without any issue.

@vandman Are you by any chance hosting your nextcloud instance on a shared hoster? I guess it won't work there. Most shared hosters block the exec calls this app relies on.

@dakeeper Yeah sure, here is the link: https://github.com/cloud-py-api/mediadc/releases/tag/v0.3.7 if you run Alpine use the musllinux tar.gz, if you use any glibc linux use the manylinux tar.gz. Then you need to extract the files in the mediadc directory in your appdata folder in your instances folder.

[vandman]

Hi, I am hosting everything on my server. The server run latest Debian and all stack are launched via docker compose. In oc_storages table, there is no reference to local storage, all are minio. regards Pierre

[dakeeper]

@dakeeper Yeah sure, here is the link: https://github.com/cloud-py-api/mediadc/releases/tag/v0.3.7 if you run Alpine use the musllinux tar.gz, if you use any glibc linux use the manylinux tar.gz. Then you need to extract the files in the mediadc directory in your appdata folder in your instances folder.

copyied the files.... Still "pending".... :(

[DerKnerd]

@vandman The app itself uses nextclouds apis to access the files so no need to check the nextcloud tables. But then I am not sure what the issue is.

@dakeeper Did you restart the job? I needed to restart the job to actually make it pick up the files. And could you post your path and directory content so I can compare it?

[vandman]

Hi, I think the issue is that I put manually the binaries in the local storage, not in the minio server. So nextcloud does not even see them as there is not local storage in the 'oc_storages' table.

I do not know how to send the binaries into minio through nextcloud. I can not just send them to minio because the file and there metadata need to be registered in the database.

Regards Pierre

[dakeeper]

Hi @DerKnerd

@dakeeper Did you restart the job? I needed to restart the job to actually make it pick up the files. And could you post your path and directory content so I can compare it?

I just restarted the jobs again, stillt pending. Then i checked the logs...i think theres something else wrong. it seems that the py api cant find the correct path?

ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, *params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '///vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'dbname']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbname} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '/**//vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'dbuser']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbuser} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '/**//vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'dbpassword']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbpassword} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '/**//vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'dbhost']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbhost} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '/**//vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'dbtype']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbtype} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '/**//vhosts/dake***.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'datadirectory']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {datadirectory} value in NC config. ERROR:nc_py_api.mediadc:main::Unable to parse config or connect to database. Does occ works?`

[dakeeper]

Hi @DerKnerd

@dakeeper Did you restart the job? I needed to restart the job to actually make it pick up the files. And could you post your path and directory content so I can compare it?

I just restarted the jobs again, stillt pending. Then i checked the logs...i think theres something else wrong. it seems that the py api cant find the correct path?

ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '///vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'dbname']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbname} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '///vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'dbuser']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbuser} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '///vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'dbpassword']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbpassword} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '///vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'dbhost']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbhost} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '///vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'dbtype']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {dbtype} value in NC config. ERROR:nc_py_api:occ:php_call:php_call exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/nc_py_api/occ.py", line 57, in php_call result = subprocess.run([_PHP_PATH, params], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/php8.1', '///vhosts/dake.de/httpdocs/occ', '--no-warnings', 'config:system:get', 'datadirectory']' returned non-zero exit status 1. ERROR:nc_py_api:config::Cant find {datadirectory} value in NC config. ERROR:nc_py_api.mediadc:main::Unable to parse config or connect to database. Does occ works?`

Finally it`s running !

Seems it has something to do with apcu caching. I`m running Ubuntu 22.04.3 LTS with Plesk. PHP 8.1.2 I added apc.enable_cli=1 in the /etc/php/8.1/mods-available/apcu.ini Now it seems to do something. The logs now looks like this:

DEBUG:nc_py_api.mediadc:task:process_task:Processing task: id=11 DEBUG:nc_py_api.mediadc:task:analyze_and_lock:Processing new task. DEBUG:nc_py_api.mediadc:task:analyze_and_lock:Task locked. DEBUG:nc_py_api.mediadc:task:init_task_settings:Image hamming distance: 26 DEBUG:nc_py_api.mediadc:task:init_task_settings:Video hamming distance between 4 frames: 104 DEBUG:nc_py_api.mediadc:task:init_task_settings:Hashing algo: dhash DEBUG:nc_py_api.mediadc:task:start_background_thread:Starting background thread. DEBUG:nc_py_api.mediadc:images:process_images:calculating hash for image: fileid = 130390 DEBUG:nc_py_api.mediadc:images:hash_image_data:Exception during image processing: module 'PIL.Image' has no attribute 'ANTIALIAS' DEBUG:nc_py_api.mediadc:images:process_images:calculating hash for image: fileid = 130391 DEBUG:nc_py_api.mediadc:images:hash_image_data:Exception during image processing: etc...

[DerKnerd]

@vandman The app doesn't lookup in the table for anything. If you check the code it runs based on a self composed path. Binaries in minio wouldn't work. These tables are empty for me too. Even if you use minio, many apps store their configuration, logs and other helpers in the appdata folder.

[DerKnerd]

@dakeeper Perfect :)

[dakeeper]

@DerKnerd

Maybe theres another issue. The task completed without any issues but found 0 duplicates. This cant be correct.

I lowered the detection to 60% an will start another run. Before i start this i copy a image to my desktop, renamed it, and uploaded it to the same folder. So this is definately a Duplicate.

The run finished with 0 duplicates found. This ist definately wrong.

Maybe something to do with: DEBUG:nc_py_api.mediadc:images:hash_image_data:Exception during image processing: module 'PIL.Image' has no attribute 'ANTIALIAS'

Any clues?

[vandman]

Hi, I think the problem comes from this file: occ.py for whatever reason, _OCC_PATH is wrong. I had to change it even if I set SERVER_ROOT to the right path in my env. So for the time being, replace /var/www/nextcloud by /var/www/html. Regards Pierre

[dakeeper]

Hi, I think the problem comes from this file: occ.py for whatever reason, _OCC_PATH is wrong. I had to change it even if I set SERVER_ROOT to the right path in my env. So for the time being, replace /var/www/nextcloud by /var/www/html. Regards Pierre

@vandman In terms of wich error?

[dakeeper]

DEBUG:nc_py_api.mediadc:images:hash_image_data:Exception during image processing: module 'PIL.Image' has no attribute 'ANTIALIAS'

Hello again, found the error.

ANITALIAS was removed from Pillow 10.0.0 onwards and is replaced by LANCZOS. In /apps/mediadc/python/imagehash.py i changed Image.ANTIALIAS to Image.LANCZOS I tested it in the code for dhash and it worked.

Maybe we need an official update for this, but till then this seems to be a workaround quick and dirty.

Best Regards, Holger