sebastiencastermans
commented
1 year ago Hi, I have the same issue and... the same request! I vote for that.
Open apoorv569 opened 1 year ago
sebastiencastermans
commented
1 year ago Hi, I have the same issue and... the same request! I vote for that.
zybster
commented
1 year ago Would really be great if this could be added - FaceRecognition is a very nice addon
antmarra
commented
1 year ago Please add pdlib to AIO, or let us know how to do this.
froman753
commented
1 year ago Same request, definitely vote for this!
enoch85
commented
1 year ago It's also used in the Nextcloud VM but that repo doesn't work. Would be super useful with "an official" release from something like PECL.
rastographics
commented
1 year ago Same issue...I want to install face recognition on nextcloud AIO so adding my vote to this.
tola5
commented
1 year ago one more plus from here
enoch85
commented
1 year ago Please stop adding your "+1" comments, it notifies all that's posted here, or that subscribes to this issue. Add your 👍 in the first post instead.
Thanks!
stalsbyt
commented
1 year ago +1
enoch85
commented
1 year ago Please stop adding your "+1" comments, it notifies all that's posted here, or that subscribes to this issue. Add your +1 in the first post instead.
Thanks!
@stalsbyt PLEASE! :)
JeCheeseSmith
commented
1 year ago I am using
Nextcloud-AIOand want to install theFace Recognitionapp which requires this extension as a dependency. Please add this to PECL so me and anybody else can use this.I got recommended to request this here, nextcloud/all-in-one#1551
I would like to use Face Recognition too on Nextcloud AIO. I'm a newbie to any of this but I would like too use the API of Face Recognition to build a genetic tree with the faces found.
szaimen
commented
1 year ago cc @stalker314314 @matiasdelellis what is your take on this?
artur90226
commented
1 year ago +1
jrmarincallejero
commented
1 year ago Un voto mas
matiasdelellis
commented
1 year ago Doubt.. I'm no docker expert... 🤔
Why does it seem so complicated to do something like this?
RUN wget https://github.com/goodspb/pdlib/archive/master.zip \
&& mkdir -p /usr/src/php/ext/ \
&& unzip -d /usr/src/php/ext/ master.zip \
&& rm master.zip
RUN docker-php-ext-install pdlib-masterYou could also install dlib compiling, and it would be important to do it, (to optimize it for hardware), with 5 more lines of code.
ps: https://github.com/matiasdelellis/facerecognition/wiki/Docker
Zoey2936
commented
1 year ago If we included all dependencies ever requested in AIO, then Nextcloud AIO would be unnecessary big after some time and installing decencies by default would mean additionally Things to maintain and the docker built could easily break. That is the reason AIO only supports adding PHP extension using pecl. And also is nearly every PHP extension requested until now (except this extension here) available over pecl. (https://github.com/nextcloud/all-in-one/discussions/1159#discussioncomment-3655824)
matiasdelellis
commented
1 year ago Hello everyone, Personally, I see Docker as a way to do dirty stuff in a safe environment, and keep my server clean. So the argument doesn't convince me... 🤦🏻 🙈
I don't know.. I'm probably wrong.. 🤔
However, I think there is an opportunity for improvement in the optimization of the extension, and that is worth it. 😄 😬
Ok okey.. Well. During the day, I was doing tests, and I was able to compile dlib as a submodule of the same extension.
So, both dlib and the extension compile as native architecture, and are optimized for the hardware used.
About PECL, which is the main issue here, I'm not sure if I can do it, since I don't own the extension, and would prefer not to keep a fork, but in the meantime I invite you to try the changes since it will be essential to be build as a PECL package.
matiasdelellis
commented
1 year ago Well, I guess I have something interesting... As I said, I will not upload to pecl soon.. but I have a working package to test and probably just use.. 😄
sudo apk install libpng openblas lapack
wget https://github.com/matiasdelellis/pdlib/releases/download/v1.1.0/pdlib-1.1.0.tgz
sudo pecl install ./pdlib-1.1.0.tgzIn short, you must install dependencies (NEXTCLOUD_ADDITIONAL_APKS 😉 😉 wink...), download the package 🤷🏻♂️ , and install it from package..
It's not the same as before???!?!??? 🙈 😅 😅 😆
Well, now being dlib in the same code, many problems are minimized, and it is always optimized when compiling it.. 😉
So, you can adapt the AIO scripts to download the tarball and install them???
I imagine something like that.. .
-e NEXTCLOUD_ADDITIONAL_PHP_EXTENSIONS="https://github.com/matiasdelellis/pdlib/releases/download/v1.1.0/pdlib-1.1.0.tgz"
similar to what shivammathur/setup-php@v2 does.. 🤔
Zoey2936
commented
1 year ago @szaimen could you please have a look?
matiasdelellis
commented
1 year ago Hi everyone,
Maybe not even have to change the scripts, but I need someone to test it. Pecl can install the package from a URL.. 🎉
[matias@nube ~]$ sudo pecl install https://services.delellis.com.ar/data/facerecognition/pdlib-1.1.0.tgz
downloading pdlib-1.1.0.tgz ...
Starting to download pdlib-1.1.0.tgz (3,078,356 bytes)
.............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................done: 3,078,356 bytes
WARNING: channel "pecl.php.net" has updated its protocols, use "pecl channel-update pecl.php.net" to update
1389 source files, building
running: phpize
Configuring for:
PHP Api Version: 20200930
Zend Module Api No: 20200930
Zend Extension Api No: 420200930
building in /var/tmp/pear-build-rootGQ6ovZ/pdlib-1.1.0
running: /var/tmp/pdlib/configure --with-php-config=/usr/bin/php-config
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
....So, -e NEXTCLOUD_ADDITIONAL_PHP_EXTENSIONS="https://services.delellis.com.ar/data/facerecognition/pdlib-1.1.0.tgz", probably works directly. 😄
EDIT: Change the url to a private server, since PECL doesn't like github redirects. 😅
EDIT 2: It will fail, because it does some character-only validation. https://github.com/nextcloud/all-in-one/blob/c23b7aa3969a5f029e3cc9f9abc6be34e97a9ec0/Containers/mastercontainer/start.sh#L207-L212
Beyond that, I would think it should work. 🤔
szaimen
commented
1 year ago Wow, thanks a lot for this nice new aproach @matiasdelellis ! :)
I wonder how much work would it be to publish this now on PECL? With that it would work out-of-the-box in AIO directly and would be prefered from a UX perspective since the user would not need to maintain and change this link when a new release comes out.
As for libpng openblas lapack are these runtime dependencies or only needed for compiling?
goodspb
commented
1 year ago sorry guys, I lost track of PHP now. Plz give me some time to figure out how to upload it to PECL. 😭
matiasdelellis
commented
1 year ago Hi @goodspb If you're willing to upload to PECL (Theoretically, it shouldn't be difficult to be accepted.. and you have a lot of support from here.. 😬), I can do a pull request with the changes.
I would love to help you to improve and extend it, but I don't want to be responsible for another project forking this. 😄
About the current proposal:
My TODO list about that:
Now I don't remember anything else relevant to this discussion.. 🤔
Well, If you want, I start the PR.. 😃
Zoey2936
commented
1 year ago @matiasdelellis I've tried installing it from inside the nextcloud-aio-nextcloud container, but it does not work (I've installed build-base and autoconf already):
samssausages
commented
1 year ago +1, want it for Nexcloud AIO support
likt0r
commented
1 year ago +1 fro support in AIO would be great, thx for your work :)
wuast94
commented
1 year ago +1 for support
johnnyq
commented
1 year ago +1 for support
XiaoKe89
commented
1 year ago +1 for support
szaimen
commented
1 year ago Hi @goodspb do you have a comment on the offer of @matiasdelellis above?
matiasdelellis
commented
1 year ago Hello everyone, Please, be patient. My main work includes some open source projects, but nothing related to Nextcloud, and my free time is distributed among many projects. 😅
The same goes for Malvin. 😉
I haven't even been able to test NC26 yet, and when I do I'll focus on Face Recognition, pdlib and Quick Notes. 😬
Regards, 😄
jcampos79
commented
1 year ago One more vote here
miguelzabala
commented
1 year ago Please do it.
sunnyd24
commented
1 year ago +1 pdlib dependencies for Ubuntu 22.04 LTS running AIO
bau-tech
commented
1 year ago One more vote
sunnyd24
commented
1 year ago facerecognition new version available: 0.9.20
Error: App "Face Recognition" cannot be installed because the following dependencies are not fulfilled: The library pdlib is not available.
facerecognition updated
shukryshuk
commented
1 year ago up
sudo add pdlib to PECL && sudo thank you very much
jockefardig
commented
1 year ago Yet another vote +1
sunnyd24
commented
1 year ago Updated to Nextcloud AIO v6.2.1 (still an issue)
This app (Face Recognition) cannot be installed because the following dependencies are not fulfilled: The library pdlib is not available.
eldertek
commented
1 year ago I vote for
PrivatHomeServer
commented
1 year ago I have tested both solutions "recognize" and "face recognition". For me, recognize is far from replacing face recognition. Reasons - recognition rate, false detection rate, no possibility to edit it via metadata, APPs (nkming)...
After changing PHP from 7.4 to 8.x I decided to use Nextcloud AOI and I am disappointed. This function is essential for replacing Google Photos in combination with nextcloud. But as it is, it is hardly usable.
--> I would be very happy to receive instructions, support or whatever.
szaimen
commented
1 year ago Hi, you could check if you can use the external model but I guess this does not eliminate the need for pdlib @matiasdelellis ?
matiasdelellis
commented
1 year ago Hi, you could check if you can use the external model but I guess this does not eliminate the need for pdlib @matiasdelellis ?
Hello everyone, I was denied to free my application from the use of PDLIB 😅 , but finally I could make a 100% native php implementation using, only requireing the external model to do the most arduous part of faces discoveries. 😬
See:
Please do not take this as an abandonment of this issue. It is still recommended to use PDLIB for performance reasons, and soon I hope to upload the PR that I promised to use PECL. 😄
To do this reliable, we should give the external model a bit of love. ❤️
sunnyd24
commented
1 year ago @matiasdelellis Many thanks for your efforts here to create a native PHP implementation as a workaround. I have updated the Plugin and it now can be enabled.
Now, just to get my head around how to configure AIO with Face Recognition plugin with models and PHP memory settings. I have read up that CNN68/model 1 and PHP memory of between 2-4 GB should be fine, but not clear where to apply these settings.
Is this a one time setup in AIO using bash or needs to be configured in AIO nextcloud docker settings?
Is there a list of all model Id's somewhere?
cd nextcloud/ # or whatever your path to Nextcloud is
sudo -u www-data php occ app:enable facerecognition # or ./occ app:enable facerecognition
sudo -u www-data php occ face:setup -M MEMORY # or ./occ face:setup --memory MEMORY
sudo -u www-data php occ face:setup -m MODEL_ID # or ./occ face:setup -m MODEL_IDThe php memory limit in AIO can be adjusted like so: https://github.com/nextcloud/all-in-one#how-to-adjust-the-php-memory-limit-for-nextcloud
Also see https://github.com/nextcloud/all-in-one#how-to-run-occ-commands how to run occ commands in AIO.
sunnyd24
commented
1 year ago The php memory limit in AIO can be adjusted like so: https://github.com/nextcloud/all-in-one#how-to-adjust-the-php-memory-limit-for-nextcloud
Also see https://github.com/nextcloud/all-in-one#how-to-run-occ-commands how to run occ commands in AIO.
@szaimen Thanks!
For others with similar AIO related questions OR new to face recognition here's a quick start guide for us AIO users which took me a while to figure out:
NEXTCLOUD MASTER CONTAINER CONFIG =============================
Open Nextcloud AIO interface, and stop containers.
docker stop nextcloud-aio-mastercontainer
docker rm nextcloud-aio-mastercontainerAmend the PHP memory limit to be anything from 1024M to 2048M should suffice in --env NEXTCLOUD_MEMORY_LIMIT=4096M \ to docker run command. I used 4096M as I was unsure how much is required.
docker run [your amended multi-line docker run command with updated PHP memory limit]
Open Nextcloud AIO interface, and start containers.
FASTEST METHOD using Docker
======================
docker build -t facerecognition https://github.com/matiasdelellis/facerecognition-external-model.git
openssl rand -base64 32
echo EnTeRYoURgEnErAtEdLoNgKeYhErE= > api.key
NOTE: change the key above generated in output of openssl rand -base64 32 and copy it to clipboard.
sudo docker run --rm --detach -i -p 8080:5000 -e API_KEY="EnTeRYoURgEnErAtEdLoNgKeYhErE" facerecognition
NOTE: change 8080 to any external port available on host PC where Docker is running
CONFIG
======
sudo docker run -it --rm --volume nextcloud_aio_nextcloud:/var/www/html:rw alpine sh -c "apk add --no-cache nano && nano /var/www/html/config/config.php"
$CONFIG = array (
...........................................
'facerecognition.external_model_url' => 'http://192.168.1.52:8080',
'facerecognition.external_model_api_key' => 'EnTeRYoURgEnErAtEdLoNgKeYhErE',
...............................
);NOTE: Add the above lines starting with 'facerecognition' within array in config.php, change host IP address and port, and be careful edigint his file as this can break Nextcloud from starting if done incorrectly.
sudo docker exec --user www-data -it nextcloud-aio-nextcloud php occ face:setup -m 5
sudo docker exec --user www-data -it nextcloud-aio-nextcloud php occ face:setup -M 4GNOTE: change 4G to be anything between 2G and 4G of memory.
sudo docker exec --user www-data -it nextcloud-aio-nextcloud php occ face:setup
NOTE: just checks configuration and provides a table to see that settings have been applied.
Go here for advanced settings such as enabling face recognition on group/external/shared folders
USAGE
=====
sudo docker exec --user www-data -it nextcloud-aio-nextcloud php occ face:background_job --user_id USERNAME
NOTE: change USERNAME to your nextcloud username you want to run face recognition on. You can also append --timeout 18000, to stop after 5 hours. However, for first run you may want to omit timeout as for my initial library test of 573 photos, it took approx 7 hours or so to complete, that's approx 44 seconds per photo (mind you I did have my temporary files settings at max of 3325x2494 (4x3))!
sudo docker exec --user www-data -it nextcloud-aio-nextcloud php occ user:setting USERNAME facerecognition full_image_scan_done false
NOTE: Another useful command to have if you need to start a full image scan from scratch due to advanced settings you have changed, it will not do it automatically! Remember to change USERNAME to your nextcloud username you want to to rescan.
REFERENCE =========
Apologies if I've made any errors in the above, it's the best I could capture on the go, feel free to let me know and I will update the above for other users Thanks @szaimen for the PHP memory limit commands reminder!
szaimen
commented
1 year ago Hi, thanks for this guide! However I think you missed actually adjusting the php memory limit for the Nextcloud container itself or is this not need to be adjusted when using the external model @matiasdelellis?
If it is, see https://github.com/nextcloud/all-in-one#how-to-adjust-the-php-memory-limit-for-nextcloud
matiasdelellis
commented
1 year ago In principle, the external model only requires 128 MB of ram, since the large memory consumption is done by the analysis of the images (specifically the search for faces) and another process does it.. However, the clustering of faces may require more memory.. I don't have much statistics about it, but there are reports of 2 GB with 50 thousand faces of single user.
szaimen
commented
1 year ago Btw another advice: @sunnyd24 you could also add the container to the nextcloud-aio network. Then you do not need to expose the sevice on the host and instead can point directly at the container name...
szaimen
commented
1 year ago Also my personal recommendation for the analysis_image_area: https://github.com/nextcloud/vm/blob/5a245496ce055d5c1697b689ede6e398548f7603/old/face-recognition.sh#L138
I am using
Nextcloud-AIOand want to install theFace Recognitionapp which requires this extension as a dependency. Please add this to PECL so me and anybody else can use this.I got recommended to request this here, https://github.com/nextcloud/all-in-one/discussions/1551