ctf0 / Laravel-Media-Manager

A "Vuejs & Laravel" Media Manager With Tons of Features
MIT License
832 stars 181 forks source link

[BUG] request takes too long #150

Closed vntrungld closed 4 years ago

vntrungld commented 4 years ago

What version of Laravel are you using? (nothing lower than v6).

v6

is it an issue related to installation or usage ?

Usage

what is the error you are getting ?

extreme slow when you try to get files

Extra Info "attaching a screen shot would be very helpful"

When I try to get 34 files from a folder and it takes 2.4 min image

When I try to debug it by this image

And the result: image It's still slow but faster than normal

But when I tried to dive deeper, I saw that the function iterator_to_array makes the request slow extra 2min for just transforming from PHP Object to Array. Do we have a different function to use instead of iterator_to_array?

Do You Know A Solution For this Issue ?

Not yet

ctf0 commented 4 years ago

i believe its part of https://github.com/jhofm/flysystem-iterator which was suppose to make things faster.

also what driver r u using, aws, google, public ?

vntrungld commented 4 years ago

Thanks for your reply. I'm using digital ocean spaces

ctf0 commented 4 years ago

can u make a repo for me to test with ?

ctf0 commented 4 years ago

https://github.com/ctf0/Laravel-Media-Manager/issues/144

kmarkovych commented 3 years ago

I have a similar issue. Using DO S3 and Laravel 8. In my folder structure, I have /media/files folder which contains 16 000+ files. Set 'get_folder_info' => false and got some performance improvement. The main issue is in \Vendor\Mediamanager\GetContent::getFolderContent, it asks 'mimetype', 'visibility', 'timestamp', 'size' by separate HTTP queries, in my case it's 'mimetype' and 'visibility' so a have 32 000 HTTP queries.

ctf0 commented 3 years ago

@kvmarkovich this is not true for 2 reasons

  1. we use listWith which get all the relevant info in the same request "at least thats why we use it in the first place" https://github.com/ctf0/Laravel-Media-Manager/blob/c53c5fe75ec693e8b1886acac80bcaacce60913f/src/App/Controllers/Modules/GetContent.php#L97-L101

  2. all the files at the same level will be fetched in the same request, so if the 16,000 files are at the same level then we will make only one request to list all of them except that it will take some time for the request to finish.

however there is a downside with the current solution due to pagination which we do over the whole collection instead of the original listing which is not originally available in the filesystem plugin , thats why its highly recommended to use the cache adaptor.

u can test the prev points using a local disk on ur machine with ex.100 file at the same level and check the network tab.

kmarkovych commented 3 years ago

I wish you know better code you work with... You can check how it works one more time. If you won't have any success I'll explain to you how it works.

ctf0 commented 3 years ago

am probably missing ur point, so yeah explain to me how it works