Documentation - More Details & usage for boost with file / images

[carstengrimm]

Currently, I am experimenting to implement boost with plenty of images e.g. as an alternative comparable to

if($file = $site->file('file://'.$uuid)){
   // do something
}

but using something like:

if($file = boost($fileuuid)){

}
// or
if($file = boost('file://'.$fileuuid)){

}

Didn't seem to work. Also, my question would be, if I call an image via UUID - if the performance would be increased in a one by one image scenario.

[bnomei]

the boost index is currently only tracking pages.

boost does sometyhing similar to the regular uuid resolver. lookup uuid to path. BUT boost stores the diruri not the id. thats why its a bit faster. kirby does not do this so it doesnt have to update the index when items get sorted. boost does have to update then.

i do not think you would gain much unless you have a very fragmented pages tree. if you have few pages but lots of files you would not gain a noticeable amount of improvement in speed.

the idea is from an old plugin of mine that got merged into boost

https://github.com/bnomei/kirby3-bolt#usage

// lets assume 1000 pages: 10*10*10
$id = 'this-page-has/ten-siblings/in-every-subfolder';
$page = page($id); // kirby core
$page = bolt($id); // ~10x faster lookup

// can lookup beginning at a certain page as well
$page = $somePage->bolt($idInTree);

// it's even faster when you look up based on a directory name
$page = bolt('1_this-page-has/5_ten-siblings/3_in-every-subfolder');

[bnomei]

the bigger issue with the image is that kirby will load the parent page slower than boost would (using the old bolt code).

if i were to add a tracker for files i would store the parent diruri and try to solve that first and foremost.

what is you dataset/setup like exactly? how many pages and files? what does the pages tree look like?

[carstengrimm]

so far i am just playing around, looking to see how the performance could hold up.

Basically as an idea, having an image gallery where the file can be bought as like stock photo, or wallpaper.

Which could be 100, 1000 or even 10000 photos.

File Structure

- category 1
-- subcategory 1
---files list
-- subcategory 2
---files list
-- subcategory 3
---files list

- category 2
-- subcategory 1
.....
.....

so far i have been playing around to see the performance, while using LAPSE to cache/store the files collection (let's say 500 to 600 images) each subcategory which will be paginated and displayed ... like 50 per page which then could be acting like a cart ....

the cart function could be using the uuid as key to make sure the picture exists, as well as only keep one occurance (digital download, there's no quantity)

the basic test functionality is already there and working. I am just wondering if with as many files as 10000+ the UUID will become a bottleneck.

        $image = $site->file('file://'.$uuid);
        if($image){
            if(!isset($cart[$uuid])){
                $cart[$uuid] = 1;            
            }
            $session->set('cart', $cart);
        }

In the Cart and calculations and whatever, the $cart will be run through,

    foreach($cart as $key => $item){
        if($image = $site->file('file://'.$key)){     
            $total += $image->page()->parent()->price()->float(); 
        }
    }

So let's say there's 10 Files in the Cart, it will run though each File via UUID to confirm it's actually available, as well as getting the price from a parent page.

in a later scenario, the file uuid will saved in an order, and the download functionality would access the file via the uuid as well to trigger a secured download.

[bnomei]

i had an idea how to solve it simple enough and will add a lookup for files later today.

[bnomei]

i added support for looking up files.

i assume you did set the core uuid cache to apcu already to get max performance there.

on dev-setup with filecache in unittesting... so boost will only get even faster in production...

10000 call to get the same file

boost(): 0.68
site()->file(): 1.09

[bnomei]

personally i would create a cache that updates on the parent page update hook to cache the price for all images. i would not cache it for more than 1day just to be save but i might create a lots of hit for apcu write then.

'hooks' => [
        'page.update:after' => function (Kirby\Cms\Page $newPage, Kirby\Cms\Page $oldPage) {
            if ($newPage->template()->name() === 'myPageWithPrice') {
               foreach($newPage->index()->images() as $image) {
                  kirby()->cache('pricesPerImage')->set($image->uuid()->toString(), $newPage->price()->toFloat(), 24*60*60);
              }
         }
        }
    ]

instead

$total += $image->page()->parent()->price()->float(); 

get for cache and use 0 for nonexisting

$total += kirby()->cache('pricesPerImage')->get($key, 0); 

TIP: you can also set your custom cache to apcu or use lapse with apcu.

[bnomei]

this means you would have NO file lookup at all. maybe you should add hooks for file and page delete to flush the unneeded caches but its not really that important imho.

[carstengrimm]

I've updated via composer and was trying out to replace

$image = $site->file('file://'.$uuid);
$image = boost($uuid);

and it seems not to work, is the usage correct to use boost($uuid) e.g. it is already identifing automatically that this uuid will be a file?

since kirby introduced uuids, i find it very straight forward using the uuid instead of diruri, or uuid, while diruri und uids change, the uuids doesn't.

to your comments above, i am trying out different caching methods, but locally my setup does not allow me to try out apcu, so i am usually doing tests with no caching / debug on, or sometimes sqlite driver, but locally sqlite having sqlite being locked error

[bnomei]

you need to have the schema for the file.

boost('file://'.$uuid);

i decided to do it this way because kirby stores them with schema in their pages and files fields.

[carstengrimm]

in your updated documentation you write boost($uuid) for images, which seems missleading if it's supposed to be boost('file://'.$uuid) according to your last comment

is 4.0.0. backwards compatible? as far as i can see composer wants kirby v4 to be installed.

i am still stuck with using boost with files...

Infos:

Kirby 3.9.7 Boost 2.4.1 PHP 8.2.11

site()->boost();
// boostmark showing like 3109 results

if($file = $site->file('file://'.$uuid){
 // works
}

if($file = boost($uuid){
 dump($file); // empty / null / no dump
}
if($file = boost('file://'.$uuid){
 dump($file); // empty / null / no dump
}

tried with debug on, debug off, sqlite, file cache etc.pp. i am having page "has boost" models for the pages which include the images

[bnomei]

i added that files feature for the k4 version and plugin 4.x. any reason you will not use k4?

but i think it should be backwards compatible and i will test soon (apart from the composer conflict i set as a precaution).

kirby docs refer to the UUID as including the schema. https://getkirby.com/docs/guide/uuids

[bnomei]

using boost to resolve the image from an uuid will be faster by the amount gained from using boost (and bolt under the hood) to resolve the parent of the file.

like i did show above even repeated calls or for that matter calls to files on the same parent can be expected to be a teeny-tinsy bit faster - but it might add up if you had thousands per single request.

[carstengrimm]

i added that files feature for the k4 version and plugin 4.x. any reason you will not use k4?

but i think it should be backwards compatible and i will test soon (apart from the composer conflict i set as a precaution).

kirby docs refer to the UUID as including the schema. https://getkirby.com/docs/guide/uuids

Eventually I will upgrade, but since it's only in beta I am waiting for a stable release