bnomei / kirby3-boost

Boost the speed of Kirby by having content files of pages cached and a fast lookup based on uuids
https://sakila-with-boost.bnomei.com/
MIT License
51 stars 0 forks source link

Not sure where to start #3

Closed psntr closed 2 years ago

psntr commented 2 years ago

Thanks for this new plugin.

I'm eager to make it work with a new project that would handle on a single request a lot of pages. But I'm having trouble to simply use BoostID.

Maybe I'm confused with your other plugin AutoID which was really straightforward and easy to use. With BoostID I seem not able to create and ID out of the box by following your instruction here.

I have installed:

I have the config.php with:

return [
    'smartypants' => true,
    'debug' => false,
    'bnomei.boost.cache' => ['type' => 'sqlite']
];

In my models folder I have the following file article.php:

class ArticlePage extends \Bnomei\BoostPage
{
}

and finally in the panel I have article.yml:

columns:
  - width: 2/3
    fields:
      boostid:
        type: boostid
      headline:
        type: writer
        marks:
          - italic
      text: fields/blocks
      source:
        type: textarea

  - width: 1/3
    sections:
      meta:
        type: fields
        fields:
          date:
            type: date
            time: false
            default: now
          category: tags/category

When in article.php I try to output the boostID nothing is return, also the article.txt doesn't register the boostID. Finally in the panel if I check the article, there is the following error: SQLite3Stmt::execute(): Unable to execute statement: UNIQUE constraint failed: cache.id

Sorry for the long message, but maybe I don't understand how it differs from autoID where the ID was naturally register on the .txt file.

bnomei commented 2 years ago

no problem. lets take this step by step.

1) lets use apcu or memcached as cachedrivers depending on which is available. just to make sure we do not have a problem with sqlite. we can debug that later.

'bnomei.boost.cache' => ['type' => 'apcu']

2) boost will not recreate missing ids on its own like the autoid plugin. so if you add boost to an existing project you have to call something like this in an template.

kirby()->impersonate('kirby');
site()->boost();
// same as
foreach(site()->index(true) as $page) {
   $page->boost();
}

the $page->boost() will force an id if it does not exist yet. that was added in an recent version (make sure you have most current).

please report back if that did work and then we will take care of the sqlite issue.

psntr commented 2 years ago

Thank you for your reply!

So I'm a on local version now with MAMP and don't have APCU nor memcached, so I try now with: 'bnomei.boost.cache' => ['type' => 'file'] and executed successfully the manual call for the ids to be populated, now indeed they are shown in the articles.

Ready for the next stage

bnomei commented 2 years ago

about sqlite... it either works now (because no ids are missing/empty) or something is not right with sqlite.

can you print me your sqlite version?

print_r(SQLite3::version());

1) then try enabling

'bnomei.boost.cache' => ['type' => 'sqlite']

2) access any page with boost model in preferable your frontend (better debugging than in panel) and tell me what happens.

psntr commented 2 years ago

Indeed, now that the IDs are populated, the previous error message doesn't show anymore after turning the boost cache to sqlite. — *edit: the error now only appear on time to time in the backend. SQLite3Stmt::execute(): Unable to execute statement: UNIQUE constraint failed: cache.id — This is the SQLite version running on my machine: Array ( [versionString] => 3.32.3 [versionNumber] => 3032003 )

bnomei commented 2 years ago

i think i found the issue and managed to fix it. thanks for pointing me into that direction. you need to update the sqlite cache driver.

https://github.com/bnomei/kirby3-sqlite-cachedriver/releases/tag/v1.4.0

psntr commented 2 years ago

Thank you so much for looking into that and indeed, after the update the error is gone!

There was actually another thing I wanted to ask you last time and I hope it might help other as well.

It's regarding the example you made in your collections.

For instance, if I am in the panel, and inside article post type and want to have a relation the post type member, but by using the default collections example, both article and member are going to be shown.

If I edit the boostidpages.php as an example:

return function ($site) {
    $drafts = option('bnomei.boost.drafts');
    // Here I filter the pages I want by the filterBy method to be shown in the my member page
    return $site->index($drafts)->filterBy('template', 'article')
    ->filter(function ($page) {
        return $page->hasBoost() === true;
    });
};

and as far as I understand, this method is slower than using the boostidkvs.php method because we rely on the $site->index()?

Is it possible to execute filterBy or any other methods in boostidkvs.php?

bnomei commented 2 years ago

yes. every time you use the collection it will filter again and thats not the most performant solution but having boost is allready a big win. you could do a few things but caching tends to get complex quick.

1) cache the filtering with raw kirby https://forum.getkirby.com/t/caching-pages-collections/13021/2?u=bnomei

2) use my lapse plugin to cache the filter (which takes care if rhe modification check) https://forum.getkirby.com/t/help-needed-troubleshooting-slow-panel-query-multiselect-field/23685/8?u=bnomei

3) in addtion to 2) you could wrap it in a static class to make sure the collection is only built once per request. https://forum.getkirby.com/t/remember-this-caching-in-kirby/23705/4?u=bnomei

bnomei commented 2 years ago

Is it possible to execute filterBy or any other methods in boostidkvs.php?

no. but you could create another collection with same code and filter in that.

bnomei commented 2 years ago

Is it possible to execute filterBy or any other methods in boostidkvs.php?

no. but you could create another collection with same code and filter in that.

sorry thats nonsense. ignore it. the kv is from cache. you can not filter.

psntr commented 2 years ago

Thank you for information, it's good to know. At my level and currently project I will go with the boostidpages.php but will definitely try the (2) Lapse+KV method seems to be the easiest the implement.

Again thank you for the help! Highly appreciated

psntr commented 2 years ago

While continuing to explore the plugin, I just encountered this error after playing around:

SQLite3::exec(): database is locked

I guess this is a work in progress for you:

    private function applyPragmas(string $pragmas)
    {
        // TODO: recover from SQLite3::exec(): database is locked
        foreach ($this->options[$pragmas] as $pragma) {
            $this->database->exec($pragma);
        }
    }

Is there a way to unlock it manually?

bnomei commented 2 years ago

yeah that can happen if plugin did not properly close the connection due to a timeout etc. but i encountered it rarely. since everything in boost is just a cache you can simply delete the sqlite file.

or you could try sqlite3 mydata.db ".dump" | sqlite3 new.db or sqlite3 broken.db ".recover" | sqlite3 new.db https://stackoverflow.com/questions/18259692/how-to-recover-a-corrupt-sqlite3-database

psntr commented 2 years ago

Thank you for the explanation!