PhrozenByte
commented
4 years ago We could also simply store the parsed contents of a page (both YAML and Markdown) in simple PHP files. However, in practice we don't need the data of all pages at a time anyway. Most of the time we just need the data of the requested page and the titles of pages shown in the navigation. That's it.
Thus we're planning to overhaul the page discovery process and implement a directory-tree-based page discovery with Pico 3.0. We did some groundwork for this with Pico 2.0 by implementing Pico's page tree and allowing pages and directories to be hidden. But we didn't change anything about how Pico 2.0 discovers pages; this is planned for Pico 3.0.
This alone should solve most performance issues that arise with many pages. Since we're also planning to explicitly support caching plugins, there should be no reason why adding a caching functionality utilizing a SQLite database shouldn't be possible.
marcus-at-localhost
Hi, I was thinking a bit about the issue of filesystem access as bottleneck for larger sites (like described here: https://github.com/picocms/Pico/issues/490)
I love Pico as a simple blogging engine, but I can see a slow down as the site grows.
Wouldn't be the use of SQLite (probably as plugin) a solution to store all (YAML) metadata already parsed as JSON and therefor access it faster for listing pages? I know, this is against the "Flat File Philosophy" of Pico, but at the end - SQLite is also just a single file in the file system with the advantage of indexes and maybe even to provide a search functionality (although, it's probably better to use an external full text search).
Still, this needs a file system lookup if a file has changed (this could be done once after publishing/changing an article) and I wonder if this would be the same as loading the DB etc.
This is more a rhetorical question what you as dev think about this approach in general and what your experience is with working with the file system.
Thanks for your thoughts.