[Feature Request / Discussion] Reducing the load time of the main page

[RadianttK]

Context

• Currently the Jimaku home page loads in every entry from the database all at once, much the same way as Kitsunekko does.
  • As Kitsunekko does not have a search bar, this is useful as it gives the user the option to search for a particular entry using ctrl + f.
• Since Jimaku has a proper search bar, it's not as necessary to load in every single entry when loading the main page.
• Currently when loading Jimaku it takes me approximately ~7 seconds to load on average.
  • I think it would significantly improve the user experience if this was reduced.

Ideas for possible solutions

1. Deferred loading

   • Only load entries when the user initiates a search.
   • This reduces the initial load time significantly as it avoids fetching and rendering all entries at once.

2. Data caching

   • Cache subtitle entries locally in the browser after the initial load.
   • Subsequent visits or searches can then use the cached data, reducing the need for frequent database queries.
     • I assume this would also require a mechanism to periodically update the cached data to reflect changes in the database, such as modified or deleted entries.

3. Incremental loading

   • Only load a portion of the subtitle entries initially.
     • As the user scrolls, dynamically load additional entries from the database in smaller batches.

4. Data pagination

   • Implement pagination for the main page to display entries in manageable chunks.
   • Only a set number of subtitles would load on a page, and the user could then click to go to the next page to see more.
   • This distributes the load evenly and prevents overwhelming the browser with a large amount of data at once.

5. Server-side caching

   • Server-side caching mechanisms (e.g., Memcached or Redis) to store frequently accessed data and reduce database load.

[Rapptz]

I'm aware of this but I'm not planning on changing this any time soon. I'm not really optimizing for the bad internet use case currently. Again, I want to stress that the search is entirely client sided. The reason it works is because it loads everything at once. There is no server side search component, the API's search mechanism is completely different from the one on the site.

That being said:

1. This would not work, because again, the search is client sided. It just does DOM filtering.
2. This is already a thing for logged out users. Logged in users have no caching set up since the template is dynamic for those users (i.e. the username display and how the site itself changes).
3. This is possible, but the moment you start searching the experience is diminished completely.
4. Same as above.
5. Already done for logged out users.

For logged out users there's already caching and compression set up so it's not a big deal for those users. One thing I might explore is server side on-the-fly compression to reduce the amount of data being transferred regardless of being logged in or out, but other than that I'm not sure what else to do in the interim.

[RadianttK]

Yeah, I agree deferred loading would not work with client side searching. The searching would need to be moved to the server side, although I imagine that would be a fairly large undertaking.

I didn't realise that the caching is already being done for logged out users.

• This alone seems to reduce the loading time to around ~2 seconds for me.
• How much work do you think it would be to apply this for logged in users?

[Rapptz]

I've enabled on-the-fly compression for all responses as a trial. I'm not sure it's giving me much of a benefit personally but it might be better for people with subpar connections:

Before:

After:

You'll see that for me it's actually slower (in terms of DOM load), I suspect that's because the server has to do more work but it's negligible so it doesn't matter much.

[RadianttK]

Hmm, interesting. For me it reduced the load time from ~7 seconds to ~2.5 seconds, the same as when I am not logged in. That's quite a large improvement!

I tried pinging jimaku.cc just in Command Prompt and got an average round trip time of 237ms, so I assume it's basically as optimised as it can get now.

Thanks!

[Rapptz]

Yeah, ping is another factor. Right now the server is hosted in US Central so for me the ping isn't too bad (around 25ms). Anyway, I'm glad this helped your loading speeds.