Large delay when accessing /history

[LunNova]

See https://github.com/foosel/OctoPrint/issues/1241#issuecomment-243376615

Causing print stuttering issues when accessing the web interface with lots of history entries.

Is it possible to avoid accessing /history until the history tab is actually opened? That would be an immediate quick fix as it shouldn't need that data when not on the history tab.

[LunNova]

Tried to fix this: https://github.com/nallar/OctoPrint-PrintHistory/commit/19fe06ddf8935736f3aa3c5c1686b1c489463860

Haven't been able to test that code yet as I don't have octoprint set up here.

[foosel]

@imrahil any ETA when this will be fixed? I keep getting reports from people with performance issues on their machines with this plugin installed, this is starting to become a real issue.

[foosel]

@imrahil I have marked the plugin as disabled in the repository for now (see commit message above) which makes it show up as incompatible in the built-in repository browser and displays a message on the plugin's repo page. I'll happily re-enable it once this issue is taken care of! I just don't want more people to run into this (apparently very serious and sometimes even print ruining) issue. I'll also have to put out a more public notice about this, since it is really hurting some people with huge histories and that's now backfiring on OctoPrint. I hope you understand this - it's really not to discourage you or anything!

[imrahil]

Patch from @nallar looked good so I use it. Thanks Gina for good motivator :)

[foosel]

Perfect, re-enabled the plugin in the repo and sent a follow-up tweet.

Since that /history endpoint appears to be quite resource intensive for large amounts of historical data, I'd suggest to look into setting some caching headers on the outgoing response to allow the browser to aggressively cache it. At least the Last-Modified header based on when the last log was added. The last modification time of the history file itself would probably already be enough. Take a look at e.g. here for how I do this in OctoPrint. Maybe even switching to some sort of SQLite DB or similar, because a single YAML file only scales so much :/

[imrahil]

aghhh... can't get it working.... your link @foosel (with_revalidation_checking in settings) doesn't work for plugins (or I don't understand something) I found similar use case in AnnouncementPlugin here - but still get Flask errors or my view() function is completely ignored... current version - https://gist.github.com/imrahil/41c93715ba19fc70de50e630ca9e54f4

[foosel]

From the looks of it, you are decorating the view there successfully. And then you try to call the view result (a response object). It's not a function so that will fail. I'd think the exception you are seeing should tell you something to the tune of it not being a callable too.

But: I'm actually in vacation mode, so I haven't tested this claim, just looked at your excerpt. I can definitely tell you though that this stuff works with plugins just fine, or none of the bundled plugins in 1.3.0 utilising it would still work ;)

[imrahil]

huh... today this code works like expected :) magic! added 304 HTTP response - aggressive caching is awesome. Added "Force refresh" button in UI new release is under way!

[imrahil]

latest version migrated to SQLite database so large histories should be handled with easy now :)