[BUG] LOG viewer crashed

KITT3000 commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior

I checked the downloaded LOG file, then I didn't reload it to my LOG, I just used the program to change active mods, etc. I deleted the file with this LOG file and now when I try to check, nothing wants to load in this window, I can't click any button in this browser window, after a short while the window turns completely white. I remember that the log was about 50MB and there was also a problem with viewing it, scrolling was terribly laggy.

Expected Behavior

No response

Steps To Reproduce

No response

Environment

- Mod Assistant Version: 2.9.0
- OS: Win 10 22H2
- Virtualized: No

Anything else?

No response

jtsage commented 1 year ago

Huh. gross. So, my notes below, but included cause you might be interested.

1.) The log is read sync right now - this was great for testing, but super bad for production. Needs fixed.

2.) A 50MB log is bad, as you know - but it shouldn't crash. There are two ok-ish options here, both with their downsides:

Truncate the log file and only return the last couple MB or so. The downside is you only have part of the log in the window then - fine if your running game is spamming, but bad for loading a log from discord.
Turn off the color regexs when it is massive, this should speed things greatly.

I'm leaning towards a two-fold solution -

Scenario A : you load a single massive log, it displays, but without color.
Scenario B : you load the game and it gets massive. Once it hits 10MB or so, it starts only sending the end of the log to the file. I think I can do this by throwing a "is updating" flag in the bit that watches the open log file for changes.

Thoughts?

KITT3000 commented 1 year ago

Currently I can't do anything to the window, nor change filters or anything else, only minimize the window and close it. As for the solution, I noticed that usually the end of the file is repeated sections with "LUA call stack", I think it would be best to hide it somehow (show only one section and collapse the rest with the possibility of expanding), or cut/stop loading the file after several such entries.

jtsage commented 1 year ago

Aye. the issue is, the render portion takes forever to do it's thing - it's actually reading and sending the file to the window very, very quickly - then the bit to parse through it is super slow. I'm working on some ways to greatly speed that up, including some automated truncation for stuff like that

jtsage commented 1 year ago

sigh. auto-close. This is why you don't use "Fix" in a commit message. lol.

Anyway, what I ended up finding is that it was neither the file IO nor the regex parsing that was the limiting factor here - it's that asking chromium to display a few hundred thousand table rows is bad. So, what it actually does is render the 20 or so that you can see in the window. And when you scroll, it grabs the appropriate lines that you see now, etc, etc - the bonus is, the filtering and finding code is much, much cleaner, as it is operating on a data structure rather than a DOM structure. In my tests, the 51Mb log file I made by pasting my regular log a few dozen times together ran every bit as fast as a tiny log file did - with the caveat that it took around a second to load when it hit the window. If your running, constantly updating log file is that big, you might notice the lag at some point, but the debouncer also doesn't load the log that fast (I think there is a 5 second delay built in there).

Anyway, this will be in the next version, whenever that may be (probably a day or two, I don't have much else cooking, but I do want to update the dependencies again)

KITT3000 commented 12 months ago

I found some large LOG files so you can test them yourself logs.zip

jtsage commented 12 months ago

you rock, thank you!

jtsage commented 12 months ago

https://github.com/FSGModding/FSG_Mod_Assistant/assets/605986/e57657c4-4806-45b2-b3d5-ee865b7fd72e

not too shabby now.

KITT3000 commented 12 months ago

Nice

jtsage commented 12 months ago

If you are feeling adventurous - https://github.com/FSGModding/FSG_Mod_Assistant/releases/tag/v2.9.92

The new log viewer is in there. The other huge update is the mod parser is multi-threaded and runs in the background now. Feels faster, and it is way less resource intensive.

jtsage commented 11 months ago

Update release in 3.0.0 - lemme know if you still have issues. Seems pretty sorted to me - thanks for your continued interest in testing and improving this :)

FSGModding / FSG_Mod_Assistant