Getting dashboard crash when new savegame is generated in Stellaris 3.10.4, log added below

[Abolohit]

2023-12-15 17:41:29,712 - stellarisdashboard.parsing.saveparser - INFO - Reading save file C:\Users\Abolohit\Documents\Paradox Interactive\Stellaris\save games\unitednationsofearth-15512622\ironman.sav. 2023-12-15 17:41:31,197 - stellarisdashboard.parsing.saveparser - INFO - Parsed save file C:\Users\Abolohit\Documents\Paradox Interactive\Stellaris\save games\unitednationsofearth-15512622\ironman.sav in 1.485 seconds. 2023-12-15 17:41:32,167 - stellarisdashboard.parsing.timeline - INFO - unitednationsofearth-15512622 2205.01.28 Processing Gamestate 2023-12-15 17:41:32,387 - stellarisdashboard.parsing.timeline - INFO - unitednationsofearth-15512622 2205.01.28 Gamestate for same date already exists in database. Aborting... Traceback (most recent call last): File "stellarisdashboard__main.py", line 32, in File "stellarisdashboard__main__.py", line 24, in main File "stellarisdashboard\cli.py", line 84, in f_monitor_saves File "stellarisdashboard\dashboard_app\visualization_data.py", line 95, in get_current_execution_plot_data File "stellarisdashboard\dashboard_app\visualization_data.py", line 238, in update_with_new_gamestate File "stellarisdashboard\dashboard_app\visualization_data.py", line 301, in extract_data_from_gamestate File "stellarisdashboard\datamodel.py", line 678, in rendered_name File "stellarisdashboard\game_info.py", line 233, in render_name File "stellarisdashboard\game_info.py", line 245, in get_global_renderer File "stellarisdashboard\config.py", line 438, in localization_files File "stellarisdashboard\config.py", line 447, in File "stellarisdashboard\config.py", line 470, in _localization_file_matches_language File "encodings\cp1252.py", line 23, in decode UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 72: character maps to [7304] Failed to execute script 'main__' due to unhandled exception!

[chennin]

Look a lot like #125. Can you please attach the save?

Is your game in English, or another language?

Guessing 7304 is a Unicode ID; does this character look familiar in your game?

[Abolohit]

My game is in Engish, and that character does not look familiar to me, seems to be Chinese, which I don't know. I had deleted the previous save, here is a new save, alongwith an updated log: ironman.zip

2023-12-19 12:03:56,697 - stellarisdashboard.parsing.saveparser - INFO - Reading save file C:\Users\Abolohit\Documents\Paradox Interactive\Stellaris\save games\unitednationsofearth-15512622\ironman.sav. 2023-12-19 12:03:59,840 - stellarisdashboard.parsing.saveparser - INFO - Parsed save file C:\Users\Abolohit\Documents\Paradox Interactive\Stellaris\save games\unitednationsofearth-15512622\ironman.sav in 3.144 seconds. 2023-12-19 12:04:02,829 - stellarisdashboard.parsing.timeline - INFO - unitednationsofearth-15512622 2241.11.11 Processing Gamestate 2023-12-19 12:04:02,963 - stellarisdashboard.parsing.timeline - INFO - unitednationsofearth-15512622 2241.11.11 Gamestate for same date already exists in database. Aborting... Traceback (most recent call last): File "stellarisdashboard__main.py", line 32, in File "stellarisdashboard__main__.py", line 24, in main File "stellarisdashboard\cli.py", line 84, in f_monitor_saves File "stellarisdashboard\dashboard_app\visualization_data.py", line 95, in get_current_execution_plot_data File "stellarisdashboard\dashboard_app\visualization_data.py", line 238, in update_with_new_gamestate File "stellarisdashboard\dashboard_app\visualization_data.py", line 301, in extract_data_from_gamestate File "stellarisdashboard\datamodel.py", line 678, in rendered_name File "stellarisdashboard\game_info.py", line 233, in render_name File "stellarisdashboard\game_info.py", line 245, in get_global_renderer File "stellarisdashboard\config.py", line 438, in localization_files File "stellarisdashboard\config.py", line 447, in File "stellarisdashboard\config.py", line 470, in _localization_file_matches_language File "encodings\cp1252.py", line 23, in decode UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 1690: character maps to [21216] Failed to execute script 'main__' due to unhandled exception!

[chennin]

Thanks. It looks like an encoding error, but I can't reproduce the problem on Linux in en_US.UTF-8. Someone else might need to reproduce it on Windows.

[MichaelMakesGames]

Ah, I believe we need to explicitly open the locale files with utf-8 encoding. Apparently Python defaults to cp1252 on windows computers.

The first line of this func in config.py:

def _localization_file_matches_language(file_path: pathlib.Path, language: str):
    with open(file_path, "r") as f:
        first_line = f.readline()
        match = re.match(f"^\ufeff\s*{language}\s*:\s*$", first_line)
        return match is not None

I think just adding encoding="utf_8" would fix this, but I'm also on Linux so can't readily confirm. @eliasdoehne do you have the project set up on a Windows machine?

[eliasdoehne]

@eliasdoehne do you have the project set up on a Windows machine?

Yes, I have it set up in windows and can test this. (maybe I can try it today, but more likely after Christmas)

[eliasdoehne]

I just tried it and the fix seems to work. Before I reproduced the error

  File "....\stellaris-dashboard\stellarisdashboard\config.py", line 470, in _localization_file_matches_language
    first_line = f.readline()
  File "...\AppData\Local\Programs\Python\Python310\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 1690: character maps to <undefined>

With encoding="utf8" the page loads fine.

Insterestingly, while reading the save file, there are some warnings about some unexpected list data type, I will try to take a look at that in the next days.

2023-12-30 21:54:13,605 - stellarisdashboard.parsing.timeline - WARNING - debuggame 2246.01.10 country_power_projection: Found unexpected type list with value [1.48198, 1.48198].
2023-12-30 21:54:13,634 - stellarisdashboard.parsing.timeline - WARNING - debuggame 2246.01.10 country_base: Found unexpected type list with value [3.25, 0.25].
...

[kathith1234]

Hello. I have what appears to be the same issue, so I'll drop my log and save file below just in the case it's any help. Dashboard crashes whenever the game saves. This one is at the very beginning of an ironman run, though it seems to have the same issue on any.

Save file: ironman.zip

Log: 2024-02-14 19:48:48,254 - stellarisdashboard.parsing.timeline - INFO - kavariantribunal5_-319139440 2200.01.01 Processed Gamestate in 0.672 s, writing changes to database Traceback (most recent call last): File "stellarisdashboard__main.py", line 32, in File "stellarisdashboard__main__.py", line 24, in main File "stellarisdashboard\cli.py", line 84, in f_monitor_saves File "stellarisdashboard\dashboard_app\visualization_data.py", line 95, in get_current_execution_plot_data File "stellarisdashboard\dashboard_app\visualization_data.py", line 238, in update_with_new_gamestate File "stellarisdashboard\dashboard_app\visualization_data.py", line 301, in extract_data_from_gamestate File "stellarisdashboard\datamodel.py", line 678, in rendered_name File "stellarisdashboard\game_info.py", line 233, in render_name File "stellarisdashboard\game_info.py", line 245, in get_global_renderer File "stellarisdashboard\config.py", line 438, in localization_files File "stellarisdashboard\config.py", line 447, in File "stellarisdashboard\config.py", line 470, in _localization_file_matches_language File "encodings\cp1252.py", line 23, in decode UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 1690: character maps to [4640] Failed to execute script 'main__' due to unhandled exception!

[MichaelMakesGames]

Hey @eliasdoehne, any update on releasing a fix for this?

I also want to bring up again my offer to help maintain this project. I could do just bug fixes and releases, or could potentially do some feature work too. Let me know if you want to discuss.

[eliasdoehne]

Hi @MichaelMakesGames , you're right, I really can't commit the necessary time to keep the project going. I may still be able to put a few hours here and there on a weekend and maybe at some point I can be a bit more involved again (though probably not this year). So thanks a lot for offering some help.

I just invited you to the repo so you should be able to merge PRs and create releases without being blocked by me. I'll also remove the approval requirement from the master branch. In general, I don't mind if you want to build new features, I'm pretty open about the direction to take the project and I think you already had some great ideas.

Let me know if you need more access or have any questions about the setup.

[MichaelMakesGames]

Thanks! The benefits of open source :)

I have a busy couple weeks with some family stuff, so probably won't do much here, but I'll try to test + release for Stellaris 3.12 if I find time.

In the short term, I expect to just do maintenance and fixes. If I work on any features, I'll create GitHub issues or discussions so that you (or others) have a chance to provide input

[MichaelMakesGames]

@eliasdoehne Can you make me a collaborator for the Steam mod too? (I sent you a friend request on Steam.)

[eliasdoehne]

@eliasdoehne Can you make me a collaborator for the Steam mod too? (I sent you a friend request on Steam.)

Done, you should have access there.