Parsing log files doesn't handle encoding

[realgdubz]

Hi, I am trying to use your bot I used it in the past and loved it, so many awesome features. I tried setting it up again for my server, but I can't seem to get it to work. I don't know what I'm doing wrong I was hoping you could help me. I am using Python 3.10 and I have also tried other versions and no luck. It looks like the bot is starting normal, but the rcon doesn't show the amount of players and logs don't work also.

C:\Users\DonJulioPC\Downloads\zomboi-master>python zomboi.py Python-dotenv could not parse statement starting at line 39 [2022-11-21 12:36:44] [INFO ] discord.client: logging in using static token [2022-11-21 12:36:45] [INFO ] discord.gateway: Shard ID None has connected to Gateway (Session ID: ffbae5a4eb20d92eb71a7fa1a9553f6d). 2022-11-21 12:36:47,613:INFO:zomboi: We have logged in as Nature's Vengeance Status#8484 2022-11-21 12:36:47,614:INFO:zomboi: channel connected 2022-11-21 12:36:47,614:INFO:zomboi: Loading user history... 2022-11-21 12:36:47,615:INFO:zomboi: User history loaded 2022-11-21 12:36:47,615:INFO:zomboi: Loading Perk history... 2022-11-21 12:36:47,616:INFO:zomboi: Perk history loaded 2022-11-21 12:36:47,616:ERROR:zomboi: Map path None not found and/or no suitable default 2022-11-21 12:36:47,618:INFO:zomboi: Checking rcon to see if the player counter is correct 2022-11-21 12:36:47,731:INFO:zomboi: Synced players successfully!

[UrbanSide]

Bot working,but chat not work

[realgdubz]

Nothing is working.

[UrbanSide]

only the chat does not work for me ... the map, perks, the entry and exit of players work ... but the chat does not (((

[realgdubz]

Can you tell me what version of python you are using?

[realgdubz]

Its very strange, bot doesn't put out any help commands or show any logs, nor the players online with correct rcon info

[UrbanSide]

The most recent, 3.11 like ..

[UrbanSide]

The bot i use is from the fork https://github.com/a7xpadd3r/zomboi , but this one also worked, except for the chat.

[realgdubz]

Oh wow I didn't even know there was a fork of it. I will try that one, and post back thanks for the fast reply!

[UrbanSide]

Ok,but I still don’t understand how to start the chat, I don’t understand English well, maybe I didn’t setup bot correctly.

[realgdubz]

I found the issue, So in my env I have a custom path I don't use the default zomboid path for some reason it doesn't like when I use a different hard drive or my path. my log path: E:\Server1_Data\Zomboid\Logs\

[UrbanSide]

Working?Does the chat work?

[UrbanSide]

Can you help if it's not difficult to set up? My game is stored along the path D:\Servers\PZ_Vanilla and the folder with configs and saves along the path D:\Servers\PZ_Vanilla\UData\Zomboid Config file is like this:

# Sample configuration file. Rename to ".env" and customise with your settings

# The token for your discord bot
DISCORD_TOKEN="TOKEN"

# The id for the channel you want notifications and chat
# If this is a channel name it will use the first channel it finds
# matching that name, or it can be the discord channel ID which
# you can get by right clicking on the channel -> copy ID
# or from the channel link, e.g: https://discord.com/channels/<server_id>/<channel_id>
CHANNEL="822380630167519244"

# Set to "True" if you want Admin logs to be sent to the channel below.
# These logs give info about what players are doing and could be useful for server admins
# Defaults to False unless this is set to True
ADMIN_LOGS="True"

# The id for the channel that you want admin/other logs to be sent to
ADMIN_CHANNEL="1044374078062075944"

# The notifications for the above channel
JOINS=True
DISCONNECTS=True
DEATHS=True
PERKS=True
CREATECHAR=True

# RCON Host
RCON_HOST="localhost"

# The password for rcon on the server (used to relay chat from discord to game)
RCON_PASSWORD="PASSS"

# The port to use for the rcon connection, default is 27015
RCON_PORT=27015

# Path to Project Zomboid Logs folder
# Leave empty to try default paths
LOGS_PATH="D:\Servers\PZ_Vanilla\UData\Zomboid\Logs"

# Path to the project zomboid maps folder
# leave empty to try the default paths
MAPS_PATH="D:\Servers\PZ_Vanilla\media\maps"

# Path to the project zomboid saves folder
# leave empty to try the default paths
# i.e. ~/Zomboid/Saves/Multiplayer/server_name
SAVES_PATH="D:\Servers\PZ_Vanilla\UData\Zomboid\Saves\Multiplayer\SmollNet"

# Roles ID allowed to use commands, separated by ";"
ROLESID_SERVERCMD="763704069310906368"
ROLESID_PLAYERCMD="763704069310906368"
ROLESID_VEHICLECMD="763704069310906368"
ROLESID_ACCESSLEVEL="763704069310906368"

# Roles ID allowed to config this bot. ALLOW AT YOUR OWN RISK! Administrators can use this without restrictions.
ROLESID_BOTCONFIG="763704069310906368"

[UrbanSide]

Or is the chat still not working?

[realgdubz]

[2022-11-23 01:05:54] [ERROR ] discord.client: Ignoring exception in on_ready Traceback (most recent call last): File "C:\Users\DonJulioPC\AppData\Local\Programs\Python\Python38\lib\site-packages\discord\client.py", line 409, in _run_event await coro(*args, **kwargs) File "zomboi.py", line 68, in on_ready await zomboi.add_cog(UserHandler(zomboi, logPath)) File "C:\Users\DonJulioPC\Desktop\zomboi-master\zomboi-master\users.py", line 39, in init self.loadHistory() File "C:\Users\DonJulioPC\Desktop\zomboi-master\zomboi-master\users.py", line 115, in loadHistory for line in f: File "C:\Users\DonJulioPC\AppData\Local\Programs\Python\Python38\lib\encodings\cp1252.py", line 23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0] UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 707: character maps to

[realgdubz]

My env file is also like yours

[UrbanSide]

Hmm ... I repaired it with some kind of file from the answers in the errors ....

[UrbanSide]

https://dropmefiles.com/rEmu1 its my bot files,Try it, it might work, just use your own config file

[realgdubz]

Just tried that, and same error for some strange reason :(

[realgdubz]

This was on 3.8 going to try 3.11

[realgdubz]

Okay updated to 3.11 and same error did some testing it looks like its a file in my save causing this because changing to server1_data to sever_test causes no errors.

[realgdubz]

2022-11-23 01:22:23,134:INFO:zomboi: We have logged in as Nature's Vengeance Status#8484 2022-11-23 01:22:23,134:INFO:zomboi: channel connected 2022-11-23 01:22:23,134:INFO:zomboi: Loading user history... [2022-11-23 01:22:23] [ERROR ] discord.client: Ignoring exception in on_ready Traceback (most recent call last): File "C:\Users\DonJulioPC\AppData\Local\Programs\Python\Python311\Lib\site-packages\discord\client.py", line 409, in _run_event await coro(*args, **kwargs) File "C:\Users\DonJulioPC\Desktop\zomboi-master.zip_pass_123\zomboi-master\zomboi.py", line 68, in on_ready await zomboi.add_cog(UserHandler(zomboi, logPath)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\DonJulioPC\Desktop\zomboi-master.zip_pass_123\zomboi-master\users.py", line 39, in init self.loadHistory() File "C:\Users\DonJulioPC\Desktop\zomboi-master.zip_pass_123\zomboi-master\users.py", line 115, in loadHistory for line in f: File "C:\Users\DonJulioPC\AppData\Local\Programs\Python\Python311\Lib\encodings\cp1252.py", line 23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 707: character maps to

[UrbanSide]

UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 707: character maps to

Hmm ... damn magic ...

[UrbanSide]

I don’t understand anything ... Everything works for me ... except for the chat ... it doesn’t work for you in general ...

[UrbanSide]

@JonnyPtn Can you help us?) Your bot is awesome, but something is wrong with it...

[realgdubz]

@JonnyPtn Yes please, I run a zomboid community with over 400 people in our discord, and we love your bot, and would love it working again

[JonnyPtn]

Hey, not sure I understand the issue here as it got lost in the chat a little bit - based on your original message the only bit that stands out is: Python-dotenv could not parse statement starting at line 39 so what's line 39 in your env file?

[realgdubz]

LOGS_PATH="E:\Server1_Data\Zomboid\Logs\"

[realgdubz]

when i do LOGS_PATH="E:\Server1_Data\Zomboid\Logs"

I get this error.

[realgdubz]

2022-11-23 15:36:18,954:INFO:zomboi: Loading user history... [2022-11-23 15:36:19] [ERROR ] discord.client: Ignoring exception in on_ready Traceback (most recent call last): File "C:\Users\DonJulioPC\AppData\Local\Programs\Python\Python311\Lib\site-packages\discord\client.py", line 409, in _run_event await coro(*args, **kwargs) File "C:\Users\DonJulioPC\Desktop\zomboi-master\zomboi-master\zomboi.py", line 68, in on_ready await zomboi.add_cog(UserHandler(zomboi, logPath)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\DonJulioPC\Desktop\zomboi-master\zomboi-master\users.py", line 39, in init self.loadHistory() File "C:\Users\DonJulioPC\Desktop\zomboi-master\zomboi-master\users.py", line 115, in loadHistory for line in f: File "C:\Users\DonJulioPC\AppData\Local\Programs\Python\Python311\Lib\encodings\cp1252.py", line 23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 707: character maps to

[realgdubz]

Lets say If i change the path to when i do LOGS_PATH="E:\Server2_Data\Zomboid\Logs" a folder thats empty the bot starts no errors.

[JonnyPtn]

Looks like there is a character in your user log file that is failing to decode. I think there might be a fix I can try for this, although it would be helpful to work out which character it is to properly test - are you able to upload that user log file to pastebin or similiar?

[realgdubz]

Your right I deleted the logs and the bot is working now, one thing that is a problem thought is Traceback (most recent call last): File "C:\Users\DonJulioPC\AppData\Local\Programs\Python\Python311\Lib\site-packages\discord\ext\commands\bot.py", line 1347, in invoke await ctx.command.invoke(ctx) File "C:\Users\DonJulioPC\AppData\Local\Programs\Python\Python311\Lib\site-packages\discord\ext\commands\core.py", line 986, in invoke await injected(*ctx.args, **ctx.kwargs) # type: ignore ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\DonJulioPC\AppData\Local\Programs\Python\Python311\Lib\site-packages\discord\ext\commands\core.py", line 199, in wrapped raise CommandInvokeError(exc) from exc discord.ext.commands.errors.CommandInvokeError: Command raised an exception: HTTPException: 400 Bad Request (error code: 50035): Invalid Form Body In content: Must be 2000 or fewer in length.

[realgdubz]

I can't look up some players stats

[JonnyPtn]

Yeah that's #8 - discord has a limit of 2000 chars per message which the bot hits sometimes.

If it's OK with you I'll modify this issue to represent the problem with encoding in the log files, as that's something I'd like to fix or at least handle gracefully. Do you have any idea what might have caused the issue? e.g. you do use any non-english languages on your server or any special characters in usernames?

[realgdubz]

That is fine, I appreciate the fast response. Also for the bot hitting 2000 characters is there a way to have the bot maybe send 2 messages so that doesn't happen? Appreciate your hard work.

[JonnyPtn]

Yeah that's likely how that problem needs to be solved - #45 was an attempt to fix it but hasn't really been tested, so if you are able to test that it would be a great help