converting error - Githubissues

getteq commented 5 months ago

I don't know what the hell is going on, but suddenly it failed to convert it

I use docker image and this is what my server's staus is https://palworld.chanhome.xyz As you see this, container failed to convert save file. I've purged and re installed this container several times so my monitoring services lost its backup data..... but before 2weeks ago, this palworld-server-tool just working great.
Now I can monitor information like steamid guid, player name, but i can't monitor their inventory and other things. I really whish to use this again, so please give me some hint for me.

*Note, I use auto save repair tool (https://github.com/YDKK/palworld-save-repair) which made by ydkk and actually several bugged sav file already purged Maybe i missed something, but i already tried something about broken player's sav file. IDK which file actually make PST tool bugged

docker log

This my server's save file and IDK why it's not able to convert it. [[file deleted]]

zaigie commented 5 months ago

I downloaded your game archive file and ran it in my own docker container without any problems, here is the picture after running...

Combined with your log, the conversion always ends in a very short time with no error message, and I have not encountered this situation.

To further understand the problem, please tell me the server hardware configuration you use, the system architecture, and the environment variables you specify when docker run

getteq commented 5 months ago

It's far different that what I expected! My server ran on my nas server which consited by i5 10400f h410m ddr4 64gb OS:OMV6(NAS OS) ssd samsung evo 250gb

스크린샷 2024-03-17 185058 스크린샷 2024-03-17 185234

I'll check file permissin right now....

zaigie commented 5 months ago

If you have access to the container's shell, try running the archive parser directly to see its output. After entering the interactive terminal of the container, you can run the following command:

# docker exec -it ......
./sav_cli -f /game/SavedGames/0/......./Level.sav -o structure.json

getteq commented 5 months ago

[SAV-CLI] 2024/03/17 - 10:12:00 | INFO | Converting... Traceback (most recent call last): File "sav_cli.py", line 31, in File "structurer.py", line 185, in convert_sav FileNotFoundError: [Errno 2] No such file or directory: '/game/SavedGames/0/C396 1BA1EDEF4C529F3EA01E61DEB12C/Level.sav' [3120] Failed to execute script 'sav_cli' due to unhandled exception! root@openmediavault:~#

......maybe I made very simple mistake when i purge docker before and make again. I'll check what file location actually made it is....

getteq commented 5 months ago

wait a minute is your directory ./sav_cli -f /game/SavedGames/0/......./Level.sav -o structure.json is correct? not ./sav_cli -f /game/SaveGames/0/......./Level.sav -o structure.json? I changed it to ./sav_cli -f /game/SaveGames/0/......./Level.sav -o structure.json and succesfully working now

getteq commented 5 months ago

스크린샷 2024-03-17 192206

I don't know what the hell is going on now but yeah it's converting correctly

zaigie commented 5 months ago

It was weird, it seemed like it ended without warning or error

getteq commented 5 months ago

It was weird, it seemed like it ended without warning or error

https://palworld.chanhome.xyz And you know, actually my server is still online, I just double checked file permission and file location now and still can not read sav file.And basically my sav file is able to work on your system right?

getteq commented 5 months ago

Just I wonder, 2024/03/17 - 05:59:20 | ERROR | task/task.go:71 | error waiting for command: signal: killed What does it means? I think it is the main cause of this problem, I already purged currupted save file because I've already see this (https://github.com/zaigie/palworld-server-tool/issues/181) And maybe sav file is clean i think....

zaigie commented 5 months ago

Your problem is different from #181, it will show a more obvious error, but yours does not have any error, error waiting for command: signal: killed actually means "ended unexpectedly without warning/error".

So I think the root cause of the problem may be a system difference, and further, it may be a third-party library that I use to parse the archive, palworld-save-tools.

zaigie commented 5 months ago

Of course, the easiest way is to put pst on any other machine, and deploy PST-Agent to the machine with palserver.

https://github.com/zaigie/palworld-server-tool/blob/main/README.en.md#agent-deployment

getteq commented 5 months ago

Yeah... basically my server's os is Open media vault6, so it is not desirable os to run server actually.... I think my case is very very rare case to find out maybe.
If things not working really well, you know, oracle free tier may help me. I'll try later.

getteq commented 5 months ago

Of course, the easiest way is to put pst on any other machine, and deploy PST-Agent to the machine with palserver.

https://github.com/zaigie/palworld-server-tool/blob/main/README.en.md#agent-deployment

Ok, so basically it's my machine's problem. IDK what actually cause such problem, May be I just need to reboot my server now since i run this server without reboot for 5month actually. But in VM on my free google cloud account it's successfully convert it locally. I'll just consider this issue as natural disaster. I'll try PST agent method and leave result later

getteq commented 5 months ago

Your problem is different from #181, it will show a more obvious error, but yours does not have any error, error waiting for command: signal: killed actually means "ended unexpectedly without warning/error".

So I think the root cause of the problem may be a system difference, and further, it may be a third-party library that I use to parse the archive, palworld-save-tools.

Of course, the easiest way is to put pst on any other machine, and deploy PST-Agent to the machine with palserver.

https://github.com/zaigie/palworld-server-tool/blob/main/README.en.md#agent-deployment

http://20.249.172.245:8080/ Not Perfect solution, but at least working thx

getteq commented 5 months ago

Can't reproduce on other machine but can bypass the bug. Maybe not a big deal actually. For me, I solved it at least.

pepsizerosugar commented 5 months ago

@getteq, @zaigie

The reason is that cmd.Wait() in 'Decode' at line 75 of /internal/tool/save.go generates an error due to out of memory.

If you have set memory options in Docker execution options, you must delete those settings or let loose a bit.

The current deployment guide to "-m 256M --memory-swap=4G" as optional, and if you set that option, you must delete it and run the Docker container again.

getteq commented 5 months ago

@getteq, @zaigie

The reason is that cmd.Wait() in 'Decode' at line 75 of /internal/tool/save.go generates an error due to out of memory.

If you have set memory options in Docker execution options, you must delete those settings or let loose a bit.

The current deployment guide to "-m 256M --memory-swap=4G" as optional, and if you set that option, you must delete it and run the Docker container again.

I'll check it in my old system.

getteq commented 5 months ago

@getteq, @zaigie

The reason is that cmd.Wait() in 'Decode' at line 75 of /internal/tool/save.go generates an error due to out of memory.

If you have set memory options in Docker execution options, you must delete those settings or let loose a bit.

The current deployment guide to "-m 256M --memory-swap=4G" as optional, and if you set that option, you must delete it and run the Docker container again.

You were right, and I can reproduce error when I added "-m 256M --memory-swap=4G" make such error again.

zaigie commented 5 months ago

This parameter that restricts memory operation was originally a solution submitted by a contributor to prevent excessive memory consumption in a version below v0.5.2.

It seems that it is no longer needed today when the archive parsing tool is optimized.

I will modify README.md to remind everyone to use this optional parameter carefully or not.

Thanks for your attempt and solution.

getteq commented 5 months ago

Well I think I find out what actually heppend in my system Screenshot_20240319_105122_Chrome when converting, ram usage rise up to 500-600mb and it cause OOM.

getteq commented 5 months ago

Actually, I tried to reproduce this error in many other case and in AWS lightsail instance, I can make very simmillar error Instance: 1GBRAM, 2 vCPU, 40GB SSD

Amazon Linux 2

서울, 영역 A (ap-northeast-2a)

2024/03/19 - 02:02:08 | INFO | source/http.go:15 | downloading sav.zip from http://chanhome.xyz:5975/sync 2024/03/19 - 02:02:08 | INFO | source/http.go:52 | sav.zip downloaded and extracted [SAV-CLI] 2024/03/19 - 02:02:09 | INFO | Converting... Traceback (most recent call last): File "sav_cli.py", line 31, in File "structurer.py", line 188, in convert_sav File "palworld_save_tools/gvas.py", line 131, in read File "palworld_save_tools/archive.py", line 291, in properties_until_end File "palworld_save_tools/archive.py", line 304, in property File "palworld_save_tools/archive.py", line 415, in struct File "palworld_save_tools/archive.py", line 442, in struct_value File "palworld_save_tools/archive.py", line 291, in properties_until_end File "palworld_save_tools/archive.py", line 301, in property File "structurer.py", line 65, in skip_decode File "palworld_save_tools/archive.py", line 165, in read MemoryError [40] Failed to execute script 'sav_cli' due to unhandled exception! 2024/03/19 - 02:02:15 | ERROR | task/task.go:71 | error waiting for command: exit status 1 2024/03/19 - 02:02:15 | INFO | task/task.go:73 | Sav sync done 2024/03/19 - 02:03:08 | INFO | task/task.go:42 | Scheduling Rcon sync...

getteq commented 5 months ago

Since this instance only has 1gb ram and share ram with other thing like nginx and etc, I think this is just caused by out of memory

MapleEve commented 5 months ago

Although the parsing library for this reference has been optimized for reading and writing as much as possible, the system will still use as much memory and virtual memory as possible when parsing archives larger than 20MB. The current conservative approach is to allocate 2GB of backup memory (including virtual memory).

zaigie / palworld-server-tool

converting error #193