search

cheahjs / palworld-save-tools

Tools for converting Palworld .sav files to JSON and back
MIT License
790 stars 71 forks source link

Investigation: Can numpy improve parsing performance #156

Open cheahjs opened 7 months ago

cheahjs commented 7 months ago

Investigate if using numpy to read and parse data structures could be faster than the current solution.

magicbear commented 7 months ago

no, you may check for the way that I used to load it by multiprocessing. MPMapPropertyProcess MPMapProperty

magicbear commented 7 months ago

24% performace from struct.Struct to int.from_bytes (like numpy)

def perf(): ... t1 = time.time() ... for _ in range(100000000): ... struct.Struct("I").unpack(b"\x00\x00\x00\x00") ... print(time.time()-t1) ...

perf() 8.79693603515625 def perf(): ... t1 = time.time() ... for _ in range(100000000): ... int.from_bytes(b"\x00\x00\x00\x01", byteorder="little") ... print(time.time()-t1) ...

perf()

AntiMoron commented 7 months ago

a suggestion: rewrite in rust. [[[[[