Currently chrome-pak-customizer reads the whole pak file for unpacking, while allocating all the memory needed of pak file for packing. On early stage of this project hard disk drive (HDD) is widely used, which good at sequential reading or writing while has a slow random access speed, so the tradeoff is to read and write the whole pak file and then operate it in memory.
Nowadays solid-state drive (SSD) is taking over, which could have much better random access speed than HDDs, so maybe it is time for making the tradeoff optional, using mmap or MapViewOfFile can create a view of file managed by operating system, so the file is automatically read and written on reading and writing memory, which could save memory or make better performance in some cases.
Proposal
Add an command-line parameter to use mmap for reading and writing pak files whenever available.
Print availablity of mmaping for help.
Print a warning if the parameter is on and mmap is unavailable.
Use mmap for reading and writing pak files if the parameter is on and mmap is available by OS.
Unsafe could be used if needed.
Non-Goals
It is not a goal to implement this on branch develop or master
It is not a goal to make the parameter on by default.
It is not a goal to implement block-based reading and writing for pak files.
It is not a goal to use mmap for non-pak files, such as contents inside pak files and pak-index.ini
Target: branch 3.x
Description
Currently chrome-pak-customizer reads the whole pak file for unpacking, while allocating all the memory needed of pak file for packing. On early stage of this project hard disk drive (HDD) is widely used, which good at sequential reading or writing while has a slow random access speed, so the tradeoff is to read and write the whole pak file and then operate it in memory. Nowadays solid-state drive (SSD) is taking over, which could have much better random access speed than HDDs, so maybe it is time for making the tradeoff optional, using
mmaporMapViewOfFilecan create a view of file managed by operating system, so the file is automatically read and written on reading and writing memory, which could save memory or make better performance in some cases.Proposal
Non-Goals
pak-index.ini