search

OGRECave / ogre

scene-oriented, flexible 3D engine (C++, Python, C#, Java)
https://ogrecave.github.io/ogre/
MIT License
3.99k stars 979 forks source link

zip archive always loaded into RAM #2400

Open paroj opened 2 years ago

paroj commented 2 years ago

currently, we load the complete zip archive into RAM when opening: https://github.com/OGRECave/ogre/blob/aa7604254ee07a9e95d339652eca0ee1aaf9c2f5/OgreMain/src/OgreZip.cpp#L104

this increases the loading time, if the majority of the contents are not read.

this is a zip API limitation, which only allows passing buffers when externally opening the file. The underlying miniz API actually allows passing read callbacks: https://github.com/richgel999/miniz/blob/5a38f5564a092462817b97014f4477c8f0783f8f/miniz_zip.h#L173-L175

using these would allow seeking in the file, which improves loading time in the above scenario.

tritonas00 commented 2 years ago

Is this the reason we experience slower cache generation in RoR with Ogre 13 also?

paroj commented 2 years ago

probably. Although generation should not be noticeable slower. update should be - i.e. if you only want to read some file metadata without reading the whole file. Maybe you triggering some bad behavior where a zip file is loaded twice? Previously, you would only pay on reading, but now you pay regardless.

tritonas00 commented 2 years ago

eh dont really know, here we parse https://github.com/tritonas00/rigs-of-rods/blob/12d8ece21d0453f66803e356fd2cebe6943e1875/source/main/resources/CacheSystem.cpp#L997

i remember though if we dont unload at the end, we had massive ram usage