descawed / tesconvert

A tool for converting player characters between The Elder Scrolls III: Morrowind and The Elder Scrolls IV: Oblivion
0 stars 0 forks source link

Lazy record loading #15

Closed descawed closed 4 years ago

descawed commented 4 years ago

Improve performance by deferring parsing of records until they're needed. This has reduced total conversion time by 80+% (average time before change: 78,408ms; after change: 11,703ms). Lazy-loaded records must be explicitly finalized before their fields can be accessed. Currently, this is handled transparently by the plugin, so API consumers should not need to worry about this if they're accessing records via plugins.

This functionality is implemented by both Tes3Record and Tes4Record, but only Tes4Plugin takes advantage of it, where avoiding unnecessarily decompressing compressed records makes a significant impact. Not all Tes3Records can be loaded lazily because we need to know their ID, which is a field. I have a branch, lazy-tes3, that implements lazy loading for record types that do not have IDs. That branch shows no discernible difference in performance despite deferring processing of 60-80% of records per plugin, so I won't be merging that at this time.