The current document loader does some bespoke on-disk caching. This should be reworked or moved outside of the core module as an optional module -- and its implementation should be replaced with something less custom, if we decide we want to keep using it. It is a premature optimization at this point and may create a number of foreseen issues (inability to invalidate caches, etc.) and unforeseen issues that other utility modules we could make use of may help us mitigate.
The current document loader does some bespoke on-disk caching. This should be reworked or moved outside of the core module as an optional module -- and its implementation should be replaced with something less custom, if we decide we want to keep using it. It is a premature optimization at this point and may create a number of foreseen issues (inability to invalidate caches, etc.) and unforeseen issues that other utility modules we could make use of may help us mitigate.