Also adds a root metadata file that loaders can optionally use to store non-file specific metadata. In this case git commit hash.
Time perf from loading goes from 10 seconds to 1 second on our github repo on first load, and subsequent loads become free unless there has been a new commit.
Writing the files into memory however take longer, but there is nothing we can do about that here.
Adds an efficient git loader
Also adds a root metadata file that loaders can optionally use to store non-file specific metadata. In this case git commit hash.
Time perf from loading goes from 10 seconds to 1 second on our github repo on first load, and subsequent loads become free unless there has been a new commit.
Writing the files into memory however take longer, but there is nothing we can do about that here.