There's going to have to be a bit of refactoring on the Itinero side before this can be used all that effectively, since the only ways Itinero currently lets you create ArrayBase<T> instances seem to be:
Create a new empty instance and then copy from an arbitrary Stream, or
Create an Array<T> using an abstract accessor.
I still haven't quite figured out the best approach to integrate this in Itinero, but I think this is solid enough to commit to Reminiscence. My current thought is that RouterDb could have methods like Deserialize / DeserializeAndAddContracted, but which accept file paths instead of arbitrary Streams. The inner types would be updated as appropriate, of course. Still thinking about it, though, since the downside is that we'd either have to accept a lot of code duplication (eww) or add even more weird logic in the existing methods around where they check for non-null "profile" instances...
The way I did this was:
Create an exact copy of NativeMemoryArray.cs to help with the diff for the next step.
Split NativeMemoryArray<T> into two parts:
An abstract base class for arrays backed by contiguous virtual memory, and
The implementation from #18 that uses IUnmanagedMemoryAllocator.
Add another subclass that uses a pointer to the data from a "real" memory-mapped file.
The new array class can work in either of two slightly different ways:
If you just pass in the path to a file, then we interpret it to mean that you want to use the entire file for your data. In this mode, CanResize is true (because we can support that by resizing the file).
If you pass in the path to a file with an offset / length, then we support that too, but CanResize will be false, because resizing could mess up offsets for other readers of the same file.
The weakest part of this one: I'm not 100% sure about the values of FileAccess, FileShare, and MemoryMappedFileAccess that I used here. Maybe we should be a bit more restrictive...
There's going to have to be a bit of refactoring on the Itinero side before this can be used all that effectively, since the only ways Itinero currently lets you create
ArrayBase<T>instances seem to be:Stream, orArray<T>using an abstract accessor.I still haven't quite figured out the best approach to integrate this in Itinero, but I think this is solid enough to commit to Reminiscence. My current thought is that
RouterDbcould have methods likeDeserialize/DeserializeAndAddContracted, but which accept file paths instead of arbitraryStreams. The inner types would be updated as appropriate, of course. Still thinking about it, though, since the downside is that we'd either have to accept a lot of code duplication (eww) or add even more weird logic in the existing methods around where they check for non-null"profile" instances...The way I did this was:
NativeMemoryArray.csto help with the diff for the next step.NativeMemoryArray<T>into two parts:IUnmanagedMemoryAllocator.The new array class can work in either of two slightly different ways:
CanResizeis true (because we can support that by resizing the file).CanResizewill be false, because resizing could mess up offsets for other readers of the same file.The weakest part of this one: I'm not 100% sure about the values of
FileAccess,FileShare, andMemoryMappedFileAccessthat I used here. Maybe we should be a bit more restrictive...