Closed airbreather closed 5 years ago
using System;
using Itinero;
using Reminiscence.Arrays;
public sealed class MyArrayFactory : IArrayFactory
{
public ArrayBase<T> CreateMemoryBackedArray<T>(long size) => ArrayCreatorCache<T>.CreateNewArray(size);
private sealed class ArrayCreatorCache<T>
{
public static readonly Func<long, ArrayBase<T>> CreateNewArray = Create();
private static Func<long, ArrayBase<T>> Create()
{
if (typeof(T) == typeof(byte) ||
typeof(T) == typeof(sbyte) ||
typeof(T) == typeof(short) ||
typeof(T) == typeof(ushort) ||
typeof(T) == typeof(int) ||
typeof(T) == typeof(uint) ||
typeof(T) == typeof(long) ||
typeof(T) == typeof(ulong) ||
typeof(T) == typeof(float) ||
typeof(T) == typeof(double))
{
var types = new[] { typeof(IUnmanagedMemoryAllocator), typeof(long) };
var ctor = typeof(NativeMemoryArray<>).MakeGenericType(typeof(T)).GetConstructor(types);
var allocator = DefaultUnmanagedMemoryAllocator.Instance;
return x => (ArrayBase<T>)ctor.Invoke(new object[] { allocator, x });
}
return x => new MemoryArray<T>(x);
}
}
}
This is great! Especially the RouterDb.Deserialize(Stream)
... Give me some time to review this but is looking very good...
Give me some time to review this but is looking very good...
Have you had the opportunity to take a look at this one yet?
Following this, I have a proper implementation of ArrayBase<T>
using "true" memory-mapped files which is almost ready-to-go.
Following both of those, I can work towards updating Itinero proper to take advantage of the new types.
According to my testing, I/O for serializing / deserializing Itinero objects is significantly faster with this, to the extent that I'm satisfied that this is worth pursuing on that basis alone.
There are occasionally some other places that see speedups as well, primarily during the initial build and contraction hierarchy creation (which is what I was targeting with these improvements), but these are very minor (presumably because
netcoreapp2.2
has a much smarter JIT thannetcoreapp2.0
, which is what I was originally targeting when I embarked on this roughly a year and a half ago).I just did one run for each row because the times can be so long. All files were entirely served from the filesystem cache for these tests. Timings were captured on
netcoreapp2.2
with server GC. The only difference between "vanilla" and "new array" is whether or not the test injected a customIArrayFactory
implementation.denmark-latest.osm.pbf
(2019-01-19) (from here)RouterDb.LoadOsmData
denmark-latest.osm.pbf
(2019-01-19) (from here)RouterDb.AddContracted(Profile)
RouterDb.Deserialize(Stream)
RouterDb.DeserializeAndAddContracted(Stream)
I did not observe any differences in timings for actual calculations worth discussing.