search

mapbox / vector-tile-cs

Parses vector tiles with C#.
BSD 3-Clause "New" or "Revised" License
57 stars 37 forks source link

Performance profiling #15

Closed wilhelmberg closed 8 years ago

wilhelmberg commented 8 years ago

Do some performance profiling to see if and where there are bottle necks.

wilhelmberg commented 8 years ago

Results looking good, without any optimizations yet - just as is.

decoding 210 z14 streets+terrain tiles covering DC:

runs:100
tiles per run:210
min [ms]:981
max [ms]:1795
avg [ms]:1098,01
StdDev:128,91
overall [ms]:109801
springmeyer commented 8 years ago

Working on companion C++ benchmarks at https://github.com/mapbox/vector-tile/pull/14

springmeyer commented 8 years ago

Just moved fixtures to a new repo. Add them like:

git submodule add https://github.com/mapbox/mvt-bench-fixtures.git bench/mvt-bench-fixtures
wilhelmberg commented 8 years ago

Removing just one debug print statement dropped processing duration from ~110 seconds to ~83.

It's a pity that we are stuck at .Net Framework 3.5 because of Unity. Just switching to 4.5.2 reduced time to ~59 seconds.

brnkhy commented 8 years ago

Oh yes, debug/exception/warning logs hits the performance quite hard in Unity, especially if it's spamming.

We're supposed to get C# compiler upgrade this month so it'll probably be better. Actually 5.5 beta is released so you can try in that as well. See; https://unity3d.com/unity/roadmap

wilhelmberg commented 8 years ago

Oh yeah! @brnkhy thanks for the information, would be really nice to have .Net 4.6. Also much better support for profiling in Visual Studio.

.NET Profile Upgrade Scripting Upgrade .Net profile to 4.6 enabling access to the latest .Net functionality and APIs.

debug/exception/warning logs hits the performance quite hard in Unity

Actually I'm not yet testing from within Unity just with Visual Studio.

brnkhy commented 8 years ago

@BergWerkGIS more details on it; (it has the new GC as well) https://forum.unity3d.com/threads/upgraded-mono-net-in-editor-on-5-5-0b9.438359/

I'm not expecting the full runtime release anytime soon though.

wilhelmberg commented 8 years ago

@springmeyer this looks pretty close to the ~23 secs we got with C++ 😏 

runs:100
tiles per run:210
min [ms]:179
max [ms]:269
avg [ms]:209,44
StdDev:20,67
overall [ms]:20944

Turns out the C# version also calculated WGS84 coordinates.

As I see it the C++ bench doesn't do that, right? https://github.com/mapbox/vector-tile/pull/14/files#diff-e21ec455a183f748ecfe2b6f5233f589R32

Once I commented WGS84 calculation (+ some minor changes) I got above results.

EDIT start And of course switching from Debug to Release resulted in the biggest improvement. EDIT end

Also I'm going to leave the current approach with Array.Copy as is. Profiling didn't show this as bottleneck. Also @brnkhy pointed out: copying arrays isn't that much as it's all on same memory page.

brnkhy commented 8 years ago

@BergWerkGIS I mean it's just faster than what people generally expect. I'm not really experienced in performance stuff but there's also this class Buffer for that I believe (I never tested it) https://msdn.microsoft.com/en-us/library/system.buffer.aspx Edit: Just did a quick search and people say it depends on the case (data type & length) a lot so got to try and compare against array.copy I guess

wilhelmberg commented 8 years ago

@brnkhy I haven't used System.Buffer either and we will most certainly revisit performance later on (after refactoring for lazy decoding) but I think for now it's good enough - matched the C++ speed.

Even on AppVeyor it's "ok" fast:

wilhelmberg commented 8 years ago

Speed improvements have been merged https://github.com/mapbox/vector-tile-cs/pull/16 110 secs -> 20 secs.

Going to revisit CPU and memory profiling after these have been implemented: https://github.com/mapbox/vector-tile-cs/issues/12 https://github.com/mapbox/vector-tile-cs/issues/7 https://github.com/mapbox/vector-tile-cs/issues/3