search

itinero / reminiscence

A library for cross-platform memory mapping.
MIT License
9 stars 6 forks source link

[Request]: VS2017 Support #7

Closed airbreather closed 7 years ago

airbreather commented 7 years ago

The projects use the .xproj / project.json format from the VS2015 preview era, so they can't be used directly with VS2017. VS2017's auto-upgrade process, however, doesn't seem to understand what you're doing here, and the upgraded project won't build.

My VS2015 is broken at the moment (reinstalling doesn't work), so I can't confirm that I can get it working in that version, but it looks like you've set a precedent by upgrading itinero/routing, so I feel OK requesting this.

Here's the MSBuild detailed output from trying to build the auto-upgraded version, in case that helps you figure this out.

xivk commented 7 years ago

Ok, it's on my TODO list, wasn't really urgent because the builds can still be used via nuget and the current state of the project is ok for Itinero.

But nonetheless the projects needs an update, could take more than a week, leaving on holiday.

airbreather commented 7 years ago

The reason I bring it up is because I'm seeing Reminiscence stuff show up in a big way while measuring what's making edge-based contraction take so long. I built a routerdb file for planet-170306.osm.pbf, and I thought it might be interesting to see what happens when taking that whole thing and building a contracted graph for it... I just killed the process today, after roughly two weeks on my really nice gaming computer. I was using the NoCache profiles because I was getting OutOfMemoryException with the default (even with 64 GiB of physical RAM, a gigantic swap file, and evil tricks).

I was going to see what I can do to make it better. One thought was to try System.IO.MemoryMappedFiles as an alternative implementation of some of the stuff in there. Ideally, it wouldn't make enough of a difference to justify going much further with it, but I suspect that the OS's native memory management will do much better than a slightly naive managed wrapper around a reasonably well-organized managed wrapper around the Win32 I/O APIs.

Another thought was to allocate the gigantic things in unmanaged memory with Marshal.AllocHGlobal and write a thin wrapper around it for managed code to be able to use nicely.

Anyway, I've been able to get what I need for .NET 4.5 locally, of course... if I couldn't even get that far, then my quest would be pretty hopeless.

And of course, in general, I'm probably supposed to restrict the input data to just what's within the geographical area that's relevant to the actual application or query. Ideally, though, I'd like to get to a place where I could build one RouterDb, do one contraction (per relevant vehicle profile of course), and leverage that around the world for all the regional solutions.

airbreather commented 7 years ago

I'm going to start working on this. I got my VS2015 working at some point, but it turns out that System.Runtime.CompilerServices.Unsafe will not work without C# 7.0 features (it's not just creating methods with by-ref returns, as I had hoped).

xivk commented 7 years ago

@airbreather Any ideas on improving this library, that can use a lot of work to be honest are very welcome. There is also this:

http://adamsitnik.com/Span/

I'm preparing work on v2 so it's the right time to make big changes.

airbreather commented 7 years ago

Well I'd submitted #8 a while back. Looks like there are conflicts now, though. I'll try to address them.

Yes I'm aware of Span<T>. In fact, I'm pretty sure reminiscence was exactly what I'd had in mind when I'd commented on an early iteration of it back here.

I was waiting to get back to itinero and reminiscence stuff until my hiatus was over and until after that PR, or something like it, got merged.

airbreather commented 7 years ago

The issue in this title was addressed by d308506, so I'm going to close this issue and continue further unrelated discussion on another issue that I'll open shortly.