search

itinero / routing

The routing core of itinero.
Apache License 2.0
221 stars 70 forks source link

Question: How to apply the "Memory mapping" on the server which doesn't have enough memory? #224

Closed johnnyzhangzhao closed 5 years ago

johnnyzhangzhao commented 5 years ago

Hi,

According to the section "Memory mapping and mobile devices" at http://www.itinero.tech/2017/03/20/stablerelease/, the itinero is using a very simple type of memory mapping strategy to make sure it can run well on a server which has less memory. For example, I'm building the routing index for the countries - Germany, France, Austria and Switzerland locally, the memory required is fine for me, because my computer has enough memory, but I would like to put the built RouterDb to an AWS EC2 server to let it serving as RESTful service, you know it's so expensive if keep a big memory EC2 for a long time, thus I was thinking that if possible to not load all network data into memory, but an index of the records in RouterDb, the index file can has an reference to the RouterDb on disk and has some required properties, such as weight for calculation. Is it possible or you guys have the similar strategy already to avoid a big memory required? If yes, would you give me some hits here? Any built-in APIs that I can use?

Thanks, Johnny

johnnyzhangzhao commented 5 years ago

Hello, according to the docs mentioned at https://docs.itinero.tech/docs/itinero/basic-concepts/routerdb.html, I have supported "Memory mapping" in my application, but really slow, for example, if the routerdb is only for Texas, it takes around 1.8 seconds to route from north of Texas to the south of Texas, is there any balance between "Load all in to memory" and "Memory Mapping"? I have 8GB RAM of my server, i can load all indexes in memory.

Besides, is there any way to sort the records in routerDB with GeoHash? in that case, we can make sure the coordinates of 2 records are adjacent are saved adjacent on disk.

Thanks, Johnny

xivk commented 5 years ago

You should contract the network for the profiles you are using, that should make routing faster even when using memory mapping.

Yes it should be possible to load only the indexes for example by tuning this:

https://github.com/itinero/routing/blob/develop/src/Itinero/RouterDbProfile.cs#L30

These profiles can be used to control what is being loaded and how.

Apologies for answering this so late, feel free to reopen if you have any more questions.