JSON parser: slow because FormatException (fix included)

[kluiverjh]

The Convert.ToDouble is used to see if it is a number or a json treeleaf, Using FormatException handler has large performance penalty. Loading json file 1000 bytes took 60 sec; with TryParse 4 msec (in hololens emulator)

Fix included in attachement

JSON.zip

[kluiverjh]

In unity the titles load very fast (with FormatException method), performance problem is with mobile devices

[brnkhy]

Wow and I was still trying to improve mesh generation performance. I mean I knew parser is slow but it works in another thread but 60sec vs 4msec !! Thanks a lot man, I would have never looked into that :) Fixing asap

[kluiverjh]

Looks like every thrown exception cause a very large bottleneck

[brnkhy]

Even debug logs are killing unity really, so no surprise with exceptions

[kluiverjh]

Great work btw; very nice library! Very clear structure, and thereby very easy to understand. Thanks for sharing it!

[brnkhy]

Oh no I didn't wrote the Json parser, it's Matt Schoen and his license and everything is still on top of that file :)

[brnkhy]

pushed this on dev branch

[kluiverjh]

My compliments where for the complete library :) How to go from geojon to meshes in unity. And use other libraries is better as reinventing the wheel!

[brnkhy]

Ah thank you! It could have been a lot better with dependency injection container, proper use of unirx reactive properties and a message bus but I decided to keep everything simple so that everyone can understand/follow what's going on :) (unity community has so many junior developers)

[erikvullings]

Yeah, nice work!

I've also linked back to your project. You can get your adapted code for the Hololens version here.

BTW, there is one additional fix we included: caching files on the Hololens cannot use your folder, and we had some issues with the streams too. An are you also aware of the Mantle plugin in Unity.

[brnkhy]

And I was dreaming of porting mapzengo on hololens... man it would be awesome. I'll surely check your project if you send me one (hololens), deal? :)

also thanks! I'm working quite hard so it's great people are loving&using it. I think I saw mantle a few months ago, impressive tool but nothing jaw dropping proc generation wise. I love watching those videos again though, getting some new ideas already :)

[erikvullings]

Sure, why not. I still have 50 hololenses lying around in my garage ;-) But the emulator isn't too bad either.

One of the open issues is the building generation: Seems like some are only partially rendering, and they contain no open courtyards? Do you recognise this, or is OSM not correct?

We are also looking at identifying individual items - now that all buildings, roads etc are grouped together into one game object, what would be the best way to proceed?

Any suggestions would be appreciated!

[brnkhy]

me want hololens ;_;

Not sure if I understood that first question correct but current mesh generation code doesn't support holes so cant visualize courtyards. But that was only because of the old and "less capable" triangulation code. Now that we have something much much powerful (which supports multiple meshes, edge definitions, holes and all sort of other stuff), we should be easily able to add holes as well. that being said, I have bigger ideas for mesh generation so it might take time to get it all done. Or maybe I'll just do the holes things and release it before moving on.... dunno.

well, for one; you can disable merging meshes (layer based as well). Then everything will be an individual gameobject that you can click, hover, drag etc. But that willl only work if you use a close up cam like pokemon go. If you move your cam further away, you'll see huge performance hits.

So if you want huge maps and individual selection and whatever, you have two choices; 1) make everything flat. If you dont need 3D buildings, you can simplify it all to 2Dish polygons on y=0. Then you'll be able to get the mouse position (where ray collides y=0) and find the polygon (building) containing that position from osm data. you won't use mesh for anything doing that (just visualization) so it won't matter if it's merged or not. 2)if you need 3d buildings.... well I just realized same thing would probably work on them as well. I'm just making up this stuff on the fly, so it might be wrong but; you can get ray/object collision point. This will be a point on any building (and any floor). Then set y to 0, maybe push it a little further inside the building and do the same "polygon contains" check for that point. It should work. sounds sensible. it's 3am here though and I slept 4 hours last night..... so yeah.... I might be totally horribly missing something :)

[erikvullings]

I hope you've had a good night's rest... Four hours is not uncommon for me either, but not something I would generally recommend.

Thanks for sharing your thoughts. The collision approach was something I expected to work too. The mesh part was rather interesting. Could you elaborate a bit on that? What are your plans...

[brnkhy]

I'm also working overtime these days (and next few months) so yeah, quite exhausting days...

I'll try to explain that in detail when I have some time, in a little rush at the moment, and realized that the usage of the new triangulation library isnt really that straight forwards and I'm having trouble finding my way around it.

[brnkhy]

ops I totally forgot this erik, I'm sorry. well, at the moment, the triangulation algorithm runs for every single building separately and then I merge those vertex & triangle lists to create one huge mesh. That's quite inefficient especially if you have a good triangulation library which can handle "islands". the idea is to give all buildings (roofs) at once and run it just once. you'll have problem with building heights then but I'm sure we can find a work around for that.

but the main problem is, even though current triangulation library is quite powerful, I can't even use it properly and cut some holes... First I'll have to find how "holes" work in that lib, then I'll check "islands". I'm closing this as it's a json parser thing, can talk in another issue if you have questions ;)