OneB1t / VcMOSTRenderMqb

Way to write custom data or VNC stream to Virtual cockpit for MQB platform
The Unlicense
38 stars 4 forks source link

vertice and texture cooridnates normalized mode #2

Open grajen3 opened 1 month ago

grajen3 commented 1 month ago

I was playing with VNC client with some generated images being served and noticed that center of my generated content is not in the center of VC (and overall some blurriness)

image Is my current stuff - but that green box with navigation cue had to be rendered off center by trial and error (and even now it doesn't feel centered hehe). Worse part is that "your location" on the map is not centered - it is a bit to the right off center, but this part I can't control myself unless I add custom clipping after the fact.

I suspect those to be "at fault": https://github.com/OneB1t/VcMOSTRenderMqb/blob/4f726cf6ac5b53074988a704ac04dbf8fe89968a/opengl-render-qnx/opengl.cc#L88-L113 I assume this is some clipping that attempts to remove some "junk" from screen recordings tailored to nav app(s) you use?

Would you accept a change that allow to use fully normalized coordinates? so vertices would be from -1 to 1 and tex coordinates from 0 to 1 and that could be toggle with CLI switch like --no-clipping or something along the lines (as to not change default behavior)?

Unless my understanding is wrong and this is not just for screenrecording, but also for VC display mapping?

OneB1t commented 1 month ago

Yes this is there to clip unused parts of image. I usually run windows version where you can see perfectly what will be rendered. To get rid of this behaviour just use

GLfloat landscapeVertices[] = { -1.0f, 1.0f, 0.0f, // Top Left 1.0f, 1.0f, 0.0f, // Top Right 1.0f, -1.0f, 0.0f, // Bottom Right -1.0f, -1.0f, 0.0f // Bottom Left };

I will add no-clip option for next releases

grajen3 commented 1 month ago

I usually run windows version where you can see perfectly what will be rendered.

Yeah, did that today (had to setup visual studio heh) and will be using that instead of vnc client I used for testing before :)

Not sure if it's just me but windows app seems unresponsive? it does work and renders stuff, I just can't move window or stuff like that

OneB1t commented 1 month ago

this is what is rendered: image

yes the application is somehow unresponsive no clue why i never cared to fix it it probably requires some sleep in order to deal with windowing system events

OneB1t commented 1 month ago

unresponsive window fixed in https://github.com/OneB1t/VcMOSTRenderMqb/commit/4546d582d7040fdf73ae75fa32686c0e2acecf59

OneB1t commented 1 month ago

@grajen3 this "Oil" temp is nice can you also try to add battery voltage? 😈 It should be inside persistence channels on this address:

keyValue value int per 0x01a000d7 4 label "Kl.30 vbat [V]" poll 500

grajen3 commented 1 month ago

Oh the oil stuff from my screenshot is all inside LSD - reading it there and using BAP stuff to set it as one of navigation related data fields, but I will actually be dropping that if the google map rendering prove to be working fine ... some day :) )

I'm not doing any rendering of the google map on the car's hw (other than relying on your VNC client) and all the rendering is on the phone right now, so I don't even know how would I use those persistance channels ( I saw https://github.com/OneB1t/VcMOSTRenderMqb/issues/1 about it, but this seems like something accessible from car itself that could be exposed with some custom server if needed)

I was thinking about replacing what I have now with BAP and just render oil (and whatever else) on the phone somewhere on top of the map and get that data from exlap (using same thing as https://github.com/jilleb/mqb-pm so there is something for android already built that makes reading from exlap easier and not something that has to be done from scratch) - oil temp is for sure available from exlap and I'm pretty sure battery voltage is as well ( this has reference to it https://github.com/jilleb/mqb-pm/blob/c47ca0ed3e277ac72fa525803f6c9c5b3aaeed5b/app/src/main/java/com/mqbcoding/stats/DashboardFragment.java#L2691-L2696 )

OneB1t commented 1 month ago

There is a way to draw over map image but my font is quite ugly one :-/ maybe later i will invest more time into nicer font.

image

Car itself is serving exlap data over port 25010 so it is possible to connect there over WIFI and read car data from that endpoint.

grajen3 commented 1 month ago

I would just like to do all the rendering in single place - given that I have map and I already custom render navigation cue (green box stuff that mimics google maps style) it just make sense to me to render rest of the things on the phone too (if data is available and it is).

Also think the API for drawing is a bit nicer with Android than dealing with OpenGL directly - so it's just "easier" to work with

OneB1t commented 1 month ago

On android it will be light years better for font rendering and also for exlap client (as there are libraries which can be used). Exlap also expose consumption data, dsg oil data, performance data (on cars which have that). etc.. There is small python software on my repo developed by @jilleb https://github.com/OneB1t/VcMOSTRenderMqb/blob/main/RESEARCH/exlapResearch/exlap_client_v2.py which can be easily transformed into java app (chatGPT)

grajen3 commented 1 month ago

There is small python software on my repo developed by @jilleb https://github.com/OneB1t/VcMOSTRenderMqb/blob/main/RESEARCH/exlapResearch/exlap_client_v2.py which can be easily transformed into java app (chatGPT)

I think I'll just copy most of https://github.com/martoreto/aauto-vex-vag/tree/master/app/src/main/java/com/github/martoreto/aauto/vex/vag and just adjust transport stuff (this is using android auto custom vehicle extensions transport AFAIK which would require android auto connection to work, but nothing in my custom renderer rely on actually runing android auto so it would be pointless limitation if I can connect to it over 25010 port)

grajen3 commented 1 month ago

Slowly getting there :) image

grajen3 commented 1 month ago

And battery voltage reading and rendering on the phone working: image

Should print oil temp next to battery voltage, but so far I was testing without even starting engine - I'd need warm engine to verify because we don't get any values until it's 50 deg - will check on my next trip.

I'll try clean up my code a bit and share the project, but the part that syncs google maps navigation with this custom renderer is pretty wonky so probably will omit that at the start at least (I'm using additional server for that which make running it not the greatest for other possibly interested in playing with it and would need to redo some things to avoid need for it)

grajen3 commented 1 month ago

Oil temp is working too:

image

Just in case it might be helpful for you folks working on exlap separately - https://gist.github.com/grajen3/f0884a9f4cbcc044a42c88ea958472db is what I used for exlap using socket over 25010 port ( connect() method starts the entire thing) - overall still pretty hacky due to copying stuff from https://github.com/martoreto/aauto-vex-vag/tree/master/app/src/main/java/com/github/martoreto/aauto/vex/vag (i.e. this one by default would subscribe for everything and I added some hardcoded filter to just subscribe to stuff I'm interested in rendering possibly) but it works

OneB1t commented 1 month ago

As you are progressing quite nicely, I think I will focus my efforts on seeing if it is possible to increase the framerate of the video stream from MIB to VC. It is currently limited to 10 fps, but I hope there might be some flexibility.

The exlap client in plain C was tough, and I had been waiting to tackle it during the long winter nights. However, if there is a way to program everything on top of the phone, that would be a much easier and faster approach."