aurora-opensource / xviz

A protocol for real-time transfer and visualization of autonomy data
http://xviz.io
Apache License 2.0
1.01k stars 228 forks source link

XVIZ live conversion + visualization #531

Open enginbaglayici opened 4 years ago

enginbaglayici commented 4 years ago

Hello, I want to generate .glb files from sensor data and directly send it to Uberviewer to visualize data in real time. I can generate .glb files but I need some ideas to visualize this data with a frequency. If it possible, what would be the steps to achieve this?

Here's what I've done and found for live mode and the results:

I'd appreciate if you provide an example or specification for live mode operations. Thanks for your interest.

twojtasz commented 4 years ago

It is a fair to say that a live mode has partial support, as you have experienced.

To address your bullet points

In general, a live mode is extremely useful, but large amounts of data exists off-line that are easier to work with and equally valuable. Priorities and resources is the name of the game.

Let me see if I can help remove some complexity for you.

You are working with the existing components, which is great build an understanding of the system, but to make it work for you I expect you to replace components that don't fit. The layers try to isolate a responsibility. XVIZSource/XVIXSink operate on a named data source, it could be from file or something else but assumes some structure w/o saying memory or file or socket or db.

The next layer is the XVIZReader/Writer. These conform to the structure you mention of index, metadata, data (0-frame.json, 1-frame.json, {2,*}-frame.??? - respectively). You could create a component that returned some minimally viable data where it is not relevant to you instance.

The next layer is the XVIZProvider, which is very concrete about Index, Metadata, Message and does not care if the underlying interface follows the convention (0-frame.json, etc) since it relies on the XVIZReader.

What this means is you could create an XVIZProvider and XVIZReader that was entirely independent of the existance of an index or metadata (just send valid but largely ignored values as necessary).

I would love to get XVIZ to a level of maturity where what you want came out of the box ready to use, but we are building to that. In the mean time we are trying to provide appropriate levels of abstraction such that most elements can be worked around while still leveraging other layers provided.

Hope this helps give you some ideas. Ping me on slack or post a PR and I can provided assistence.

VijeetBenni commented 4 years ago

Hello Eugin, I followed same the steps you mentioned. but when I run yarn start-live-local, The UI comes up but says Missing Data:

How were you able to visualize .glb files in real time? Right now latency is not matter of concern for me.

VijeetBenni commented 4 years ago

Anyways., I am able to visualize now. But the .glb files are generated in realtime, so for testing I kept 1 file in the data folder and later started copying 10 files in the folder. Only first frame is displayed after that nothing.

How to send the glb files one after the other through xviz server?

enginbaglayici commented 4 years ago

Hello! Actually I am not working on this issue for a while but I was exactly at that point. I also wanted to send glb files to xviz server in real time but I could only display the data that I put before I start streaming.. If you figure out how to solve this and display continously, the problem will be pretty much solved. Next thing will be synchronizing the data which seems difficult while using js for xviz conversion.. Hopefully python support will solve this issue.

VijeetBenni commented 4 years ago

Hello Engin, Thanks for the reply. I am able to display it continuously. But yes synchronization is the next task now which is difficult. I wanted to know how are you converting the data into XVIZ ? I meant how did you integrate the conversion and the server part?

enginbaglayici commented 4 years ago

Hello,

I am actually doing those separately. I convert json files to xviz glb files. Then I start streaming and finally visualize.

VijeetBenni commented 4 years ago

Thanks Engin.

didibaba commented 4 years ago

Is there any difference between sending glb and json directly in live mode?

heshiwen1989 commented 4 years ago

Hello,

I am actually doing those separately. I convert json files to xviz glb files. Then I start streaming and finally visualize.

Hello, Can you Offer you solution with more information? My work is similary with you;

mahaogithub commented 4 years ago

@VijeetBenni hello, Can you tell me your solution? I want to do real-time visualization.

mahaogithub commented 4 years ago

@VijeetBenni Hello, How do I send GLB files one by one through the xviz server?Have you solved the problem,Could you tell me your solution?

enginbaglayici commented 4 years ago

@heshiwen1989 I recommend you guyz to join Slack channel and ask your questions in detail. You can also find similar issues of people and answers to these issues. https://join.slack.com/t/streetscapexviz/shared_invite/zt-71l8ybk0-DKJHSCamkgZ81L7QmhLoHw

xibei08 commented 1 year ago

@VijeetBenni how you fix the UI missing data issue for live mode. i also meet the same issue and cannot fix it. thanks a lot, looking forward your relply