search

PickNikRobotics / data_tamer

C++ library for Fearless Timeseries Logging
MIT License
219 stars 21 forks source link

Can't view mcap files in foxglove #19

Open jlack1987 opened 4 months ago

jlack1987 commented 4 months ago

It seems that mcap files generated by the mcap sink can't be viewed in Foxglove Studio. It gives an Unsupported encoding "data_tamer" error. I see that the profile field in the message is set to data_tamer, but i'm not sure if that's what's causing the problem or not.

jlack1987 commented 3 months ago

Ok I have looked more into this and can confirm the reason foxglove can't view the data is because it's not using one of foxgloves supported encodings/schema's. It's currently using a custom datatamer schema which I assume only plotjuggler is setup to read

facontidavide commented 3 months ago

This is expected. The data tamer protocol is NOT supposed to work with foxglove.

Said that, if the foxglove team want to, I will be happy to support them with the development of a parser 😉

jlack1987 commented 3 months ago

Would it maybe be better(or at least as good? idk) to add support for one of their supported protocols? I may be able to put some time in on that and see if I can get something working if you think it'd be worth it

facontidavide commented 2 months ago

The current protocol is very, very similar to ROS1. The only difference is that you should remove the ability of disabling a variable.

I can prepare a preliminary branch with the changes, but I need other people to test it, since I have little bandwidth

jlack1987 commented 2 months ago

Yeah I just meant that it'd probably be easier to make data tamer log data in a format that foxglove natively supports rather than try and add something to foxglove to support a custom data tamer protocol. I could totally be wrong though as i'm not all that familiar with the specifics of either

jlack1987 commented 1 month ago

I went ahead and created a foxglove websocket sink and an mcap sink that both work with foxglove here. If you all are interested in incorporating this functionality in data tamer itself let me know and I can incorporate it however you'd like and make any suggested changes. If not no problem I can keep it there. It doesn't support the active variable masking stuff though. It's still new but I've been testing with it and it has been functional for me.

facontidavide commented 1 month ago

If I understand correctly, looking at the code, the way this works is converting it to a brutally large JSON message...

https://gitlab.com/jlack/data_tamer_tools/-/blob/main/src/foxglove_mcap_sink.cpp?ref_type=heads#L75

So, even if I believe that it is good, this goes against the idea of "efficiency" in the library...

Maybe it is still useful, but I am uncertain about including this.

jlack1987 commented 1 month ago

Yeah I actually very much agree with you. I like the data tamer API and very much want to use it but I want to use it with foxglove and the data tamer schema isn't supported by foxglove so I just picked one of the protocols that foxglove supported and went with that. I'm open to suggestions on other avenues to be more efficient though, so if you have some suggestions on what I could do to maximize efficiency in a way that foxglove still supports throw them my way 😊

henrygerardmoore commented 1 month ago

@facontidavide Should I add a link to this in the README? I'm sure others would find this really useful. I'm unfamiliar with foxglove studio so I don't have any suggestions regarding efficiency.

jlack1987 commented 1 month ago

I agree w/ @facontidavide about the inefficiencies. It should probably wait until I can figure out how to make it more efficient. I have opened a discussion on the foxglove discussion board where i'm trying to figure out how to make it more efficient given the schema parsing abilities foxglove already has. Ideally they would support the data tamer style schema even if through some other more standard mechanism, but I can't currently see how to make that work.