search

LimelightVision / limelight-feedback

Feature Requests and Issue Tracking for the Limelight Ecosystem
1 stars 0 forks source link

Utilize a Structured Format for Sending Data #4

Closed Gold872 closed 6 months ago

Gold872 commented 8 months ago

Currently, the JSON format is extremely slow. My team relies on using it to retrieve data from the limelight because our pose estimation uses lots of information to determine the standard deviation measurements, including how many tags are seen, the distance to the tags, and we also log the poses of the tags that are visible. Right now, this is only possible by using the JSON dump, which can take a long time to parse, especially if there's more than 1 limelight.

There's 2 ideas I have to solve this:

  1. Use NT4's structured data support, such as structs or protobuf This would allow for much easier readability in log viewing applications such as AdvantageScope, and also proves to be much faster than a giant json string which gets passed by Jackson, which is known to take away 3-4 milliseconds every robot loop.

  2. Make a custom schema that packs data with raw bytes This is the fastest possible way of sending data via network tables. It would be a bunch of raw bytes that are in a custom format, and get unpacked by the robot program.

JosephTLockwood commented 8 months ago

We also have an issue with this. The readings from our multiple LLs took over 20ms to parse the readings (if all see tags). Not only this, but we were attempting to subscribe to the LL values, which could return multiple readings per LL. Once this happened, we experienced a snowball effect that clogged up our NT, and after a while, the processing of JSONs took nearly a second in some cases. This is also the issue in #7 , and I feel that it is currently the most significant issue for anyone attempting to use multiple LLs.

bhjelstrom commented 8 months ago

In 2024.1, all botpose arrays in networktables contain the following [tx,ty,tz,r,p,y,latency,tagcount, max tag span in meters, average tag distance in meters, average tag area]

I believe this obviates JSON for robot localization

Gold872 commented 8 months ago

Is there any way of accessing which tags are being used? My team relies on being able to see the 3D poses of all of the tags in view to easily tune pose estimators. I believe this still requires the json dump. I think having a topic where the tag IDs are published can solve this

bhjelstrom commented 7 months ago

@Gold872 Can you let me know exactly what you want to see?

what about a new entry that looks like: [tx,ty,tz,r,p,y,latency,tagcount, max tag span in meters, average tag distance in meters, average tag area, ]. Each tag entry would have id, txnc, tync, ta, distance, and ambiguity.

bhjelstrom commented 7 months ago

@Gold872 https://downloads.limelightvision.io/images/limelight3g_2024_3_4.zip https://docs.limelightvision.io/docs/docs-limelight/software-change-log#limelight-os-202432-32024

bhjelstrom commented 6 months ago

Feel free to re-open if this change and megatag2 do not have what you need