Custom Vision Processing

[jsmmr2]

Doing some research about processing vision and detecting objects. So far I have found 2 things:

• Axon
• WPILibPi

Axon and WPILibPi are both developed by WPI, although Axon was only developed to v0.1.0 Prerelease, and hasn't had any signs of life for a few years now. Axon is WPI's machine learning tool, meant for vision processing. WPILibPi had its latest release in 2023, which is a lot newer. WPILibPi is a Raspberry Pi OS-based Raspberry Pi image that includes C++, Java, and Python libraries required for vision coprocessor development for FRC (e.g. opencv, cscore, ntcore, robotpy-cscore, pyntcore, Java 17, etc) which are all used in things like Limelight OS and Photonvision.

I will look at PhotonVision docs again and see what has changed since last year.

[jsmmr2]

I have found the following:

• PhotonVision does have built-in object detection, and uses an NPU that can be found in the Orange Pi 5/5+, and PhotonVision ships with note detection :)
• YOLOv5 is used as a "You Only Look Once" approach to vision processing. This allows high speed and high accuracy results. This means we could potentially train our own models
• WPILibPi is a headless, fast booting pi image, similar to PhotonVision
• I am not sure that WPILibPi has built in vision processing for anything... (video)

I will research and see if I can change the way PhotonVision handles processing and change it to use the Pi CPU?

[jsmmr2]

On a sidenote, I found the Limelight 3G

[jsmmr2]

@Captagonal the Limelight 3G has 1280x800@120fps, and has a 240fps mode! The Limelight 3G has faster AprilTag detection on the fly, and better pose estimation on the fly. This may be something we look into buying.