✨[FEATURE REQUEST] AI Vision Sensor

[nightwolf3140]

Vex AI vision sensor:

My team recently got the new AI vision sensors, and Ive been really excited by their potential for use in high stakes for detecting rings.

I was wondering if there were any plans to add this in the near future to be used during this season.

Thanks!

[Rocky14683]

It is work in progress.

[djava]

Assuming that the most recent branch is aivision-rewrite, it seems like things are pretty stalled (last commit 10/24). What still needs to happen here, are there material blockers or just haven't had time/interest? @KdotDevelopment @Rocky14683

[Rocky14683]

@thiccaxe just did not push his code. His current working on documentation and testing.

[djava]

Ok, you should ask him to do that; it'd be good to start having eyes on it, probably. Also so that there's more than one copy of it in the world.

[thiccaxe]

see https://github.com/purduesigbots/pros/tree/pros-4/feature/aivision

Some of the current issues:

• ai vision sensor doesn't seem to remember its configuration in the case that it momentarily loses connection (and then reconnects). This effectively means that users will have to replicate something similar #641 in their codebase to check if configuration was lost, and if so, re-update it. This shouldn't be too big of a deal, but it effectively boils down to an additional API call every X milliseconds to check if the vision sensor was reset

• Unfortunately, if you want to use april tags, we only have what I believe are the coordinates of the corners of the april tag and not its angle. I will look into having a pros function to output the center and angle, if possible.

[thiccaxe]

example main.cpp

pros::c::aivision_enable_detection_types(2, pros::aivision_mode_type_e_t::E_AIVISION_MODE_TAGS | pros::aivision_mode_type_e_t::E_AIVISION_MODE_OBJECTS);

pros::c::aivision_enable_detection_types(2, pros::aivision_mode_type_e_t::E_AIVISION_MODE_TAGS);
pros::c::aivision_set_tag_family(2, pros::aivision_tag_family_e_t::TAG_16H5);
pros::c::aivision_set_overlay(2, true);
while (true) {
  uint32_t resp = (uint32_t) pros::c::aivision_get_object_count(2);
  if (resp > 0 && resp != PROS_ERR) {
    printf("detected objects: %ld\n", resp);
    for (size_t idx = 0; idx < resp; idx++) {
      pros::aivision_object_s_t obj = pros::c::aivision_get_object(2, idx);
      printf("%d %s\n", idx, aivision_object_str(&obj));
    }
  }
  pros::delay(20);                               // Run for 20 ms then update   
}

[djava]

Some of the current issues:

• ai vision sensor doesn't seem to remember its configuration in the case that it momentarily loses connection (and then reconnects). This effectively means that users will have to replicate something similar ✨[FEATURE REQUEST] Momentary Motor Disconnect Resilience #641 in their codebase to check if configuration was lost, and if so, re-update it. This shouldn't be too big of a deal, but it effectively boils down to an additional API call every X milliseconds to check if the vision sensor was reset

The only real opinions given in #641 are that this isn't really PROS's responsibility to fix. This shouldn't be a blocker, as it's already an issue for literally every device?

• Unfortunately, if you want to use april tags, we only have what I believe are the coordinates of the corners of the april tag and not its angle. I will look into having a pros function to output the center and angle, if possible.

This shouldn't be a blocker, as it can be added later? Make the current API denote corner in the name somewhere if needed.

[ion098]

@thiccaxe

Some of the current issues:

• ai vision sensor doesn't seem to remember its configuration in the case that it momentarily loses connection (and then reconnects). This effectively means that users will have to replicate something similar ✨[FEATURE REQUEST] Momentary Motor Disconnect Resilience #641 in their codebase to check if configuration was lost, and if so, re-update it. This shouldn't be too big of a deal, but it effectively boils down to an additional API call every X milliseconds to check if the vision sensor was reset

Is this not an issue with the old vision sensor as well? Also, could this be solved by storing the configuration PROS side and resending it on every API call? (Or does VEXos mess with that somehow?)

[thiccaxe]

pros::c::aivision_reset(3);
pros::c::aivision_disable_detection_types(3, pros::aivision_mode_type_e_t::E_AIVISION_MODE_ALL);
pros::aivision_color_s_t color_s;
color_s.id = 1;
color_s.red = 18;
color_s.green = 87;
color_s.blue = 171;
color_s.hue_range = 10.0;
color_s.saturation_range = 0.2;
pros::c::aivision_set_color(3, &color_s);
color_s.id = 2;
color_s.red = 192;
color_s.green = 23;
color_s.blue = 81;
color_s.hue_range = 10.0;
color_s.saturation_range = 0.2;
pros::c::aivision_set_color(3, &color_s);
pros::aivision_code_s_t code_s;
code_s.id = 1;
code_s.length = 2;
code_s.c1 = 1;
code_s.c2 = 2;
pros::c::aivision_set_code(3, &code_s);
pros::c::aivision_enable_detection_types(3,
    pros::aivision_mode_type_e_t::E_AIVISION_MODE_TAGS |
    pros::aivision_mode_type_e_t::E_AIVISION_MODE_COLORS);
pros::c::aivision_set_tag_family(3, pros::aivision_tag_family_e_t::TAG_16H5);
pros::c::aivision_set_usb_bounding_box_overlay(3, true);