cortictechnology / cep

CEP is a software platform designed for users that want to learn or rapidly prototype using standard A.I. components.
MIT License
166 stars 21 forks source link

Support for LEGO EV3 31313 #36

Closed dhruvmsheth closed 1 year ago

dhruvmsheth commented 1 year ago

@michaelhwn Can this be modified to support Mindstorms EV3 too? I guess CAIT is supported for EV3. How should I go about to integrate it with EV3 31313? I'm trying to use it with an OAK-D camera and a custom object detection model.

Thank you so much for your help! Hoping to hear back from you soon.

dhruvmsheth commented 1 year ago

For example in the face tracking example:

https://github.com/cortictechnology/cep/blob/c8d3e145ea8ccc58c5ab49d9fb132356d60a666d/examples/cait/example_tracking_face.py

Can the part with "hub_name": "Robot Inventor: A8:E2:C1:9B:EF:3B" be replaced with EV3: 172.20.1.73 and made compatible after pairing connecting ev3 with raspberry pi?

michaelhwn commented 1 year ago

Hi @dhruvsheth-ai, as we do not have plans to support the Mindstorms EV3, there isn't a ready-to-use flow in CEP to control the EV3. However, we have implemented a very basic EV3 control worker that you can use as a starting point to extend CEP for a complete EV3 control flow. I have outlined some steps below that you can refer to if you decide to do that:

  1. Extend the ev3_control.py worker and implement all the motor/sensor function you need.
  2. Add the ev3 control worker into the CEP's module_config. It will look like this:
    "control": {
        "robot_inventor_control": "RobotInventorControl",
        "ev3_control": "EV3Control",
        "control_service": {
            "class": "ControlService",
            "friendly_name": "Control Service",
            "worker_list": [
                "robot_inventor_control",
                "ev3_control"
            ]
        }
    }
  3. Modify and extend the core interface of CAIT so that it can take in an EV3 address for initialization.
  4. Modify/Add any motor/sensor control function in the core module so that the control commands match your implementation in Step 1. One example of such function can be found here.
  5. Setup your EV3 Hub according to this guide.
  6. Modify the example you referenced so that it works with the changes you made in Step 1-4.

Please don't hesitate to contact us if you have any other issues.

dhruvmsheth commented 1 year ago

Hi @michaelhwn for the detailed info! We'll try it out this week and let you know how it goes!

dhruvmsheth commented 1 year ago

Hello @michaelhwn I was trying to pair EV3 via bluetooth to raspberry pi through the instructions given here however the EV3 wouldn't pair with the Raspberry pi. It momentarily connects (2 seconds) and disconnects right after that. I tried connecting EV3 via bluetooth to my smartphone and it paired successfully and displayed the 'network connection' prompt after connecting. However, this isn't the case with Raspberry Pi. I tried following these guides too: http://www.abrowndesign.com/2018/10/25/sending-bluetooth-messages-from-raspberry-pi-to-lego-ev3-stock-firmware/ however, it didn't work. I'm not sure if this is an issue that's related to CEP in any way, but I thought of asking you if you might have faced a similar issue in the past. Thanks for all your help!

Edit: (This is the error:) image It connects for a second and then immediately disconnects.

dhruvmsheth commented 1 year ago

@michaelhwn If you have any idea about this error, any help would be appreciated :D I have been trying to resolve it since past few days but haven't succeeded in doing so.

michaelhwn commented 1 year ago

Hi @dhruvsheth-ai, sorry for the late reply, we were on vacation for the past few weeks. I haven't met this problem before. During the development of our previous project CAIT, we were using an earlier version of OS (Raspbian) and the Bluetooth connection was stable. We haven't tried using the EV3 with the latest Raspberry Pi OS.

dhruvmsheth commented 1 year ago

@michaelhwn No issues, thanks for the help!