isl-org / OpenBot

OpenBot leverages smartphones as brains for low-cost robots. We have designed a small electric vehicle that costs about $50 and serves as a robot body. Our software stack for Android smartphones supports advanced robotics workloads such as person following and real-time autonomous navigation.
https://www.openbot.org
MIT License
2.85k stars 540 forks source link

Can't get the motors working with app control #51

Closed parixit closed 4 years ago

parixit commented 4 years ago

Basic setup but can't get the motors running with app control. Any help is appreciated!

IMG_3960

https://youtu.be/muJt_BANYZw

thias15 commented 4 years ago

1) Are the pins D5/D6 and D9/D10 connected to the L298N and are the jumpers to enable PWM in place?

2) Does the LED light blink every time you move the joystick? This indicates the message is sent to the Arduino. If not try reconnecting the USB and verify the baud rate.

3) Can you try to pull the code and recompile the Android app. Early on there was a bug which would skip intermediate control signals on some compilers due to type conversion.

thias15 commented 4 years ago

I assume your issue is solved by (3). Please update the issue after pulling the latest code from the repo and testing.

parixit commented 4 years ago

Hello @thias15 just did a new pull and build of the app and still the same issue. I created a video of the issue: https://www.youtube.com/watch?v=muJt_BANYZw&feature=youtu.be

Are the pins D5/D6 and D9/D10 connected to the L298N and are the jumpers to enable PWM in place? Yes

Does the LED light blink every time you move the joystick? This indicates the message is sent to the Arduino. If not try reconnecting the USB and verify the baud rate. Yes, the LED light blinks every time I move the joystick

BrandenSebring commented 4 years ago

@parixit the issue may be in the phone itself. We have found that older phones (such as the note 4) did not send the "bluetooth information correctly" from the controller to the phone then out via usb otg serial to the board. When we changed the phone to a more powerful note 8 everything worked correctly, as far as controller support and motors actually moving.

learning the AI part now, with collected data

Also in the application when using the controller we keep drive mode on "game" and processing on the cpu. other wise the controller acts funny... (using 8bitdo contoller they made for nintendo switch)

and @thias15 thank you and all involved so much for this project!!!

thias15 commented 4 years ago

@BrandenSebring good point. The oldest phone we have tested was about 3 years old. @parixit what phone are you using? One thing you could try is to echo the control sent to the Arduino back to the phone and log it.

Here is a quick and dirty way to do it.

In the Arduino code replace lines 316-328:

void sendVehicleData()
{
  Serial.print(getVoltage());
  Serial.print(","); 
  Serial.print(counter_left);
  Serial.print(","); 
  Serial.print(counter_right);
  Serial.print(","); 
  Serial.print(distance_cm);
  Serial.println(); 
  counter_left = 0;
  counter_right = 0;
}

with

void sendVehicleData()
{
  Serial.print(getVoltage());
  Serial.print(","); 
  Serial.print(ctrl_left);
  Serial.print(","); 
  Serial.print(ctrl_right);
  Serial.print(","); 
  Serial.print(distance_cm);
  Serial.println(); 
  counter_left = 0;
  counter_right = 0;
}

replace lines 181-183:

    if (ctrl_rx) {
      processCtrlMsg(inChar);
    }

with

    if (ctrl_rx) {
      processCtrlMsg(inChar);
      sendVehicleData();
    }

Now connect everything and press the X button on the controller. The phone will start logging (you will hear the voice feedback "start logging"). Then move the joysticks to generate control signals (in game mode the trigger is used for throttle and joystick for steering). After some time stop the logging by pressing the X button again. Now, go to the log folder and look at the vehicle log file. In place of the speed sensor readings you will now see the control signals that were received. Can you post this here?

parixit commented 4 years ago

@thias15 I am using a LG Rebel 4 (L212VL) which was released in 2018. More info on the phone: https://www.phonemore.com/specs/lg/rebel-4/

I am using a PS4 controller and I hit the Square button to start testing. The X button on the controller doesn't do anything for me.

What's weird is I don't see new zip files created for my test today. Anything I have to do in the app?

Screen Shot 2020-09-21 at 3 57 07 PM

Yes, I updated the firmware code with your edits.

thias15 commented 4 years ago

Logging was working before but not anymore? Logging should be triggered by the X button on the PS4 controller. The circle and square buttons control the indicators.

Do you hear "logging started" and then "logging stopped"? You don't need to do anything besides starting the log and the stopping it which will zip all the data. If the buttons are not working properly, you can also toggle logging the the app.

parixit commented 4 years ago

Yes, I do hear logging started and logging stopped. But I hit the Square button on my PS4 controller to start logging. The X button does not trigger logging. It looks like logging was working before but now I don't see new files in OpenBot folder.

thias15 commented 4 years ago

The phone is on the lower end and only supports Android 7/8. We have tested the Nokia 2.2 which has similar performance as your phone but has Android 9. It worked, but was not able to run the AI models perfectly. You may have a hard time running the AI models at a decent speed as well. Do you have access to another phone?

thias15 commented 4 years ago

Hm. It seems like there is an issue with the button mapping. How about if you toggle logging in the app. Does that work?

parixit commented 4 years ago

I am an iPhone user and this is a burner phone I use to test. Not sure if I want to buy a high end Android phone for this as it defeats the purpose of a low cost bot.

parixit commented 4 years ago

What is weird is when I manually toggle logging in the app (not with a controller) it doesn't say "Logging started", "Logging stopped". It does say it when I press the square button on the PS4 controller.

thias15 commented 4 years ago

Yes, makes sense. Not saying you should buy a new phone, but maybe some in the family has one to try.

thias15 commented 4 years ago

One thing you could do is to print the joystick commands received inside the GameController class. Then you keep the phone connected to the laptop and look at the debug output.

thias15 commented 4 years ago

When you use the button in the app there is no audible feedback. It is just meant for operation via game controller when the screen is not visible.

parixit commented 4 years ago

One thing you could do is to print the joystick commands received inside the GameController class. Then you keep the phone connected to the laptop and look at the debug output.

Can you walk me through this? I see GameController.java in Android Studio.

thias15 commented 4 years ago

Are you already on the Slack channel?

thias15 commented 4 years ago

Actually, even easier. Just add gameController.processButtonInput(event); at the top of the function public boolean dispatchKeyEvent (KeyEvent event) in NetworkActivity.java.

thias15 commented 4 years ago

You should then see toast messages in the app when you press different button. What message shows up when you press the "X" button or the other buttons?

parixit commented 4 years ago

No not in the slack channel, do you need to add me? If so please do add me. So, like this? public boolean dispatchKeyEvent (KeyEvent event) { // Check that the event came from a game controller gameController.processButtonInput(event);

thias15 commented 4 years ago

Yes.

thias15 commented 4 years ago

For checking what controls are being sent out, add the LOGGER to line 985 in CameraActivity.java

  protected void sendControlToVehicle(ControlSignal vehicleControl) {
    LOGGER.d("Control: " + message);**
    if ((usbConnection != null) && usbConnection.isOpen() && !usbConnection.isBusy()) {
      String message = String.format("c%d,%d\n",
              (int) (vehicleControl.getLeft() * speedMultiplier),
              (int) (vehicleControl.getRight() * speedMultiplier));
      usbConnection.send(message);
    }
  }
thias15 commented 4 years ago

Then compile and run app. Keep it connected to USB of the laptop and connect the BT controller. Select 6:Logcat from the bottom, and enter "Control:" in the search window. Now when you move joysticks on the game controller, you should see the messages being sent in the Window. Can you post them here?

Screenshot 2020-09-21 at 23 07 06
thias15 commented 4 years ago

Are you already on the Slack channel?

Join Slack

parixit commented 4 years ago

A/libc: Fatal signal 11 (SIGSEGV), code 1, fault addr 0x8c080000 in tid 15642 (org.openbot), pid 15642 (org.openbot) Changes were not applied. We were unable to deploy your changes: MISSING_AGENT_RESPONSES

thias15 commented 4 years ago

Can you restart Android studio and try again?

parixit commented 4 years ago

Ok, an update on where we are. Big thank you to @thias15 for walking me through. Turned on logging and for some strange reason my PS4 controller is triggering logging with the square button instead of the X. Will try to another usb cable and to try to send a signal from the Arduino to the pins directly.

BrandenSebring commented 4 years ago

@parixit when we used the older phone(note 4) , it would log start and stop with different button (we heard it) and the "blinkers" would get hit with gas or r1 buttons. This sounds like something in android is mapping the buttons differently.

parixit commented 4 years ago

I finally got the motors to move! It turns out I had a bad L298N board. I am pasting firmware code for the Arduino Nano to test the motors. If others are having the same issue this helped me troubleshoot.
`//This is to test the motors //It will send a signal to move the motors

//set up pins

define PIN_PWM1 5

define PIN_PWM2 6

define PIN_PWM3 9

define PIN_PWM4 10

void setup() { // put your setup code here, to run once:

//Outputs pinMode(PIN_PWM1,OUTPUT); pinMode(PIN_PWM2,OUTPUT); pinMode(PIN_PWM3,OUTPUT); pinMode(PIN_PWM4,OUTPUT);

}

void loop() { // put your main code here, to run repeatedly: analogWrite(PIN_PWM1, 128); analogWrite(PIN_PWM2, 0); analogWrite(PIN_PWM3, 128); analogWrite(PIN_PWM4, 0); delay(5000);

}`