Steering wheel joystick with Carla

[sandman]

Hi, Has anyone successfully managed to set up (and calibrate) a steering wheel and brake joystick (like the Logitech Driving Force GT for example) with Carla?

I'm trying to set one up right now and would like to learn of any pitfalls.

[sandman]

So I got it to work: https://gist.github.com/sandman/366e45d3a836da1a2a90fe9eeccd689a

pygame makes this super easy.

[sandman]

Sorry for the 'hacky' code. I can refine it over time. But essentially the joystick input depends on the model and so it could be difficult to make a generalized implementation. But using jstest to query the joystick model, a proper commit can be made.

[felipecode]

Hey @sandman , thanks for the interest. I did this myself also ! For the Logitech G29, using pygame, in a similar way than you. But the code is even more hacky than yours, so we didn't release. The problem is, as you said, each joystick is different, so you need to re-adapt the keys. And people would want to configure the mapping from joystickinput to steering in a way they like like. A jstest version would probably be better. We could eventually think of something running over the client to provide support for driving with joysticks in a nice way, joining what you said on issue #89

[sandman]

Hey @felipecode , nice that you got it to work! How about simply adding a short HOWTO: in the Documentation and people who are interested can get working on it? I can help with this.

Another issue I am facing now is the calibration of the steering but mainly the throttle and brake pedals. I think it is not so easy! Any ideas how to do it scientifically?

[zhangjun-xyz]

@sandman Which steering wheel did you use? Logitech G920 or Logitech G29? Can you offer more details? Thanks!

[sandman]

@zhangjun-xyz I'm using the Logitech Driving Force GT. But another model should work just as well. You can check out my gist posted earlier for the code.

[zhangjun-xyz]

@sandman Thanks for your kind reply. :+1:

[fbuchinger]

Hi @sandman, I would also be interested in using my Logitech G25 steering wheel with Carla. Unfortunately your gist doesn't seem to work anymore with Carla 0.81 and Python 2.7.11:

File "carla_manual_control.py", line 42, in <module>
    from carla import CARLA
ImportError: cannot import name CARLA

To me it seems the Python Client API has changed. Or is this a Python 2.7 vs 3.3 thing?

[sandman]

Hi @fbuchinger , You need pygame to get the steering wheel running. I am upgrading now to CARLA 0.81 - will test it and post an update here later today.

[sandman]

Here's an updated gist that works in 0.8.1: https://gist.github.com/sandman/24970f0a6fe4d40293ff86159ee740cb

Note that only the throttle and steering commands are used from the steering wheel. For additional commands (like reverse, scene change etc.) you would need to map the joystick input to the corresponding control. This can be done in a straightforward way with jstest.

[fbuchinger]

Thank you! Will check it out tonight.

[sandman]

By the way, currently its very difficult to steer the car comfortably as the wheel & throttle are not properly calibrated and there is no official kernel support for Force Feedback. I have a naive linear steering control that does not perform well in practice. If you have a solution for this, please share with us!

[fbuchinger]

@sandman seems that there are undefined globals in lines 246 to 254:

>python manual_control_steering_wheel.py
STARTING in a moment...

Starting new episode...
Joystick ID: 0 Init status: 1 Axis(1): 0
Steering angle: 0.000000 Throttle: 0.000000 Brake: 0.000000
Traceback (most recent call last):
  File "manual_control_steering_wheel.py", line 461, in <module>
    main()
  File "manual_control_steering_wheel.py", line 450, in main
    game.execute()
  File "manual_control_steering_wheel.py", line 163, in execute
    self._on_loop()
  File "manual_control_steering_wheel.py", line 246, in _on_loop
    print('reverse: %d toggle: %d' % (reverse, toggle))
NameError: global name 'reverse' is not defined

Script ran after I commented out the mentioned lines. Nevertheless I couldn't start the car properly due to the low frame rate (~7 fps).

I was also wondering whether https://github.com/schmerdy/openxc-wheel could be useful for the steering wheel support in carla.

[fbuchinger]

Regarding better steering wheel support, I'd recommend to check out the logitech-g27 library for nodejs. It offers a more advanced steering wheel control routine that differentiates between coarse and fine movements: https://github.com/ForestMist/logitech-g27/blob/master/code/data-map.js#L254 The library also supports force feedback control (see scripts in test folder).

[felipecode]

Hey guys any progress on that ? We would be interested into incorporating an interface for human driving as a PR. CHeers !

[fbuchinger]

Hi @felipecode! I think @sandman's script is a step in the right direction, but not yet ready for a PR. I had to comment out some lines to get it to run (see above), as a consequence it doesn't have reverse gear logic.

But the bigger issue is the lack of compatibility between the different (Logitech) steering wheels. My Logitech G25 wheel has a completely different pedal/button layout than @sandman's Logitech GT - as a consequence, I could only use the steering wheel itself to control the car, but not the pedals, gear shift, button etc.

OpenXC's wheel.py class clearly shows that PyGame's Joystick control is too low-level/primitive for true steering wheel control. Unfortunately this class only works for G27 wheels and is unmaintained. I'm afraid you would have to invest quite some work to get it to run with other Logitech Wheels.

[sandman]

Hi @fbuchinger, I agree that the script I provided is just a quick hack to map the essential controls for driving. Note that not all driving controls may be necessary in CARLA. For example: Assuming automatic transmission, the necessary controls include throttle, brake, steering and reverse. Hence there is no need to comprehensively map all controls from all wheels IMO. The focus can be to support a few wheels completely rather than all wheels completely. What do you think @felipecode ?

Coming to the real issue which is steering wheel calibration. CARLA takes input values from -1.0 to 1.0 unlike games like ETS/GTA etc. which have a higher range. Same applies for the throttle and brake. So the effective range is limited, making the wheel very sensitive. It would be great if we could come up with a smart solution for this..

[fbuchinger]

Hi @sandman, I agree with you that we should only port the necessary controls wheel, throttle, brake and reverse. Unfortunately, the pedal layout differs quite a lot between the different wheels.

The most simple solution i can think of would be a wheel.ini file with sections for every wheel that maps the input to every command (inspired by openxc's wheel_config.py):

[G27 Racing Wheel]
steering_wheel = 0
accelerator = 1
brake = 2
clutch = 3
gear_1 = 12
gear_reverse = 22

[G25 Racing Wheel]
#config for Logitech G25

(the section title corresponds to pygame's joystick name)

In order to create mappings for unknown wheels, we could add a cli option -m or --mapwheel. It would interactively ask the user to perform the required actions:

> python manual_control.py --mapwheel
Unknown wheel 'Logitech G25' detected.
Please press accelerator pedal.
Axis 2 configured for accelerator.
Please press brake pedal.
....

The mapwheel command would simply append the configuration to the ini file.

What do you think?

[felipecode]

Hello Guys, I like the @fbuchinger, solution for having different wheels. It think mapping steering , throttle, brake reverse gear and hand brake is enough.

About the wheel sensitivity, i didn't see any problems when I was working with it. You just need a good mapping function from the wheel output signal to -1,1. Some soft exponential function with dead zones maybe ?

To create the autocentering and force feedback i was using this library https://github.com/MadCatX/FFBChecker. Maybe we could find a cleaner solution ? We will have some people working on this in fulltime in house for a while. @rohitgajawada and @madan96 Do you already have a fork where we can contribute ? We will make one if not. Thanks a lot guys !

[fbuchinger]

Hi @felipecode,

thanks for the feedback!

About autocentering/force feedback: the node.js driver for the Logitech G27 supports setting these, it is directly built upon node-hid, which provides low-level access to USB-based human input devices.

Maybe you could use PyUSB for a similar approach (which would be pretty low-level though and requires testing of each wheel).

AFAIK there is no steering wheel for for CARLA, @sandman just created a gist containing his modified script.

[felipecode]

Some folks released a first version for joystick control for logitech G29. It was very based on @sandman GIST. https://github.com/rohitgajawada/carla/tree/intern/PythonClient The example can be run on "human_controller.py" . It uses the ini files as proposed by @fbuchinger . On this fork, I can see the following issues:

1. Manual control and human control now are duplicate code. We will try to join both.
2. We need an integrated solution to have the autocentering and the force feedback without external dependencies. Thanks @fbuchinger for the suggestions.
3. Pygame is very slow #421.
4. Having a HUD as described in issue #89 would be cool. Has anyone worked on that ?

Cheers

[fbuchinger]

@felipecode thanks to you and your team for implementing the prototype so quickly! Will check it out tonight.

Regarding (3): I think it's important to find out why PyGame is slow in our scenario. Is the slowness caused by the CPU/GPU? Then it could make sense to use another 3D engine within Pygame (ModernGL comes to my mind). If the poor performance is caused by IO (e.g. blocking socket.read() calls that delay image downloads), then we should switch to an async networking framework (e.g. Twisted or Python 3 asyncio).

[fbuchinger]

Another thought on the performance issue: have you tried to run the client script on pypy instead of cpython? Pypy 6 was just released and offers full pygame compatibility along with up to 30x speedup: http://renesd.blogspot.co.at/2018/03/pygame-on-pypy-usable.html?m=1 (in some cases it might also be slower though)

[fbuchinger]

I did a first check on humancontrol.py and added the Logitech G25 to the wheel config: https://github.com/fbuchinger/carla/commit/40ad9ab5b3723f4200da570566b0ded8f44e2f4a Unfortunately, the code runs into an exception when connecting to carla server - any hints on that?

[madan96]

Could you please post the exception you're getting? It'd be easier to debug that way.

[fbuchinger]

@madan96 here is my python 2.7 traceback when running https://github.com/fbuchinger/carla/blob/intern/PythonClient/human_control.py

> python human_control.py
INFO: listening to server localhost:2000

Welcome to CARLA manual control.

Use ARROWS or WASD keys for control.

    W            : throttle
    S            : brake
    AD           : steer
    Q            : toggle reverse
    Space        : hand-brake
    P            : toggle autopilot

    R            : restart level

STARTING in a moment...

Starting new episode...
Traceback (most recent call last):
  File "human_control.py", line 486, in <module>
    main()
  File "human_control.py", line 478, in main
    game.execute()
  File "human_control.py", line 235, in execute
    self._on_loop()
  File "human_control.py", line 274, in _on_loop
    measurements, sensor_data = self.client.read_data()
  File "c:\Users\fbu\dev\carla\PythonClient\carla\client.py", line 127, in read_data
    return pb_message, dict(x for x in self._read_sensor_data())
  File "c:\Users\fbu\dev\carla\PythonClient\carla\client.py", line 127, in <genexpr>
    return pb_message, dict(x for x in self._read_sensor_data())
  File "c:\Users\fbu\dev\carla\PythonClient\carla\client.py", line 175, in _read_sensor_data
    yield self._parse_sensor_data(data)
  File "c:\Users\fbu\dev\carla\PythonClient\carla\client.py", line 180, in _parse_sensor_data
    return parser.name, parser.parse_raw_data(data[4:])
  File "c:\Users\fbu\dev\carla\PythonClient\carla\client.py", line 196, in parse_image
    return sensor.Image(frame_number, width, height, image_type, fov, data[24:])
  File "c:\Users\fbu\dev\carla\PythonClient\carla\sensor.py", line 149, in __init__
    assert len(raw_data) == 4 * width * height
AssertionError

[fbuchinger]

@madan96 any updates on my error? Will try to look into it again next weekend. BTW I noticed your "intern" branch (which i forked) has disappeared. Should I wait for your merge to develop and then re-create a fork based on develop?

[madan96]

@fbuchinger Sorry for the delay. I was trying to reproduce the error but couldn't do it. My client side is updated with the master and I'm using the binaries available on GitHub for running the server side. Since the error occurs while receiving sensor data, there's a bit of confusion that it might be due to the server side. Which binary are you using for CARLA server? Did you do a build from master branch?

[fbuchinger]

@madan96 my bad, I was still using the 0.8.1 release. Now the error has gone. Which branch are you using now for the joystick support?

[madan96]

@fbuchinger For now the code is still in https://github.com/rohitgajawada/carla/tree/intern/PythonClient. But we are working on improving the code and we'll soon try to have it on https://github.com/carla-simulator/carla/tree/CS1_client_side_agents

[fbuchinger]

Just saw that 0.83 was released on Friday. Will test my code with that version and submit a PR against the intern branch then.

[imran514]

I am trying to use Logitech G920, can any give me some input where should start from? Does @fbuchinger code still works ? can i use it for G920 too ?

[fbuchinger]

Hi @imran514! At the moment you would have to download human_control.py and wheel.ini from Rohits CARLA fork. Then you start CARLA and the human_control.py client.

Now steer the wheel and push the throttle/brake pedals. Grep for the Joystick ID: log output and take note of the axis ID for the wheel and each pedal. Then open wheel_config.ini and adjust them.

Next, push the button on your steering wheel that should act as a handbrake and take note of its jsButton ID. Then switch to the reverse gear and do the same (Hint: watch out for the log line after Steer Cmd to obtain the button ids). Then add both button ids to wheel_config.ini and you should be good to go.

[rohanpansare]

@fbuchinger and Rohit, Thanks for the scripts. I am able to map it to Fanatec joystick and get it working as intended. But, I did observe an odd behavior where the vehicle loses focus even when window is focussed on client and it does not respond to any controls after a few seconds. And that behavior was also common when I ran the manual_control.py script for WASD controls. I do see the logs and their event values getting changed as you mentioned above but the car doesn't respond to any input after a while. Do you know why that may be happening?

[germanros1987]

Hey...how about finishing a PR of this...? :P

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[sangeetsu]

@imran514 Did you get the Logitech G920 to work? If so please explain what you did.

[germanros1987]

checkout the branch features/steering_wheel_control https://github.com/carla-simulator/carla/compare/features/steering_wheel_control . I just added code to use the steering wheel in manual mode.

On Wed, Feb 13, 2019 at 11:25 AM sangeetsu notifications@github.com wrote:

@imran514 https://github.com/imran514 Did you get the Logitech G920 to work? If so please explain what you did.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/carla-simulator/carla/issues/88#issuecomment-463332840, or mute the thread https://github.com/notifications/unsubscribe-auth/Aku6zI70ILeYvdiQZaQanLFq7jasJ9fgks5vNGaNgaJpZM4RCLfY .

[sangeetsu]

@germanros1987 With a little help from this http://www.draisberghof.de/usb_modeswitch/bb/viewtopic.php?f=2&t=2510 and your code, I got the G920 working on Carla. Thank you!

[sangeetsu]

After updating to 0.9.4, I'm getting the following error Traceback (most recent call last): File "manual_control_steeringwheel.py", line 855, in <module> main() File "manual_control_steeringwheel.py", line 847, in main game_loop(args) File "manual_control_steeringwheel.py", line 784, in game_loop world.tick(clock) File "manual_control_steeringwheel.py", line 176, in tick self.hud.tick(self, clock) File "manual_control_steeringwheel.py", line 421, in tick 'Map: % 20s' % world.world.map_name, AttributeError: 'World' object has no attribute 'map_name' This happens when I run manual_control_steeringwheel.py Can someone explain the error? Thank you. EDIT(fix) - map_name was removed from world, use world.get_map().name (c/o @nsubiron)

[mallela]

@felipecode I'm having trouble understanding the parameters to be set and the effect it will have on force feedback/autocentering in FFBChecker. Is there any helpful link you know of that discusses the parameter description? I'm using the Logitech G29. I experimented with some autocentering and gain values but don't understand the other parameters (replay length/delay, repeat, attack level/length fade level/length).

[damascenodiego]

Dear members,

I managed to set values to the FF_AUTOCENTER parameter of the Logitech steering wheel using the python-evdev library. I found this parameter digging the code of FFBChecker that was suggested by @felipecode

Here it follows the code to test the autocenter feature:

import evdev from evdev import ecodes, InputDevice device = evdev.list_devices()[0] evtdev = InputDevice(device) val = 65535 # val \in [0,65535] evtdev.write(ecodes.EV_FF, ecodes.FF_AUTOCENTER, val)

Now, my question is: How can we use this parameter to make the driving experience a little more realistic?

[walzimmer]

@damascenodiego Thank you for the code snippet. It will set force feedback to the maximum if setting val = 65535. We need to add this code into the game loop of manual_steering_wheel.py and make it dependent on the steering wheel angle and the velocity. E.g.

device = evdev.list_devices()[0]
evtdev = InputDevice(device)
while True:
  if steering_angle > 0.75 and velocity > 100:
    val = 65535
  elif steering_angle > 0.5 and velocity > 50:
    val = 65535 * 0.5
  else:
    val = 0
  evtdev.write(ecodes.EV_FF, ecodes.FF_AUTOCENTER, val)

[damascenodiego]

That's exactly my idea. :)

I'll reverse my code which uses updated libraries updated to the versions that Carla is using now and I'll do my push request.

On Fri, 5 Jul 2019, 11:35 Walter Zimmer, notifications@github.com wrote:

@damascenodiego https://github.com/damascenodiego Thank you for the code snippet. It will set force feedback to the maximum if setting val =

65535. We need to add this code into the game loop of manual_steering_wheel.py and make it dependent on the steering wheel and the velocity. E.g. device = evdev.list_devices()[0] evtdev = InputDevice(device) while True: if steering_angle > 0.75 and velocity > 100: val = 65535 elif steering_angle > 0.5 and velocity > 50: val = 65535 * 0.5 else: val = 0 evtdev.write(ecodes.EV_FF, ecodes.FF_AUTOCENTER, val)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/carla-simulator/carla/issues/88?email_source=notifications&email_token=AAL5HIDV4C3PTHMYZVASMPTP54PVXA5CNFSM4EIIW7MKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZJGAZI#issuecomment-508715109, or mute the thread https://github.com/notifications/unsubscribe-auth/AAL5HIDERUXZB62R23B6WL3P54PVXANCNFSM4EIIW7MA .

[soulslicer]

Hey is there a version that works with the newest version of CARLA?

[soulslicer]

@damascenodiego

Have you written any code to map the force feedback correctly?

[damascenodiego]

Hi everyone, I have implemented some primary implementation of force feedback using a previous version of the Carla script manual_control_steeringwheel.py.

This can be found in my repo's fork.

The idea was pretty simple:

1. I imported the evdev library to use the FF_AUTOCENTER feature.
2. I implemented a function mapping the car speed to the FF_AUTOCENTER parameter.

The FF_AUTOCENTER allows setting the steering wheel to the center at some specific intensity defined by the numeric value passed as parameter.

The function was defined with the help of @hlsa by running some empirical/manual tests.

[soulslicer]

Does it behave like in real life? Like if I am turning and halfway I stop, the steering should stop turning. And if i continue to press on the acceleration, the wheel should turn automatically

[damascenodiego]

No, unfortunately, it doesn't. This is a very primitive version that just centers the wheel accordingly to the speed. The faster you are, the higher is the value set to FF_AUTOCENTER up to some speed limit.

[damascenodiego]

Unfortunately, I didn't try this code on Windows. The only issues I had with pedals happened due to the switch button. Depending on its position, the mapping between commands and buttons/pedals will change and hence, it may not work properly.

On Wed, Jan 15, 2020 at 3:39 PM aakashsehgal notifications@github.com wrote:

@damascenodiego https://github.com/damascenodiego I tried this with carla 0.9.6 (Windows) and it works...a small correction : line 527 should be world.world.get_map().name The steering works but somehow the pedals dont seem to work. I have rechecked the connections but didnt help. Any ideas ? -- Diego Damasceno