Questions about modifying image in sensor

[Doctor-TOTORO]

CARLA version: 0.9.15

Platform/OS: windows11

Problem I have experienced: I want to modify the image from sensor and make some attacks on the vehicle in CARLA, but seems that the attribute of image cannot be changed. So I tried generate a carla.Image object, but it was still failed.

What I expected to happen: I'm trying to do some black-box attack in CARLA to observe the influence on camera sensor, hopefully I can generate a new carla.Image object and replace the orignial one, or I can just modfiy the attribute of the orignial image directly. So that the image perceived by sensor would be changed, and I can implement the black-box attack in CARLA.

[EmmettJZK]

I have reviewed your request, and I want to confirm if I understand it correctly: you want to perform image attacks on visual sensors, similar to adversarial examples. The result of the attack is that the perception/control algorithms receiving the image sensor data will behave differently.

Therefore, your requirement is that the visual sensor can generate/transmit images altered by the attack algorithm based on your algorithm. If that is the case, you can implement your algorithm and modify the image data in the file Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Sensor/PixelReader.cpp within the CARLA project path. If you want to achieve the alteration during the rendering process, you can modify the Unreal Engine code.

[EmmettJZK]

Of course, you can also add a callback function containing the tampering algorithm in sensor.listen() in the PythonAPI to achieve the tampering effect.

Hope this helpful.