search

gazebosim / gz-sensors

Provides numerous sensor models designed to generate realistic data from simulation environments.
https://gazebosim.org
Apache License 2.0
52 stars 53 forks source link

Image visualization / LED Strip #427

Open rafal-gorecki opened 2 months ago

rafal-gorecki commented 2 months ago

Hello, Is there currently an option to visualize camera data like in gazebo classic? image

azeey commented 2 months ago

I know there's an "Image Display" plugin you can use. Is that sufficient?

image

rafal-gorecki commented 2 months ago

Hello @azeey,

Thank's for replay.

What I really care about is simulating an LED strip. I was planning to make a clever workaround that instead of writing the entire functionality from scratch, I would take a camera and display the image from the camera on the robot, but i notice this function is not available for gazebo ignition.

If you have any advice on how to start your adventure with such a task, I can try to do it. However, I have little experience when it comes to writing the gazebo code itself, so I would like to ask for a little mentoring and if you have and idea how to start this project what to base on.

azeey commented 2 months ago

What I really care about is simulating an LED strip.

I'm not quite sure what you want to do. Maybe a Projector would help?

rafal-gorecki commented 2 months ago

Thank you @azeey, Projector looks interesting.

I would like to explain even better what I have in mind when they write an LED strip. On the robot, I would like to add a link that represents the LED strip. The difference between a projector and an LED strip is that the projector sends an image onto the space it encounters. However, the LED strip would be a permanent element that can be attached to the robot and which displays the image from the topic.

bperseghetti commented 1 month ago

@rafal-gorecki Don't think that would be a great way to show LEDs IMO, you can actually "control LEDs" easily if you are on harmonic.

Here is one example showing spotlights from various object:

https://github.com/gazebosim/gz-sensors/assets/10233412/ee3c73a0-e9da-4325-a107-0eb0466f68c7

Here is how we control LEDs one our ground based vehicles for status lighting (seamlessly through the posix of our Zephyr RTOS based autopilot the same way the real LEDs are controlled):

https://github.com/gazebosim/gz-sensors/assets/10233412/cdf72255-6d99-4875-9017-96600a8ff7a1

rafal-gorecki commented 1 month ago

Hello @bperseghetti and thank you, That's look great! Could you share a demo that can be inspired or what's the name of this plugin?

bperseghetti commented 1 month ago

Hello @bperseghetti and thank you, That's look great! Could you share a demo that can be inspired or what's the name of this plugin?

I sorta hate "one off" plugins, so I extended the use of material color and lights to make it far more versatile than a plugin IMO (it also works with ros_gz bridge):

rafal-gorecki commented 1 month ago

Thanks a lot @bperseghetti. I will try this and let you know about results. ;)

bperseghetti commented 1 month ago

Also if you want you can run this in an ipython notebook for ros_gz based control, just make sure to change world name in it (from default to whatever your world name is) and change the name of the entities you want to control:

import rclpy
import numpy as np
from rclpy.node import Node
from ros_gz_interfaces.msg import MaterialColor, Light
from rclpy.qos import QoSProfile
from rclpy.qos import qos_profile_sensor_data
# ros2 run ros_gz_bridge parameter_bridge /world/default/light_config@ros_gz_interfaces/msg/Light@gz.msgs.Light
# ros2 run ros_gz_bridge parameter_bridge /world/default/material_color@ros_gz_interfaces/msg/MaterialColor@gz.msgs.MaterialColor
class lmTester(Node):

    def __init__(self):
        super().__init__('lm_tester_node')

        self.MaterialColorTopic = '/world/default/material_color'
        self.lightTopic = '/world/default/light_config'
        timer_period = 0.05
        self.timer = self.create_timer(timer_period, self.timerCallback)
        self.i=0
        self.delta=0.75
        self.ranges2=np.hstack((np.linspace(1-self.delta,1+self.delta,25),np.flip(np.linspace(1-self.delta,1+self.delta,25))))
        self.MaterialColorPub = self.create_publisher(MaterialColor,'{:s}'.format(self.MaterialColorTopic), 1)
        self.lightPub = self.create_publisher(Light,'{:s}'.format(self.lightTopic), 1)
        self.last_button = 0

    def timerCallback(self):

        if self.i < 49:
            self.i+=1
        elif self.i >= 49:
            self.i=0
        val2=self.ranges2[self.i]

        msgM=MaterialColor()
        msgM.entity_match=1
        msgM.entity.name="led_0"
        msgM.diffuse.r = 1.0
        msgM.diffuse.g = .1-.1*np.sin(2*np.pi*self.i/49+np.pi/2)
        msgM.diffuse.b = .1-.1*np.sin(2*np.pi*self.i/49+np.pi/2)
        msgM.diffuse.a = 1.0
        msgM.emissive=msgM.specular=msgM.ambient=msgM.diffuse
        self.MaterialColorPub.publish(msgM)
        msgM.entity.name="led_1"
        self.MaterialColorPub.publish(msgM)
        msgM.entity.name="led_2"
        self.MaterialColorPub.publish(msgM)

        msgL=Light()
        msgL.name="led_0"
        msgL.type=1
        msgL.diffuse.r = 1.0
        msgL.diffuse.g = .1-.1*np.sin(2*np.pi*self.i/49+np.pi/2)
        msgL.diffuse.b = .1-.1*np.sin(2*np.pi*self.i/49+np.pi/2)
        msgL.diffuse.a = 1.0
        msgL.specular=msgL.diffuse
        msgL.spot_inner_angle = 1.25+np.sin(2*np.pi*self.i/49)
        msgL.spot_outer_angle = 1.5+np.sin(2*np.pi*self.i/49)
        msgL.spot_falloff = 0.0
        msgL.attenuation_constant = 1.0
        msgL.attenuation_linear = 1.0
        msgL.attenuation_quadratic = 0.0
        msgL.intensity=5.0
        msgL.range=100.0
        self.lightPub.publish(msgL)
        msgL.name="led_1"
        self.lightPub.publish(msgL)
        msgL.name="led_2"
        self.lightPub.publish(msgL)

        return 

if __name__ == '__main__':
    rclpy.init()
    lmMT = lmTester()
    rclpy.spin(lmMT)
    lmMT.destroy_node()
    rclpy.shutdown()
rafal-gorecki commented 1 month ago

Hi @bperseghetti,

What version of ROS are you running? Currently I am using Humble to which I have installed the gz_sim in the Harmonic version. I have a problem uploading an image using ros_gz_bridge, the data is not transferred to the simulator.

Did you build ros_gz_bridge from source or do you run your project some other way?

bperseghetti commented 1 month ago

I'm running Humble with Harmonic, look at using these to install with apt and make sure to get rid of any old ros_gz otherwise that would be for fortress or garden potentially:

sudo apt-get install ros-humble-ros-gzharmonic*
bperseghetti commented 1 month ago

But also I don't project an image to gz from ros only transport images from gz to ros.

rafal-gorecki commented 1 month ago

Hi @azeey I already checked markers. What do you think about creating such carmera image from markers?

rafal-gorecki commented 1 month ago

I use markers and it's work, but it seems that the solution is not the best, except for the computational costs. The results look like this :)

led_strips_vis

Thank you @azeey and @bperseghetti for your support.

scpeters commented 1 week ago

related: a feature request for camera frustum visualization https://github.com/gazebosim/gz-sim/issues/2401

bperseghetti commented 1 week ago

I use markers and it's work, but it seems that the solution is not the best, except for the computational costs. The results look like this :)

Thank you @azeey and @bperseghetti for your support.

Not sure if you want it on or off, but you can remove the green pyramidal bounding box on that visualization if you want to (as a heads up).