search

gazebosim / gz-sim

Open source robotics simulator. The latest version of Gazebo.
https://gazebosim.org
Apache License 2.0
663 stars 260 forks source link

[Bug]: Camera image does not show the actual color of the mesh #1435

Closed hardesh closed 2 years ago

hardesh commented 2 years ago

Environment

Description

mjcarroll commented 2 years ago

When you applied the material, how did you do it?

Can you upload the model and sdf somewhere so that it can be reproduced?

hardesh commented 2 years ago

Hi @mjcarroll,

I applied the material using UV unwrap in Blender.

Here's the model apriltag_block.zip

hardesh commented 2 years ago

Hey @mjcarroll, is there any update on this? curious to know the details

also, can you point me towards a design doc if it exists? want to know more about the workings of the simulator

mjcarroll commented 2 years ago

is there any update on this? curious to know the details

On my list to take a look at this morning.

also, can you point me towards a design doc if it exists?

Probably the best jumping-off point is here: https://ignitionrobotics.org/docs/all/architecture

mjcarroll commented 2 years ago

I'm having trouble reproducing, this is what it looks like with fortress binaries on my system:

From the GUI Viewer image

From the sensor image viewer image

The camera parameters are:

            <sensor name="camera" type="camera">
                <always_on>1</always_on>
                <update_rate>20</update_rate>
                <camera name="camera">
                    <horizontal_fov>1.0472</horizontal_fov>
                    <lens>
                        <intrinsics>
                            <!-- fx = fy = width / ( 2 * tan (hfov / 2 ) ) -->
                            <fx>1108.5</fx>
                            <fy>1108.5</fy>
                            <!-- cx = ( width + 1 ) / 2 -->
                            <cx>640.5</cx>
                            <!-- cy = ( height + 1 ) / 2 -->
                            <cy>480.5</cy>
                            <s>0</s>
                        </intrinsics>
                    </lens>
                    <image>
                        <width>1280</width>
                        <height>960</height>
                        <format>R8G8B8</format>
                    </image>
                    <clip>
                        <near>0.01</near>
                        <far>300</far>
                    </clip>
                    <noise>
                        <type>gaussian</type>
                        <mean>0</mean>
                        <stddev>0.007</stddev>
                    </noise>
                </camera>
            </sensor>
hardesh commented 2 years ago

I'm using the sensor like this. Am I missing something?

<sensor name="color" type="camera">
        <pose>0 0 0 0 0 0</pose>
        <update_rate>6</update_rate>
        <always_on>0</always_on>
        <camera>
          <horizontal_fov>1.22</horizontal_fov>
          <image>
            <width>640</width>
            <height>480</height>
          </image>
          <clip>
            <near>0.1</near>
            <far>50</far>
          </clip>
          <lens>
            <intrinsics>
              <fx>615.9603271484375</fx>
              <fy>616.227294921875</fy>
              <cx>419.83026123046875</cx>
              <cy>245.1431427001953</cy>
              <s>0</s>
            </intrinsics>
          </lens>
        </camera>
      </sensor>
mjcarroll commented 2 years ago

The only difference I'm seeing is: <format>R8G8B8</format>, but given that you are seeing other color, that seems unlikely to be it.

How are you loading the model, is it possible that the GUI/server side is using a cached version?

hardesh commented 2 years ago

I'm loading the model using the create node in ros_ign_gazebo. I don't think it's loading a cached version. I had made some changes in the model and those were directly reflected in gazebo after building

iche033 commented 2 years ago

can you check the first of ignition-rendering6 you have?

One thing to try is updating to the latest version, which is currently 6.3.1. Also make sure that it's running with the ogre2 render engine. There is also an ~/.ignition/rendering/ogre2.log to indicate if there're any issues (look for ogre exceptions)

Kakcalu13 commented 2 years ago

Hi,

I saw this and figured I could help.

So, I tested it on my sdf. See the result image

What is your version on Fortress? You can check by ign gazebo --version

I looked into your camera paramter. I think it's because you are calling camera instead of rgb_camera.

See here from my code:

      <sensor name="rgbd_camera" type="rgbd_camera">
          <update_rate>100</update_rate>
          <visualize>true</visualize>
          <always_on>1</always_on>
          <topic>IR0</topic>
          <camera>
            <horizontal_fov>1.2</horizontal_fov>
            <image>
              <width>1</width>
              <height>1</height>
              <format>L_INT8</format>
            </image>
            <clip>
              <near>0.02</near>
              <far>10.0</far>
            </clip>
          </camera>
      </sensor>

Obviously, you will need to adjust this to your own setting, but this is official sample from ign's rgb camera.

Can you let us know if it's fixed on your end? I humbly encourage you to discuss more about this in gazebo community since this is related to code isssues, not ign bug.

hardesh commented 2 years ago

Hi @Kakcalu13, Thanks for letting me know. I haven't had a chance to take a look at this :sweat_smile: . It's on my TODO list. I'll try again on Monday

hardesh commented 2 years ago

Just installed ign on fresh ubuntu 22! ign gazebo --version is giving me 

Ignition Gazebo, version 6.9.0
Copyright (C) 2018 Open Source Robotics Foundation.
Released under the Apache 2.0 License.

Still facing the same issue

hardesh commented 2 years ago

Hi @iche033, Checked the log. Found some relevant lines related to the model:

Added resource location '/home/harshal/sim_ws/install/movus_ignition_sim/share/movus_ignition_sim/models/apriltag_block/meshes' of type 'FileSystem' to resource group 'General' with recursive option
23:12:27: Initialising resource group General
23:12:27: Added resource location '/home/harshal/sim_ws/install/movus_ignition_sim/share/movus_ignition_sim/models/apriltag_block/meshes/' of type 'FileSystem' to resource group 'General'
23:12:27: Texture memory budget exceeded. Stalling GPU.
23:12:27: Stalling was not enough. Freeing memory.
23:12:27: Can't assign material scene::Material(65429) because this Material does not exist. Have you forgotten to define it in a .material script?
23:12:27: Can't assign material scene::Material(65428) because this Material does not exist. Have you forgotten to define it in a .material script?
23:12:27: Vertex Shader: 100000000VertexShader_vs
Fragment Shader: 100000000PixelShader_ps
 GLSL validation result : 
active samplers with a different type refer to the same texture image unit
23:12:27: Vertex Shader: 100000001VertexShader_vs
Fragment Shader: 100000001PixelShader_ps
 GLSL validation result : 
active samplers with a different type refer to the same texture image unit

I'm can't understand what exactly is happening though

Kakcalu13 commented 2 years ago

Just installed ign on fresh ubuntu 22! ign gazebo --version is giving me

Ignition Gazebo, version 6.9.0
Copyright (C) 2018 Open Source Robotics Foundation.
Released under the Apache 2.0 License.

Still facing the same issue

I wonder if it is because of ubuntu 22. :thinking:

Let me know if the code I gave you changed anything

hardesh commented 2 years ago

I wonder if it is because of ubuntu 22. thinking

I was facing the same issue on Ubuntu 20

Also, your snippet did not work

hardesh commented 2 years ago

Hi everyone,

I just checked the world file today and found out that I was using OGRE1 engine. Extremely sorry for wasting everyone's time. Closing this issue.

Is there any doc where I can read about the difference between OGRE1 and OGRE2?

Kakcalu13 commented 2 years ago

Here is the doc

Here is example on ogre and ogre2

Speak of ogre1/ogre2...I wonder if it is possible to run ogre1 while camera and everything else on ogre2 or highest quality. :thinking: I think this is great way to experiment!

hardesh commented 2 years ago

Thanks for sharing!