Detect crops/plants/vegetation with the laser scanner

[Sagexs]

hi,

I would like to detect the edge of the crop.

i tried to modifyed the modROS.lua to make the lidar look down by changing line 309:

https://github.com/tud-cor/FS19_modROS/blob/642ddf9c7fbbcb6ad53aea7ad8f8e52b80050c29/modROS.lua#L308-L309

to:

localDirectionToWorld(self.instance_veh.cameraNode, i_laser_dx, delta_y * i -0.15, i_laser_dz)

and line 340:

https://github.com/tud-cor/FS19_modROS/blob/642ddf9c7fbbcb6ad53aea7ad8f8e52b80050c29/modROS.lua#L339-L340

to:

local scanning_plane_pitch = -math.atan2((delta_y * i-0.15) , radius)

instead of getting the ground i get the output that you can see in the image.

Any ideas what is causing this?

[gavanderhoorn]

I don't immediately recognise what we're seeing here. The scale of the markers is also difficult to figure out from your screenshot.

The virtual laser scanner is attached to a node in the FarmSim19 model itself. Changing the orientation of the scanner would perhaps be easier / more transparent by either adding a new node with the desired orientation instead of directly editing the lines responsible for emitting the scans.

@tckarenchiang: what do you think?

---

Edit: it would also be much easier to reason about your changes if you show the original code before showing your modifications. I've edited your post to include references to the original lines.

[gavanderhoorn]

@Sagexs: I believe this might be caused by:

https://github.com/tud-cor/FS19_modROS/blob/642ddf9c7fbbcb6ad53aea7ad8f8e52b80050c29/lua/mod_config.lua#L32

by default, the laser scanner ignores returns caused by the terrain. We do this to make the simple demos in fs_mod_ros work.

The circular patterns you see could be lines at the maximum scanning distance of the laser, but I'm not sure.

If you set that to false and reload FarmSim, the terrain should start showing up.

Note: depending on the other settings for the laser scanner, not ignoring terrain could lead to quite a jump in CPU usage of FarmSim.

[Sagexs]

ignore_terrain

Unfortunately, I want to recognise exactly the ground or the edge of the crop, which is why I have set the parameter to true.

can you give me a hint how to change the "node in the FarmSim19"? I changed the maximum range of the scanner, but nothing in the data changed.

it looks like, that directly in front of the scanner the data are correct, and only gets wrong if the y-Value is increasing.

Ps: i also updated the question with a second image

[gavanderhoorn]

First: I believe it would be good to remember Farming Simulator is a game, not a purpose-built simulator for agricultural scenarios.

modROS tries to see how far we can take that game and use it for something it was not created for (ie: as a simulator for use with ROS).

This will inevitably mean there will be things which we cannot do. Whether what you describe (detect crops) is possible will be something to figure out.

Having written that:

I want to recognise exactly the ground or the edge of the crop

I'm not sure whether plants/crops show up on raycast results. @tckarenchiang: would you happen to know?

It could be the raycast mask needs to be changed for that:

https://github.com/tud-cor/FS19_modROS/blob/642ddf9c7fbbcb6ad53aea7ad8f8e52b80050c29/modROS.lua#L93-L99

right now we limit it pretty severely.

@tckarenchiang: could you add a reference to where we found those bits defined? Perhaps plants/crops have a bit that could cause them to show up.

can you give me a hint how to change the "node in the FarmSim19"?

I actually wrote:

The virtual laser scanner is attached to a node in the FarmSim19 model itself.

FarmSim vehicle models consist of many so called nodes, similar to TF frames in ROS. You can see those if you open the vehicle in the Giants Editor (you can download it at gdn.giants-software.com).

One of those nodes is used to attach the virtual laser scanner to in modROS. My suggestion was to see whether it would be 'cleaner' to change the base transform instead of changing the scanning planes directly.

[tckarenchiang]

1 Tilt laser scanner downwards issue (if you are using gmapping/amcl) If you want to tilt the laser downwards, you can tune this parameter to a negative number. The larger the number is the more tilted the laser scanners are

[gavanderhoorn]

@tckarenchiang: would you suggest to change the direction vectors of each individual ray (as @Sagexs is doing IIUC), or would it be better to change the orientation of the "entire laser scanner", such that the fov will point downwards automatically?

[Sagexs]

Here you can see that there is something fundamentally wrong with the calculation of the lidar beams. In real life, lidar beams can never cross because they are at an angle to each other and then rotate. In the picture I have positioned myself sideways against a wall. The beams seem to be calculated in a plane.

The error must be here but I can't see something wrong

https://github.com/tud-cor/FS19_modROS/blob/642ddf9c7fbbcb6ad53aea7ad8f8e52b80050c29/modROS.lua#L306-L310

maybe the problem is not in the .lua but in the way of ploting it into rviz.

[gavanderhoorn]

It would help if you could show a screenshot of the situation in FarmSim as well.

And steps to reproduce it (ie: which level, which vehicle, any local changes to the .lua you've made, etc).

maybe the problem is not in the .lua but in the way of ploting it into rviz.

the .py script taking the data produced by modROS isn't really doing very much. It's a straight JSON->ROS msg conversion.

[Sagexs]

to reproduce it, you can use the original version of the script. Place the vehicle in an alley where you can see a wall on all sides. Set a Decay time > 0 of the scan msgs in Rviz.

It seems like for each layer a new scan msg is produce. If you now plot this scan msg directly into the frame you will get a wrong position of the points cause only the first layer is in a plane. Thats way in driver of mutli layer lidars they use own msgs par example http://docs.ros.org/en/noetic/api/velodyne_msgs/html/msg/VelodyneScan.html . Another way to get around this problem would be to use a pointcloud as message type.

What do you thing? Do i have a thinking error here?

[tckarenchiang]

what we are trying to simulate is not a multi-ring lidar but a multiple single ring laserscanners using sensor_msgs/LaserScan. So we basically have 5 laser scanners which share the same origin (raycastNode) but oriented slightly differently.

lidar beams can never cross because they are at an angle to each other and then rotate

Because the LS_FOV is set as 360 degrees (angle_max- angle_min), the beams cross each other.

[gavanderhoorn]

I believe what @tckarenchiang writes is the cause of the confusion here @Sagexs.

The 5 planes of the 5 virtual laser scanners are not parallel, but all contain the same origin. This causes them to 'cross' if you will at the point of the origin, causing the scan which is 'lowest' in front of the vehicle to become the 'highest' at the back.

Does that correspond to what you observe?

An alternative arrangement could be to keep all laser scan planes parallel to each other by not changing the pitch of the planes, but their Z value. That should not be too difficult I believe @tckarenchiang ?

---

Edit: I don't believe any of this influences the ability to detect the ground and/or crops. That would probably only be determined by the raycast mask (ie: whether FarmSim supports that).

[tckarenchiang]

An alternative arrangement could be to keep all laser scan planes parallel to each other by not changing the pitch of the planes, but their Z value. That should not be too difficult I believe @tckarenchiang ?

No, it shouldn't be. I'll create a PR for this later.

[tckarenchiang]

@Sagexs I've created a PR regarding your issue. If you are interested, you could test that and see if it solves your problem.

[gavanderhoorn]

@Sagexs: with #20 and #23 merged, we're now in a better position wrt the virtual laser scanner I believe.

The remaining issue appears to be detecting crops.

I've updated the title of your issue to reflect that.

[gavanderhoorn]

Discussion about the collision mask we're currently using for the laser scanner in #35.

[tckarenchiang]

As discussed in #35, the laser scanner can't be used for detection of crops/vegetation. I'll close this issue.

---

And as @gavanderhoorn mentioned, if we make some progress with #22, the detection of crops might be possible.

[gavanderhoorn]

This seems to be one area where the limitations of using a game engine (designed for real-time graphics and limited physics) become apparent.

raycastClosest(..) is really for collision detection and for 'occasional' detection of closest objects. @tckarenchiang found a way to use it to implement a virtual laser scanner, but it doesn't appear to work for all objects in the game (lack of collision geometry is likely the cause).