Only string and Float32 messages are available in blueprints

[ab3nd]

I'm using Unreal Engine 4.18.3 on Windows 10, with ROS and rosbridge running in WSL. I am able to create a topic by having a Topic blueprint object with an Event On Construct that uses an Init to create a topic called "/unreal/target_location". If I set the Message Type to string, and convert the location of the character to a string before sending it, I can send the desired position of a character in the world every tick by using a Send Message and connecting it to the On Tick Event in a blueprint.

Ideally, the Message Type of the Topic would be a geometry_msgs/Pose, rather than a string, but the only options I have available when creating the topic are string and float32.

I see in Topic.cpp at line 171-175, if there are no supported message types (SupportedMessageTypes.Num() == 0), then the string and float32 message types are added, so I assume that something is going wrong with my build, and no message types are added.

I have tried closing the editor, deleting the Binaries and Intermediate directories of the plugin (and of the whole project), and restarting the editor, which forces a rebuild of the plugin. This does not correct the problem.

[Sanic]

Hi!

I'm rarely working with Blueprints, so i can't really comment on that. Maybe the others can help here.

[DarioMazzanti]

Hi, I am testing the plugin on UE 4.23 to compare it with the solution my team has been using for the last year or so - and it seems like this plugin might be a good replacemente for that!

Anyway, I just tried doing what you are asking for, so, even if I am not among the plugin developers, I might be able to help - I hope it's not too late :)

I will just copy-paste the parts to be modified - they can easily be found in the .h and .cpp file.

Header In Topic.h you can modify the enum defining the available topics in your Blueprint by adding the datatype you need, as follows:

UENUM(BlueprintType, Category = "ROS")
enum class EMessageType : uint8
{
    String = 0,
    Float32 = 1,
        // added the following type:
    PoseStamped = 2
};

Then, you can scroll down in the same file, and add the UFUNCTION to be implemented in Blueprint after the existing ones (OnStringMessage, OnFloat32Message), like so:

UFUNCTION(BlueprintImplementableEvent, Category = ROS)
void OnPoseStampedMessage(const FVector& Position, const FRotator& Rotation);

Source

Then you can move to the Topic.cpp file. You can add the new message type to the supported ones within the constructor:

if (SupportedMessageTypes.Num() == 0)
    {
        SupportedMessageTypes.Add(EMessageType::String, TEXT("std_msgs/String"));
        SupportedMessageTypes.Add(EMessageType::Float32, TEXT("std_msgs/Float32"));
// this is the line to be added
        SupportedMessageTypes.Add(EMessageType::PoseStamped, TEXT("geometry_msgs/PoseStamped"));
    }

Finally, move to the UTopic::Subscribe() function and add a case to the switch:

case EMessageType::PoseStamped:
            {
                auto ConcretePoseStampedMessage = StaticCastSharedPtr<ROSMessages::geometry_msgs::PoseStamped>(msg);
                if (ConcretePoseStampedMessage.IsValid())
                {
                    const FVector Pos = FVector(ConcretePoseStampedMessage->pose.position.x,
                        ConcretePoseStampedMessage->pose.position.y,
                        ConcretePoseStampedMessage->pose.position.z);

                    const FQuat Rot = FQuat(ConcretePoseStampedMessage->pose.orientation.x,
                        ConcretePoseStampedMessage->pose.orientation.y,
                        ConcretePoseStampedMessage->pose.orientation.z,
                        ConcretePoseStampedMessage->pose.orientation.w);

                    TWeakPtr<UTopic, ESPMode::ThreadSafe> SelfPtr(_SelfPtr);
                    AsyncTask(ENamedThreads::GameThread, [this, Pos, Rot, SelfPtr]()
                    {
                        if (!SelfPtr.IsValid()) return;
                        OnPoseStampedMessage(Pos, Rot.Rotator());
                    });
                }
                break;
            }

Important things to notice

I am just trying the plugin, so I don't know if internally it is already dealing with the change of basis required to properly exchange geometry data between UE4 and ROS. So, the way the code above is filling the position and rotation variables fed to the blueprint could definitely be wrong!

Also, I didn't test the blueprint/code, but it seems to be receiving a test PoseStamped message fine! This might be a good starting point for your own implementation :)

[sparky002]

This works. Good job. Thank you so much for the help. This should be a part of the ReadMe of this plugin. That and be integrated into the code base. I managed to setup a publishing node in blueprints using that as a starting point and knowledge gained from working with a new subscribing topic. Ideally, with more time, I would like to expand this to blueprint all the existing message types for subscribing and publishing along with including examples on setting up blueprints for passing this information around.

[DominikAUT]

Realy cool, thank you. Is it also possible to use bson with blueprints? For my project it is necessary to create a Videostream from UE to ROS and my C++ knowledge is not the best.

[sparky002]

Realy cool, thank you. Is it also possible to use bson with blueprints? For my project it is necessary to create a Videostream from UE to ROS and my C++ knowledge is not the best.

I believe the code-iai vision plugin handles videostreaming from UE4 to ROS. I got it to work once but it was fairly resource hungry as I roughly recall. https://github.com/code-iai/ROSIntegrationVision

[basvangeuns]

Hi, how do I know what to put in the UFUNCTION for Twist for example?

Then, you can scroll down in the same file, and add the UFUNCTION to be implemented in Blueprint after the existing ones (OnStringMessage, OnFloat32Message), like so:

UFUNCTION(BlueprintImplementableEvent, Category = ROS)
void OnPoseStampedMessage(const FVector& Position, const FRotator& Rotation);

And I also have no idea how to get this right..

Finally, move to the UTopic::Subscribe() function and add a case to the switch:

case EMessageType::PoseStamped:
          {
              auto ConcretePoseStampedMessage = StaticCastSharedPtr<ROSMessages::geometry_msgs::PoseStamped>(msg);
              if (ConcretePoseStampedMessage.IsValid())
              {
                  const FVector Pos = FVector(ConcretePoseStampedMessage->pose.position.x,
                      ConcretePoseStampedMessage->pose.position.y,
                      ConcretePoseStampedMessage->pose.position.z);

                  const FQuat Rot = FQuat(ConcretePoseStampedMessage->pose.orientation.x,
                      ConcretePoseStampedMessage->pose.orientation.y,
                      ConcretePoseStampedMessage->pose.orientation.z,
                      ConcretePoseStampedMessage->pose.orientation.w);

                  TWeakPtr<UTopic, ESPMode::ThreadSafe> SelfPtr(_SelfPtr);
                  AsyncTask(ENamedThreads::GameThread, [this, Pos, Rot, SelfPtr]()
                  {
                      if (!SelfPtr.IsValid()) return;
                      OnPoseStampedMessage(Pos, Rot.Rotator());
                  });
              }
              break;
          }

Maybe the answer is really simple but I'm new to ROS :)

[DarioMazzanti]

Hi, how do I know what to put in the UFUNCTION for Twist for example?

I don't have a project setup with this code at the moment, but based on the Twist message structure (link) the UFUNCTION should be something like:

UFUNCTION(BlueprintImplementableEvent, Category = ROS)
void OnTwistMessage(const FVector& Linear, const FVector& Angular);

The way this UFUNCTION is defined is up to you, in the end: this is an Event you can implement in your blueprint, and based on your needs you could specify less or more arguments. More on that once we get to the following part.

And I also have no idea how to get this right..

Supposing Twist has been added to the EMessageType enum as Twist, you need to add a case for it in your code. That case should be similar to the following:

case EMessageType::Twist:
            {
                auto ConcreteTwistMessage = StaticCastSharedPtr<ROSMessages::geometry_msgs::Twist>(msg);
                if (ConcreteTwistMessage.IsValid())
                {
                    const FVector Linear = FVector(ConcreteTwistMessage->linear.x,
                        ConcreteTwistMessage->linear.y,
                        ConcreteTwistMessage->linear.z);

                        const FVector Angular = FVector(ConcreteTwistMessage->angular.x,
                            ConcreteTwistMessage->angular.y,
                            ConcreteTwistMessage->angular.z);

                    TWeakPtr<UTopic, ESPMode::ThreadSafe> SelfPtr(_SelfPtr);
                    AsyncTask(ENamedThreads::GameThread, [this, Linear, Angular, SelfPtr]()
                    {
                        if (!SelfPtr.IsValid()) return;
                        OnTwistMessage(Linear, Angular);
                    });
                }
                break;
            }

As you can see, the modified parts are the ones regarding the fields of your message. In this case, geometry_msgs::Twist (that's a link to the sourcecode) is made of Angular and Linear, which are geometry_msgs::Vector3 types. All these are ROS messages, which are also defined by this Unreal plugin. Looking at both the ROS documentation and the plugin code, it is possible to understand how these messages are structured.

Going back to the UFUNCTION, you see how it is called by this code, specifically this part:

AsyncTask(ENamedThreads::GameThread, [this, Linear, Angular, SelfPtr]()
                    {
                        if (!SelfPtr.IsValid()) return;
                        OnTwistMessage(Linear, Angular);
                    });

Now, let's imagine you only need Linear, and you don't care about Angular for your application: in that case, you could define your UFUNCTION to have only the const FVector& Linear argument, and call it as OnTwistMessage(Linear);. On the other hand, if you need to pass additional information to your blueprint, you can also do that: you just need to define the UFUNCTION the way you need it, e.g. adding a string parameter, a bool, etc.

Anyway, I hope this code works, since I cannot test it at the moment :P

A not so well written note about the AsyncTask Also, the AsyncTask portion is an interesting one. This is because Events defined in Blueprints cannot run in any thread, you want them in the GameThread (I don't know about other threads actually).

The AsyncTask allows you to schedule the code you specify for execution in the thread you need. In this case that thread is the GameThread. This is important: the above c++ code is running in its own thread, if I remember correctly. So, if you were to just call OnTwistMessage right after retrieving the message, you might have some issues (crashes, errors, etc). By using AsyncTask you are ensuring that the Event declared as UFUNCTION will be execute in the GameThread. As you see, the AsyncTask arguments include the thread you want the specified code to run into, the passed arguments, and finally the code.

[basvangeuns]

@DarioMazzanti Thank you very much! I understand how it works now. After editing Topic.h & Topic.cpp the option still doesn't show up in UE4. Is there anything I have to do after editing the files? Like compiling the Topic.cpp file?

[DarioMazzanti]

@DarioMazzanti Thank you very much! I understand how it works now. After editing Topic.h & Topic.cpp the option still doesn't show up in UE4. Is there anything I have to do after editing the files? Like compiling the Topic.cpp file?

That's right, whenever you change your code in cpp you need to compile it, otherwise the binaries you will be running will be outdated. That can be a good reason for your new enum option not to show up in UE4.

In UE4, if you are changing code inside a plugin you need to compile your project from your IDE, and not from the editor. This is because plugins are compiled as libraries, and are loaded by UE4 when you launch it. So, you have to close the editor, compile your project (which will include the plugins you are working on) and then launch UE4. This should load the latest version of your code!

There are certain languages which don't need to be compiled (interpreted languages, e.g. python, and most of js/web stuff), but that's not the case for cpp. This is actually a huge topic, and I am definitely no expert about it! I think Epic presented some sort of "live code" feature, but I never tried it and at the time it didn't work for plugins.

[basvangeuns]

@DarioMazzanti Thank you! Didn't know. It works! :smiley: