search

jmmorato / openddsharp

OpenDDS wrapper for .NET languages
http://www.openddsharp.com
GNU Lesser General Public License v3.0
52 stars 12 forks source link

OpenDDSharp with development on Windows but targeting Linux & docker #148

Closed iisfaq closed 1 year ago

iisfaq commented 1 year ago

I have a Windows machine running Visual Studio 2022.

I want to create a publisher and subscriber that will run as Linux docker containers.

I am using "OpenDDSharp - IDL Project Template (Jose Morato)" new project template.

This creates a .CSProj project with the following

 <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
    <LangVersion>7.3</LangVersion>
    <Platforms>x64;x86</Platforms>
    <Configurations>Debug;Release</Configurations>
    <EnableDefaultCompileItems>false</EnableDefaultCompileItems>
    <RuntimeIdentifiers>win-x64;win-x86;linux-x64;osx-x64</RuntimeIdentifiers>
  </PropertyGroup>

If I try to compile the project I get this error

MSB4044 The "SetEnvVar" task was not given a value for the required parameter "Value".

dbl-clicking on the error open the OpenDDSharp.Standard.IdlGenerator.targets file highlighting this line

<SetEnvVar Variable="DDS_ROOT" Value="$(DDS_ROOT)" />

It comes from this block


 <!--Set Windows environment variables-->
  <UsingTask TaskName="OpenDDSharp.BuildTasks.SetEnvVar" AssemblyFile="$(MSBuildThisFileDirectory)..\tools\OpenDDSharp.BuildTasks.dll" />
  <Target Name="SetEnvVarTarget" BeforeTargets="PreBuildEvent">
    <Message Text="Set OpenDDS environment variables:" Importance="High" />
    <Message Text="DDS_ROOT=$(DDS_ROOT)" Importance="High" />
    <Message Text="ACE_ROOT=$(ACE_ROOT)" Importance="High" />
    <Message Text="TAO_ROOT=$(TAO_ROOT)" Importance="High" />
    <SetEnvVar Variable="DDS_ROOT" Value="$(DDS_ROOT)" />
    <SetEnvVar Variable="ACE_ROOT" Value="$(ACE_ROOT)" />
    <SetEnvVar Variable="TAO_ROOT" Value="$(TAO_ROOT)" />
  </Target>

This tracks back to the following

<!--Custom OpenDDS and ACE/TAO variables-->
  <PropertyGroup Label="OpenDDS Home" Condition="'$(IsWindows)'=='true' And '$(RuntimeIdentifier)'=='win-x86'">
    <DDS_ROOT>$(MSBuildThisFileDirectory)..\tools\DDS_x86</DDS_ROOT>
  </PropertyGroup>
  <PropertyGroup Label="ACE Home" Condition="'$(IsWindows)'=='true' And '$(RuntimeIdentifier)'=='win-x86'">
    <ACE_ROOT>$(MSBuildThisFileDirectory)..\tools\ACE_x86</ACE_ROOT>
  </PropertyGroup>
  <PropertyGroup Label="TAO Home" Condition="'$(IsWindows)'=='true' And '$(RuntimeIdentifier)'=='win-x86'">
    <TAO_ROOT>$(MSBuildThisFileDirectory)..\tools\ACE_x86\TAO</TAO_ROOT>
  </PropertyGroup>

In my case the RuntimeIdentifier is null since it is not set in the csproj,

So I change the CSPROJ and remove the <RuntimeIdentifiers> tag and replace it with this

<RuntimeIdentifier>linux-x64</RuntimeIdentifier>

Then $(RuntimeIdentifier)=linux-x64 but the project is being compiled on Windows

I think the following changes to the IsWindows, IsLinux and IsOSX variables fixes this.


<IsWindows Condition="'$(RuntimeIdentifier)'=='win-x86'">true</IsWindows>
    <IsWindows Condition="'$(RuntimeIdentifier)'=='win-x64'">true</IsWindows>
    <IsLinux Condition="'$(RuntimeIdentifier)'=='linux-x64'">true</IsLinux>
    <IsOSX Condition="'$(RuntimeIdentifier)'=='osx-x64'">true</IsOSX>
iisfaq commented 1 year ago

Nope - more issues.

C:\Users\ccrowe.nuget\packages\openddsharp.standard.idlgenerator\0.8.22127.96\build\OpenDDSharp.Standard.IdlGenerator.targets(151,4): error MSB6004: The specified task executable location "C:\WINDOWS\system32\sh" is invalid.


<Target Name="OpenDDSharpIdlWrapperLinux" BeforeTargets="PreBuildEvent" Inputs="@(IdlFiles)" Outputs="@(IdlFiles->'%(RootDir)%(Directory)..\%(filename)TypeSupport.cs')" Condition="'$(IsLinux)'=='true'">
      <Message Text="Generating OpenDDSharp %(IdlFiles.Identity) wrapper..." Importance="High" />
      <Exec ToolExe="sh" Command="&quot;$(MSBuildThisFileDirectory)../tools/DDS_Linux/bin/openddsharp_idl&quot; &quot;%(IdlFiles.Filename).idl&quot; -cwrapper" WorkingDirectory="$([System.IO.Path]::GetFullPath('$(IntermediateOutputPath)NativeProject'))" IgnoreExitCode="false" ContinueOnError="false" />
  </Target>

Can I even do this step from Windows? I think I am out of my depth here.

iisfaq commented 1 year ago

I am wondering if this part could in fact open a docker container copy the openddsharp_idl and my .IDL compile it and then return it to the next stage?

I will see if I can get this to work

Both Linux & Docker all new to me

iisfaq commented 1 year ago

I got a little further on with a linux docker image


root@098a745caa7b:/tools# export DDS_ROOT=/tools/DDS_Linux
root@098a745caa7b:/tools# export ACE_ROOT=/tools/ACE_Linux
root@098a745caa7b:/tools# export TAO_ROOT=/tools/ACE_Linux/TAO
root@098a745caa7b:/tools# export LD_LIBRARY_PATH=/tools/DDS_Linux/lib:/tools/ACE_Linux/lib
root@098a745caa7b:/tools# "/tools/DDS_Linux/bin/openddsharp_idl" HelloWorldDockerIDL.idl -cwrapper -v

/tools/DDS_Linux/bin/openddsharp_idl: preprocessing HelloWorldDockerIDL.idl
/tools/DDS_Linux/bin/openddsharp_idl: spawning: cc -D__TAO_IDL=0x060516 -I. -I/tools/ACE_Linux/TAO -I/tools/ACE_Linux/TAO/tao -E -D__TAO_IDL_IDL_VERSION=0x040000 -D__TAO_IDL_FEATURES="tao/idl_features.h" -I/tools/ACE_Linux/TAO/orbsvcs -D__OPENDDS_IDL=0x032000 -D__OPENDDS_IDL_HAS_FIXED -I/tools/DDS_Linux -D__OPENDDS_IDL_HAS_ANNOTATIONS /tmp/tao-idli_dEn1pq.cpp
/tools/DDS_Linux/bin/openddsharp_idl: preprocessor "cc" returned with an error

It looks to be spawing somethig called cc but I do not know what cc is or where it is supposed to exist.

jmmorato commented 1 year ago

I see and understand your issue... basically, the IsWindows, IsLinux... variables are used in two different checks:

I will fix it in my next PR but, for the moment, the only option is to compile and run in a Linux machine (your previous comment)

/tools/DDS_Linux/bin/openddsharp_idl: preprocessor "cc" returned with an error

cc is the C preprocessor, required for the native generated code and for the includes in the IDL files. Have you installed in your Linux docker image? Best would be to install sudo apt install build-essential to have all the C/C++ compile tools.

iisfaq commented 1 year ago

Thank you very much for the help….

iisfaq commented 1 year ago

I spent a lot of time today trying to get everything working with the IDL compilation in a Linux docker image.

I have an IDL project (linux and docker)

I create a new ASP.NET MVC application with docker for Linux enabled. I set the runtime_identifer in the project csproj to linux-x64


<PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
    <UserSecretsId>ef01d071-4b05-4ba2-b3d5-ac6faaf0471e</UserSecretsId>
    <DockerDefaultTargetOS>Linux</DockerDefaultTargetOS>
    **<RuntimeIdentifier>linux-x64</RuntimeIdentifier>**
  </PropertyGroup>

I add a project reference to the IDL project above. I also added a nuget reference to OpenDDSharp.Native

The web site runs and when I call

Ace.Init();

I get this

image

2023-02-01 20:38:19 fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1] 2023-02-01 20:38:19 An unhandled exception has occurred while executing the request. 2023-02-01 20:38:19 System.DllNotFoundException: Unable to load shared library 'OpenDDSWrapper' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libOpenDDSWrapper: cannot open shared object file: No such file or directory 2023-02-01 20:38:19 at OpenDDSharp.Ace.UnsafeNativeMethods.NativeInit() 2023-02-01 20:38:19 at OpenDDSharp.Ace.Init() 2023-02-01 20:38:19 at WebApplication1.Controllers.HomeController.Index() in C:\Projects\Energy\DDS\WebApplication1\Controllers\HomeController.cs:line 19 2023-02-01 20:38:19 at lambda_method1(Closure , Object , Object[] ) 2023-02-01 20:38:19 at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments) 2023-02-01 20:38:19 at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync() 2023-02-01 20:38:19 at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) 2023-02-01 20:38:19 at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync() 2023-02-01 20:38:19 --- End of stack trace from previous location --- 2023-02-01 20:38:19 at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context) 2023-02-01 20:38:19 at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) 2023-02-01 20:38:19 at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync() 2023-02-01 20:38:19 --- End of stack trace from previous location --- 2023-02-01 20:38:19 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.gAwaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) 2023-02-01 20:38:19 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context) 2023-02-01 20:38:19 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) 2023-02-01 20:38:19 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync() 2023-02-01 20:38:19 --- End of stack trace from previous location --- 2023-02-01 20:38:19 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope) 2023-02-01 20:38:19 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.gAwaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope) 2023-02-01 20:38:19 at Microsoft.AspNetCore.Routing.EndpointMiddleware.g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger) 2023-02-01 20:38:19 at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context) 2023-02-01 20:38:19 at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

Is there anything that jumps out at you that I am missing?

Chris

jmmorato commented 1 year ago

you should set LD_LIBRARY_PATH environment variable pointing to the execution folder in order to find the "DLLs". Linux does not load libraries (shared objects .so) from the process working directory by default

iisfaq commented 1 year ago

The LD_LIBRARY_PATH was very helpful and this now works for loading the libOpenDDSWrapper and there is no exception for the ace.Init() in my main application.

I have found that after the build task which executes the following command

cmake --build . --config Release

I can see a libHelloWorldDockerIDLWrapper.so in the HelloWorldDockerIDL/obj/x64/Debug/netstandard2.0/linux-x64/NativeProject

It never gets copied to the bin directory or any where else and it never gets to my actual application which is referencing the IDL project.

Is this libHelloWorldDockerIDLWrapper.so file supposed to exist only in the obj folder in the IDL project? It must need to be copied to my main project, but I am not sure of the build task that actually copies this file.

So my project structure is like this

The main application has a number of .so files (which came from the OpenDDSharp.Native package) in the \bin\Debug\net6.0\linux-x64 folder along with my hello worldDockerIDL.dll

I assume that if my libHelloWorldDockerIDLWrapper.so file was copied then everything might work.

Maybe all I need is to copy the libHelloWorldDockerIDLWrapper.so file to the correct bin output folder of the HelloWorldDockerIDL project when my docker compile is finished - like a step #6 below.

Here is my script which runs in my linux Docker container at the moment to do the compiling.

cd /nativeproject
echo "Step #1 of 5 in docker compiler"
"/tools/DDS_Linux/bin/openddsharp_idl" "HelloWorldDockerIDL.idl" -cwrapper -v
echo "Step #2 of 5 in docker compiler"
cmake -DCMAKE_PREFIX_PATH=/tools/DDS_Linux -DOPENDDS_ACE=/tools/ACE_Linux -DOPENDDS_TAO=/tools/ACE_Linux/TAO -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=TRUE -H/nativeproject -B/nativeproject
echo "Step #3 of 5 in docker compiler"
cmake --build . --config Release
echo "Step #4 of 5 in docker compiler"
"/tools/DDS_Linux/bin/openddsharp_idl" "HelloWorldDockerIDL.idl" -csharp -PHelloWorldDockerIDL -v  
echo "Step #5 of 5 in docker compiler"
cp *TypeSupport.cs /idlprojectroot/
echo "Steps all completed docker compiler"

Hope you have an idea that may help.

Chris

iisfaq commented 1 year ago

Well I copied the libHelloWorldDockerIDLWrapper.so into the main projects bin\x64\Debug\netstandard2.0\linux-x64 folder but it just displays this

Unhandled exception. System.DllNotFoundException: Unable to load shared library 'HelloWorldDockerIDLWrapper' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libHelloWorldDockerIDLWrapper: cannot open shared object file: No such file or directory

But it exists in the same bin folder as the other .so files

Chris

jmmorato commented 1 year ago

The library should exist in the same folder that you pointed your LD_LIBRARY_PATH (executable output folder).

When using dotnet to compile the IDL projects, the package reference post-action should copy the native libraries in the output folder, but it seems that you are trying to compile the library by yourself so perhaps some dependencies are missing.

Try to execute ldd libHelloWorldDockerIDLWrapper.so and check the output. Also as indicated in the error message, setting the LD_DEBUG environment variable, for instance to 1, could help to debug that kind of situations

iisfaq commented 1 year ago

Thanks I will see what 8 can find.

Can you tell me how you compile this say in a Linux virtual machine.

I thought vs.net is not available for Linux.

How would I be able to compile this like you do on Linux.

Have you provided instructions already online?

Cheers

Chris

On Thu, 2 Feb 2023 at 6:06 PM, jmorato @.***> wrote:

The library should exists in the same folder that you pointed your LD_LIBRARY_PATH (executable output folder).

When using dotnet to compile the IDL projects, the package reference post-action should copy the native libraries in the output folder, but it seems that you are trying to compile the library by yourself so perhaps some dependencies are missing.

Try to execute ldd libHelloWorldDockerIDLWrapper.so and check the otput, also as indicated in the error message, setting the LD_DEBUG environment variable, for instance to 1, could help to debug that kind of situations

— Reply to this email directly, view it on GitHub https://github.com/jmmorato/openddsharp/issues/148#issuecomment-1413157104, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABMO4UQN7IYIHCVAIMR6UBDWVM6GXANCNFSM6AAAAAAULS7TJQ . You are receiving this because you authored the thread.Message ID: @.***>

jmmorato commented 1 year ago

install dotnet SDK v6.x https://learn.microsoft.com/en-us/dotnet/core/install/linux-ubuntu then use dotnet build dotnet publish commands with the linux-x64 runtime identifier

iisfaq commented 1 year ago

Thanks I will see how I go. Might have been an easier way to go than the way I started, but I have learned a lot with your help 😀

Cheers

Chris

On Thu, 2 Feb 2023 at 6:58 PM, jmorato @.***> wrote:

install dotnet SDK v6.x https://learn.microsoft.com/en-us/dotnet/core/install/linux-ubuntu then use dotnet build dotnet publish commands with the linux-x64 runtime identifier

— Reply to this email directly, view it on GitHub https://github.com/jmmorato/openddsharp/issues/148#issuecomment-1413194919, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABMO4UV24TTQ2KBNIBLJVRDWVNEJ3ANCNFSM6AAAAAAULS7TJQ . You are receiving this because you authored the thread.Message ID: @.***>

iisfaq commented 1 year ago

I decided to try a virtual machine. I am running Ubuntu 22.04 in OracleBox virtual box

I can now build my two applications on Linux

However when I run the code on Linux I am getting a fatal error.

dotnet /app/build/HelloWorldLinuxPublisherService.dll

info: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[62]
      User profile is available. Using '/home/ccrowe/.aspnet/DataProtection-Keys' as key
      repository; keys will not be encrypted at rest.

info: HelloWorldDockerPublisherService.Worker[0]
      ParticipantService.Instance.GetDomainParticipantFactory

info: HelloWorldDockerPublisherService.Worker[0]
      CreateParticipant

info: HelloWorldDockerPublisherService.Worker[0]
      MyTopic

malloc(): invalid size (unsorted)
Aborted (core dumped)

The line that causes the malloc() error is

HelloWorld.MyTopicTypeSupport support = new HelloWorld.MyTopicTypeSupport();

My IDL file is pretty basic

module HelloWorld
{
    @topic
    struct MyTopic
    {
    @key long Key;
    long long DateTimeTicks;
    string<128> Message;
    };
};

Have you ever seen this type of error before?

Chris

iisfaq commented 1 year ago

I decided to start clean again

It now is working without issue on a single linux virtual machine with publisher and subscriber using your vs.net templates.

Chris

iisfaq commented 1 year ago

After a bit more work I have it working by manually running the dockerFile for both the Publisher and Subscriber projects from the windows machine command line.

This creates a Linux docker image and runs each app after compiling them.

I can't hit F5 in Visutal Studio since the projects must target linux becuase of the targets file (OpenDDSharp.Standard.IdlGenerator.targets) currently can't support multiple runtime identifiers and since it checks for Windows it wants to build a windows version not Linux.

But again thank you very much for your help. I am glad I got to where I wanted with this which was to be able to do exactly what I have done.

Hopefully you will have some time to look at the runtimeIdentifiers tag in the csproj files some time soon along with the IsWindows, IsLinux etc.

cheers

chris

jmmorato commented 1 year ago

Glad to read that you achieved your goal!

I will keep this issue open to fix:

Thanks for reporting!

jmmorato commented 1 year ago

The runtime constant checks have been fixed and merged on this PR: https://github.com/jmmorato/openddsharp/pull/159

However, it is still required to compile the IDL project in the target OS due to the required native code compilation .