search

carla-simulator / carla

Open-source simulator for autonomous driving research.
http://carla.org
MIT License
11.18k stars 3.61k forks source link

Carla UnrealEngine changed files paths after using a newer Carla version #7714

Open PatrickPromitzer opened 4 months ago

PatrickPromitzer commented 4 months ago

Platform Ubuntu 20.04

I am currently working with the Carla d version 0.9.15, and have the source version 0.9.14 and 0.9.13 as a backup. I tested the Carla version 0.9.15.2, to see if some bugs (updated download link of the "Content.tar.gz" file) are fixed.

I tried to create a new Carla package with the Carla source version 0.9.15, and noticed that some paths in my local "CarlaUnreal/UnrealEngine" installation changed to the folder of the 0.9.15.2 installation path. For example: /apps/UnrealEngine_4.26/Engine/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/Link-libUE4Editor-CarlaUE4.so.link.sh line:12: mv "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.so.temp" "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.so" line 13: chmod 644 "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.debug"

/apps/UnrealEngine_4.26/Engine/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/libUE4Editor-CarlaUE4.so.psym Line7: FILE 4 /apps/carla_source_0.9.15/Unreal/CarlaUE4/Source/CarlaUE4/CarlaUE4.cpp

After trying to create the Carla package, "make launch" didn't work anymore (it worked the last time it worked on my source installation).

Problem you have experienced: The files in my local "CarlaUnreal/UnrealEngine" repository changed, or was linked to the local Carla repository.

What you expected to happen: I expect the files in the CarlaUnrealEngine installation will not getting changed, except with a manual "git pull" call or other manual update. or Update the paths with each "make package" call.

Blyron commented 4 months ago

Hey Patrick,

In dev version of CarlaUE4 you should not have anything related to CarlaUnreal folder as it only belongs to ue5-dev. It seems like you have a missmatch between versions. We recommend to have carla clones two times if you wanna work with ue4 and ue5 versions.

We are gonna release a document about how to work with Carla UE5 in next weeks. For now, we suggest you to do switch branch to dev, or an specific branch, do a git clean -fdx and continue working as you were until now.

Please, do not mix 'dev' and 'ue5-dev'

Apologizes Aaron

PatrickPromitzer commented 4 months ago

Hi Aaron,

I didn't switch branch or use the UE5 branch.

I cloned the repository with 

git clone --depth 1 -b carla https://github.com/CarlaUnreal/UnrealEngine.git /apps/UnrealEngine_4.26

a few months ago, and didn't change the UnrealEngine_4.26 file for months, too. image Jän = January Mai = May

i did the same with the Carla source version

git clone --depth 1 -b 0.9.15 hhttps://github.com/carla-simulator/carla.git /apps//apps/carla_source_0.9.15

image Jän = January Mai = May

I did the same with Carla 0.9.15.2, but deleted the folder a week after installing it.

I can switch to the dev branch, but if the normal branch doesn't change without me updating it manually, I want to keep it that way to prevent breaking something.

I want to be sure this behavior is documented (as an issue), if someone has this kind of problem in the future.

BR, Patrick

Blyron commented 4 months ago

Then this is so strange... nothing should have changed

PatrickPromitzer commented 4 months ago

I believe the problem is, that the files I mention before have an absolute path to the Carla source version. At some point 2 files are generated, and it doesn't update with just using "make launch"

I can look into it a bit deeper, if I have some free time.

PatrickPromitzer commented 4 months ago

The problematic file is this one

/apps/UnrealEngine_4.26/Engine/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/Link-libUE4Editor-CarlaUE4.so.link.sh

Line 2
#!/bin/sh
# Automatically generated by UnrealBuildTool
# *DO NOT EDIT*

set -o errexit
"/apps/UnrealEngine_4.26/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v17_clang-10.0.1-centos7/x86_64-unknown-linux-gnu/bin/clang++" -fuse-ld=lld -rdynamic -shared -Wl,--gdb-index -Wl,-rpath='${ORIGIN}' -Wl,-rpath-link='${ORIGIN}' -Wl,-rpath='${ORIGIN}'/.. -Wl,-rpath='${ORIGIN}'/../../../Engine/Binaries/ThirdParty/Qualcomm/Linux -Wl,-rpath='${ORIGIN}'/../../../Engine/Binaries/ThirdParty/OpenVR/OpenVRv1_5_17/linux64 -Wl,-rpath='${ORIGIN}'/../../../Engine/Binaries/ThirdParty/PhysX3/Linux/x86_64-unknown-linux-gnu -Wl,--disable-new-dtags -Wl,--as-needed -Wl,--hash-style=gnu -Wl,--build-id -target x86_64-unknown-linux-gnu "--sysroot=/apps/UnrealEngine_4.26/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v17_clang-10.0.1-centos7/x86_64-unknown-linux-gnu" -B"/apps/UnrealEngine_4.26/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v17_clang-10.0.1-centos7/x86_64-unknown-linux-gnu/usr/lib/" -B"/apps/UnrealEngine_4.26/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v17_clang-10.0.1-centos7/x86_64-unknown-linux-gnu/usr/lib64/" -L"/apps/UnrealEngine_4.26/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v17_clang-10.0.1-centos7/x86_64-unknown-linux-gnu/usr/lib/" -L"/apps/UnrealEngine_4.26/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v17_clang-10.0.1-centos7/x86_64-unknown-linux-gnu/usr/lib64/" -o "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.so" -Wl,@"/apps/carla_source_0.9.15/Unreal/CarlaUE4/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/CarlaUE4/libUE4Editor-CarlaUE4.so.response" -Wl,--start-group -lpthread -ldl -lc -lsqlite3 -lUE4Editor-Engine -lUE4Editor-PhysXVehicles -lUE4Editor-Carla -lUE4Editor-CoreUObject -lUE4Editor-Core -lUE4Editor-InputCore -lUE4Editor-UnrealEd -Wl,--unresolved-symbols=ignore-in-shared-libs -Wl,--end-group -lrt -lm -nodefaultlibs -LThirdParty/Linux/LibCxx/lib/Linux/x86_64-unknown-linux-gnu/ ThirdParty/Linux/LibCxx/lib/Linux/x86_64-unknown-linux-gnu/libc++.a ThirdParty/Linux/LibCxx/lib/Linux/x86_64-unknown-linux-gnu/libc++abi.a -lm -lc -lpthread -lgcc_s -lgcc
"/apps/UnrealEngine_4.26/Engine/Binaries/Linux/dump_syms" -c -o "/apps/UnrealEngine_4.26/Engine/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/libUE4Editor-CarlaUE4.so.psym" "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.so"
"/apps/UnrealEngine_4.26/Engine/Binaries/Linux/BreakpadSymbolEncoder" "/apps/UnrealEngine_4.26/Engine/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/libUE4Editor-CarlaUE4.so.psym" "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.sym"
"/apps/UnrealEngine_4.26/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v17_clang-10.0.1-centos7/x86_64-unknown-linux-gnu/bin/llvm-objcopy" --strip-all "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.so" "/apps/UnrealEngine_4.26/Engine/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/libUE4Editor-CarlaUE4.so_nodebug"
"/apps/UnrealEngine_4.26/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v17_clang-10.0.1-centos7/x86_64-unknown-linux-gnu/bin/llvm-objcopy" --only-keep-debug "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.so" "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.debug"
"/apps/UnrealEngine_4.26/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v17_clang-10.0.1-centos7/x86_64-unknown-linux-gnu/bin/llvm-objcopy" --add-gnu-debuglink="/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.debug" "/apps/UnrealEngine_4.26/Engine/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/libUE4Editor-CarlaUE4.so_nodebug" "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.so.temp"
mv "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.so.temp" "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.so"
chmod 644 "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.debug"

After searching for the UE4_ROOT in BuildTools bash files, I found the following

/apps/carla_source_0.9.15/Util/BuildTools/BuildCarlaUE4.sh

Line 197
${UE4_ROOT}/GenerateProjectFiles.sh -project="${PWD}/CarlaUE4.uproject" -game -engine -makefiles

/apps/UnrealEngine_4.26/GenerateProjectFiles.sh

line 27-29
    cd Engine/Build/BatchFiles/Linux
    bash ./GenerateGDBInit.sh
    bash ./GenerateProjectFiles.sh "$@"

/apps/UnrealEngine_4.26/Engine/Build/BatchFiles/Linux/GenerateProjectFiles.sh

Line 25 - 30
if [ -f "$BASE_PATH/../../../Source/Programs/UnrealBuildTool/UnrealBuildTool.csproj" ]; then
    xbuild "$BASE_PATH/../../../Source/Programs/UnrealBuildTool/UnrealBuildTool.csproj" /property:Configuration="Development" /verbosity:quiet /nologo /p:NoWarn=1591
fi

# pass all parameters to UBT
mono "$BASE_PATH/../../../Binaries/DotNET/UnrealBuildTool.exe" -projectfiles "$@"

/apps/UnrealEngine_4.26/Engine/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/Link-libUE4Editor-CarlaUE4.so.link.sh

Line 2
#!/bin/sh
# Automatically generated by UnrealBuildTool
# *DO NOT EDIT*

set -o errexit
"/apps/UnrealEngine_4.26/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v17_clang-10.0.1-centos7/x86_64-unknown-linux-gnu/bin/clang++" -fuse-ld=lld -rdynamic -shared -Wl,--gdb-index -Wl,-rpath='${ORIGIN}' -Wl,-rpath-link='${ORIGIN}' -Wl,-rpath='${ORIGIN}'/.. -Wl,-rpath='${ORIGIN}'/../../../Engine/Binaries/ThirdParty/Qualcomm/Linux -Wl,-rpath='${ORIGIN}'/../../../Engine/Binaries/ThirdParty/OpenVR/OpenVRv1_5_17/linux64 -Wl,-rpath='${ORIGIN}'/../../../Engine/Binaries/ThirdParty/PhysX3/Linux/x86_64-unknown-linux-gnu -Wl,--disable-new-dtags -Wl,--as-needed -Wl,--hash-style=gnu -Wl,--build-id -target x86_64-unknown-linux-gnu "--sysroot=/apps/UnrealEngine_4.26/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v17_clang-10.0.1-centos7/x86_64-unknown-linux-gnu" -B"/apps/UnrealEngine_4.26/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v17_clang-10.0.1-centos7/x86_64-unknown-linux-gnu/usr/lib/" -B"/apps/UnrealEngine_4.26/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v17_clang-10.0.1-centos7/x86_64-unknown-linux-gnu/usr/lib64/" -L"/apps/UnrealEngine_4.26/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v17_clang-10.0.1-centos7/x86_64-unknown-linux-gnu/usr/lib/" -L"/apps/UnrealEngine_4.26/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v17_clang-10.0.1-centos7/x86_64-unknown-linux-gnu/usr/lib64/" -o "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.so" -Wl,@"/apps/carla_source_0.9.15/Unreal/CarlaUE4/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/CarlaUE4/libUE4Editor-CarlaUE4.so.response" -Wl,--start-group -lpthread -ldl -lc -lsqlite3 -lUE4Editor-Engine -lUE4Editor-PhysXVehicles -lUE4Editor-Carla -lUE4Editor-CoreUObject -lUE4Editor-Core -lUE4Editor-InputCore -lUE4Editor-UnrealEd -Wl,--unresolved-symbols=ignore-in-shared-libs -Wl,--end-group -lrt -lm -nodefaultlibs -LThirdParty/Linux/LibCxx/lib/Linux/x86_64-unknown-linux-gnu/ ThirdParty/Linux/LibCxx/lib/Linux/x86_64-unknown-linux-gnu/libc++.a ThirdParty/Linux/LibCxx/lib/Linux/x86_64-unknown-linux-gnu/libc++abi.a -lm -lc -lpthread -lgcc_s -lgcc
"/apps/UnrealEngine_4.26/Engine/Binaries/Linux/dump_syms" -c -o "/apps/UnrealEngine_4.26/Engine/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/libUE4Editor-CarlaUE4.so.psym" "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.so"
"/apps/UnrealEngine_4.26/Engine/Binaries/Linux/BreakpadSymbolEncoder" "/apps/UnrealEngine_4.26/Engine/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/libUE4Editor-CarlaUE4.so.psym" "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.sym"
"/apps/UnrealEngine_4.26/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v17_clang-10.0.1-centos7/x86_64-unknown-linux-gnu/bin/llvm-objcopy" --strip-all "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.so" "/apps/UnrealEngine_4.26/Engine/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/libUE4Editor-CarlaUE4.so_nodebug"
"/apps/UnrealEngine_4.26/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v17_clang-10.0.1-centos7/x86_64-unknown-linux-gnu/bin/llvm-objcopy" --only-keep-debug "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.so" "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.debug"
"/apps/UnrealEngine_4.26/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v17_clang-10.0.1-centos7/x86_64-unknown-linux-gnu/bin/llvm-objcopy" --add-gnu-debuglink="/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.debug" "/apps/UnrealEngine_4.26/Engine/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/libUE4Editor-CarlaUE4.so_nodebug" "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.so.temp"
mv "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.so.temp" "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.so"
chmod 644 "/apps/carla_source_0.9.15/Unreal/CarlaUE4/Binaries/Linux/libUE4Editor-CarlaUE4.debug"

I didn't looked up the second problematic file for now. /apps/UnrealEngine_4.26/Engine/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/libUE4Editor-CarlaUE4.so.psym

Blyron commented 3 months ago

Sorry for being late @PatrickPromitzer. By the way, thanks for the community contributions you are making we really appreciate it. How do you suggest to fix this issue?

PatrickPromitzer commented 3 months ago

You are Welcome.

From the code I saw, I am not sure if you could stop creating the problematic file. (I am not deep enough in the UnrealEngine code to see if you could shift it into the Carla project path. At some point I probably try to work the the C++ code itself.)

I have 3 ideas, but someone have to check witch one works

1.) Generate the files each time you need it If it doesn't take too long, it would be the easiest way to just generate the file each time you use the make file from Carla. As example for

You could just replace the "/apps/carla_source_0.9.15/" with the right path, too. (I did that to make it work)

2.) move the file into the project folder (maybe not possible) A better way (but more complex), would be if the script is in the Carla Project folder instead of the UnrealEngine folder. I don't think it is possible, but it would be cleaner.

3.) use a different file name for each installation The current file has the name "Link-libUE4Editor-CarlaUE4.so.link.sh" which is pointing to the project folder. If you have a more unique name, for example "Link-libUE4Editor-CarlaUE4-v0_9_15.so.link.sh", it would not be completely fail save, but better with different versions.