search

carla-simulator / carla

Open-source simulator for autonomous driving research.
http://carla.org
MIT License
11.58k stars 3.73k forks source link

Crash when package branch ue5-dev on Ubuntu22.04 #8030

Closed anschen1994 closed 1 week ago

anschen1994 commented 4 months ago

Crash when package with branch dev-ue5 on Ubuntu22.04

Thanks the community for the contribution of Carla with UE5. We follow the instruction: https://github.com/carla-simulator/carla/blob/ue5-dev/UE5_BUILD_README.md step by step and all operations are on Ubuntu22.04. We can build successfully until the package stage. Specificly,

  1. Configure;
  2. Build Carla;
  3. Build and install the Python API
  4. Launch the Editor The above four procedures have no problems.

The last step --- package can not build successfully. The ERROR Log are shown in the attached file.

Can anyone help us fix this problem?

anschen1994 commented 4 months ago

The Log is archived in the following content. Log.zip

The core trace info is shown in the figure: image

qhaas commented 3 months ago

I'm unable to cmake --build Build --target package as well in Ubuntu 22.04. Following the documentation results in LogInit: Display: Failure - 26616 error(s), 217757 warning(s)

Source versions:

Unreal Engine itself, cmake --build Build, cmake --build Build --target carla-python-api-install, and cmake --build Build --target launch work, but not the final cmake --build Build --target package to create the packaged release.

Oddly, I get an error Out of memory. (despite having 256GB of RAM) if I try to run cmake --build Build --target package without first cmake --build Build --target launch, which leads me to believe that the build order dependency for the package target is missing something.

I assume the docs mean CarlaSetup.sh instead of Setup.sh. I had to install additional dependencies beyond what is deployed by that script to get this far:

Curiously, started getting errors about too my FDs open after deploying libwebsockets-dev, so used ulimits to cap nproc / nofile to 1024 based on a hint I found online with another project.

I set CARLA_UNREAL_ENGINE_PATH and CARLA_PATH to the root of unreal engine and carla respectively.

I suspect that the asset content commit is out-of-sync with the carla src commit based on many warnings like this:

LogInit: Display: Warning/Error Summary (Unique only)
LogInit: Display: -----------------------------------
LogInit: Display: LogEditorAssetSubsystem: Error: GetPathNameForLoadedAsset failed: The Asset is not valid.
LogInit: Display: LogEditorAssetSubsystem: Error: LoadAsset failed: Can't convert the path '/NonL.L' because it does not map to a root.
LogInit: Display: LogConsoleManager: Warning: Setting the console variable 'r.Streaming.PoolSize' with 'SetByScalability' was ignored as it is lower priority than the previous 'SetByProjectSetting'. Value remains '4000'
LogInit: Display: LogScript: Warning: Script Msg: A null object was passed as a world context object to UEngine::GetWorldFromContextObject().
LogInit: Display: LogUObjectGlobals: Warning: [AssetLog] /mnt/carla-src/Unreal/CarlaUnreal/Plugins/CarlaTools/Content/OnroadMapGenerator/BP_OpenDriveToMap.uasset: Failed to load '/Game/Carla/Static/Static/SM_LightBox2': Can't find file.
...
LogInit: Display: LogScript: Warning: Script Msg: Attempted to access index 0 from array 'CallFunc_GetAllActorsOfClassWithTag_OutActors_1' of length 0 in '/Game/Carla/Blueprints/BP_GeneralSceneSettings.BP_GeneralSceneSettings_C:ExecuteUbergraph_BP_GeneralSceneSettings'!
LogInit: Display: LogScript: Warning: Script Msg called by: BP_GeneralSceneSettings_C /Game/Carla/Maps/Town10HD_Opt.Town10HD_Opt:PersistentLevel.BP_GeneralSceneSettings_C_1

Update: I rolled back carla and its content to the earlier tagged builds from their ue5 branches, keeping carla's ue5 fork at the same commit noted above. cmake --build Build --target package worked after rerunning the prior cmake calls, but I had to rerun it multiple times due to process Engine/Binaries/Linux/UnrealTraceServer fork hanging, even having tokill it by PID when it gets into a bad state and CTRL-C doesn't stop it. This could be an issue with it being in a container and the files being mounted from a CEPH filesystem, I haven't tried it on a 'bare metal' Ubuntu 22.04 deployment and an EXT4 filesystem.

$ cmake -G Ninja -S . -B Build --toolchain=$PWD/CMake/LinuxToolchain.cmake -DCMAKE_BUILD_TYPE=Release -DENABLE_ROS2=ON -DBUILD_CARLA_UNREAL=ON -DCARLA_UNREAL_ENGINE_PATH=$CARLA_UNREAL_ENGINE_PATH
...
$ cmake --build Build
...
$ cmake --build Build --target carla-python-api-install
...
$ cmake --build Build --target launch
(CTRL -C to terminate once it tries to launch, if no GUI, will complain about SDL lacking a device)
...
$ cmake --build Build --target package
...
(repeat: CTRL-C to interrupt UnrealTraceServer deadlock(?) and/or kill it by PID when the build stalls, then rerun the 'package' target above, had to do this a few times)
...
********** COMPRESSING PACKAGE COMPLETED **********
...
$ file Build/Package/CARLA.tar.gz 
Build/Package/CARLA.tar.gz: gzip compressed data, last modified: Sat Aug 24 15:37:56 2024, from Unix, original size modulo 2^32 1350616576

Update 2: Works with the head of the ue5-dev branch of carla itself at time of writing (commit aa9d95eb21d03abe143471d9a7e854a794e5533d) too, as long as one uses the ue5devStable0.1 tag of the content, lending evidence that the main culprit is an out-of-sync content commit. Unfortunately, anytime I try to run what is built, it crashes without meaningful error (shipping mode). When building as debug mode with stack traces enabled, I see that an assertion fails: Assertion failed: FInternationalization::Get().IsInitialized() == true [File:./Runtime/Core/Private/Internationalization/TextFormatter.cpp][Line: 1056] FInternationalization is not initialized. An FText formatting method was likely used in static object initialization - this is not supported.

Blyron commented 3 weeks ago

Without modifying any cmakelists or cmake scripts. You should be able to launch cmake --build Build --target package. Make sure you update latest content too