search

adamrehn / pixel-streaming-linux

Issue tracker repository for Pixel Streaming for Linux
https://adamrehn.com/articles/pixel-streaming-in-linux-containers/
22 stars 7 forks source link

UE4Editor can't create new project - CUDA library resolution failure #42

Closed bramford closed 3 years ago

bramford commented 3 years ago

Environment: Debian Buster 11 (same issue on Ubuntu 18.04 LTS)

I successfully compiled 4.25-pixelstreaming after:

  1. Creating symlinks for all necessary files to /usr/local/cuda
  2. Adding one additional library path to UnrealEngine/Engine/Plugins/CUDA/Source/CUDA.Build.cs: 
    PublicSystemLibraryPaths.Add("/usr/local/cuda/cudart");

Runnin UE4Editor works but when I try to create a new project, it gets stuck and 90% then returns to 80% and works its way back to 90% only to rinse and repeat.

The logs lead me to believe that the cuda library path is not used by the runtime despite compilation working on the same machine:

[2021.03.23-02.57.01:866][333]Running /home/bramford/Documents/git/github/ImmortalEmperor/UnrealEngine/Engine/Binaries/DotNET/Un
realBuildTool.exe  -projectfiles -project="/home/bramford/Documents/git/github/newgameplus/LinuxPixelStreaming/LinuxPixelStreami
ng.uproject" -game -engine -progress                                                                                            
[2021.03.23-02.57.01:966][333]                                                                                                  
[2021.03.23-02.57.01:966][333]Running Mono...                                                                                   
[2021.03.23-02.57.01:966][333]                                                                                                  
[2021.03.23-02.57.02:066][333]Fixing inconsistent case in filenames.                                                            
[2021.03.23-02.57.02:066][333]Setting up Mono                                                                                   
[2021.03.23-02.57.02:066][333]~/Documents/git/github/ImmortalEmperor/UnrealEngine/Engine ~/Documents/git/github/ImmortalEmperor/
UnrealEngine/Engine/Binaries/Linux                                                                                              
[2021.03.23-02.57.02:467][333]Discovering modules, targets and source code for project...                                       
[2021.03.23-02.57.07:672][333]Generating data for project indexing...                                                           
[2021.03.23-02.57.08:678][333]WARNING: Library 'cudart_static' was not resolvable to a file when used in Module 'CUDA', assuming
 it is a filename and will search library paths for it. This is slow and dependency checking will not work for it. Please update
 reference to be fully qualified alternatively use PublicSystemLibraryPaths if you do intended to use this slow path to suppress
 this warning.                                                                                                                  
[2021.03.23-02.57.08:678][333]WARNING: Library 'cuda' was not resolvable to a file when used in Module 'CUDA', assuming it is a 
filename and will search library paths for it. This is slow and dependency checking will not work for it. Please update referenc
e to be fully qualified alternatively use PublicSystemLibraryPaths if you do intended to use this slow path to suppress this war
ning.                                                                                                                           
[2021.03.23-02.57.17:960][333]Generating data for project indexing... 76%                                                       
[2021.03.23-02.57.20:386][333]Generating data for project indexing... 100%                                                      
[2021.03.23-02.57.20:386][333]Discovering modules, targets and source code for project...                                       
[2021.03.23-02.57.22:088][333]Generating data for project indexing...                                                           
[2021.03.23-02.57.22:391][333]WARNING: Library 'cudart_static' was not resolvable to a file when used in Module 'CUDA', assuming
 it is a filename and will search library paths for it. This is slow and dependency checking will not work for it. Please update
 reference to be fully qualified alternatively use PublicSystemLibraryPaths if you do intended to use this slow path to suppress
 this warning.                                                                                                                  
[2021.03.23-02.57.22:391][333]WARNING: Library 'cuda' was not resolvable to a file when used in Module 'CUDA', assuming it is a 
filename and will search library paths for it. This is slow and dependency checking will not work for it. Please update referenc
e to be fully qualified alternatively use PublicSystemLibraryPaths if you do intended to use this slow path to suppress this war
ning.                                                                                                                           
[2021.03.23-02.57.32:472][333]Generating data for project indexing... 89%                                                       
[2021.03.23-02.57.33:482][333]Generating data for project indexing... 100%                                                      
[2021.03.23-02.57.33:482][333]Writing project files...                                                                          
[2021.03.23-02.57.33:582][333]Writing project files... 100%                                                                     
[2021.03.23-02.57.33:683][333]Discovering modules, targets and source code for project...                                       
[2021.03.23-02.57.35:184][333]Generating data for project indexing...                                                           
[2021.03.23-02.57.35:487][333]WARNING: Library 'cudart_static' was not resolvable to a file when used in Module 'CUDA', assuming
 it is a filename and will search library paths for it. This is slow and dependency checking will not work for it. Please update
 reference to be fully qualified alternatively use PublicSystemLibraryPaths if you do intended to use this slow path to suppress
 this warning.                                                                                                                  
[2021.03.23-02.57.35:487][333]WARNING: Library 'cuda' was not resolvable to a file when used in Module 'CUDA', assuming it is a 
filename and will search library paths for it. This is slow and dependency checking will not work for it. Please update referenc
e to be fully qualified alternatively use PublicSystemLibraryPaths if you do intended to use this slow path to suppress this war
ning.                                                                                                                           
[2021.03.23-02.57.45:573][333]Generating data for project indexing... 84%                                                       
[2021.03.23-02.57.46:984][333]Generating data for project indexing... 100%                                                      
[2021.03.23-02.57.46:984][333]Discovering modules, targets and source code for project...                                       
[2021.03.23-02.57.48:686][333]Generating data for project indexing...                                                           
[2021.03.23-02.57.48:989][333]WARNING: Library 'cudart_static' was not resolvable to a file when used in Module 'CUDA', assuming
 it is a filename and will search library paths for it. This is slow and dependency checking will not work for it. Please update
 reference to be fully qualified alternatively use PublicSystemLibraryPaths if you do intended to use this slow path to suppress
 this warning.                                                                                                                  
[2021.03.23-02.57.48:989][333]WARNING: Library 'cuda' was not resolvable to a file when used in Module 'CUDA', assuming it is a 
filename and will search library paths for it. This is slow and dependency checking will not work for it. Please update referenc
e to be fully qualified alternatively use PublicSystemLibraryPaths if you do intended to use this slow path to suppress this warning.                                                                                                                           [2021.03.23-02.57.59:081][333]Generating data for project indexing... 87%                                                       
[2021.03.23-02.58.00:295][333]Generating data for project indexing... 100%                                       
[2021.03.23-02.58.46:442][333]Discovering modules, targets and source code for project...                                       
[2021.03.23-02.58.48:043][333]Generating data for project indexing...                                                           
[2021.03.23-02.58.48:347][333]WARNING: Library 'cudart_static' was not resolvable to a file when used in Module 'CUDA', assuming
 it is a filename and will search library paths for it. This is slow and dependency checking will not work for it. Please update
 reference to be fully qualified alternatively use PublicSystemLibraryPaths if you do intended to use this slow path to suppress
 this warning.                                                                                                                  
[2021.03.23-02.58.48:347][333]WARNING: Library 'cuda' was not resolvable to a file when used in Module 'CUDA', assuming it is a 
filename and will search library paths for it. This is slow and dependency checking will not work for it. Please update referenc
e to be fully qualified alternatively use PublicSystemLibraryPaths if you do intended to use this slow path to suppress this war
ning.                                                                                                                           
[2021.03.23-02.58.58:436][333]Generating data for project indexing... 87%                                                       
[2021.03.23-02.58.59:647][333]Generating data for project indexing... 100%                                                      
[2021.03.23-02.59.05:252][333]Discovering modules, targets and source code for project...                                       
[2021.03.23-02.59.06:954][333]Generating data for project indexing...                                                  
[2021.03.23-02.59.07:257][333]WARNING: Library 'cudart_static' was not resolvable to a file when used in Module 'CUDA', assuming
 it is a filename and will search library paths for it. This is slow and dependency checking will not work for it. Please update
 reference to be fully qualified alternatively use PublicSystemLibraryPaths if you do intended to use this slow path to suppress
 this warning.                                                                                                                  
[2021.03.23-02.59.07:257][333]WARNING: Library 'cuda' was not resolvable to a file when used in Module 'CUDA', assuming it is a 
filename and will search library paths for it. This is slow and dependency checking will not work for it. Please update referenc
e to be fully qualified alternatively use PublicSystemLibraryPaths if you do intended to use this slow path to suppress this war
ning.                                                                                                                           
[2021.03.23-02.59.17:236][333]Generating data for project indexing... 82%                                                       
[2021.03.23-02.59.19:050][333]Generating data for project indexing... 100%                                                      
[2021.03.23-02.59.19:050][333]Writing project files...                                                                          
[2021.03.23-02.59.20:953][333]Writing project files... 100%                                                                     
[2021.03.23-02.59.21:256][333]LogSlate: Window 'Unreal Project Browser' being destroyed                                  
[2021.03.23-02.59.21:319][334]Cmd: QUIT_EDITOR                                                                                  
[2021.03.23-02.59.21:320][334]LogCore: Engine exit requested (reason: UUnrealEdEngine::CloseEditor())                           
[2021.03.23-02.59.21:331][334]LogCore: Engine exit requested (reason: EngineExit() was called; note: exit was already requested)
perl: warning: Setting locale failed.                                                                                           
perl: warning: Please check that your locale settings:                                                                          
        LANGUAGE = "en_NZ:en",                                                                                                  
        LC_ALL = (unset),                                                                                                       
        LC_NUMERIC = "en_US",                                                                                                   
        LANG = "en_NZ.UTF-8"                                                                                                    
    are supported and installed on your system.                                                                                 
perl: warning: Falling back to a fallback locale ("en_NZ.UTF-8").                                                               
[2021.03.23-02.59.21:385][334]LogEditorSessionSummary: EditorSessionSummary sent report. Type=Shutdown, SessionId={9EA140E4-5833
-4674-B00E-62B32D9A9403}                                                                                                        
[2021.03.23-02.59.21:393][334]LogSlate: Slate User Destroyed.  User Index 0, Is Virtual User: 0                 
[2021.03.23-02.59.21:393][334]LogExit: Preparing to exit.
bramford commented 3 years ago

I managed to quash the WARNINGs by changing the lines from PublicAdditionalLibraries.Add to PublicSystemLibraries.Add:

PublicSystemLibraries.Add("cudart_static");
PublicSystemLibraries.Add("cuda");

The same end result failure occurs. Here's the log:

[2021.03.23-21.40.41:561][ 39]Running /home/bramford/Documents/git/github/ImmortalEmperor/UnrealEngine/Engine/Binaries/DotNET/UnrealBuildTool.exe  -projectfiles -project="/home/bramford/Documents/git/github/newgameplus/PixelStreamingLinux3/
PixelStreamingLinux3.uproject" -game -engine -progress                                                                                                                                                                                          
[2021.03.23-21.40.41:661][ 39]                                                                                                                                                                                                                  
[2021.03.23-21.40.41:661][ 39]Running Mono...                                                                                                                                                                                                   
[2021.03.23-21.40.41:661][ 39]                                                                                                                                                                                                                  
[2021.03.23-21.40.41:761][ 39]Fixing inconsistent case in filenames.                                                                                                                                                                            
[2021.03.23-21.40.41:762][ 39]Setting up Mono                                                                                                                                                                                                   
[2021.03.23-21.40.41:762][ 39]~/Documents/git/github/ImmortalEmperor/UnrealEngine/Engine ~/Documents/git/github/ImmortalEmperor/UnrealEngine/Engine/Binaries/Linux                                                                              
[2021.03.23-21.40.42:162][ 39]Discovering modules, targets and source code for project...                                                                                                                                                       
[2021.03.23-21.40.45:365][ 39]Generating data for project indexing...                                                                                                                                                                           
[2021.03.23-21.40.55:739][ 39]Generating data for project indexing... 78%                                                                                                                                                                       
[2021.03.23-21.40.57:856][ 39]Generating data for project indexing... 100%                                                                                                                                                                      
[2021.03.23-21.40.57:856][ 39]Discovering modules, targets and source code for project...                                                                                                                                                       
[2021.03.23-21.40.59:558][ 39]Generating data for project indexing...                                                                                                                                                                           
[2021.03.23-21.41.09:835][ 39]Generating data for project indexing... 89%                                                                                                                                                                       
[2021.03.23-21.41.10:744][ 39]Generating data for project indexing... 100%                                                                                                                                                                      
[2021.03.23-21.41.10:744][ 39]Writing project files...                                                                                                                                                                                          
[2021.03.23-21.41.10:947][ 39]Writing project files... 100%                                                                                                                                                                                     
[2021.03.23-21.41.10:947][ 39]Discovering modules, targets and source code for project...                                                                                                                                                       
[2021.03.23-21.41.12:549][ 39]Generating data for project indexing...                                                                                                                                                                           
[2021.03.23-21.41.22:834][ 39]Generating data for project indexing... 87%                                                                                                                                                                       
[2021.03.23-21.41.24:041][ 39]Generating data for project indexing... 100%                                                                                                                                                                      
[2021.03.23-21.41.24:041][ 39]Discovering modules, targets and source code for project...                                                                                                                                                       
[2021.03.23-21.41.25:743][ 39]Generating data for project indexing...                                                                                                                                                                           
[2021.03.23-21.41.36:029][ 39]Generating data for project indexing... 87%                                                                                                                                                                       
[2021.03.23-21.41.37:243][ 39]Generating data for project indexing... 100%                                                                                                                                                                      
[2021.03.23-21.42.16:784][ 39]Discovering modules, targets and source code for project...                                                                                                                                                       
[2021.03.23-21.42.18:386][ 39]Generating data for project indexing...                                                                                                                                                                           
[2021.03.23-21.42.28:772][ 39]Generating data for project indexing... 87%                                                                                                                                                                       
[2021.03.23-21.42.29:983][ 39]Generating data for project indexing... 100%                                                                                                                                                                      
[2021.03.23-21.42.35:689][ 39]Discovering modules, targets and source code for project...                                                                                                                                                       
[2021.03.23-21.42.37:291][ 39]Generating data for project indexing...                                                                                                                                                                           
[2021.03.23-21.42.47:679][ 39]Generating data for project indexing... 84%                                                                                                                                                                       
[2021.03.23-21.42.49:092][ 39]Generating data for project indexing... 100%                                                                                                                                                                      
[2021.03.23-21.42.49:092][ 39]Writing project files...                                                                                                                                                                                          
[2021.03.23-21.42.51:097][ 39]Writing project files... 100%                                                                                                                                                                                     
[2021.03.23-21.42.51:355][ 39]LogSlate: Window 'Unreal Project Browser' being destroyed                                                                                                                                                         
[2021.03.23-21.42.51:389][ 40]Cmd: QUIT_EDITOR                                                                                                                                                                                                  
[2021.03.23-21.42.51:389][ 40]LogCore: Engine exit requested (reason: UUnrealEdEngine::CloseEditor())                                                                                                                                           
[2021.03.23-21.42.51:399][ 40]LogCore: Engine exit requested (reason: EngineExit() was called; note: exit was already requested)                                                                                                                
perl: warning: Setting locale failed.                                                                                                                                                                                                           
perl: warning: Please check that your locale settings:                                                                                                                                                                                          
        LANGUAGE = "en_NZ:en",                                                                                                                                                                                                                  
        LC_ALL = (unset),                                                                                                                                                                                                                       
        LC_NUMERIC = "en_US",                                                                                                                                                                                                                   
        LANG = "en_NZ.UTF-8"                                                                                                                                                                                                                    
    are supported and installed on your system.                                                                                                                                                                                                 
perl: warning: Falling back to a fallback locale ("en_NZ.UTF-8").                                                                                                                                                                               
[2021.03.23-21.42.51:469][ 40]LogEditorSessionSummary: EditorSessionSummary sent report. Type=Shutdown, SessionId={7C5FF12E-3306-4C77-B727-111B97B56C99}                                                                                        
[2021.03.23-21.42.51:475][ 40]LogSlate: Slate User Destroyed.  User Index 0, Is Virtual User: 0                                                                                                                                                 
[2021.03.23-21.42.51:476][ 40]LogExit: Preparing to exit.
bramford commented 3 years ago

I also suspected that the cuda library path was not known during runtime so I tried patching the RPATH with this command:

patchelf --force-rpath --set-rpath '${ORIGIN}:${ORIGIN}/..:${ORIGIN}/../../../Engine/Binaries/ThirdParty/Qualcomm/Linux:${ORIGIN}/../../../Engine/Binaries/ThirdParty/OpenVR/OpenVRv1_5_17/linux64:${ORIGIN}/../../../Engine/Binaries/ThirdParty/PhysX3/Linux/x86_64-unknown-linux-gnu:/usr/local/cuda/lib64:/usr/local/cuda/cudart' UE4Editor

Didn't help

lukehb commented 3 years ago

@ImmortalEmperor