GPUOpen-Drivers / AMDVLK

AMD Open Source Driver For Vulkan
MIT License
1.69k stars 160 forks source link

AMDVLk build failed for linux-distro (Photon) #316

Closed shivania2 closed 1 year ago

shivania2 commented 1 year ago

I am trying to build AMDVLK for linux-distro (Photon). I have built and installed dxc and other dependencies. But while building AMDVLK, I am getting following error

`root@photon-1234 [ ~/dummy/dummy1/drivers/builds/Release64 ]# cmake --build . [3/2119] Building C object compiler/llpc/llvm/lib/Support/CMakeFiles/LLVMSupport.dir/regfree.c.o cc1: warning: command-line option ‘-faligned-new=0’ is valid for C++/ObjC++ but not for C [4/2119] Building C object compiler/llpc/llvm/lib/Support/CMakeFiles/LLVMSupport.dir/regerror.c.o cc1: warning: command-line option ‘-faligned-new=0’ is valid for C++/ObjC++ but not for C [5/2119] Building C object compiler/llpc/llvm/lib/Support/CMakeFiles/LLVMSupport.dir/regstrlcpy.c.o cc1: warning: command-line option ‘-faligned-new=0’ is valid for C++/ObjC++ but not for C [6/2119] Building C object compiler/llpc/llvm/lib/Support/CMakeFiles/LLVMSupport.dir/regcomp.c.o cc1: warning: command-line option ‘-faligned-new=0’ is valid for C++/ObjC++ but not for C [7/2119] Building C object compiler/llpc/llvm/lib/Support/CMakeFiles/LLVMSupport.dir/regexec.c.o cc1: warning: command-line option ‘-faligned-new=0’ is valid for C++/ObjC++ but not for C [56/2119] Generating pipelines/g_internal_shaders.h, pipelines/g_GpuRtLibrary.h FAILED: gpurt/src/pipelines/g_internal_shaders.h gpurt/src/pipelines/g_GpuRtLibrary.h /root/dummy/dummy1/drivers/builds/Release64/gpurt/src/pipelines/g_internal_shaders.h /root/dummy/dummy1/drivers/builds/Release64/gpurt/src/pipelines/g_GpuRtLibrary.h cd /root/dummy/dummy1/drivers/builds/Release64/gpurt/src && /usr/bin/python3.10 /root/dummy/dummy1/drivers/gpurt/tools/CompileRTShaders.py --vulkan --spirv --outputDir /root/dummy/dummy1/drivers/builds/Release64/gpurt/src/pipelines --shaderConfig /root/dummy/dummy1/drivers/gpurt/tools/RTShaders.xml --whiteListPath /root/dummy/dummy1/drivers/gpurt/tools/strip_whitelist.txt /root/dummy/dummy1/drivers/gpurt/src/shaders Compilation failed for shader GpuRtLibrarySw Output log: Compiling GpuRtLibrary.hlsl -> GpuRtLibrarySw_spv.h... Subprocess invocation failed. Error code -6. Additional details follow: Environment: {'SHELL': '/bin/bash', 'XDG_CONFIG_DIRS': '/etc/xdg/', 'HISTSIZE': '1000', 'GOHOSTOS': 'linux', 'PWD': '/root/dummy/dummy1/drivers/builds/Release64/gpurt/src', 'LOGNAME': 'root', 'XDG_SESSION_TYPE': 'tty', 'HOME': '/root', 'LANG': 'en_US.UTF-8', 'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arc=01;31:.arj=01;31:.taz=01;31:.lha=01;31:.lz4=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.tzo=01;31:.t7z=01;31:.zip=01;31:.z=01;31:.dz=01;31:.gz=01;31:.lrz=01;31:.lz=01;31:.lzo=01;31:.xz=01;31:.zst=01;31:.tzst=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.alz=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.cab=01;31:.wim=01;31:.swm=01;31:.dwm=01;31:.esd=01;31:.jpg=01;35:.jpeg=01;35:.mjpg=01;35:.mjpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.webp=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.ogv=01;35:.ogx=01;35:.aac=00;36:.au=00;36:.flac=00;36:.m4a=00;36:.mid=00;36:.midi=00;36:.mka=00;36:.mp3=00;36:.mpc=00;36:.ogg=00;36:.ra=00;36:.wav=00;36:.oga=00;36:.opus=00;36:.spx=00;36:.xspf=00;36:', 'SSH_CONNECTION': '10.104.5.10 62267 10.197.103.184 22', 'GOROOT': '/usr/lib/golang', 'XDG_SESSION_CLASS': 'user', 'TERM': 'xterm-256color', 'USER': 'root', 'SHLVL': '1', 'INPUTRC': '/etc/inputrc', 'XDG_SESSION_ID': 'c82', 'XDG_RUNTIME_DIR': '/run/user/0', 'SSH_CLIENT': '10.104.5.10 62267 22', 'XDG_DATA_DIRS': '/usr/share/', 'PATH': '/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin', 'HISTIGNORE': '&:[bf]g:exit', 'SSH_TTY': '/dev/pts/59', 'GOOS': 'linux', 'OLDPWD': '/root/dummy/dummy1/drivers/builds/Release64', 'GOPATH': '/usr/share/gocode', 'GOHOSTARCH': 'amd64', 'BASH_FUNC_pathappend%%': '() { pathremove $1 $2;\n local PATHVARIABLE=${2:-PATH};\n export $PATHVARIABLE="${!PATHVARIABLE:+${!PATHVARIABLE}:}$1"\n}', 'BASH_FUNC_pathprepend%%': '() { pathremove $1 $2;\n local PATHVARIABLE=${2:-PATH};\n export $PATHVARIABLE="$1${!PATHVARIABLE:+:${!PATHVARIABLE}}"\n}', 'BASH_FUNCpathremove%%': '() { local IFS=\':\';\n local NEWPATH;\n local DIR;\n local PATHVARIABLE=${2:-PATH};\n for DIR in ${!PATHVARIABLE};\n do\n if [ "$DIR" != "$1" ]; then\n NEWPATH=${NEWPATH:+$NEWPATH:}$DIR;\n fi;\n done;\n export $PATHVARIABLE="$NEWPATH"\n}', '': '/usr/bin/python3.10'} Command arguments: ['spirv-remap', '--strip-white-list', '/root/dummy/dummy1/drivers/gpurt/tools/strip_whitelist.txt', '--strip-all', '-i', 'GpuRtLibrarySw.spv', '-o', '.'] Expected empty output: False Working directory: /root/dummy/dummy1/drivers/builds/Release64/gpurt/src/pipelines/GpuRtLibrarySw Stdout:

Stderr: double free or corruption (!prev) Converting compiled output...Failure Compilation failed for shader GpuRtLibrarySwDev`

Flakebi commented 1 year ago

That looks like a double free in spirv-remap.

Can you try to reproduce this outside the build system?

  1. In your build folder, run /usr/bin/python3.10 /root/dummy/dummy1/drivers/gpurt/tools/CompileRTShaders.py --vulkan --spirv --outputDir /root/dummy/dummy1/drivers/builds/Release64/gpurt/src/pipelines --shaderConfig /root/dummy/dummy1/drivers/gpurt/tools/RTShaders.xml --whiteListPath /root/dummy/dummy1/drivers/gpurt/tools/strip_whitelist.txt /root/dummy/dummy1/drivers/gpurt/src/shaders --interm-only (adding the --interm-only argument to keep the .spv file)
  2. Then run spirv-remap --strip-white-list /root/dummy/dummy1/drivers/gpurt/tools/strip_whitelist.txt --strip-all -i gpurt/src/pipelines/GpuRtLibrarySw/GpuRtLibrarySw.spv -o .

The spirv-remap command should create a file called GpuRtLibrarySw.spv in your build folder.

shivania2 commented 1 year ago

I am getting the error opening file for read while executing second command. Complete logs are following root@photon-1234 [ ~/amdvlk/drivers/builds/Release64 ]# /usr/bin/python3.10 /root/dummy/dummy1/drivers/gpurt/tools/CompileRTShaders.py --vulkan --spirv --outputDir /root/dummy/dummy1/drivers/builds/Release64/gpurt/src/pipelines --shaderConfig /root/dummy/dummy1/drivers/gpurt/tools/RTShaders.xml --whiteListPath /root/dummy/dummy1/drivers/gpurt/tools/strip_whitelist.txt /root/dummy/dummy1/drivers/gpurt/src/shaders --interm-only Launching threads for raytracing shader compilation... Raytracing shader compilation completed with no errors. Launching threads for raytracing shader compilation... Raytracing shader compilation completed with no errors. root@photon-1234 [ ~/amdvlk/drivers/builds/Release64 ]# spirv-remap --strip-white-list /root/dummy/dummy1/drivers/gpurt/tools/strip_whitelist.txt --strip-all -i gpurt/src/pipelines/GpuRtLibrarySw/GpuRtLibrarySw.spv -o . error opening file for read:

Flakebi commented 1 year ago

error opening file for read:

That means the file the python script should create was removed again (gpurt/src/pipelines/GpuRtLibrarySw/GpuRtLibrarySw.spv).

Which amdvlk version are you using? Can you try checking out the dev branch in the drivers/gpurt repository?

A change in gpurt from 4 weeks ago fixed not removing folders when --interm-only is used. I have this version (gpurt commit 714a028):

$ sha256sum tools/CompileRTShaders.py
e0150f3dceae258c6ad8adac6ef5a27baa6cdb131abae527562a6ae6b5d94549  tools/CompileRTShaders.py
shivania2 commented 1 year ago

Hi Flakebi, I am getting the similar issue for dev branch on gpurt commit 714a028. If i run the command you shared in dev branch then second command is ended in core dump. Coredump stack trace: `(gdb) where

0 0x00007ffff670a041 in raise () from /lib/libc.so.6

1 0x00007ffff66f3536 in abort () from /lib/libc.so.6

2 0x00007ffff674b5a8 in __libc_message () from /lib/libc.so.6

3 0x00007ffff6752fea in malloc_printerr () from /lib/libc.so.6

4 0x00007ffff675475c in _int_free () from /lib/libc.so.6

5 0x00007ffff6c080bf in gnu_cxx::new_allocator<char const*>::deallocate (t=, __p=, this=0x7ffff76c0718 <spv::InstructionDesc+598008>)

at /usr/include/c++/10.2.0/ext/new_allocator.h:120

6 std::allocator_traits<std::allocator<char const*> >::deallocate (n=, p=, __a=...) at /usr/include/c++/10.2.0/bits/alloc_traits.h:492

7 std::_Vector_base<char const, std::allocator<char const> >::_M_deallocate (n=, p=,

this=0x7ffff76c0718 <spv::InstructionDesc+598008>) at /usr/include/c++/10.2.0/bits/stl_vector.h:354

8 std::_Vector_base<char const, std::allocator<char const> >::~_Vector_base (this=0x7ffff76c0718 <spv::InstructionDesc+598008>, __in_chrg=)

at /usr/include/c++/10.2.0/bits/stl_vector.h:335

9 std::vector<char const, std::allocator<char const> >::~vector (this=0x7ffff76c0718 <spv::InstructionDesc+598008>, __in_chrg=)

at /usr/include/c++/10.2.0/bits/stl_vector.h:683

10 spv::OperandParameters::~OperandParameters (this=0x7ffff76c0700 <spv::InstructionDesc+597984>, __in_chrg=)

at /usr/src/debug/glslang-1.3.231.1-1.ph4.x86_64/SPIRV/doc.h:169

11 spv::InstructionParameters::~InstructionParameters (this=0x7ffff76c06f0 <spv::InstructionDesc+597968>, __in_chrg=)

at /usr/src/debug/glslang-1.3.231.1-1.ph4.x86_64/SPIRV/doc.h:219

12 __tcf_0 () at /usr/src/debug/glslang-1.3.231.1-1.ph4.x86_64/SPIRV/doc.cpp:1463

13 0x00007ffff670ccf6 in __cxa_finalize () from /lib/libc.so.6

14 0x00007ffff6bf9903 in __do_global_dtors_aux () from /lib/libSPVRemapper.so

15 0x00007fffffffe070 in ?? ()

16 0x00007ffff7fe2423 in _dl_fini () from /lib64/ld-linux-x86-64.so.2`

shivania2 commented 1 year ago

My system has glslang version 1.3.231.1. Can you please share the glslang version of your system

Flakebi commented 1 year ago

I am getting the similar issue for dev branch on gpurt commit 714a028. If i run the command you shared in dev branch then second command is ended in core dump.

Good, so it does seem to be a problem in spirv-remap. My spirv-remap is from glslang 1.3.239.

Btw, you can format code blocks by surrounding them with tripple backticks

nice multiline code

shivania2 commented 1 year ago

I have upgraded to glslang 1.3.239.0, getting similar issue. Seems like issue is not related to glslang.