Closed miguelapg closed 4 years ago
ccache gcc -D_FORTIFY_SOURCE=0 -D__USE_MINGW_ANSI_STDIO=1 -mthreads -mtune=generic -O2 -pipe -fopenmp -DLIBTWOLAME_STATIC -DCACA_STATIC -DMODPLUG_STATIC -DCHROMAPRINT_NODLL -DZMQ_STATIC -DLIBXML_STATIC -DKVZ_STATIC_LIB -IC:/PROGRA~1/NVIDIA GPU Computing Toolkit/CUDA/v11.0/include -c -o /tmp/ffconf.Vx32iG1r/test.o /tmp/ffconf.Vx32iG1r/test.c
gcc.exe: error: GPU: No such file or directory
gcc.exe: error: Computing: No such file or directory
gcc.exe: error: Toolkit/CUDA/v11.0/include: No such file or directory
seems quotation rules has an issue
Who has to fix it then?
haven't been able to look into it yet
Seems that the default path chosen by Nvidia on Windows, namely "NVIDIA GPU Computing Toolkit", has blank spaces so as there is a quotation issue, it doesn't work
One thing I am wondering about is at what point is that path inserted, since running the individual commands on a local machine that would normally produce the cudapath results in
PS > cygpath -sm $env:CUDA_PATH
C:/PROGRA~1/NVIDIA~2/CUDA/v10.2
with no spaces importantly. It seems maybe something went wrong with yours since although it has the first part in the correct format, the second part isn't shortened correctly
I have the latest and updated version of media-autobuild_suite.
In my MSYS2 ...
$ cygpath -sm $env:CUDA_PATH cygpath: cannot create short name of CUDA_PATH
need to change that to
$ cygpath -sm "$CUDA_PATH"
OK. This is the output now ...
$ cygpath -sm "$CUDA_PATH" C:/PROGRA~1/NVIDIA GPU Computing Toolkit/CUDA/v11.0
What does this say? $ ls "$CUDA_PATH"
$ ls "$CUDA_PATH" bin extras include lib libnvvp nvml nvvm Sanitizer src
Weird. So it does exist, but for some reason cygpath isn't converting the whole path correctly.
Can you do command -v cygpath.exe
?
$ command -v cygpath.exe /usr/bin/cygpath.exe
$ /usr/bin/cygpath.exe -V cygpath (cygwin) 3.1.6 Path Conversion Utility Copyright (C) 1998 - 2020 Cygwin Authors
I won't be able to be close to my computer for the next hours. Sorry.
Drop me a line if you need more information and I will paste here whatever you want to know
If you need anything let me know.
Have you some plan to fix this issue?
Can you add a right quotation to the configure line of FFmpeg or send some warning to somebody else to fix the issue?
@wiiaboo do you think this would work?
diff --git a/build/media-suite_helper.sh b/build/media-suite_helper.sh
index 28aa18c..a30fb68 100644
--- a/build/media-suite_helper.sh
+++ b/build/media-suite_helper.sh
@@ -885,8 +885,8 @@ do_changeFFmpegConfig() {
echo -e "${orange}FFmpeg and related apps will depend on CUDA SDK to run!${reset}"
local fixed_CUDA_PATH
fixed_CUDA_PATH="$(cygpath -sm "$CUDA_PATH")"
- do_addOption "--extra-cflags=-I$fixed_CUDA_PATH/include"
- do_addOption "--extra-ldflags=-L$fixed_CUDA_PATH/lib/x64"
+ do_addOption "--extra-cflags=-I\"$fixed_CUDA_PATH/include\""
+ do_addOption "--extra-ldflags=-L\"$fixed_CUDA_PATH/lib/x64\""
fi
if enabled cuda-nvcc; then
local fixed_CUDA_PATH_UNIX
I will try it, but t have my own doubts
I just can check it if you add it to the repository
I've added it here https://github.com/1480c1/media-autobuild_suite/tree/cuda if you do want to try it
I will do it
It failed. I replaced the new media-suite_helper.sh with the one provided by you
Likely error (tail of the failed operation logfile):
CPPFLAGS: -D_FORTIFY_SOURCE=0 -D__USE_MINGW_ANSI_STDIO=1
CFLAGS: -mthreads -mtune=generic -O2 -pipe
CXXFLAGS: -mthreads -mtune=generic -O2 -pipe
LDFLAGS: -pipe -static-libgcc -static-libstdc++ -L/local64/lib -L/mingw64/lib
../configure --prefix=/local64 --bindir=/local64/bin-video --pkg-config-flags=--static --cc=ccache gcc --cxx=ccache g++ --disable-autodetect --enable-amf --enable-bzlib --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-iconv --enable-lzma --enable-zlib --enable-sdl2 --enable-ffnvcodec --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libdav1d --enable-libaom --disable-debug --enable-fontconfig --enable-libass --enable-libbluray --enable-libfreetype --enable-libmfx --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libwavpack --enable-libwebp --enable-libxml2 --enable-libzimg --enable-libshine --enable-gpl --enable-avisynth --enable-libxvid --enable-libopenmpt --enable-version3 --enable-chromaprint --enable-decklink --enable-frei0r --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libfdk-aac --enable-libflite --enable-libfribidi --enable-libgme --enable-libgsm --enable-libilbc --enable-libkvazaar --enable-libmodplug --enable-librtmp --enable-librubberband --enable-libtesseract --enable-libxavs --enable-libzmq --enable-libzvbi --enable-openal --enable-libvmaf --enable-libcodec2 --enable-libsrt --enable-ladspa --enable-librav1e --enable-libglslang --enable-vulkan --enable-libnpp --enable-libopenh264 --enable-openssl --extra-cflags=-fopenmp --extra-libs=-lgomp --extra-cflags=-DLIBTWOLAME_STATIC --extra-libs=-lstdc++ --extra-cflags=-DCACA_STATIC --extra-cflags=-DMODPLUG_STATIC --extra-cflags=-DCHROMAPRINT_NODLL --extra-libs=-lstdc++ --extra-cflags=-DZMQ_STATIC --extra-libs=-lpsapi --extra-cflags=-DLIBXML_STATIC --extra-libs=-liconv --disable-w32threads --extra-cflags=-DKVZ_STATIC_LIB --enable-nonfree --extra-cflags=-IC:/PROGRA~1/NVIDIA GPU Computing Toolkit/CUDA/v11.0/include --extra-ldflags=-LC:/PROGRA~1/NVIDIA GPU Computing Toolkit/CUDA/v11.0/lib/x64 --extra-version=g06fef1e9f1+3
Unknown option "GPU"
The log is here ...
One thing that wasn't asked yet is that you show us the output of printf '"%s"\n' "$CUDA_PATH"
$ printf '"%s"\n' "$CUDA_PATH" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0"
fixed_CUDA_PATH="$(cygpath -sm "$CUDA_PATH")"
Supposedly that shouldn't work since cygpath is not working well right?
Well, I've seen this issue happen before, but I don't really know how to fix it.
@wiiaboo do you think it would work if we inject the flags quoted in the CFLAGS instead of using the configure option?
I'm not sure how to forcefully make cygpath produce a shorter path intentionally, might need to find a way to manually generate one if a path has spaces or is too long
it seems that cygpath just sends the output of GetShortPathNameW
as is after replacing the slashes, so in this case it seems it might be an issue with windows' API
or ...
an alternative is that windows haven't generated a short name for the full path yet.
Sorry. I closed this issue unintentionally
do you think it would work if we inject the flags quoted in the CFLAGS instead of using the configure option?
Sure, might work.
If you want to code some test I could check it
Sorry, haven't yet had time to try to make a change for this due to real life stuff, going to try in a second, going to first see if injecting is CFLAGS is better or try making a small dumb program that will set a shortpath for the cuda_path
@miguelapg can you try the same branch? I force pushed it there
Yes, of course, this is the result ...
CPPFLAGS: -D_FORTIFY_SOURCE=0 -D__USE_MINGW_ANSI_STDIO=1
CFLAGS: -mthreads -mtune=generic -O2 -pipe -I'C:/PROGRA~1/NVIDIA GPU Computing Toolkit/CUDA/v11.0/include'
CXXFLAGS: -mthreads -mtune=generic -O2 -pipe
LDFLAGS: -pipe -static-libgcc -static-libstdc++ -L'C:/PROGRA~1/NVIDIA GPU Computing Toolkit/CUDA/v11.0/lib/x64' -L/local64/lib -L/mingw64/lib
../configure --prefix=/local64 --bindir=/local64/bin-video --pkg-config-flags=--static --cc=ccache gcc --cxx=ccache g++ --disable-autodetect --enable-amf --enable-bzlib --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-iconv --enable-lzma --enable-nvenc --enable-zlib --enable-sdl2 --enable-ffnvcodec --enable-nvdec --enable-cuda-llvm --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libdav1d --enable-libaom --disable-debug --enable-fontconfig --enable-libass --enable-libbluray --enable-libfreetype --enable-libmfx --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libwavpack --enable-libwebp --enable-libxml2 --enable-libzimg --enable-libshine --enable-gpl --enable-avisynth --enable-libxvid --enable-libopenmpt --enable-version3 --enable-chromaprint --enable-decklink --enable-frei0r --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libfdk-aac --enable-libflite --enable-libfribidi --enable-libgme --enable-libgsm --enable-libilbc --enable-libkvazaar --enable-libmodplug --enable-librtmp --enable-librubberband --enable-libtesseract --enable-libxavs --enable-libzmq --enable-libzvbi --enable-openal --enable-libvmaf --enable-libcodec2 --enable-libsrt --enable-ladspa --enable-librav1e --enable-libglslang --enable-vulkan --enable-opencl --enable-opengl --enable-libnpp --enable-libopenh264 --enable-openssl --extra-cflags=-fopenmp --extra-libs=-lgomp --extra-cflags=-DLIBTWOLAME_STATIC --extra-libs=-lstdc++ --extra-cflags=-DCACA_STATIC --extra-cflags=-DMODPLUG_STATIC --extra-cflags=-DCHROMAPRINT_NODLL --extra-libs=-lstdc++ --extra-cflags=-DZMQ_STATIC --extra-libs=-lpsapi --extra-cflags=-DLIBXML_STATIC --extra-libs=-liconv --disable-w32threads --extra-cflags=-DKVZ_STATIC_LIB --enable-nonfree --extra-version=g9ce787d61e+3
ccache gcc is unable to create an executable file.
If ccache gcc is a cross-compiler, use the --enable-cross-compile option.
Only do this if you know what cross compiling means.
C compiler test failed.
Creating diagnostics file...
All relevant logs have been anonymously uploaded to https://0x0.st/i3g8.zip
$ cygpath -sm "$CUDA_PATH" C:/PROGRA~1/NVIDIA GPU Computing Toolkit/CUDA/v11.0
$ ls "$CUDA_PATH" bin CUDA_Toolkit_Release_Notes.txt doc EULA.txt extras include lib libnvvp nvml nvvm Sanitizer src tools
$ command -v cygpath.exe /usr/bin/cygpath.exe
$ /usr/bin/cygpath.exe -V cygpath (cygwin) 3.1.6 Path Conversion Utility Copyright (C) 1998 - 2020 Cygwin Authors
$ printf '"%s"\n' "$CUDA_PATH" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0"
I found this article about how to create short paths in windows ...
https://peoplesofttutorial.com/how-to-create-short-path-for-windows-folders-or-directories/
Hope this helps too. I don't know if you can use this from MSYS2
doesn't seem we can use that article since we can't require using admin privilege for this suite.
could you run the mintty link in the suite and copy and paste this into there?
git apply <<'EOF'
diff --git a/build/media-suite_helper.sh b/build/media-suite_helper.sh
index 0dbc243..99a1d5a 100644
--- a/build/media-suite_helper.sh
+++ b/build/media-suite_helper.sh
@@ -886,8 +886,8 @@ do_changeFFmpegConfig() {
echo -e "${orange}FFmpeg and related apps will depend on CUDA SDK to run!${reset}"
local fixed_CUDA_PATH
fixed_CUDA_PATH="$(cygpath -sm "$CUDA_PATH")"
- CFLAGS+=" -I'$fixed_CUDA_PATH/include'"
- LDFLAGS+=" -L'$fixed_CUDA_PATH/lib/x64'"
+ CFLAGS+=" -I\"$fixed_CUDA_PATH/include\""
+ LDFLAGS+=" -L\"$fixed_CUDA_PATH/lib/x64\""
fi
if enabled cuda-nvcc; then
local fixed_CUDA_PATH_UNIX
EOF
and rerun?
It didn't work, unfortunately.
CPPFLAGS: -D_FORTIFY_SOURCE=0 -D__USE_MINGW_ANSI_STDIO=1
CFLAGS: -mthreads -mtune=generic -O2 -pipe -I"C:/PROGRA~1/NVIDIA GPU Computing Toolkit/CUDA/v11.0/include"
CXXFLAGS: -mthreads -mtune=generic -O2 -pipe
LDFLAGS: -pipe -static-libgcc -static-libstdc++ -L"C:/PROGRA~1/NVIDIA GPU Computing Toolkit/CUDA/v11.0/lib/x64" -L/local64/lib -L/mingw64/lib
../configure --prefix=/local64 --bindir=/local64/bin-video --pkg-config-flags=--static --cc=ccache gcc --cxx=ccache g++ --disable-autodetect --enable-amf --enable-bzlib --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-iconv --enable-lzma --enable-nvenc --enable-zlib --enable-sdl2 --enable-ffnvcodec --enable-nvdec --enable-cuda-llvm --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libdav1d --enable-libaom --disable-debug --enable-fontconfig --enable-libass --enable-libbluray --enable-libfreetype --enable-libmfx --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libwavpack --enable-libwebp --enable-libxml2 --enable-libzimg --enable-libshine --enable-gpl --enable-avisynth --enable-libxvid --enable-libopenmpt --enable-version3 --enable-chromaprint --enable-decklink --enable-frei0r --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libfdk-aac --enable-libflite --enable-libfribidi --enable-libgme --enable-libgsm --enable-libilbc --enable-libkvazaar --enable-libmodplug --enable-librtmp --enable-librubberband --enable-libtesseract --enable-libxavs --enable-libzmq --enable-libzvbi --enable-openal --enable-libvmaf --enable-libcodec2 --enable-libsrt --enable-ladspa --enable-librav1e --enable-libglslang --enable-vulkan --enable-opencl --enable-opengl --enable-libnpp --enable-libopenh264 --enable-openssl --extra-cflags=-fopenmp --extra-libs=-lgomp --extra-cflags=-DLIBTWOLAME_STATIC --extra-libs=-lstdc++ --extra-cflags=-DCACA_STATIC --extra-cflags=-DMODPLUG_STATIC --extra-cflags=-DCHROMAPRINT_NODLL --extra-libs=-lstdc++ --extra-cflags=-DZMQ_STATIC --extra-libs=-lpsapi --extra-cflags=-DLIBXML_STATIC --extra-libs=-liconv --disable-w32threads --extra-cflags=-DKVZ_STATIC_LIB --enable-nonfree --extra-version=gea8f8d28d0+3
ccache gcc is unable to create an executable file.
If ccache gcc is a cross-compiler, use the --enable-cross-compile option.
Only do this if you know what cross compiling means.
C compiler test failed.
All relevant logs have been anonymously uploaded to https://0x0.st/i3EY.zip
If you cannot solve this I can install the Nvidia Toolkit in a folder without blank spaces, and you could update the installation manual with some warning. It's just an idea.
Yeah. Probably will need to require people to reinstall to a shorter path if the path produced by windows has a space, it doesn't seem quoting it helps at all since even using single or double results in gcc seeing them as separate arguments.
I will try to see if making a single small program just to add a short path to it without admin permissions can work, else, I will just add an error at the time of processing the CUDA_PATH variable if there's a space and add a note about it in the README.md
Thank you!! The Toolkit by Nvidia allows a custom installation where you can choose a different path
Hmm, it seems trying to create a small program to do that is very hit or miss depending on the system's setup.
I'll just go the error and note route for this one
the small test program I made for future reference
#define WIN32_LEAN_AND_MEAN
#include <stdio.h>
#include <windows.h>
int main() {
PTSTR filename = TEXT("D:/media-autobuild_suite/build/media-autobuild_suite.ini"),
shortname = TEXT("D:/MEDIA-~1/build/MEDIA-~1.INI");
WIN32_FIND_DATA temp;
HANDLE hFile = FindFirstFile(filename, &temp);
printf("%d\n", SetFileShortName(
hFile,
shortname));
DWORD length = GetShortPathName(filename, NULL, 0);
PTSTR buffer = malloc(length * sizeof(*buffer));
length = GetShortPathName(filename, buffer, length);
printf("long name = %s shortname = %s, %s, %s; \n", filename, buffer, temp.cFileName, temp.cAlternateFileName);
free(buffer);
}
gcc ./temp.c; .\a.exe
0
long name = D:/media-autobuild_suite/build/media-autobuild_suite.ini shortname = D:/media-autobuild_suite/build/media-autobuild_suite.ini, media-autobuild_suite.ini, ;
attempting to get it to work with UNICODE was just a plain nope.
Actually, before I commit any doc changes etc, can you retry downloading from the same branch and rerun?
changes this time around
diff --git a/build/media-suite_helper.sh b/build/media-suite_helper.sh
index 5927f9f..a1b3952 100644
--- a/build/media-suite_helper.sh
+++ b/build/media-suite_helper.sh
@@ -886,8 +886,8 @@ do_changeFFmpegConfig() {
echo -e "${orange}FFmpeg and related apps will depend on CUDA SDK to run!${reset}"
local fixed_CUDA_PATH
fixed_CUDA_PATH="$(cygpath -sm "$CUDA_PATH")"
- do_addOption "--extra-cflags=-I$fixed_CUDA_PATH/include"
- do_addOption "--extra-ldflags=-L$fixed_CUDA_PATH/lib/x64"
+ do_addOption "--extra-cflags=-I${fixed_CUDA_PATH// /\\ }/include"
+ do_addOption "--extra-ldflags=-L${fixed_CUDA_PATH// /\\ }/lib/x64"
fi
if enabled cuda-nvcc; then
local fixed_CUDA_PATH_UNIX
It doesn't work either.
CPPFLAGS: -D_FORTIFY_SOURCE=0 -D__USE_MINGW_ANSI_STDIO=1
CFLAGS: -mthreads -mtune=generic -O2 -pipe
CXXFLAGS: -mthreads -mtune=generic -O2 -pipe
LDFLAGS: -pipe -static-libgcc -static-libstdc++ -L/local64/lib -L/mingw64/lib
../configure --prefix=/local64 --bindir=/local64/bin-video --pkg-config-flags=--static --cc=ccache gcc --cxx=ccache g++ --disable-autodetect --enable-amf --enable-bzlib --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-iconv --enable-lzma --enable-nvenc --enable-zlib --enable-sdl2 --enable-ffnvcodec --enable-nvdec --enable-cuda-llvm --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libdav1d --enable-libaom --disable-debug --enable-fontconfig --enable-libass --enable-libbluray --enable-libfreetype --enable-libmfx --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libwavpack --enable-libwebp --enable-libxml2 --enable-libzimg --enable-libshine --enable-gpl --enable-avisynth --enable-libxvid --enable-libopenmpt --enable-version3 --enable-chromaprint --enable-decklink --enable-frei0r --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libfdk-aac --enable-libflite --enable-libfribidi --enable-libgme --enable-libgsm --enable-libilbc --enable-libkvazaar --enable-libmodplug --enable-librtmp --enable-librubberband --enable-libtesseract --enable-libxavs --enable-libzmq --enable-libzvbi --enable-openal --enable-libvmaf --enable-libcodec2 --enable-libsrt --enable-ladspa --enable-librav1e --enable-libglslang --enable-vulkan --enable-opencl --enable-opengl --enable-libnpp --enable-libopenh264 --enable-openssl --extra-cflags=-fopenmp --extra-libs=-lgomp --extra-cflags=-DLIBTWOLAME_STATIC --extra-libs=-lstdc++ --extra-cflags=-DCACA_STATIC --extra-cflags=-DMODPLUG_STATIC --extra-cflags=-DCHROMAPRINT_NODLL --extra-libs=-lstdc++ --extra-cflags=-DZMQ_STATIC --extra-libs=-lpsapi --extra-cflags=-DLIBXML_STATIC --extra-libs=-liconv --disable-w32threads --extra-cflags=-DKVZ_STATIC_LIB --enable-nonfree --extra-cflags=-IC:/PROGRA~1/NVIDIA\ GPU\ Computing\ Toolkit/CUDA/v11.0/include --extra-ldflags=-LC:/PROGRA~1/NVIDIA\ GPU\ Computing\ Toolkit/CUDA/v11.0/lib/x64 --extra-version=g6e951d0cf8+3
ccache gcc is unable to create an executable file.
If ccache gcc is a cross-compiler, use the --enable-cross-compile option.
Only do this if you know what cross compiling means.
C compiler test failed.
All relevant logs have been anonymously uploaded to https://0x0.st/i3ld.zip
I guess no-paths is always the better option
When I add this in MSYS2 it goes to the right folder ...
$ cd C:/PROGRA~1/NVIDIA\ GPU\ Computing\ Toolkit/CUDA/v11.0/include
mm@MM /c/PROGRA~1/NVIDIA GPU Computing Toolkit/CUDA/v11.0/include
$ pwd
/c/PROGRA~1/NVIDIA GPU Computing Toolkit/CUDA/v11.0/include
I don't understand why this doesn't happen in the compiling time.
It's working adding this too ...
$ cd /c/PROGRA~1/NVIDIA\ GPU\ Computing\ Toolkit/CUDA/v11.0/include
$ pwd
/c/PROGRA~1/NVIDIA GPU Computing Toolkit/CUDA/v11.0/include
Here you have the output ...
CPPFLAGS: -D_FORTIFY_SOURCE=0 -D__USE_MINGW_ANSI_STDIO=1 CFLAGS: -mthreads -mtune=generic -O2 -pipe CXXFLAGS: -mthreads -mtune=generic -O2 -pipe LDFLAGS: -pipe -static-libgcc -static-libstdc++ -L/local64/lib -L/mingw64/lib ../configure --prefix=/local64 --bindir=/local64/bin-video --pkg-config-flags=--static --cc=ccache gcc --cxx=ccache g++ , etc ccache gcc is unable to create an executable file. If ccache gcc is a cross-compiler, use the --enable-cross-compile option. Only do this if you know what cross compiling means. C compiler test failed.
The full logs are placed at ... https://0x0.st/i3fi.zip
As you can see, it's impossible to compile ffmpeg.
Regards, Mapg