search

RPCS3 / rpcs3

PS3 emulator/debugger
https://rpcs3.net/
GNU General Public License v2.0
14.83k stars 1.87k forks source link

Guix package: build offline without the git submodules #7508

Closed Ambrevar closed 3 years ago

Ambrevar commented 4 years ago

(Sorry if this belonds to a discord discussion, but Discord won't let me log in.)

Quick summary

I'm trying to package rpcs3 for guix.

Details

The package definition I've got so far is here: https://issues.guix.gnu.org/issue/39644

It's mostly inspired from the Nix definition.

It fails with the following output:

starting phase `configure'
source directory: "/tmp/guix-build-rpcs3-0.0.8.drv-0/source" (relative from build: "../source")
build directory: "/tmp/guix-build-rpcs3-0.0.8.drv-0/build"
running 'cmake' with arguments ("../source" "-DCMAKE_BUILD_TYPE=RelWithDebInfo" "-DCMAKE_INSTALL_PREFIX=/gnu/store/0p9v13dy10abknx6j48yn9dwwx614x56-rpcs3-0.0.8" "-DCMAKE_INSTALL_LIBDIR=lib" "-DCMAKE_INSTALL_RPATH_USE_LINK_PATH=TRUE" "-DCMAKE_INSTALL_RPATH=/gnu/store/0p9v13dy10abknx6j48yn9dwwx614x56-rpcs3-0.0.8/lib" "-DCMAKE_VERBOSE_MAKEFILE=ON" "-DUSE_SYSTEM_LIBPNG=ON" "-DUSE_SYSTEM_FFMPEG=ON" "-DBUILD_LLVM_SUBMODULE=OFF" "-DWITH_LLVM=OFF" "-DUSE_NATIVE_INSTRUCTIONS=OFF")
-- The C compiler identification is GNU 9.2.0
-- The CXX compiler identification is GNU 9.2.0
-- Check for working C compiler: /gnu/store/3pa3gmb9np87iz7sjrs405kbbj9bx74m-gcc-9.2.0/bin/gcc
-- Check for working C compiler: /gnu/store/3pa3gmb9np87iz7sjrs405kbbj9bx74m-gcc-9.2.0/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /gnu/store/3pa3gmb9np87iz7sjrs405kbbj9bx74m-gcc-9.2.0/bin/c++
-- Check for working CXX compiler: /gnu/store/3pa3gmb9np87iz7sjrs405kbbj9bx74m-gcc-9.2.0/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at Vulkan/CMakeLists.txt:8 (add_subdirectory):
  The source directory

    /tmp/guix-build-rpcs3-0.0.8.drv-0/source/Vulkan/glslang

  does not contain a CMakeLists.txt file.

CMake Error at asmjitsrc/CMakeLists.txt:8 (include):
  include could not find load file:

    /tmp/guix-build-rpcs3-0.0.8.drv-0/source/asmjit/CMakeLists.txt

-- Found PkgConfig: /gnu/store/i1cqaixp79vd3qwnyj1ll10pq6skm2wk-pkg-config-0.29.2/bin/pkg-config (found version "0.29.2") 
-- Found ZLIB: /gnu/store/qx7p7hiq90mi7r78hcr9cyskccy2j4bg-zlib-1.2.11/lib/libz.so (found version "1.2.11") 
-- Found PNG: /gnu/store/3snpwk7jl8i125bhiilvk9scqc4mnsx7-libpng-1.6.37/lib/libpng.so (found version "1.6.37") 
CMake Error at 3rdparty/CMakeLists.txt:111 (add_subdirectory):
  The source directory

    /tmp/guix-build-rpcs3-0.0.8.drv-0/source/3rdparty/pugixml

  does not contain a CMakeLists.txt file.

CMake Error at 3rdparty/CMakeLists.txt:124 (add_subdirectory):
  add_subdirectory given source "hidapi/linux" which is not an existing
  directory.

CMake Error at 3rdparty/CMakeLists.txt:125 (target_include_directories):
  Cannot specify include directories for target "hidapi-hidraw" which is not
  built by this project.

CMake Error at 3rdparty/CMakeLists.txt:151 (add_subdirectory):
  The source directory

    /tmp/guix-build-rpcs3-0.0.8.drv-0/source/3rdparty/libusb

  does not contain a CMakeLists.txt file.

CMake Error at 3rdparty/CMakeLists.txt:165 (add_subdirectory):
  The source directory

    /tmp/guix-build-rpcs3-0.0.8.drv-0/source/3rdparty/yaml-cpp

  does not contain a CMakeLists.txt file.

CMake Error at 3rdparty/CMakeLists.txt:173 (add_subdirectory):
  add_subdirectory given source "xxHash/cmake_unofficial" which is not an
  existing directory.

CMake Error at 3rdparty/CMakeLists.txt:174 (target_include_directories):
  Cannot specify include directories for target "xxhash" which is not built
  by this project.

-- Found OpenGL: /gnu/store/cxnra20zy4zdbnw563q5s3bbgan20nmg-mesa-19.3.2/lib/libGL.so   
-- Found ALSA: /gnu/store/n0src98azpkmnhaq7jrlkhgpynzpil7c-alsa-lib-1.2.1.2/lib/libasound.so (found version "1.2.1.2") 
-- Checking for module 'libpulse-simple'
--   Found libpulse-simple, version 13.0
-- Checking for module 'libevdev'
--   Found libevdev, version 1.5.9
-- Found Vulkan: /gnu/store/jmsngf3h3f20c1pddyxzycsp3bc9z95j-vulkan-loader-1.1.121/lib/libvulkan.so  
-- Found WAYLAND_CLIENT: /gnu/store/m7lpzgn381a7kjcjd3k9yn99q2kybzgk-wayland-1.17.0/lib/libwayland-client.so  
-- Found WAYLAND_SERVER: /gnu/store/m7lpzgn381a7kjcjd3k9yn99q2kybzgk-wayland-1.17.0/lib/libwayland-server.so  
-- Found WAYLAND_EGL: /gnu/store/m7lpzgn381a7kjcjd3k9yn99q2kybzgk-wayland-1.17.0/lib/libwayland-egl.so  
-- Found WAYLAND_CURSOR: /gnu/store/m7lpzgn381a7kjcjd3k9yn99q2kybzgk-wayland-1.17.0/lib/libwayland-cursor.so  
-- Found WAYLAND: /gnu/store/m7lpzgn381a7kjcjd3k9yn99q2kybzgk-wayland-1.17.0/lib/libwayland-client.so;/gnu/store/m7lpzgn381a7kjcjd3k9yn99q2kybzgk-wayland-1.17.0/lib/libwayland-server.so;/gnu/store/m7lpzgn381a7kjcjd3k9yn99q2kybzgk-wayland-1.17.0/lib/libwayland-egl.so;/gnu/store/m7lpzgn381a7kjcjd3k9yn99q2kybzgk-wayland-1.17.0/lib/libwayland-cursor.so  
-- Found OpenAL: /gnu/store/08z6y9mk3i6kzfgjsmqlhf339qqa3rrh-openal-1.20.1/lib/libopenal.so  
-- Checking for module 'sdl2'
--   Found sdl2, version 2.0.10
CMake Error at 3rdparty/CMakeLists.txt:348 (add_subdirectory):
  The source directory

    /tmp/guix-build-rpcs3-0.0.8.drv-0/source/3rdparty/FAudio

  does not contain a CMakeLists.txt file.

CMake Error at 3rdparty/CMakeLists.txt:349 (target_compile_definitions):
  Cannot specify compile definitions for target "FAudio" which is not built
  by this project.

-- RPCS3: using shared ffmpeg
-- Checking for module 'libavcodec'
--   Found libavcodec, version 58.54.100
-- Checking for module 'libavformat'
--   Found libavformat, version 58.29.100
-- Checking for module 'libavutil'
--   Found libavutil, version 56.31.100
-- Checking for module 'libswscale'
--   Found libswscale, version 5.5.100
-- Found FFMPEG or Libav: /gnu/store/5x53yxi0sczpy1lwq68as8cv82qb3bgx-ffmpeg-4.2.2/lib/libavcodec.so;/gnu/store/5x53yxi0sczpy1lwq68as8cv82qb3bgx-ffmpeg-4.2.2/lib/libavformat.so;/gnu/store/5x53yxi0sczpy1lwq68as8cv82qb3bgx-ffmpeg-4.2.2/lib/libavutil.so;/gnu/store/5x53yxi0sczpy1lwq68as8cv82qb3bgx-ffmpeg-4.2.2/lib/libswscale.so, /gnu/store/5x53yxi0sczpy1lwq68as8cv82qb3bgx-ffmpeg-4.2.2/include
-- Found GLEW: /gnu/store/nrvrdfxzxai2jpr0n12lb7dl3fq7v47f-glew-2.1.0/include (found suitable version "2.1.0", minimum required is "1.13.0") 
CMake Error at 3rdparty/CMakeLists.txt:417 (add_library):
  add_library cannot create ALIAS target "3rdparty::libusb" because target
  "usb-1.0-static" does not already exist.

CMake Error at 3rdparty/CMakeLists.txt:420 (add_library):
  add_library cannot create ALIAS target "3rdparty::pugixml" because target
  "pugixml" does not already exist.

CMake Error at 3rdparty/CMakeLists.txt:421 (add_library):
  add_library cannot create ALIAS target "3rdparty::yaml-cpp" because target
  "yaml-cpp" does not already exist.

CMake Error at 3rdparty/CMakeLists.txt:422 (add_library):
  add_library cannot create ALIAS target "3rdparty::xxhash" because target
  "xxhash" does not already exist.

CMake Error at 3rdparty/CMakeLists.txt:432 (add_library):
  add_library cannot create ALIAS target "3rdparty::faudio" because target
  "FAudio" does not already exist.

-- cotire 1.8.0 loaded.
-- Found Git: /gnu/store/khvcbw37r31xmfdalplw10xz9d4j6v8l-git-2.25.0/bin/git (found version "2.25.0") 
CMake Warning at rpcs3/git-version.cmake:33 (message):
  git not found, unable to include version.
Call Stack (most recent call first):
  rpcs3/CMakeLists.txt:6 (include)

-- Performing Test HAS_NO_PIE
-- Performing Test HAS_NO_PIE - Success
-- Performing Test COMPILER_SUPPORTS_MARCH_NATIVE
-- Performing Test COMPILER_SUPPORTS_MARCH_NATIVE - Success
-- Found X11: /gnu/store/b824dq3bccq0bhjli3li0fzi11lg1bh3-xorgproto-2019.1/include   
-- Looking for XOpenDisplay in /gnu/store/06ybqkh3lb3g7c77b74izy32grglf45x-libx11-1.6.8/lib/libX11.so;/gnu/store/j01p4wjas358cvayyjqpl449hmpanra8-libxext-1.3.4/lib/libXext.so
-- Looking for XOpenDisplay in /gnu/store/06ybqkh3lb3g7c77b74izy32grglf45x-libx11-1.6.8/lib/libX11.so;/gnu/store/j01p4wjas358cvayyjqpl449hmpanra8-libxext-1.3.4/lib/libXext.so - found
-- Looking for gethostbyname
-- Looking for gethostbyname - found
-- Looking for connect
-- Looking for connect - found
-- Looking for remove
-- Looking for remove - found
-- Looking for shmat
-- Looking for shmat - found
-- Initializing RPCS3_SRC_DIR=/tmp/guix-build-rpcs3-0.0.8.drv-0/source/rpcs3
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- CXX target rpcs3_emu cotired without unity build excluding ../../Utilities/JIT.cpp, Cell/PPUTranslator.cpp.
-- RPCS3_GIT_VERSION: unknown
-- RPCS3_GIT_BRANCH: unknown
-- CXX target rpcs3 cotired without unity build.
-- Configuring incomplete, errors occurred!
See also "/tmp/guix-build-rpcs3-0.0.8.drv-0/build/CMakeFiles/CMakeOutput.log".
See also "/tmp/guix-build-rpcs3-0.0.8.drv-0/build/CMakeFiles/CMakeError.log".
command "cmake" "../source" "-DCMAKE_BUILD_TYPE=RelWithDebInfo" "-DCMAKE_INSTALL_PREFIX=/gnu/store/0p9v13dy10abknx6j48yn9dwwx614x56-rpcs3-0.0.8" "-DCMAKE_INSTALL_LIBDIR=lib" "-DCMAKE_INSTALL_RPATH_USE_LINK_PATH=TRUE" "-DCMAKE_INSTALL_RPATH=/gnu/store/0p9v13dy10abknx6j48yn9dwwx614x56-rpcs3-0.0.8/lib" "-DCMAKE_VERBOSE_MAKEFILE=ON" "-DUSE_SYSTEM_LIBPNG=ON" "-DUSE_SYSTEM_FFMPEG=ON" "-DBUILD_LLVM_SUBMODULE=OFF" "-DWITH_LLVM=OFF" "-DUSE_NATIVE_INSTRUCTIONS=OFF" failed with status 1

I suspect that the failure is due to the submodules not being loaded (there is no network access when Guix builds). That said, I have all the dependencies. Would it be possible to replace all 3rd party dependencies with system packages?

AniLeo commented 4 years ago

Some dependencies such as LLVM have specific commits for RPCS3, that's why the system one isn't used. You can replace some with system defaults, but not all.

Ambrevar commented 4 years ago

With Guix I can choose the right commit for any dependency, that's not a problem.

AniLeo commented 4 years ago

I'm afraid I don't follow, then can't you fetch all dependencies in 3rdparty folder on their latest commits?

Ambrevar commented 4 years ago

I could do that, but it's rather tedious to do for packagers. It's easier (and good practice) if all dependencies can loaded from the system by cmake with a simple Cmake variable for instance.

Looking at the .gitmodules file, it seems that only LLVM follows a specific branch (release_60), the rest seems to target master.

AniLeo commented 4 years ago

LLVM targets master from our fork (the branch on gitmodules is wrong for some reason) hidapi also targets master from our fork

I believe the other dependencies don't have any RPCS3 specific commits, but note that ffmpeg is the one from the Vita3K fork of ffmpeg

AniLeo commented 4 years ago

I've fixed the LLVM submodule on afb594c2335b59674ddcd4978aade013a15dce2f , feel free to ask regarding the other submodules if other questions arise

Ambrevar commented 4 years ago

OK, I'll look into it again in some time. Thanks!

hcorion commented 3 years ago

Gonna close this issue, since it sounds like the issue has been fixed, feel free to reopen if it's still an issue.