Upgrade GCC version (PPU-SPU)

humbertodias commented 6 months ago

Features

[x] binutils-ppu bumped from 2.22 to 2.42
[x] ppu-gcc bumped from 7.20 to 13.2.0
[x] spu-gcc bumped from 7.20 to 9.5.0
[x] docker bumped from 16.04 to 24.04 - LTS
[x] ci/cd working again and building releases
[x] ci/cd to docker hub
[x] ci/cd badges
[x] build on ubuntu/macOS for arm64 enabled
[ ] test all PSL1GHT samples

[!NOTE]
We've selected the version 9.5.0 because it's the last GCC release to include support for SPU. https://www.phoronix.com/news/GCC-10-Drops-Cell-BE-SPU

[!TIP] Patches credits to disc-kuraudo and luizfernandonb

Result

ar --version

GNU ar (GNU Binutils) 2.42
Copyright (C) 2024 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) any later version.
This program has absolutely no warranty.

ppu-gcc --version

ppu-gcc (GCC) 13.2.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

spu-gcc --version

spu-gcc (GCC) 9.5.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Docker
```
FROM ubuntu:24.04
```
ci/cd success

https://github.com/humbertodias/ps3toolchain/actions/runs/8166663270

ci/cd docker

[!NOTE]
Don't forget to create a named DockerHub environment on your repository with two secret variables: DOCKERHUB_USERNAME and DOCKERHUB_TOKEN
badges

Fewnity commented 6 months ago

Hey cool work, I really want gcc 13 on ps3. But you just forgot to credit Darjan Krijanand and luizfernandonb :/

humbertodias commented 6 months ago

Hey cool work, I really want gcc 13 on ps3. But you just forgot to credit Darjan Krijanand and luizfernandonb :/

Yeah.. I left his name on patch's file given him the credit

Fewnity commented 6 months ago

Did you tried to compile some samples of PSL1GHT? Looks like the rsxtest sample does not work anymore with the gcc upgrade in RPCS3, didn't tried on real hardware yet. Update: Crashes on real hardware too

humbertodias commented 6 months ago

Did you tried to compile some samples of PSL1GHT? Looks like the rsxtest sample does not work anymore with the gcc upgrade in RPCS3, didn't tried on real hardware yet. Update: Crashes on real hardware too

Excellent! Indeed, I've successfully tested several tiny3D samples on rpcs3 utilizing the script.

git clone https://github.com/wargio/tiny3D
TOOLCHAIN_IMAGE=hldtux/ps3dev
#TOOLCHAIN_IMAGE=zeldin/ps3dev-docker
docker run -i -v`pwd`:/build $TOOLCHAIN_IMAGE bash -s <<EOF
ppu-gcc --version
ppu-g++ --version
spu-gcc --version
spu-g++ --version

cd tiny3D
make
cd samples
(cd fireworks3D && make pkg)
(cd fonts && make pkg)
(cd fonts_from_ttf && make pkg)
(cd ps3loadx && make pkg)
(cd spheres3D && make pkg)
(cd sprites2D && make pkg)
(cd surfaces && make pkg)
(cd tiny3d_lists && make pkg)
(cd userviewport && make pkg)
(cd yuv && make pkg)
EOF

About the rsxtest sample that you mention. Could you describe the steps to reproduce the issue?

Concerning your test on real hardware. Unfortunately, I don't have a ps3 right now. However, you could compare this PR version with the previous just changing the TOOLCHAIN_IMAGE=zeldin/ps3dev-docker and executing the script again and tell us if it was working before the upgrade.

humbertodias commented 6 months ago

Did you tried to compile some samples of PSL1GHT? Looks like the rsxtest sample does not work anymore with the gcc upgrade in RPCS3, didn't tried on real hardware yet. Update: Crashes on real hardware too

Fixed here https://github.com/ps3dev/ps3toolchain/issues/118

Fewnity commented 6 months ago

I have to retry, but to compile the rsxtest sample I was using the prebuilt ubuntu ps3toolchain. I extracted the toolchain then I compiled and installed PSL1GHT and I typed "make" in the folder rsxtest, nothing more. I tried make pkg also but same result, crash. But I found where was the problem in the log. Error: videoConfigure failed. so videoConfigure(....) in rsxutil.cpp line 96 was the problem on the emulator 🤔

I tried a tiny3D sample (Sphere3D) and it was working fine on emulator but no time to test on real hardware.

Btw why all prebuilt ps3toolchain files are gone? We can't download them now 🤔

humbertodias commented 6 months ago

I have to retry, but to compile the rsxtest sample I was using the prebuilt ubuntu ps3toolchain. I extracted the toolchain then I compiled and installed PSL1GHT and I typed "make" in the folder rsxtest, nothing more. I tried make pkg also but same result, crash. But I found where was the problem in the log. Error: videoConfigure failed. so videoConfigure(....) in rsxutil.cpp line 96 was the problem on the emulator 🤔

I tried a tiny3D sample (Sphere3D) and it was working fine on emulator but no time to test on real hardware.

Btw why all prebuilt ps3toolchain files are gone? We can't download them now 🤔

Oh.. sorry. I forgot to trigger the pipeline. Here it's ps3dev-ubuntu-latest-x86_64.tar.gz

What I did for testing

# prebuilt ps3dev for ubuntu-24.04-x86_64
wget https://github.com/humbertodias/ps3toolchain/releases/download/ubuntu-latest-04ddabf8/ps3dev-ubuntu-latest-x86_64-2024-03-20.tar.gz -O ps3dev.tar.gz
tar xvfz ps3dev.tar.gz
rm ps3dev.tar.gz

# env
export PS3DEV=`pwd`/ps3dev
export PSL1GHT=$PS3DEV
export PATH=$PATH:$PS3DEV/bin
export PATH=$PATH:$PS3DEV/ppu/bin
export PATH=$PATH:$PS3DEV/spu/bin

# psl1ght samples
git clone https://github.com/ps3dev/PSL1GHT
cd PSL1GHT
make

# compile and make pkg
cd samples/graphics/rsxtest
make
make pkg

output

make[1]: '/build/PSL1GHT/samples/graphics/rsxtest/rsxtest.self' is up to date.
building pkg ... rsxtest.pkg

Fewnity commented 5 months ago

I retried to compile the sample with your last ubuntu build (2024-03-20) and it's still crashing when launching the homebrew. Were your rsxtest.self/rsxtest.pkg working well?

TheRouletteBoi commented 5 months ago

what was wrong with that arm64 version?

Update: I'm currently trying to compile the toolchain but getting some errors with autoconf. Can you try looking into it? I'm using arm64 ubuntu 23.10 in the docker container

FROM arm64v8/ubuntu:23.10

humbertodias commented 5 months ago

what was wrong with that arm64 version?

Update: I'm currently trying to compile the toolchain but getting some errors with autoconf. Can you try looking into it? I'm using arm64 ubuntu 23.10 in the docker container

FROM arm64v8/ubuntu:23.10

The issue ocurres when installing the dependency nvidia-cg-toolkit that is only packaged for x86 architecture. That is used for nvidia/cuda stuff and required for some psl1ght samples. Fixed here https://github.com/ps3dev/ps3toolchain/pull/121/commits/4d23a177cf01f201be1f4773c8c690898c893f04 or you also could force the arch as x86 doing:

DOCKER_DEFAULT_PLATFORM=linux/amd64 docker build . -t ps3dev

Please, try again and tell us if it was solved for your architecuture

TheRouletteBoi commented 5 months ago

I got the same error. Here is the output

1396.6 automake: warning: autoconf input should be named 'configure.ac', not 'configure.in'
1396.6 configure.in:45: warning: AM_INIT_AUTOMAKE: two- and three-arguments forms are deprecated.  For more info, see:
1396.6 configure.in:45: https://www.gnu.org/software/automake/manual/automake.html#Modernize-AM_005fINIT_005fAUTOMAKE-invocation
1396.6 configure.in:29: installing './compile'
1396.6 automake: warning: autoconf input should be named 'configure.ac', not 'configure.in'
1396.6 configure.in:45: warning: AM_INIT_AUTOMAKE: two- and three-arguments forms are deprecated.
1396.6 ./lib/autoconf/general.m4:2434: AC_DIAGNOSE is expanded from...
1396.6 aclocal.m4:529: AM_INIT_AUTOMAKE is expanded from...
1396.6 configure.in:45: the top level
1396.6 configure.in:48: warning: The macro `AC_PROG_LIBTOOL' is obsolete.
1396.6 configure.in:48: You should run autoupdate.
1396.6 acinclude/libtool.m4:107: AC_PROG_LIBTOOL is expanded from...
1396.6 configure.in:48: the top level
1396.6 configure.in:80: warning: The macro `AC_HELP_STRING' is obsolete.
1396.6 configure.in:80: You should run autoupdate.
1396.6 ./lib/autoconf/general.m4:204: AC_HELP_STRING is expanded from...
1396.6 ./lib/autoconf/general.m4:1534: AC_ARG_ENABLE is expanded from...
1396.6 configure.in:80: the top level
1396.6 configure.in:128: warning: The macro `AC_TRY_RUN' is obsolete.
1396.6 configure.in:128: You should run autoupdate.
1396.6 ./lib/autoconf/general.m4:2997: AC_TRY_RUN is expanded from...
1396.6 acinclude/sdl2.m4:13: AM_PATH_SDL2 is expanded from...
1396.6 configure.in:128: the top level
1396.6 configure.in:128: warning: The macro `AC_TRY_LINK' is obsolete.
1396.6 configure.in:128: You should run autoupdate.
1396.6 ./lib/autoconf/general.m4:2920: AC_TRY_LINK is expanded from...
1396.6 acinclude/sdl2.m4:13: AM_PATH_SDL2 is expanded from...
1396.6 configure.in:128: the top level
1396.6 configure.in:141: warning: AC_OUTPUT should be used without arguments.
1396.6 configure.in:141: You should run autoupdate.
1396.6 ./config.guess: unable to guess system type
1396.6
1396.6 This script, last modified 2012-01-01, has failed to recognize
1396.6 the operating system you are using. It is advised that you
1396.6 download the most up to date version of the config scripts from
1396.6
1396.6   http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD
1396.6 and
1396.6   http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub;hb=HEAD
1396.6
1396.6 If the version you run (./config.guess) is already up to date, please
1396.6 send the following data and any information you think might be
1396.6 pertinent to <config-patches@gnu.org> in order to provide the needed
1396.6 information to handle your system.
1396.6
1396.6 config.guess timestamp = 2012-01-01
1396.6
1396.6 uname -m = aarch64
1396.6 uname -r = 6.3.13-linuxkit
1396.6 uname -s = Linux
1396.6 uname -v = #1 SMP PREEMPT Thu Sep  7 07:48:47 UTC 2023
1396.6
1396.6 /usr/bin/uname -p = aarch64
1396.6 /bin/uname -X     =
1396.6
1396.6 hostinfo               =
1396.6 /bin/universe          =
1396.6 /usr/bin/arch -k       =
1396.6 /bin/arch              = aarch64
1396.6 /usr/bin/oslevel       =
1396.6 /usr/convex/getsysinfo =
1396.6
1396.6 UNAME_MACHINE = aarch64
1396.6 UNAME_RELEASE = 6.3.13-linuxkit
1396.6 UNAME_SYSTEM  = Linux
1396.6 UNAME_VERSION = #1 SMP PREEMPT Thu Sep  7 07:48:47 UTC 2023
1396.6 configure: error: cannot guess build type; you must specify one
------
Dockerfile:39
--------------------
  37 |     WORKDIR /build
  38 |     COPY . /build
  39 | >>> RUN /build/toolchain.sh
  40 |
  41 |     FROM ubuntu:22.04 as runtime
--------------------
ERROR: failed to solve: process "/bin/bash -c /build/toolchain.sh" did not complete successfully: exit code: 1

humbertodias commented 5 months ago

I got the same error. Here is the output

1396.6 automake: warning: autoconf input should be named 'configure.ac', not 'configure.in'
1396.6 configure.in:45: warning: AM_INIT_AUTOMAKE: two- and three-arguments forms are deprecated.  For more info, see:
1396.6 configure.in:45: https://www.gnu.org/software/automake/manual/automake.html#Modernize-AM_005fINIT_005fAUTOMAKE-invocation
1396.6 configure.in:29: installing './compile'
1396.6 automake: warning: autoconf input should be named 'configure.ac', not 'configure.in'
1396.6 configure.in:45: warning: AM_INIT_AUTOMAKE: two- and three-arguments forms are deprecated.
1396.6 ./lib/autoconf/general.m4:2434: AC_DIAGNOSE is expanded from...
1396.6 aclocal.m4:529: AM_INIT_AUTOMAKE is expanded from...
1396.6 configure.in:45: the top level
1396.6 configure.in:48: warning: The macro `AC_PROG_LIBTOOL' is obsolete.
1396.6 configure.in:48: You should run autoupdate.
1396.6 acinclude/libtool.m4:107: AC_PROG_LIBTOOL is expanded from...
1396.6 configure.in:48: the top level
1396.6 configure.in:80: warning: The macro `AC_HELP_STRING' is obsolete.
1396.6 configure.in:80: You should run autoupdate.
1396.6 ./lib/autoconf/general.m4:204: AC_HELP_STRING is expanded from...
1396.6 ./lib/autoconf/general.m4:1534: AC_ARG_ENABLE is expanded from...
1396.6 configure.in:80: the top level
1396.6 configure.in:128: warning: The macro `AC_TRY_RUN' is obsolete.
1396.6 configure.in:128: You should run autoupdate.
1396.6 ./lib/autoconf/general.m4:2997: AC_TRY_RUN is expanded from...
1396.6 acinclude/sdl2.m4:13: AM_PATH_SDL2 is expanded from...
1396.6 configure.in:128: the top level
1396.6 configure.in:128: warning: The macro `AC_TRY_LINK' is obsolete.
1396.6 configure.in:128: You should run autoupdate.
1396.6 ./lib/autoconf/general.m4:2920: AC_TRY_LINK is expanded from...
1396.6 acinclude/sdl2.m4:13: AM_PATH_SDL2 is expanded from...
1396.6 configure.in:128: the top level
1396.6 configure.in:141: warning: AC_OUTPUT should be used without arguments.
1396.6 configure.in:141: You should run autoupdate.
1396.6 ./config.guess: unable to guess system type
1396.6
1396.6 This script, last modified 2012-01-01, has failed to recognize
1396.6 the operating system you are using. It is advised that you
1396.6 download the most up to date version of the config scripts from
1396.6
1396.6   http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD
1396.6 and
1396.6   http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub;hb=HEAD
1396.6
1396.6 If the version you run (./config.guess) is already up to date, please
1396.6 send the following data and any information you think might be
1396.6 pertinent to <config-patches@gnu.org> in order to provide the needed
1396.6 information to handle your system.
1396.6
1396.6 config.guess timestamp = 2012-01-01
1396.6
1396.6 uname -m = aarch64
1396.6 uname -r = 6.3.13-linuxkit
1396.6 uname -s = Linux
1396.6 uname -v = #1 SMP PREEMPT Thu Sep  7 07:48:47 UTC 2023
1396.6
1396.6 /usr/bin/uname -p = aarch64
1396.6 /bin/uname -X     =
1396.6
1396.6 hostinfo               =
1396.6 /bin/universe          =
1396.6 /usr/bin/arch -k       =
1396.6 /bin/arch              = aarch64
1396.6 /usr/bin/oslevel       =
1396.6 /usr/convex/getsysinfo =
1396.6
1396.6 UNAME_MACHINE = aarch64
1396.6 UNAME_RELEASE = 6.3.13-linuxkit
1396.6 UNAME_SYSTEM  = Linux
1396.6 UNAME_VERSION = #1 SMP PREEMPT Thu Sep  7 07:48:47 UTC 2023
1396.6 configure: error: cannot guess build type; you must specify one
------
Dockerfile:39
--------------------
  37 |     WORKDIR /build
  38 |     COPY . /build
  39 | >>> RUN /build/toolchain.sh
  40 |
  41 |     FROM ubuntu:22.04 as runtime
--------------------
ERROR: failed to solve: process "/bin/bash -c /build/toolchain.sh" did not complete successfully: exit code: 1

I got it. Well, the root cause is the file https://git.savannah.gnu.org/cgit/config.git/plain/config.guess that is supposed to guess the current architecture and isn't working as expected for ARM. I'm gonna check out it later. However, in the meanwhile just use DOCKER_DEFAULT_PLATFORM=linux/amd64 that is already working and tell us the result.

DOCKER_DEFAULT_PLATFORM=linux/amd64 docker build . -t ps3dev

or for convenience just use the prebuilt one

DOCKER_DEFAULT_PLATFORM=linux/amd64 docker run -it -v `pwd`:/build -w /build hldtux/ps3dev

zeldin commented 5 months ago

I got it. Well, the root cause is the file https://git.savannah.gnu.org/cgit/config.git/plain/config.guess that is supposed to guess the current architecture and isn't working as expected for ARM. I'm gonna check out it later.

The output config.guess timestamp = 2012-01-01 means that it wasn't using an up-to-date config.guess at all, but rather an old one probably included with one of the upstreams tarballs. Each build script is supposed to replace config.guess and config.sub with freshly downloaded ones after untaring, but it looks like that didn't happen for some reason?

zeldin commented 5 months ago

Ok, I was able to reproduce the issue on my Mac Mini M1. The problem is this script which is using this ancient config.guess without replacing it.

Compare for example this line in a different script that does replace config.guess with an up-to-date version.

humbertodias commented 5 months ago

Ok, I was able to reproduce the issue on my Mac Mini M1. The problem is this script which is using this ancient config.guess without replacing it.

Compare for example this line in a different script that does replace config.guess with an up-to-date version.

Thanks @zeldin for tracking the problem: ps3toolchain > psl1ght > SDL2_PSL1GHT > SDL2_PSL1GHT_Libs I have fixed here and now it's working for arm64 as well.

@TheRouletteBoi please, try again

How to build arm platform image

DOCKER_DEFAULT_PLATFORM=linux/arm64 docker build . -t ps3dev 
DOCKER_DEFAULT_PLATFORM=linux/arm64 docker run -it -v `pwd`:/build -w /build ps3dev

TheRouletteBoi commented 5 months ago

Ok, I was able to reproduce the issue on my Mac Mini M1. The problem is this script which is using this ancient config.guess without replacing it. Compare for example this line in a different script that does replace config.guess with an up-to-date version.

Thanks @zeldin for tracking the problem: ps3toolchain > psl1ght > SDL2_PSL1GHT > SDL2_PSL1GHT_Libs I have fixed here and now it's working for arm64 as well.

@TheRouletteBoi please, try again

How to build arm platform image
DOCKER_DEFAULT_PLATFORM=linux/arm64 docker build . -t ps3dev 
DOCKER_DEFAULT_PLATFORM=linux/arm64 docker run -it -v `pwd`:/build -w /build ps3dev

the ps3toolchain now compiles for arm64 👍 thank you @humbertodias

Fewnity commented 4 months ago

GCC 14.1 is out, maybe can we expect a version bump? 👀

ps3dev / ps3toolchain