PS3 Build

[humbertodias]

I've notice that ps3 build were made through SCons and python. https://github.com/kazzmir/paintown/releases/download/v3.6.0/paintown-ps3-3.6.0.pkg Should we keep that way or try something new?

if so, do you remember steps?

I was looking at ps3toolchain : https://github.com/ps3dev/ps3toolchain

[juvinious]

You would need to build from the 3.6 source. The instructions can be found under doc/ps3. If it works/builds still is a wild guess though.

Allot of the mugen stuff was very preliminary and experimental in that release.

[kazzmir]

We should use meson to build. I doubt the ps3 stuff will work anymore. Maybe we could test it in an emulator or something if there are any good ones.

[humbertodias]

We should use meson to build. I doubt the ps3 stuff will work anymore. Maybe we could test it in an emulator or something if there are any good ones.

I have tested the release 3.6.0 on RPCS3 and it works well.

[humbertodias]

You would need to build from the 3.6 source. The instructions can be found under doc/ps3. If it works/builds still is a wild guess though.

Allot of the mugen stuff was very preliminary and experimental in that release.

Makes sense. I'm gonna try to build the pkg using docker as a cross compiler such as you did previously with mingw

[kazzmir]

Thats cool that it works on RPCS3. The latest version of paintown now requires c++11 and threading support in gcc, so I'm not sure if the PS3 sdk PSL1ght supports that.

[humbertodias]

Thats cool that it works on RPCS3. The latest version of paintown now requires c++11 and threading support in gcc, so I'm not sure if the PS3 sdk PSL1ght supports that.

It's almost working https://github.com/kazzmir/paintown/actions/runs/7941175624/job/21683244586#step:3:1

I have created the container with embedded ps3toolchain and it comes with compiled sdl1.2. Now it's needed to add the sdl2 and other paintown dependencies. Teorically, after that we should to be ready to compile and package.

[juvinious]

Once it's building and you are able to generate a pkg we should migrate the environment over from Scons to Meson.

[humbertodias]

Once it's building and you are able to generate a pkg we should migrate the environment over from Scons to Meson.

Perfect. We already have a ps3 package from a simple c++ sdl2

./easy-compile-docker-ps3 && ./release/release-ps3

[juvinious]

Ok got it to work. Had to change the files since the ones you uploaded were uppercase. Also added an argument to allow for a different data directory.

Edit. Though, the full data directory is put into the pkg. Is there a reason we get a red box? Doesn't that mean it can't find data?

[humbertodias]

Ok got it to work. Had to change the files since the ones you uploaded were uppercase. Also added an argument to allow for a different data directory.

Edit. Though, the full data directory is put into the pkg. Is there a reason we get a red box? Doesn't that mean it can't find data?

Perfect! no problem. Actually, the red box it's an example that I´m using instead of paintown project. Because, I still didn't solve all dependencies on meson build. Meanwhile, with this sample app. I have validated the ci/cd

[juvinious]

Ah ok, now it makes sense.

[humbertodias]

Ah ok, now it makes sense.

The error occurred while attempting to compile yaml-cpp using psp-g++ Same for ps3-dev

How to reproduce

docker run --platform=linux/amd64 --rm -i -v `mktemp -d`:/source pspdev/pspdev:latest sh -s <<EOF
wget https://github.com/jbeder/yaml-cpp/archive/refs/tags/0.8.0.tar.gz
tar xvfz 0.8.0.tar.gz
cd yaml-cpp-0.8.0
env
mkdir -p build 
cd build
cmake -Wno-dev -DCMAKE_TOOLCHAIN_FILE=/usr/local/pspdev/psp/share/pspdev.cmake \
-DCMAKE_INSTALL_PREFIX=/usr/local/pspdev/psp -DBUILD_SHARED_LIBS=OFF ..
make 
make install
EOF

Error

- Build files have been written to: /yaml-cpp-0.8.0/build
[  2%] Building CXX object CMakeFiles/yaml-cpp.dir/src/contrib/graphbuilder.cpp.obj
cc1plus: error: cannot generate position-independent code for ‘-mabi=eabi’
make[2]: *** [CMakeFiles/yaml-cpp.dir/build.make:76: CMakeFiles/yaml-cpp.dir/src/contrib/graphbuilder.cpp.obj] Error 1
make[1]: *** [CMakeFiles/Makefile2:861: CMakeFiles/yaml-cpp.dir/all] Error 2

[juvinious]

Easy solution is to not include yaml. It's not required. Make sure -DHAVE_YAML is not set.

[kazzmir]

The error suggests a general issue with compiling for a specific abi though, should we remove that -meabi flag?

[humbertodias]

The error suggests a general issue with compiling for a specific abi though, should we remove that -meabi flag?

Yeah. Even removing, fail. weird.

[humbertodias]

Easy solution is to not include yaml. It's not required. Make sure -DHAVE_YAML is not set.

i'm gonna try

[humbertodias]

Easy solution is to not include yaml. It's not required. Make sure -DHAVE_YAML is not set.

I found the problem and submitted a fix https://github.com/jbeder/yaml-cpp/pull/1271

[juvinious]

Cool, hopefully they accept it but I bet they'll ask for something more generic (i.e. not ps3/psp) to check. But who knows. What I did was just not include yaml support for now for the dkp build by checking if it was downloaded or not:

if meson.is_cross_build()
  fs = import('fs')
  if fs.is_dir('.tmp/yaml-cpp')
    add_global_arguments('-DHAVE_YAML_CPP', language: 'cpp')
    subdir('.tmp/yaml-cpp')
  endif
  subdir('.tmp/SDL_gfx')
endif

[kazzmir]

We could apply a local patch to the yaml code, if upstream won't accept a change

[humbertodias]

We could apply a local patch to the yaml code, if upstream won't accept a change

Good idea! I have overpass the yaml-cpp problem. Now another with r-tech1/libgme How to reproduce

git clone https://github.com/kazzmir/paintown -b build-psp
cd paintown
./easy-compile-docker-psp

Error https://github.com/kazzmir/paintown/actions/runs/7994410565/job/21832385288?pr=93#step:3:10345

  User defined options
    Cross files: psp-cross.txt

Found ninja-1.11.1 at /usr/bin/ninja
(cd build-psp; meson configure -Dbuild_tests=false)
meson compile  -C build-psp
ninja: Entering directory `/paintown-bin/build-psp'
[1/352] Compiling C++ object src/r-tech1/libs/gme/libgme.a.p/Ay_Apu.cpp.o
[2/352] Compiling C++ object src/r-tech1/libs/gme/libgme.a.p/Data_Reader.cpp.o
[3/352] Compiling C++ object src/r-tech1/libs/gme/libgme.a.p/Ay_Cpu.cpp.o
FAILED: src/r-tech1/libs/gme/libgme.a.p/Ay_Cpu.cpp.o 
psp-g++ -Isrc/r-tech1/libs/gme/libgme.a.p -Isrc/r-tech1/libs/gme -I../src/r-tech1/libs/gme -Isrc -I../src -I/psp/include -I/pspdev/psp/include -I/pspdev/psp/sdk/include -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=c++11 -O0 -g -DUSE_SDL2=1 '-DDATA_PATH="data"' -g3 -Wextra -Wno-unused-variable -Wno-unused-function -Wno-unused-parameter -DHAVE_OGG=1 -DHAVE_MP3_MPG123=1 -DPSP -D_PSP_FW_VERSION=500 -w -MD -MQ src/r-tech1/libs/gme/libgme.a.p/Ay_Cpu.cpp.o -MF src/r-tech1/libs/gme/libgme.a.p/Ay_Cpu.cpp.o.d -o src/r-tech1/libs/gme/libgme.a.p/Ay_Cpu.cpp.o -c ../src/r-tech1/libs/gme/Ay_Cpu.cpp
../src/r-tech1/libs/gme/Ay_Cpu.cpp:104:10: error: #error "Byte order of CPU must be known"
  104 |         #error "Byte order of CPU must be known"
      |          ^~~~~
../src/r-tech1/libs/gme/Ay_Cpu.cpp: In member function ‘bool Ay_Cpu::run(cpu_time_t)’:
../src/r-tech1/libs/gme/Ay_Cpu.cpp:413:24: error: ‘R8’ was not declared in this scope; did you mean ‘RL’?
  413 |                 data = R8( opcode & 7, 0 );
      |                        ^~
      |                        RL
[4/352] Compiling C++ object src/r-tech1/libs/gme/libgme.a.p/Classic_Emu.cpp.o

[kazzmir]

https://en.wikipedia.org/wiki/PlayStation_Portable_hardware

Looks like the PSP is little endian. We just have to define BLARGG_LITTLE_ENDIAN in gme/blargg_endian.h for the PSP

[kazzmir]

Well this github issue is supposed to be about the ps3 build, but now we are discussing psp. Maybe we should move psp discussion to its own issue?

[humbertodias]

Well this github issue is supposed to be about the ps3 build, but now we are discussing psp. Maybe we should move psp discussion to its own issue?

Oh.. that's true. I'm gonna create another issue for psp

[humbertodias]

https://en.wikipedia.org/wiki/PlayStation_Portable_hardware

Looks like the PSP is little endian. We just have to define BLARGG_LITTLE_ENDIAN in gme/blargg_endian.h for the PSP

Fixed https://github.com/kazzmir/paintown/pull/93/commits/e0f5bcbe4db2aad14c3baa5f9e832c7561105999

[humbertodias]

Cool, hopefully they accept it but I bet they'll ask for something more generic (i.e. not ps3/psp) to check. But who knows. What I did was just not include yaml support for now for the dkp build by checking if it was downloaded or not:

if meson.is_cross_build()
  fs = import('fs')
  if fs.is_dir('.tmp/yaml-cpp')
    add_global_arguments('-DHAVE_YAML_CPP', language: 'cpp')
    subdir('.tmp/yaml-cpp')
  endif
  subdir('.tmp/SDL_gfx')
endif

Merged https://github.com/jbeder/yaml-cpp/pull/1271

[humbertodias]

Unfortunately the container PSL1GHT doesn't include SDL2_image|ttf|mixer|glx and I'm gonna add on ps3libraries

[humbertodias]

Unfortunately the container PSL1GHT doesn't include SDL2_image|ttf|mixer|glx and I'm gonna add on ps3libraries

Image ps3dev-sdl2 created

Content:

SDL_PSL1GHT + SDL_PSL1GHT_Libs

• sdl - 1.3.0
• sdl_mixer - 1.2.11
• sdl_image - 1.2.10
• sdl_ttf - 2.0.10
• sdl_net - 1.2.7
• sdl_gfx - 2.0.27

SDL2_PSL1GHT + SDL2_PSL1GHT_Libs

• sdl2 - 2.0.13
• sdl2_mixer - 2.8.0
• sdl2_image - 2.8.2
• sdl2_ttf - 2.22.0
• sdl2_gfx - 2.22.0

[humbertodias]

Are we going to use pthread instead of std:thread or the same approach used for file-system?

[143/361] Compiling C++ object src/r-tech1/libr-tech1.a.p/graphics_bitmap.cpp.o
FAILED: src/r-tech1/libr-tech1.a.p/graphics_bitmap.cpp.o 
ppu-g++ -Isrc/r-tech1/libr-tech1.a.p -Isrc/r-tech1 -I../paintown-bin/src/r-tech1 -Isrc -I../paintown-bin/src -I/usr/local/ps3dev/portlibs/ppu/include/SDL2 -I/usr/local/ps3dev/portlibs/ppu/include -I/usr/local/include -I/usr/local/ps3dev/ppu/include/ -I/usr/local/ps3dev/portlibs/ppu/include/ -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=c++11 -O0 -g -DUSE_SDL2=1 -g3 -Wextra -Wno-unused-variable -Wno-unused-function -Wno-unused-parameter -DCROSS_BUILD -DHAVE_OGG=1 -mhard-float -fmodulo-sched -ffunction-sections -fdata-sections -maltivec -mminimal-toc -DPS3 -DHAVE_YAML_CPP -mcpu=cell -fno-builtin -nostdlib '-DDATA_PATH="/dev_hdd0/game/paintown/data"' -fPIC -MD -MQ src/r-tech1/libr-tech1.a.p/graphics_bitmap.cpp.o -MF src/r-tech1/libr-tech1.a.p/graphics_bitmap.cpp.o.d -o src/r-tech1/libr-tech1.a.p/graphics_bitmap.cpp.o -c ../paintown-bin/src/r-tech1/graphics/bitmap.cpp
In file included from ../paintown-bin/src/r-tech1/file-system.h:6:0,
                 from ../paintown-bin/src/r-tech1/funcs.h:10,
                 from ../paintown-bin/src/r-tech1/graphics/bitmap.cpp:1:
../paintown-bin/src/r-tech1/thread.h:180:10: error: 'thread' in namespace 'std' does not name a type
     std::thread thread;
          ^~~~~~
../paintown-bin/src/r-tech1/thread.h:296:10: error: 'thread' in namespace 'std' does not name a type
     std::thread thread;
          ^~~~~~
../paintown-bin/src/r-tech1/thread.h: In destructor 'virtual Util::Future<X>::~Future()':
../paintown-bin/src/r-tech1/thread.h:219:13: error: 'thread' was not declared in this scope

https://github.com/kazzmir/paintown/actions/runs/8146492951/job/22265104133#step:3:404

[kazzmir]

it would be nice if we could continue to use std::thread, and have std::thread use some native thread implementation, or perhaps use sdl2's threading system. if that doesn't work then maybe I will write some wrapper around std::thread that can use whatever the native threading system is for each port

[kazzmir]

what version of g++ is ppu-g++ based off of? if ppu-g++ is something recent, like gcc 11 then I would expect std::thread to be there. if not, maybe ppu-g++ can be updated?

[humbertodias]

what version of g++ is ppu-g++ based off of? if ppu-g++ is something recent, like gcc 11 then I would expect std::thread to be there. if not, maybe ppu-g++ can be updated?

7.2.0

DOCKER_DEFAULT_PLATFORM=linux/amd64 docker run -it hldtux/ps3dev-sdl2 bash -c "ppu-gcc --version"
ppu-gcc (GCC) 7.2.0

[humbertodias]

what version of g++ is ppu-g++ based off of? if ppu-g++ is something recent, like gcc 11 then I would expect std::thread to be there. if not, maybe ppu-g++ can be updated?

We are using the toolchain https://github.com/ps3dev/ps3toolchain I found out a newer version - 13.2.0-1 https://aur.archlinux.org/packages/ps3-ppu-gcc

it's adapted for archlinux but, they have left the patch

I'm gonna build gcc 13.2 with that patch and later on try to compile paintown with std:thread

[humbertodias]

PR - Upgrading version - https://github.com/ps3dev/ps3toolchain/pull/121

• GNU Binutils 2.42
• ppu-gcc (GCC) 13.2.0
• spu-gcc (GCC) 9.5.0

Now we can retry to compile paintown with c11 feature std::thread

[humbertodias]

Success! After upgrading ppu-gcc to 13 we have support to std:thread

[353/361] Compiling C++ object src/openbor/libopenbor.a.p/display-character.cpp.o
[354/361] Compiling C++ object src/openbor/libopenbor.a.p/pack-reader.cpp.o
[355/361] Compiling C++ object paintown.p/src_globals.cpp.o
[356/361] Compiling C++ object paintown.p/src_factory_collector.cpp.o
[357/361] Compiling C++ object paintown.p/src_factory_font_render.cpp.o
[358/361] Compiling C++ object src/openbor/libopenbor.a.p/mod.cpp.o
[359/361] Linking static target src/openbor/libopenbor.a
[360/361] Compiling C++ object paintown.p/src_main-menu.cpp.o
[361/361] Linking target paintown

Now it's remaining only runtime issues

[kazzmir]

Awesome! How far does the game get at runtime? Does the menu show up?

[humbertodias]

I have anticipated myself. Even with c11 enabled the use of thread and lambda are not compiling. Here for example https://github.com/kazzmir/paintown/actions/runs/8181132738/job/22370408154?pr=92#step:3:318

../paintown-bin/src/r-tech1/sound/music.cpp: In constructor 'Music::Music(bool)':
../paintown-bin/src/r-tech1/sound/music.cpp:57:50: error: no matching function for call to 'std::thread::thread(void (&)(Music*), Music*)'
   57 |         musicThread = std::thread(playMusic, this);

Well, at least now thread are detected on std namespace Any thoughts on what it might be?

[humbertodias]

I think i understood what is going on. Let me try to explain and if you are gonna say if it makes sense or not.

Paintown uses std::thread with c++11 that is Object-Oriented and pthread-emb library inside the container is compiled only in c as procedural at pthread-emb-ps3

I think the problem is that. pthread-emb-ps3 should be Object-Oriented as well to match and also be compiled with g++ at 039-pthread-emb.sh

Another point is that ppu-g++ is compiled with --enable-threads at 002-gcc-newlib-PPU.sh

Testing inside the container

DOCKER_DEFAULT_PLATFORM=linux/amd64 docker run -it hldtux/ps3dev-sdl2 bash

procedural.c

#include <stdio.h>
#include <pthread.h>
#include <unistd.h> // For sleep() function

void* thread_function(void* arg) {
    int thread_num = *((int*)arg);
    printf("Thread %d is running\n", thread_num);
    sleep(1); // Simulate some work
    printf("Thread %d finished\n", thread_num);
    return NULL;
}

int main() {
    pthread_t thread1, thread2;
    int thread1_num = 1, thread2_num = 2;

    // Create threads
    pthread_create(&thread1, NULL, thread_function, (void*)&thread1_num);
    pthread_create(&thread2, NULL, thread_function, (void*)&thread2_num);

    // Wait for threads to finish
    pthread_join(thread1, NULL);
    pthread_join(thread2, NULL);

    printf("Both threads have finished\n");

    return 0;
}

ppu-gcc -std=c11 procedural.c -o procedural `pkg-config --libs --cflags pthread`

file procedural
procedural: ELF 64-bit MSB executable, 64-bit PowerPC or cisco 7500, Power ELF V1 ABI, version 1 (SYSV), statically linked, with debug_info, not stripped

oo.cpp

#include <iostream>
#include <thread>

// Function to be executed by the thread
void threadFunction(int threadID) {
    std::cout << "Thread " << threadID << " is executing" << std::endl;
}

int main() {
    // Creating a thread and passing a function to execute
    std::thread t1(threadFunction, 1);

    // Joining the thread to the main thread
    t1.join();

    // Output a message from the main thread
    std::cout << "Main thread is executing" << std::endl;

    return 0;
}

ppu-g++ -std=c++11 oo.cpp -o oo `pkg-config --libs --cflags pthread`

Result

cc1plus: warning: command-line option '-std=c11' is valid for C/ObjC but not for C++
oo.cpp: In function 'int main()':
oo.cpp:11:37: error: no matching function for call to 'std::thread::thread(void (&)(int), int)'
   11 |     std::thread t1(threadFunction, 1);
      |                                     ^
In file included from /usr/local/ps3dev/ppu/powerpc64-ps3-elf/include/c++/13.2.0/thread:45,
                 from oo.cpp:2:
/usr/local/ps3dev/ppu/powerpc64-ps3-elf/include/c++/13.2.0/bits/std_thread.h:178:5: note: candidate: 'std::thread::thread(std::thread&&)'
  178 |     thread(thread&& __t) noexcept
      |     ^~~~~~
/usr/local/ps3dev/ppu/powerpc64-ps3-elf/include/c++/13.2.0/bits/std_thread.h:178:5: note:   candidate expects 1 argument, 2 provided
/usr/local/ps3dev/ppu/powerpc64-ps3-elf/include/c++/13.2.0/bits/std_thread.h:132:5: note: candidate: 'std::thread::thread()'
  132 |     thread() noexcept = default;
      |     ^~~~~~
/usr/local/ps3dev/ppu/powerpc64-ps3-elf/include/c++/13.2.0/bits/std_thread.h:132:5: note:   candidate expects 0 arguments, 2 provided

Using -pthead

ppu-g++ -std=c++11 oo.cpp -o oo -pthread
cc1plus: warning: command-line option '-std=c11' is valid for C/ObjC but not for C++

Using -fpthread

ppu-g++ -std=c++11 oo.cpp -o oo -fpthread
ppu-g++: error: unrecognized command-line option '-pthread'; did you mean '-fpthread'?
cc1plus: warning: command-line option '-std=c11' is valid for C/ObjC but not for C++
cc1plus: warning: command-line option '-fpthread' is valid for Modula-2 but not for C++
oo.cpp: In function 'int main()':
oo.cpp:11:37: error: no matching function for call to 'std::thread::thread(void (&)(int), int)'
   11 |     std::thread t1(threadFunction, 1);
      |                                     ^
In file included from /usr/local/ps3dev/ppu/powerpc64-ps3-elf/include/c++/13.2.0/thread:45,
                 from oo.cpp:2:
/usr/local/ps3dev/ppu/powerpc64-ps3-elf/include/c++/13.2.0/bits/std_thread.h:178:5: note: candidate: 'std::thread::thread(std::thread&&)'
  178 |     thread(thread&& __t) noexcept
      |     ^~~~~~
/usr/local/ps3dev/ppu/powerpc64-ps3-elf/include/c++/13.2.0/bits/std_thread.h:178:5: note:   candidate expects 1 argument, 2 provided
/usr/local/ps3dev/ppu/powerpc64-ps3-elf/include/c++/13.2.0/bits/std_thread.h:132:5: note: candidate: 'std::thread::thread()'
  132 |     thread() noexcept = default;
      |     ^~~~~~
/usr/local/ps3dev/ppu/powerpc64-ps3-elf/include/c++/13.2.0/bits/std_thread.h:132:5: note:   candidate expects 0 arguments, 2 provided

[kazzmir]

If you are compiling c++ code try adding -std=c++11 instead of -std=c11. But essentially std::thread should be implemented on top of pthread.

[humbertodias]

If you are compiling c++ code try adding -std=c++11 instead of -std=c11. But essentially std::thread should be implemented on top of pthread.

I have solved using Util::Thread instead. Now we have the compilation working again. But on runtime it freezes and got stuck on a black screen. I'm almost sure that something is missing on ps3dev and I'm still investigating