About adding a pthread library

[bucanero]

While trying to update/build some libraries like cURL, I saw that we don't have a pthread library and we need to rely on a --disable-thread option (if available).

Looking around, I saw that in the PSP/Vita homebrew sdk they built a pthread library, based on pthread-win32: https://github.com/vitasdk/pthread-embedded

Could be possible to port this library to the PS3? from what I could check on the source code, they're building wrappers around the Mutex, Semaphores, and Threads from the SCE sdk, to offer a pthread interface. In Psl1ght we do have access to the mutex, sems and threads too, so I think this would be a possibility, right?

If anyone with proper knowledge of pthread, threads, mutex, and semaphores could look into it, I'll be happy to help. 😄 My knowledge is limited in this topic (I haven't done any mutex/sem/thread stuff since college), and I don't want to just replace code lines blind, but I feel it could be a nice addition.

From my quick check on /platform/psp/psp_osal.c, we would need to change with proper PS3 calls:

sceKernelGetThreadId
sceKernelCreateSema
sceKernelCreateThread
sceKernelStartThread
sceKernelDeleteSema
sceKernelDeleteThread
sceKernelExitDeleteThread
sceKernelExitThread
SceKernelThreadRunStatus
sceKernelReferThreadRunStatus
sceKernelReferSemaStatus
sceKernelDelayThread
sceKernelReferThreadStatus
sceKernelChangeThreadPriority
sceKernelSignalSema
sceKernelWaitSema

[bucanero]

Actually, digging a bit around, I found that @kakaroto already worked on pthread-embedded for PSL1GHT: https://github.com/kakaroto/pthread-embeded

does anyone knows if this psl1ght library works, or if it was only a PoC?

In any case, I'll try to build it and see how it goes.

[kakaroto]

pretty sure it was complete and worked, but it's been so long, and don't think it was heavily tested either.

[bucanero]

thanks @kakaroto for the information! 😄

I'll share my feedback if I can run some tests with the pthread-emb psl1ght library 👍

Edit: I confirm that the library compiles Ok with the latest PSL1GHT from master branch. The only change needed was to replace SYS_MUTEX_ATTR_PSHARED to SYS_MUTEX_ATTR_NOT_PSHARED

[bucanero]

well, I tried running a basic test app with the pthread library but it didn't work, it freezes on the pthread_create() call.

About the build, I get some warnings but I'm not sure if it can be related:

../../pte_throw.c: In function 'pte_throw':
../../pte_throw.c:85:22: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
           exitCode = (unsigned) PTHREAD_CANCELED;
                      ^
../../pte_throw.c:88:22: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
           exitCode = (unsigned) sp->exitStatus;;
                      ^
../../pte_throw.c:80:16: warning: variable 'exitCode' set but not used [-Wunused-but-set-variable]
       unsigned exitCode = 0;
                ^~~~~~~~
../../pte_throw.c: In function 'pte_get_exception_services_code':
../../pte_throw.c:141:10: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
   return (unsigned int) NULL;
          ^
cleanup.c
pthread_once.c
../../pthread_once.c: In function 'pte_once_init_routine_cleanup':
../../pthread_once.c:60:27: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
       pte_osSemaphorePost((pte_osSemaphoreHandle) once_control->semaphore, 1);
                           ^
../../pthread_once.c: In function 'pthread_once':
../../pthread_once.c:139:35: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
               pte_osSemaphorePost((pte_osSemaphoreHandle) once_control->semaphore,once_control->numSemaphoreUsers);
                                   ^
../../pthread_once.c:164:35: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
               pte_osSemaphorePend((pte_osSemaphoreHandle) once_control->semaphore,NULL);
                                   ^
pthread_num_processors_np.c
pte_getprocessors.c
pte_spinlock_check_need_init.c
pthread_timechange_handler_np.c
../../pthread_timechange_handler_np.c: In function 'pthread_timechange_handler_np':
../../pthread_timechange_handler_np.c:113:10: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
   return (void *) (result != 0 ? EAGAIN : 0);
          ^
psl1ght_osal.c: In function 'pte_osThreadCreate':
psl1ght_osal.c:239:59: warning: 'snprintf' output truncated before the last format character [-Wformat-truncation=]
   snprintf(sem_attr.name, sizeof(sem_attr.name), "cncl%04d", threadNum);
                                                           ^
psl1ght_osal.c:239:3: note: 'snprintf' output between 9 and 16 bytes into a destination of size 8
   snprintf(sem_attr.name, sizeof(sem_attr.name), "cncl%04d", threadNum);
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This is the basic test sample I tried:

#include <pthread.h>
#include <stdio.h>

/* this function is run by the second thread */
void *inc_x(void *x_void_ptr)
{
    /* increment x to 100 */
    int *x_ptr = (int *)x_void_ptr;
    while(++(*x_ptr) < 100);

    printf("x increment finished\n");

    /* the function must return something - NULL will do */
    return NULL;
}

int main()
{
    int x = 0, y = 0;

    /* show the initial values of x and y */
    printf("x: %d, y: %d\n", x, y);

    /* this variable is our reference to the second thread */
    pthread_t inc_x_thread;

    printf("creating thread...\n");

    /* create a second thread which executes inc_x(&x) */
    if(pthread_create(&inc_x_thread, NULL, inc_x, &x)) {    
        printf("Error creating thread\n");
        return 1;
    }

    printf("continue...\n");

    /* increment y to 100 in the first thread */
    while(++y < 100);

    printf("y increment finished\n");

    /* wait for the second thread to finish */
    if(pthread_join(inc_x_thread, NULL)) {
        printf("Error joining thread\n");
        return 2;
    }

    /* show the results - x is now 100 thanks to the second thread */
    printf("x: %d, y: %d\n", x, y);

    return 0;

}

[OsirizX]

I've created a branch based off of kakaroto's that should be compatible with the current psl1ght sdk. I've also removed the usage of sysDbgGetPPUThreadName as that requires debug flag to be enabled (DEX kernel). Instead it uses threadInfo struct which is a solution used in the ps2 pthread implementation which should work for most cases. Please note the commit for fixing the atomic macros will need to be pulled first before compiling against psl1ght as those have been added to the branch as well.

[bucanero]

hi @OsirizX , this is awesome! I'll try to build your branch and give it a try when I have a chance 👍

so cool to see you around the ps3 again 😃

[bucanero]

@OsirizX , I tested your updated pthread library and my basic test app worked just fine 💪

x: 0, y: 0
creating thread...
continue...
x increment finished
y increment finished
x: 100, y: 100

btw, I sent a small PR to your fork, with some minor adjustments to Makefile, and I also added the ps3-sample just in case.

On a related topic, should we create PRs to @kakaroto original repo, or you prefer to use your fork for now?

[OsirizX]

We can stick to my fork for now. I may make more changes to it and create a PR at a later time

[bucanero]

We can stick to my fork for now. I may make more changes to it and create a PR at a later time

ok, sounds good. 👍 If I have a chance I'll add a simple Readme with instructions on how to build/install the library and run the sample using PSL1GHT.

[miigotu]

Merged the sysAtomicSwap fix, seems someone's typo tool changed that at some point in time (maybe mine? Lol)

[miigotu]

Merged the sysAtomicSwap fix, seems someone's typo tool changed that at some point in time (maybe mine? Lol)

Ok nvm, @shagkur just missed them in a refactor, can't believe we haven't seen that since 2011 😂

[humbertodias]

pthread-emb added here https://github.com/ps3dev/ps3libraries/pull/64