Implement POSIX compatibile layer

[jserv]

Codezero microkernel implements a primitive POSIX compatibility layer: posix, and we had an incomplete porting for its userspace library in branch userlib.

We expect to implement primitive POSIX compatibility layer as Codezero does.

[rampant1018]

In userlib, we have port some userspace library from l4lib. But now we should port posix library from libposix, right?

I don't know where to start the task exactly. Replace user/lib/l4/syscall with the functions in libposix?

[jserv]

At present, we only have to implement a minimal set of libposix based on the existing directory user/lib . Codezero's libposix is just a reference.

OKL4 implements POSIX functionality through Iguana which provides additional operating system services on top of the sparse L4 kernel.

We don't need the complex layers like OKL4, but only the proof-of-concept work for the further compatibility verification purpose.

[rampant1018]

I just try to add a posix layer in user/lib. But I'm not sure what I did is correct.

[jserv]

As we discussed last week, we would like to implement partial APIs defined in PSE51. PSE51 or POSIX Minimal has 287 APIs and is aimed at small embedded system, with no MMU, no disk, no terminal.

For this issue, we expect to implement some pthread functions such as:

• pthread_mutex_destroy
• pthread_mutex_trylock
• pthread_mutex_lock
• pthread_mutex_timedlock
• pthread_mutex_unlock
• pthread_create
• pthread_detach
• pthread_exit
• pthread_join

Please note that we don't want to implement full PSE51 at the moment. Instead, we just want to have a primitive and incomplete POSIX compatibility layer first, then we can verify F9 internals by several conformance suites later.

Reference:

• POSIX for Embedded RTOS Applications

[rampant1018]

By now, we do not have a method to transmit our global data. So we have to design the method first. In L4, there is a syscall L4_KernelInterface which can be retrieved a global structure by every user thread. Maybe we could pass data over kip. I will try this way.

[jserv]

seL4 is released, and it utilizes musl libc to provide POSIX thread routines. We can follow the steps how seL4 implements.

[rampant1018]

I encounter a problem. How can I get the current running user's information after entering pthread_* functions. For example, if current running user is pingpong and we need pthread_create to create ipc testing thread. After calling pthread_create, how to figure out the current running user if pingpong. Without the user's information I can not identify which free_mem I can use in pthread_create.

I have a idea now. Maybe we could save the current user's information in KIP. When processing context switch, replace KIP's user field with current user. But it will increase overhead in context switch and cause complex architecture in context switch.

In short, I think we need a mechanism to identify current user. Without that, we can not determine which user should I serve in the posix library.

Have any suggestion?

[jserv]

To @georgekang, can you explain how Fiasco/L4 + L4Re maniplates threading model?

[rampant1018]

@jserv, please check https://github.com/rampant1018/f9-kernel/commit/8423fc58627d15117cbf442260a70dc6dcfae41f.

[arcbbb]

@rampant1018, I have one line patch to solve your POSIX thread exit problem. Please check https://github.com/arcbbb/f9-kernel/commit/53c1469b855ce88fc90acdbe0df0de2fccc33b43

And the other problem is that task2 is removed while being kept in the ktimer event queue, but ipc_timeout() is not able to be aware of this.

uint32_t ipc_timeout(void *data)                                                
{                                                                               
        ktimer_event_t *event = (ktimer_event_t *) data;                        
        tcb_t* from_thr = (tcb_t*) event->data;                                 
        from_thr->state = T_RUNNABLE;                                           
        sched_slot_dispatch(SSI_NORMAL_THREAD, from_thr);                       
        return 0;                                                               
}

So after sched_slot_dispatch(SSI_NORMAL_THREAD, from_thr);, task2 is dispatched to the scheduling slot again. How to deallocate resources like ktimer from removed threads is a pending issue.

[rampant1018]

@arcbbb, thanks!

I have another question about pthread internal architecture. For now, I tried to maintain a tree structure in pthread implementation which contained parent, child, sibling relation. However, there is a same structure in kernel(thread list). Is it a good idea to keep them individual?

[arcbbb]

@rampant1018 , actually I have another idea to implement pthread. That is, we implement posix thread purely in user-space, and kernel should not be aware of this. it means we will have a pthread scheduler in userspace using ktimer. then the kernel can have the minimum functionality.

[rampant1018]

It sounds interesting! Do you mean that manage pthread in userspace instead of using L4_ThreadControl()? I agree with you that idea will keep minimum functionality in kernel. However, if we do so, we need to handle context switch, interrupt and preemption, right? In other words, the complexity will raise.

By the way, I had the impression that scheduler design in sche.c has reserved a field for custom scheduler.