Open lucasAbadFr opened 7 months ago
Abstracting libc primitives: After looking at how the POSIX primitives where abstracted in core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/
to support Windows
. I think that we could do the same to support Zephyr
, with the final idea to have an implementation entirely relying on the os_
APIs.
Extending platform_api_extension.h
: The following API were added.
__wasi_errno_t
os_ioctl(os_file_handle *handle, int request, void *argp);
// Not sure for return type
__wasi_errno_t
os_poll(os_poll_file_handle *pfds, os_nfds_t nfs, int timeout);
Changing the sandboxed system primitives to use the new types and functions.
random.c
:
#elif defined(BH_PLATFORM_ZEPHYR)
static void
open_urandom(void)
{
// Not implemented
}
__wasi_errno_t
random_buf(void *buf, size_t len)
{
return __WASI_ENOSYS;
}
blocking.h
:
__wasi_errno_t
- os_poll(pollfd *pfds, os_nfds_t nfds, int timeout)
+ os_poll(os_poll_file_handle *pfds, os_nfds_t nfds, int timeout)
locking.h
: change every struct timespec
to struct os_timespec
posix.c
: change every struct pollfd
to struct os_poll_file_handle
Defining new types in platform_internal.h
: Based on the os_file_handle
type implementation we define the following types (and flags).
typedef struct zsock_pollfd os_poll_file_handle;
typedef unsigned int os_nfds_t;
// To use posix flags
#define POLLIN ZSOCK_POLLIN
#define POLLPRI ZSOCK_POLLPRI
#define POLLOUT ZSOCK_POLLOUT
#define POLLERR ZSOCK_POLLERR
#define POLLHUP ZSOCK_POLLHUP
#define POLLNVAL ZSOCK_POLLNVAL
#define FIONREAD ZFD_IOCTL_FIONREAD
typedef struct {
time_t tv_sec;
long tv_nsec;
} os_timespec;
#define CLOCK_REALTIME 1
#define CLOCK_MONOTONIC 4
typedef struct {
struct k_mutex mtx; // Mutex for exclusive access
struct k_sem sem; // Semaphore for shared access
int read_count; // Number of readers
} korp_rwlock;
Implementing the new functions in platform_socket.c
:
__wasi_errno_t
os_ioctl(os_file_handle *handle, int request, void *argp)
{
__wasi_errno_t wasi_errno = __WASI_ESUCCESS;
if(zsock_ioctl_wrapper(handle, request, argp) < 0){
wasi_errno = zephyr_to_wasi_errno(errno);
}
return wasi_errno;
}
__wasi_errno_t
os_poll(os_poll_file_handle *fds, os_nfds_t nfs, int timeout)
{
__wasi_errno_t wasi_errno = __WASI_ESUCCESS;
int rc = 0;
rc = zsock_poll(fds, nfs, timeout)
if(rc < 0){
wasi_errno = zephyr_to_wasi_errno(errno);
}
switch(rc){
case 0:
wasi_errno = __WASI_ETIMEOUT;
break;
case -1:
wasi_errno = zephyr_to_wasi_errno(errno);
break;
default:
break;
}
return wasi_errno;
}
compilation errors The compilation still fail with the same warnings as before. The unknown flags errors have disappeared. But the warning about the struct timespec
and struct pollfd
still persist.
locking.h
:
/home/user/wasm-micro-runtime/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/locking.h:200:12: error: variable 'ts' has initializer but incomplete type
200 | struct os_timespec ts = {
| ^~~~~~~~~~~
/home/user/wasm-micro-runtime/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/locking.h:201:10: error: 'struct os_timespec' has no member named 'tv_sec'
201 | .tv_sec = (time_t)(timeout / 1000000000),
| ^~~~~~
/home/user/wasm-micro-runtime/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/locking.h:201:19: warning: excess elements in struct initializer
201 | .tv_sec = (time_t)(timeout / 1000000000),
| ^
/home/user/wasm-micro-runtime/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/locking.h:201:19: note: (near initialization for 'ts')
posix.c
:
/home/user/wasm-micro-runtime/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/posix.c:2153:61: error: invalid application of 'sizeof' to incomplete type 'struct os_poll_file_handle'
2153 | wasm_runtime_malloc((uint32)(nsubscriptions * sizeof(*pfds)));
| ^
/home/user/wasm-micro-runtime/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/posix.c:2177:25: error: invalid use of undefined type 'struct os_poll_file_handle'
2177 | pfds[i] = (struct os_poll_file_handle){
| ^
/home/user/wasm-micro-runtime/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/posix.c:2178:26: error: 'struct os_poll_file_handle' has no member named 'fd'
2178 | .fd = fos[i]->file_handle,
| ^~
/home/user/wasm-micro-runtime/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/posix.c:2178:31: warning: excess elements in struct initializer
2178 | .fd = fos[i]->file_handle,
|
This look like the compiler is not able to find the definition of the new types.
platform_internal.h
in the locking.h
and posix.c
files to see if it solves the issue. But it didn't work.locking.h
and posix.c
files. But the issue still persist.zephyr_file.c
file (just returning __WASI_ENOSYS
for now), and changed shared_platform.cmake
. But the issue still persist.I'm still stuck on this issue. I will try to find a solution.
If anyone is familiar with the implemenation of the sanboxed system primitives
, or the how the abstract POSIX APIs are passed to sanboxed system primitives
.
Any help on this would be appreciated.
@lucasAbadFr could you upload the patch or create a PR and change it to draft or WIP, so that others can have a try according to your work?
@wenyongh thanks for your reply.
A new zephyr sample called simpe-http
was added to demonstrate socket usage on zephyr. Refer to the sample readme for build purpose.
I've just opened a draft and noticed that I didn't adhere to the code guideline or other guidelines.
These changes will cause most tests to fail.
Please note that the work wasn't intended to be shared in this state.
Hi @wenyongh,
I managed to make a simple wasi module work (by breaking a lot of things).
I would be interested to know if it work on someone else board.
I updated the readme.md
under the new sample to give a brief overview of what was done.
Also i'm open to any tips to compile a WASI module with libc (from wasi-sdk) linked.
@lucasAbadFr I am not sure who is using zephyr but I believe there are some developers requiring libc-wasi on zephyr. I found there are many modifications in wasm_runtime_common.c
and core/iwasm/libraries/libc-wasi
and wonder whether it is necessary? It would be better to add APIs and structures in zephyr platform and leave some APIs empty (or return false), and reduce the modification on core/iwasm and make the CIs run successfully.
Thanks for the feedback.
The CI fail due to the new struct and API defined in platform_api_extension.h
, to make it pass I could just typedef the new struct in platform_internal.h
, and declare the API in their respective folder.
As for the changes in core/iwasm/libraries/libc-wasi/sandboxed-system-primitives
, they are mostly here to abstract POSIX functions and use os_
abstraction. I think this is necessary to have at least a full abstraction.
The main changes are in core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/posix.c
where I had an offset due to struct size being different on both platform.
I'm curently investigating how to solve this issue and working to maintain compatibility with POSIX platform, which is the most used/maintained one.
OK, I read the code and added some comments, it should be OK to abstract some os_xxx
structures and functions. I think maybe you can enable the basic functionality and make CI run fine first, and then we can let others help review this PR and give suggestions, like how to abstract the APIs.
Hello, I'm currently experimenting with sockets in WAMR on Zephyr, and I'm encountering some challenges.
nucleo-h563zi
board, but I'm facing compilation issues.Compilation issues
Plain compilation
I compilated the
http_client
to a wasm module and like with thesimple
sample, I included it in a header file. But when I ran it on the board I had:WAMR_BUILD_LIBC_WASI flag
After looking at the
socket-api
sample, I found that I need to use theWAMR_BUILD_LIBC_WASI
flag, so I added this flag to theCMakeLists.txt
file but it didn't work.The output of the compilation is the same described in this issue My errors concist mainly of:
libc_errno.h
not found.nfds_t
unknown type.pthread_condattr_t
unknown type &CLOCK_MONOTONIC
undeclared.Try to resolve
I browsed the issues and found this Pull Request, but applying the fix lead to more errors and just one error was fixed.
In
core/shared/platform/zephyr/shared_platform.cmake
I added the following lines:The error
libc_errno.h
was fixed but the other errors still persist. I also get all the errors from the/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/posix.c
file, mostly linked to the File System API ( e.g errors onstruct pollfd
).Test on host
I reused the
socket-api
sample to compile and run thehttp_client
module and it worked on my host machine.Runtime configuration
Am I missing something ?
My runtime configuration is the following:
How to use the address pool ?
I would like to have some help with the
libc_wasi_init(wasm_module, argc, argv, &wasi_parse_ctx)
function because if I understood it's mandatory to run a network application on WAMR.I looked at the
product-mini/plateforms/posix/main.c
&product-mini/plateforms/common/libc_wasi.c
files but didn't fully understand the implementation because it is made to parse arguments from the command line.Meanwhile, I'm trying to run the
http_client
module on the board, so I don't have the possibility to pass arguments to the module.I came up with this configuration before
wasm_runtime_full_init(&init_args)
call inmain.c
.But I'm not sure if it's the right way to use the
libc_wasi_parse
function.Should I change build policy ?
I tried to adapt the
CMakeLists.txt
file from thesimple
sample. What I changed:WAMR_BUILD_LIBC_WASI
flag.wamr-sdk
as an external projecxt and compile thelibapp_framework.a
.http_client
module.I added a custom command that use a python script to generate the
http_client.h
module.Any help will be appreciated. Thanks.