hp77 | Add Option for build without libbpf

hp77-creator commented 1 month ago

Description

Adding an option to allow bpftime to built without libbpf

Fixes #175

Type of change

[x] New feature (non-breaking change which adds functionality)
[x] This change requires a documentation update

How Has This Been Tested?

[ ] Running bpftime on MacOS

Test Configuration:

Firmware version:
Hardware:
Toolchain:
SDK:

Checklist

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules
[x] I have checked my code and corrected any misspellings

hp77-creator commented 1 month ago

TODO: had to also change path for BOOST_ROOT to opt/homebrew/include to fix boost import errors. Will have to mention same in the documentation.

hp77-creator commented 1 month ago

@yunwei37 @Officeyutong right now, I am facing issue in bpftime_shm.hpp It was giving error on sys/epoll.h not being present on system, so I added a macro-processor check but after that there are many functions which are giving error. What to do for that? Should I also make the dependent functions enclosed in a preprocessor check?

yunwei37 commented 1 month ago

Could you please provide some detailed error information?

Officeyutong commented 1 month ago

@yunwei37 @Officeyutong right now, I am facing issue in bpftime_shm.hpp It was giving error on sys/epoll.h not being present on system, so I added a macro-processor check but after that there are many functions which are giving error. What to do for that? Should I also make the dependent functions enclosed in a preprocessor check?

epoll is not supported on platforms except Linux. But in bpftime, we should have only used some types and structs defined in this header. You may find out these types and structs, define them in the code base of bpftime

hp77-creator commented 1 month ago

getting these now: @Officeyutong @yunwei37

bpftime-hp/runtime/src/handler/map_handler.hpp:37:11: error: unknown type name 'pthread_spinlock_t'; did you mean 'pthread_rwlock_t'?
        volatile pthread_spinlock_t &spinlock;
                 ^~~~~~~~~~~~~~~~~~

should i replace spinlock with rwlock? for apple systems?

Officeyutong commented 1 month ago

getting these now: @Officeyutong @yunwei37
bpftime-hp/runtime/src/handler/map_handler.hpp:37:11: error: unknown type name 'pthread_spinlock_t'; did you mean 'pthread_rwlock_t'?
        volatile pthread_spinlock_t &spinlock;
                 ^~~~~~~~~~~~~~~~~~
should i replace spinlock with rwlock? for apple systems?

You may write a spinlock. It's very simple. Don't use mutex, they are very slow

hp77-creator commented 1 month ago

@Officeyutong @yunwei37 I am getting following errors


/Users/hp77/repos/bpftime-hp/runtime/src/bpftime_shm.cpp:366:10: error: use of undeclared identifier 'EPOLLIN'
                                                                        EPOLLIN,
                                                                        ^
/Users/hp77/repos/bpftime-hp/runtime/src/bpftime_shm.cpp:382:10: error: use of undeclared identifier 'EPOLLIN'
                                                                        EPOLLIN,
                                                                        ^
/Users/hp77/repos/bpftime-hp/runtime/src/bpftime_shm.cpp:400:18: error: use of undeclared identifier 'sigtimedwait'
                        if (int sig = sigtimedwait(&to_block, &sig_info, &ts);

It is coming from sys/epoll , I looked for alternatives, kevent is one, should we go for that for BSD specific kernels?

hp77-creator commented 1 month ago

@Officeyutong @yunwei37 Also getting error on

error: use of undeclared identifier 'CLOCK_MONOTONIC_COARSE'; did you mean '_CLOCK_MONOTONIC_RAW'?
        clock_gettime(CLOCK_MONOTONIC_COARSE, &spec);
                      ^~~~~~~~~~~~~~~~~~~~~~

will try to refer this: https://stackoverflow.com/questions/41509505/clock-gettime-on-macos

hp77-creator commented 1 month ago

Also getting this:

error: use of undeclared identifier 'gettid'
                tid = gettid();

hp77-creator commented 1 month ago

also getting this:

runtime/src/bpf_helper.cpp:330:14: error: use of undeclared identifier 'bpf_test_run_opts'
        LIBBPF_OPTS(bpf_test_run_opts, run_opts, .ctx_in = context,

do we need to include LIBBPF_OPTS, it is coming from bpftime_tail_call what should I do with it?

Officeyutong commented 1 month ago

@Officeyutong @yunwei37 I am getting following errors


/Users/hp77/repos/bpftime-hp/runtime/src/bpftime_shm.cpp:366:10: error: use of undeclared identifier 'EPOLLIN'
                                                                        EPOLLIN,
                                                                        ^
/Users/hp77/repos/bpftime-hp/runtime/src/bpftime_shm.cpp:382:10: error: use of undeclared identifier 'EPOLLIN'
                                                                        EPOLLIN,
                                                                        ^
/Users/hp77/repos/bpftime-hp/runtime/src/bpftime_shm.cpp:400:18: error: use of undeclared identifier 'sigtimedwait'
                        if (int sig = sigtimedwait(&to_block, &sig_info, &ts);

It is coming from sys/epoll , I looked for alternatives, kevent is one, should we go for that for BSD specific kernels?

EPOLLIN is a constant defined in epoll headers. We didn't use epoll syscalls in bpftime, but just used constants, types, structs from epoll
sigtimedwait is a POSIX signal API. Does darwin support it?

Officeyutong commented 1 month ago

Also getting this:

error: use of undeclared identifier 'gettid'
                tid = gettid();

gettid returns the thread id of the currently executing thread. It's a linux specified syscall but I think there should be alternatives on darwin

Officeyutong commented 1 month ago

also getting this:
runtime/src/bpf_helper.cpp:330:14: error: use of undeclared identifier 'bpf_test_run_opts'
        LIBBPF_OPTS(bpf_test_run_opts, run_opts, .ctx_in = context,
do we need to include LIBBPF_OPTS, it is coming from bpftime_tail_call what should I do with it?

You may disable this helper function on darwin. It relies kernel eBPF to work.

hp77-creator commented 4 weeks ago

EPOLLIN is a constant defined in epoll headers. We didn't use epoll syscalls in bpftime, but just used constants, types, structs from epoll

I have defined the value in a separate header file for this

sigtimedwait is a POSIX signal API. Does darwin support it?

It doesn't, have created a custom implementation and added in a header file.

gettid returns the thread id of the currently executing thread. It's a linux specified syscall but I think there should be alternatives on darwin

Yea, I found some alternative in pthread_threadid_np(NULL, &tid); and using that.

You may disable this helper function on darwin. It relies kernel eBPF to work.

I have disabled this function

I am getting a lot of error in prog_array.cpp now, a lot of linux/libc related functions and constants are used.

hp77-creator commented 4 weeks ago

also what to do of bpf_object.cpp ?

Officeyutong commented 4 weeks ago

EPOLLIN is a constant defined in epoll headers. We didn't use epoll syscalls in bpftime, but just used constants, types, structs from epoll

I have defined the value in a separate header file for this

sigtimedwait is a POSIX signal API. Does darwin support it?

It doesn't, have created a custom implementation and added in a header file.

gettid returns the thread id of the currently executing thread. It's a linux specified syscall but I think there should be alternatives on darwin

Yea, I found some alternative in pthread_threadid_np(NULL, &tid); and using that.

You may disable this helper function on darwin. It relies kernel eBPF to work.

I have disabled this function

I am getting a lot of error in prog_array.cpp now, a lot of linux/libc related functions and constants are used.

prog_array.cpp is only supported on Linux (it relies on kernel eBPF). you may disable this map implementation on other platforms

hp77-creator commented 4 weeks ago

I did that, Now for some reason boost library is not recognized getting this error now:

fatal error: 'boost/interprocess/interprocess_fwd.hpp' file not found
#include <boost/interprocess/interprocess_fwd.hpp>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make[3]: *** [tools/bpftimetool/CMakeFiles/bpftimetool.dir/main.cpp.o] Error 1
make[2]: *** [tools/bpftimetool/CMakeFiles/bpftimetool.dir/all] Error 2
make[1]: *** [all] Error 2
make: *** [build-wo-libbpf] Error 2

hp77-creator commented 4 weeks ago

I have done so many changes to rectify errors now that I have lost track of my changes, will need to look at them again. I had faced this boost error before as well, setting BOOST_ROOT fixed this before. will try again.

Officeyutong commented 4 weeks ago

I have done so many changes to rectify errors now that I have lost track of my changes, will need to look at them again. I had faced this boost error before as well, setting BOOST_ROOT fixed this before. will try again.

boost may not be recognized automically. Sometimes you may need to set BOOST_ROOT manualy

Officeyutong commented 4 weeks ago

also what to do of bpf_object.cpp ?

This file is mainly used for parsing ELF files. It calls API provided by libbpf to pick up eBPF programs defined in an ELF file. It may be replaced with other libraries that can parse ELF, such as libelf.

eBPF programs are stored in program sections of an ELF file. section names would be like uprobe/XXXXX or tracepoint/syscall/XXXX or other things alike. (You may use readelf to get some details). Some eBPF programs might require linking with other eBPF programs (such as an eBPF program calling another non-inline function), but in my opinion we may temporarily not provide support for linking

You may refer to https://github.com/libbpf/libbpf/blob/42065ea6627ff6e1ab4c65e51042a70fbf30ff7c/src/libbpf.c#L7892 for how libbpf loads ELF files

hp77-creator commented 3 weeks ago

I will read more on the above. An update that I have been able to compile bpftime successfully. But when I tried running it on my system, I am getting following error:

2024-06-09 02:47:53.495] [info] Injecting to 99920
[2024-06-09 02:47:53.664] [error] Failed to inject: Module not found at "/usr/lib/libz.1.dylib"

I checked, macOS comes with a zlib installation which is symlinked to libz.1.dylib as libz.1.2.1.dylib, I thought of adding one more symlink but there are various write permission issues. I will try it again.

hp77-creator commented 3 weeks ago

fixed some issues that creeped up in PR but there is still one issue coming up when runtime is tested, CI is giving SIGSEGV and failing tests. I was thinking it was my spinlock implementation that might be doing this violation but with the new change as well, error persists. @yunwei37 @Officeyutong @viniciusd Can you folks review the PR and point out things that I might have missed. Moving it out of Draft now.

hp77-creator commented 3 weeks ago

So, some guards were missing, I have added those and regression is working fine now. Please provide your feedback @Officeyutong @yunwei37

hp77-creator commented 3 weeks ago

Also @Officeyutong @yunwei37 currently i have added all headers needed for bpf and epoll in bpftime_poll.h, I was not sure what header name to keep or how to differentiate so I have added all structs and everything into one, if it works, we can merge it like this and then as @viniciusd includes support for frida and syscalls we can separate out what is needed and what is not.

hp77-creator commented 3 weeks ago

@Officeyutong @yunwei37 should we also create a CI for macOS?

Officeyutong commented 3 weeks ago

@Officeyutong @yunwei37 should we also create a CI for macOS?

Good idea! We may migrate frida attach manager tests CI to macOS first

viniciusd commented 3 weeks ago

@Officeyutong @yunwei37 should we also create a CI for macOS?

Good idea! We may migrate frida attach manager tests CI to macOS first

That's related to something I wanted to bring up in #304, but I was trying to make sense of the current implementation and reviewing the paper before asking:

bpftime_frida_uprobe_attach_tests doesn't break at the moment. All tests pass even though I am on ARM:

$ ./bpftime/build/attach/frida_uprobe_attach_impl/bpftime_frida_uprobe_attach_tests
Randomness seeded to: 3365707513
===============================================================================
All tests passed (1025 assertions in 8 test cases)

They were expected to fail/break, right?

Officeyutong commented 3 weeks ago

@Officeyutong @yunwei37 should we also create a CI for macOS?

Good idea! We may migrate frida attach manager tests CI to macOS first

That's related to something I wanted to bring up in #304, but I was trying to make sense of the current implementation and reviewing the paper before asking:

bpftime_frida_uprobe_attach_tests doesn't break at the moment. All tests pass even though I am on ARM:
$ ./bpftime/build/attach/frida_uprobe_attach_impl/bpftime_frida_uprobe_attach_tests
Randomness seeded to: 3365707513
===============================================================================
All tests passed (1025 assertions in 8 test cases)
They were expected to fail/break, right?

It was not expected to fail. But we have never tested it on macos

hp77-creator commented 3 weeks ago

@Officeyutong @yunwei37 should we also create a CI for macOS?

Good idea! We may migrate frida attach manager tests CI to macOS first

Alright, we can take it up in another PR then Will fix this PR with requested changes and we can merge it.

Officeyutong commented 2 weeks ago

@viniciusd Remeber to approve this PR if there is no other issues. I'll merge this once you approve it

Officeyutong commented 1 week ago

Remeber to resolve conflicts @hp77-creator

hp77-creator commented 1 week ago

Alright resolving.

Officeyutong commented 1 week ago

Also, options added should be documented in https://github.com/eunomia-bpf/eunomia.dev/blob/main/docs/bpftime/documents/build-and-test.md, remember to update it

@hp77-creator

hp77-creator commented 1 week ago

Will do

eunomia-bpf / bpftime