Error in making HOMA

[sarsanaee]

Hi, here is a problem I have while making HOMA!

alireza@Alireza:~/Documents/HomaModule$ make
make -C /lib/modules/4.15.0-29-generic/build M=/home/alireza/Documents/HomaModule modules
make[1]: Entering directory '/usr/src/linux-headers-4.15.0-29-generic'
  CC [M]  /home/alireza/Documents/HomaModule/homa_input.o
In file included from /home/alireza/Documents/HomaModule/homa_input.c:3:0:
/home/alireza/Documents/HomaModule/homa_impl.h:535:8: error: unknown type name ‘__poll_t’
 extern __poll_t
        ^
scripts/Makefile.build:332: recipe for target '/home/alireza/Documents/HomaModule/homa_input.o' failed
make[2]: *** [/home/alireza/Documents/HomaModule/homa_input.o] Error 1
Makefile:1552: recipe for target '_module_/home/alireza/Documents/HomaModule' failed
make[1]: *** [_module_/home/alireza/Documents/HomaModule] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-4.15.0-29-generic'
Makefile:7: recipe for target 'all' failed
make: *** [all] Error 2

Is this a problem related to the packages in my system or is a logical error? It cannot find __poll_t ! I guess something should be included in homa_impl.h.

Thanks

[btlcmr0702]

I also mret some problems when I try to make HOMA make -C /lib/modules/4.4.0-31-generic/build M=/home/tian/libo/homa/HomaModule-master modules make[1]: Entering directory `/usr/src/linux-headers-4.4.0-31-generic' CC [M] /home/tian/libo/homa/HomaModule-master/homa_incoming.o In file included from /home/tian/libo/homa/HomaModule-master/homa_incoming.c:4:0: /home/tian/libo/homa/HomaModule-master/homa_impl.h:14:32: fatal error: linux/sched/signal.h: No such file or directory

include <linux/sched/signal.h>

                            ^

compilation terminated. make[2]: [/home/tian/libo/homa/HomaModule-master/homa_incoming.o] Error 1 make[1]: [module/home/tian/libo/homa/HomaModule-master] Error 2 make[1]: Leaving directory `/usr/src/linux-headers-4.4.0-31-generic' make: *** [all] Error 2

Dose HOMA reply on some specified packets?

[johnousterhout]

I'm terribly sorry for my slow response; I hadn't realized until today that I wasn't getting notifications for this repo.

__poll_t is used as the return value for the poll function in `struct proto_ops. I'm currently compiling against Linux v4.16, whereas I see you are using 4.15. In your version, it appears that the poll method returns an unsigned int; it should be easy to modify Homa to conform to this interface.

For btlcmr0702, it looks like your compilation error is also caused by version differences: linux/sched/signal.h was split out of linux/sched.h in a more recent version; you might be able to get around this problem by changing your #include statement to include linux/sched.h instead of linux/sched/signal.h.

I'm a newbie when it comes to working inside the Linux kernel, so it's likely I'm not doing things in the most portable way. If you find ways in which I can improve, please let me know.

-John-

[btlcmr0702]

Hi johh, Thank you so much for your reply. Actually just as you said, I changes the include file and the error just gone. But there are more errors. I also meet the problem about can't find ___pollt and other errors: /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:53:21: error: ‘sk_set_peek_off’ undeclared here (not in a function) .set_peek_off = sk_set_peek_off, ^

/home/tian/libo/homa/HomaModule-master/homa_plumbing.c:78:2: warning: initialization from incompatible pointer type [enabled by default] .hash = homa_hash, ^ /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:78:2: warning: (near initialization for ‘homa_prot.hash’) [enabled by default] /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:87:2: error: unknown field ‘diag_destroy’ specified in initializer .diag_destroy = homa_diag_destroy, ^ /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:87:2: warning: initialization makes integer from pointer without a cast [enabled by default] /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:87:2: warning: (near initialization for ‘homa_prot.slab_flags’) [enabled by default] /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:87:2: error: initializer element is not computable at load time /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:87:2: error: (near initialization for ‘homa_prot.slab_flags’) /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:102:2: error: unknown field ‘early_demux_handler’ specified in initializer .early_demux_handler = NULL, / homa_v4_early_demux_handler / ^ /home/tian/libo/homa/HomaModule-master/homa_plumbing.c: In function ‘homa_wait_ready_msg’: /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:509:48: error: macro "sk_wait_event" passed 4 arguments, but takes just 3 !list_empty(&homa_sk(sk)->ready_rpcs), &wait); ^ /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:508:7: error: ‘sk_wait_event’ undeclared (first use in this function) rc = sk_wait_event(sk, timeo, ^ /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:508:7: note: each undeclared identifier is reported only once for each function it appears in make[2]: [/home/tian/libo/homa/HomaModule-master/homa_plumbing.o] Error 1 make[1]: [module/home/tian/libo/homa/HomaModule-master] Error 2 make[1]: Leaving directory `/usr/src/linux-headers-4.4.0-31-generic' make: *** [all] Error 2

So can I have your environment for homa? Like the version of gcc or linux kernel, cause I wanna run it on my own host.

Thanks!

[sarsanaee]

Hi,

Yeah, my problem resolved thanks!

@btlcmr0702 I guess you shouldn't have this problem, just clean the directory and make it from scratch. Change the line 697 to unsigned int homa_poll(struct file *file, struct socket *sock, struct poll_table_struct *wait) in homa_plumbing.c

Also, make sure you have the same definition in homa_impl.h at line 752.

I also had a clock skewed problem in the compilation and just added these two lines below to the Makefile clean function:

clean:
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
    find . -type f | xargs touch # this 
    rm -rf $(OBJS) # and this

[johnousterhout]

I'm building HomaModule against Linux v4.16.10 using gcc v6.4. However, I only recently upgraded to gcc 6.4; I had previously been using 4.x, so I suspect that HomaModule will still compile with earlier versions of gcc.

-John-

On Thu, Oct 18, 2018 at 6:06 PM uCantSeeMe notifications@github.com wrote:

Hi johh, Thank you so much for your reply. Actually just as you said, I changes the include file and the error just gone. But there are more errors. I also meet the problem about can't find __poll_t and other errors: /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:53:21: error: ‘sk_set_peek_off’ undeclared here (not in a function) .set_peek_off = sk_set_peek_off, ^ /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:78:2: warning: initialization from incompatible pointer type [enabled by default] .hash = homa_hash, ^ /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:78:2: warning: (near initialization for ‘homa_prot.hash’) [enabled by default] /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:87:2: error: unknown field ‘diag_destroy’ specified in initializer .diag_destroy = homa_diag_destroy, ^ /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:87:2: warning: initialization makes integer from pointer without a cast [enabled by default] /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:87:2: warning: (near initialization for ‘homa_prot.slab_flags’) [enabled by default] /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:87:2: error: initializer element is not computable at load time /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:87:2: error: (near initialization for ‘homa_prot.slab_flags’) /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:102:2: error: unknown field ‘early_demux_handler’ specified in initializer .early_demux_handler = NULL, / homa_v4_early_demux_handler / ^ /home/tian/libo/homa/HomaModule-master/homa_plumbing.c: In function ‘homa_wait_ready_msg’: /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:509:48: error: macro "sk_wait_event" passed 4 arguments, but takes just 3 !list_empty(&homa_sk(sk)->ready_rpcs), &wait); ^ /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:508:7: error: ‘sk_wait_event’ undeclared (first use in this function) rc = sk_wait_event(sk, timeo, ^ /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:508:7: note: each undeclared identifier is reported only once for each function it appears in make[2]: [/home/tian/libo/homa/HomaModule-master/homa_plumbing.o] Error 1 make[1]: [module/home/tian/libo/homa/HomaModule-master] Error 2 make[1]: Leaving directory /usr/src/linux-headers-4.4.0-31-generic' make: *** [all] Error 2`

So can I have your environment for homa? Like the version of gcc or linux kernel, cause I wanna run it on my own host.

Thanks!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/PlatformLab/HomaModule/issues/1#issuecomment-431212264, or mute the thread https://github.com/notifications/unsubscribe-auth/AJzqCvDX_I42bjzRUWWLt1rOYtBhq_J-ks5umSWngaJpZM4VlZDX .

[btlcmr0702]

@sarsanaee Thanks for your help. Now the problem of __poll_t has been solved. But I still meet some problems when compile: make[1]: Entering directory/usr/src/linux-headers-4.4.0-31-generic' CC [M] /home/tian/libo/homa/HomaModule-master/homa_incoming.o CC [M] /home/tian/libo/homa/HomaModule-master/homa_outgoing.o CC [M] /home/tian/libo/homa/HomaModule-master/homa_plumbing.o /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:53:21: error: ‘sk_set_peek_off’ undeclared here (not in a function) .set_peek_off = sk_set_peek_off, ^ /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:78:2: warning: initialization from incompatible pointer type [enabled by default] .hash = homa_hash, ^ /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:78:2: warning: (near initialization for ‘homa_prot.hash’) [enabled by default] /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:87:2: error: unknown field ‘diag_destroy’ specified in initializer .diag_destroy = homa_diag_destroy, ^ /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:87:2: warning: initialization makes integer from pointer without a cast [enabled by default] /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:87:2: warning: (near initialization for ‘homa_prot.slab_flags’) [enabled by default] /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:87:2: error: initializer element is not computable at load time /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:87:2: error: (near initialization for ‘homa_prot.slab_flags’) /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:102:2: error: unknown field ‘early_demux_handler’ specified in initializer .early_demux_handler = NULL, / homa_v4_early_demux_handler / ^ /home/tian/libo/homa/HomaModule-master/homa_plumbing.c: In function ‘homa_wait_ready_msg’: /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:509:48: error: macro "sk_wait_event" passed 4 arguments, but takes just 3 !list_empty(&homa_sk(sk)->ready_rpcs), &wait); ^ /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:508:7: error: ‘sk_wait_event’ undeclared (first use in this function) rc = sk_wait_event(sk, timeo, ^ /home/tian/libo/homa/HomaModule-master/homa_plumbing.c:508:7: note: each undeclared identifier is reported only once for each function it appears in make[2]: [/home/tian/libo/homa/HomaModule-master/homa_plumbing.o] Error 1 make[1]: [module/home/tian/libo/homa/HomaModule-master] Error 2 make[1]: Leaving directory /usr/src/linux-headers-4.4.0-31-generic' make: *** [all] Error 2 Have you met the problems? This is my environment: gcc version 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.3)

THANKS!

[sarsanaee]

Hi @btlcmr0702 ,

My kernel version: 4.15.0-36-generic Also my GCC version: 5.4.0

You are using Ubuntu 14.04! If I were you I would install Ubuntu 18.04 LTS. You might face more problems even after resolving this! The current stable kernel version is 4.18.16. Mine is also OLD :)

Alireza

[btlcmr0702]

Hi @sarsanaee Thanks for your reply. It seems very hard to make and run HOMA module. If you can do that please let me know. I will try other environment to make HOMA also.

[sarsanaee]

Hi @btlcmr0702,

It is not just about running the project, you may want to read the paper itself first,

I have just read the paper when it got rejected from NSDI, and now it is accepted to SIGCOMM. In that time, I was curious to compare QUIC with this protocol. However, their applications are different and QUIC is not a data center transport protocol but both seem to be a low latency transport. This project is the implementation of a low latency transport protocol for the granular computing project. In some application domains, we have really small or tiny messages(tasks) but a large number of them and we need to handle all those messages in a timely manner without unpredictability. It is a big project that has multiple sections. For instance, when we have a large number of threads for a considerable number of incoming messages, then a load balancer is needed to manage those threads in the process scheduler(Arachne is another paper regarding this problem). This Github is also another part of the whole project since transport protocols should not impact the packet latency or have near zero impact. The total service time of each task is about a couple of microseconds(20us), and transport layer should not take more than service time. However, TCP is not a suitable solution for aiming aforementioned objective. You may also want to take a look at the DCTCP paper. These latency reqirements are getting tigher and tigher. Some people are thinking of even cached(SRAM) instead of memory(DRAM) for handling these packets(You may want to see RAMCloud and Dark Packets and the end of Network Scaling.

I'm still curious to play with it and have some measurements on it on my own and we may have some common interest on this project and can collaborate. For now, I only ran the homa_test, and it looks working. Take a look at the codes, for instance, open the homa_test.cc and check what parameters it requires. Make sure you run the make command in the HomaModule/tests directory.

Actually, after making the project successfully, according to the README.md you need to load the resulting kernel module in the kernel sudo insmod homa.ko did you do so? This project implements the whole protocol using the primitive APIs that kernel provides for the developers to build an arbitrary transport protocol.

[btlcmr0702]

Hi @sarsanaee Actually I am just reading the paper in detail. Recently, transport protocols about "token" mechanism in data center are very hot in many conference, like pHost, NDP, ExpressPass and Homa. I am very curious about the mechanism in Homa. For example, as the paper said, it can use the past traffic pattern to allocate the priority dynamically at receiver side. Followed by your advice( Thank you 👍 ), I can make the HomaModule/tests, and I will try to run the tests later. But make all of this project is not easy.

I also find another project about Homa in this lab: PlatformLab/Homa As this project said ,it is an implementation of the Homa transport protocol as a C++ userspace library. I don't know the difference between these two projects and I will read the codes further.

[johnousterhout]

Hi Alireza,

I read your posting with great interest: does it mean that you are able to build and install Homa and run some of the simple tests in homa_test? If so, that would be great news; you may be the first person besides me to do this!

-John-

On Thu, Oct 25, 2018 at 11:24 PM alireza sanaee notifications@github.com wrote:

Hi @btlcmr0702 https://github.com/btlcmr0702,

It is not just about running the project, you may want to read the paper itself first,

I have just read the paper when it got rejected from NSDI, and now it is accepted to SIGCOMM. In that time, I was curious to compare QUIC with this protocol. However, their applications are different and QUIC is not a data center transport protocol but both seem to be a low latency transport. This project is the implementation of a low latency transport protocol for the granular computing project. In some application domains, we have really small or tiny messages(tasks) but a large number of them and we need to handle all those messages in a timely manner without unpredictability. It is a big project that has multiple sections. For instance, when we have a large number of threads for a considerable number of incoming messages, then a load balancer is needed to manage those threads in the process scheduler(Arachne is another paper regarding this problem). This Github is also another part of the whole project since transport protocols should not impact the packet latency or have near zero impact. The total service time of each task is about a couple of microseconds(20us), and transport layer should not take more than service time. However, TCP is not a suitable solution for aiming aforementioned objective. You may also want to take a look at the DCTCP paper. These latency reqirements are getting tigher and tigher. Some people are thinking of even cached(SRAM) instead of memory(DRAM) for handling these packets(You may want to see RAMCloud and Dark Packets and the end of Network Scaling.

I'm still curious to play with it and have some measurements on it on my own and we may have some common interest on this project and can collaborate. For now, I only ran the homa_test, and it looks working. Take a look at the codes, for instance, open the homa_test.cc and check what parameters it requires. Make sure you run the make command in the HomaModule/tests directory.

Actually, after making the project successfully, according to the README.md you need to load the resulting kernel module in the kernel sudo insmod homa.ko did you do so? This project implements the whole protocol using the primitive APIs that kernel provides for the developers to build an arbitrary transport protocol.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/PlatformLab/HomaModule/issues/1#issuecomment-433300169, or mute the thread https://github.com/notifications/unsubscribe-auth/AJzqCmYsE9eI-KLlwG3QSS6Ms0Hn0mtiks5uoqqkgaJpZM4VlZDX .

[johnousterhout]

For now I believe all the issues in here have been resolved. Please create a new issue for any new problems building Homa.