search

sdnfv / openNetVM

A high performance container-based NFV platform from GW and UCR.
http://sdnfv.github.io/onvm/
Other
263 stars 136 forks source link

Add NFD library with multiple example NFs #98

Closed evandengdbw closed 5 years ago

evandengdbw commented 5 years ago

This pr adds a NFD library and a number of example NFs

Summary:

Internal onvm changes: Replace k = data with memcpy function in line 202 in file onvm/onvm_nflib/onvm_flow_table.h to support C++11 features.
Add NF_NAME=${NF_PATH##*/} after start_nf.sh file calculates the NF_PATH in line 27 to get the real name without NFD/ dir .

Here is the list of the new NFs added: Stateful Firewall: All outgoing flows are allowed and recorded, all incoming flows initiated by an outgoing flow are also allowed, all incoming flows without initiation are dropped. User can define the model.txt file to set allowed flow and compile it into executable NF codes.
Stateless Firewall: Allowing connections satisfying the whitelist or blocks those hitting the blacklist.
DNS Amplification Mitigation: An attacker uses a spoofed server IP to request many DNS queries that result in large answers to that DoS the server with the spoofed IP. Mitigation is by tracking if the server actually committed this request.
Heavy Hitter Detection: Keeping a counter for per flow and detecting which flows consume most bandwidth by comparing the counters with a threshold.
NAPT: NAPT is short for Network Address Port Translation. It enables mappings from tuples(address, L4 port number) to tuples(registered address and assigned port number) to complete address translation.
NAT: NAT is short for Network Address Translation(without port). Different from NAPT, it only translates IP address to another assigned one without L4 port number involved.
Super Spreader Detection: Super Spreader Detection detects and identifies super spreaders to preempt port scan attacks or DDoS attacks by increasing the counter on SYNs and decrease it on FINs.
SYN Flood Detection: SYN Flood Detection counts the number of SYNs without any matching ACK from the sender side. If one sender exceeds a certain threshold, it should be blocked.
UDP Flood Mitigation: UDP Flood Mitigation identifies source IPs that send an anomalously higher number of UDP packets and uses the statistics to categorize each packet as either attack or benign.

Usage:

This PR includes
Resolves issues
Breaking API changes
Internal API changes
Usability improvements
Bug fixes
New functionality 👍
New NF/onvm_mgr args 👍
Changes to starting NFs
Dependency updates
Web stats updates

Merging notes:

TODO before merging :

Test Plan:

Review:

(optional) << @-mention people who should review these changes >>

(optional) Subscribers: << @-mention people who probably care about these changes >>

onvm commented 5 years ago

In response to PR creation

CI Message

User not authorized to run CI, please contact one of the repo maintainers

evandengdbw commented 5 years ago

This operation is to add a new NF named stateful_firewall developed by NFD. The function and context of this NF are described in examples/stateful_firewall/Readme.md, and relative helper files are described in examples/stateful_firewall/architecture.txt. In order to support C++11 features, I have also changed some codes in onvm/onvm_nflib/onvm_flow_table.h. Thanks for your help.

koolzz commented 5 years ago

@onvm how does C++ lint work with CI?

onvm commented 5 years ago

@onvm how does C++ lint work with CI?

CI Message

Your results will arrive shortly

onvm commented 5 years ago

@onvm how does C++ lint work with CI?

CI Message

Run successful see results: [Results from nimbnode30] Median TX pps for Speed Tester: 35091593

Linter Passed

koolzz commented 5 years ago

@evandengdbw Can you please fill in the pr description a bit?

Also we were discussing the new NF additions during the our weekly meeting, we were wondering if it would make sense to create a separate examples/NFD/ folder which could have all the examples you guys have developed? If we do that the code that your NFs share can be put into that directory and not duplicated in every NF.

evandengdbw commented 5 years ago

@koolzz I think it is better creating a separate dir examples/NFD/. Then we can add other remaining NFs there.

koolzz commented 5 years ago

Yeah an NFD directory inside examples would work great, can you restructure this so that all your NFs use a shared library with common functions for your NFs? Also we use spaces instead of tabs so can you retab your files?

evandengdbw commented 5 years ago

OK, do you mean that we should change all tabs into spaces?

koolzz commented 5 years ago

OK, do you mean that we should change all tabs into spaces?

Yeah, also I've updated the linter to work for cpp files lets see if it does

koolzz commented 5 years ago

@onvm run the linter pleases

onvm commented 5 years ago

@onvm run the linter pleases

CI Message

Your results will arrive shortly

onvm commented 5 years ago

@onvm run the linter pleases

CI Message

Run successful see results: [Results from nimbnode30] Median TX pps for Speed Tester: 35161463

Linter Passed

koolzz commented 5 years ago

I appreciate the changes, I've also tried to reconfigure the linter to actually work with cpp files (again) lets see if it works(nope still being weird).

koolzz commented 5 years ago

Okay CI should work fine now @onvm

onvm commented 5 years ago

Okay CI should work fine now @onvm

CI Message

Your results will arrive shortly

onvm commented 5 years ago

Okay CI should work fine now @onvm

CI Message

Run successful see results: [Results from nimbnode30] Median TX pps for Speed Tester: 35159063

Linter Failed

examples/NFD/include/basic_classes.h:0: No #ifndef header guard found, suggested CPP variable is: _BASIC_CLASSESH [build/header_guard] [5] examples/NFD/include/basic_classes.h:18: Found C system header after C++ system header. Should be: basic_classes.h, c system, c++ system, other. [build/include_order] [4] examples/NFD/include/basic_classes.h:19: Found C system header after C++ system header. Should be: basic_classes.h, c system, c++ system, other. [build/include_order] [4] examples/NFD/include/basic_classes.h:26: Found C system header after C++ system header. Should be: basic_classes.h, c system, c++ system, other. [build/include_order] [4] examples/NFD/include/basic_classes.h:29: Do not use namespace using-directives. Use using-declarations instead. [build/namespaces] [5] examples/NFD/include/basic_classes.h:33: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] examples/NFD/include/basic_classes.h:33: Missing spaces around = [whitespace/operators] [4] examples/NFD/include/basic_classes.h:34: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] examples/NFD/include/basic_classes.h:34: Missing spaces around = [whitespace/operators] [4] examples/NFD/include/basic_classes.h:35: Missing spaces around = [whitespace/operators] [4] examples/NFD/include/basic_classes.h:45: { should almost always be at the end of the previous line [whitespace/braces] [4] examples/NFD/include/basic_classes.h:63: Missing space before { [whitespace/braces] [5] examples/NFD/include/basic_classes.h:69: Missing spaces around = [whitespace/operators] [4] examples/NFD/include/basic_classes.h:86: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] examples/NFD/include/basic_classes.h:101: An else should appear on the same line as the preceding } [whitespace/newline] [4] examples/NFD/include/basic_classes.h:101: If an else has a brace on one side, it should have it on both [readability/braces] [5] examples/NFD/include/basic_classes.h:109: An else should appear on the same line as the preceding } [whitespace/newline] [4] examples/NFD/include/basic_classes.h:109: If an else has a brace on one side, it should have it on both [readability/braces] [5] examples/NFD/include/basic_classes.h:113: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] examples/NFD/include/basic_classes.h:124: Should have a space between // and comment [whitespace/comments] [4] examples/NFD/include/basic_classes.h:125: You don't need a ; after a } [readability/braces] [4] examples/NFD/include/basic_classes.h:126: Single-parameter constructors should be marked explicit. [runtime/explicit] [5] examples/NFD/include/basic_classes.h:143: Single-parameter constructors should be marked explicit. [runtime/explicit] [5] examples/NFD/include/basic_classes.h:144: Missing space before { [whitespace/braces] [5] examples/NFD/include/basic_classes.h:146: Should have a space between // and comment [whitespace/comments] [4] examples/NFD/include/basic_classes.h:147: Should have a space between // and comment [whitespace/comments] [4] examples/NFD/include/basic_classes.h:176: Missing space before { [whitespace/braces] [5] examples/NFD/include/basic_classes.h:189: An else should appear on the same line as the preceding } [whitespace/newline] [4] examples/NFD/include/basic_classes.h:189: If an else has a brace on one side, it should have it on both [readability/braces] [5] examples/NFD/include/basic_classes.h:193: An else should appear on the same line as the preceding } [whitespace/newline] [4] examples/NFD/include/basic_classes.h:193: If an else has a brace on one side, it should have it on both [readability/braces] [5] examples/NFD/include/basic_classes.h:203: Missing spaces around = [whitespace/operators] [4] examples/NFD/include/basic_classes.h:208: Missing space before ( in if( [whitespace/parens] [5] examples/NFD/include/basic_classes.h:208: Missing space before { [whitespace/braces] [5] examples/NFD/include/basic_classes.h:209: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] examples/NFD/include/basic_classes.h:210: Missing space before ( in if( [whitespace/parens] [5] examples/NFD/include/basic_classes.h:210: Missing space before { [whitespace/braces] [5] examples/NFD/include/basic_classes.h:210: Missing space before else [whitespace/braces] [5] examples/NFD/include/basic_classes.h:212: Missing space before ( in if( [whitespace/parens] [5] examples/NFD/include/basic_classes.h:212: Missing space before { [whitespace/braces] [5] examples/NFD/include/basic_classes.h:212: Missing space before else [whitespace/braces] [5] examples/NFD/include/basic_classes.h:214: Missing space before ( in if( [whitespace/parens] [5] examples/NFD/include/basic_classes.h:214: Missing space before { [whitespace/braces] [5] examples/NFD/include/basic_classes.h:214: Missing space before else [whitespace/braces] [5] examples/NFD/include/basic_classes.h:216: Missing space before ( in if( [whitespace/parens] [5] examples/NFD/include/basic_classes.h:216: Missing space before { [whitespace/braces] [5] examples/NFD/include/basic_classes.h:216: Missing space before else [whitespace/braces] [5] examples/NFD/include/basic_classes.h:218: Missing space before ( in if( [whitespace/parens] [5] examples/NFD/include/basic_classes.h:218: Missing space before { [whitespace/braces] [5] examples/NFD/include/basic_classes.h:218: Missing space before else [whitespace/braces] [5] examples/NFD/include/basic_classes.h:220: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] examples/NFD/include/basic_classes.h:227: Missing space before { [whitespace/braces] [5] examples/NFD/include/basic_classes.h:227: Single-parameter constructors should be marked explicit. [runtime/explicit] [5] examples/NFD/include/basic_classes.h:241: Missing space before { [whitespace/braces] [5] examples/NFD/include/basic_classes.h:249: If an else has a brace on one side, it should have it on both [readability/braces] [5] examples/NFD/include/basic_classes.h:249: Missing space before { [whitespace/braces] [5] examples/NFD/include/basic_classes.h:250: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] examples/NFD/include/basic_classes.h:259: { should almost always be at the end of the previous line [whitespace/braces] [4] examples/NFD/include/basic_classes.h:261: Do not indent within a namespace [runtime/indentation_namespace] [4] examples/NFD/include/basic_classes.h:262: { should almost always be at the end of the previous line [whitespace/braces] [4] examples/NFD/include/basic_classes.h:264: { should almost always be at the end of the previous line [whitespace/braces] [4] examples/NFD/include/basic_classes.h:278: Do not indent within a namespace [runtime/indentation_namespace] [4] examples/NFD/include/basic_classes.h:282: Missing spaces around = [whitespace/operators] [4] examples/NFD/include/basic_classes.h:292: Do not indent within a namespace [runtime/indentation_namespace] [4] examples/NFD/include/basic_classes.h:296: Missing spaces around = [whitespace/operators] [4] examples/NFD/include/basic_classes.h:309: Do not indent within a namespace [runtime/indentation_namespace] [4] examples/NFD/include/basic_classes.h:310: { should almost always be at the end of the previous line [whitespace/braces] [4] examples/NFD/include/basic_classes.h:316: Do not indent within a namespace [runtime/indentation_namespace] [4] examples/NFD/include/basic_classes.h:318: { should almost always be at the end of the previous line [whitespace/braces] [4] examples/NFD/include/basic_classes.h:332: Do not indent within a namespace [runtime/indentation_namespace] [4] examples/NFD/include/basic_classes.h:334: Mismatching spaces inside () in if [whitespace/parens] [5] examples/NFD/include/basic_classes.h:338: Missing space before { [whitespace/braces] [5] examples/NFD/include/basic_classes.h:343: An else should appear on the same line as the preceding } [whitespace/newline] [4] examples/NFD/include/basic_classes.h:354: An else should appear on the same line as the preceding } [whitespace/newline] [4] examples/NFD/include/basic_classes.h:359: Namespace should be terminated with "// namespace std" [readability/namespace] [5] examples/NFD/include/basic_classes.h:332: Add #include for equal_to<> [build/include_what_you_use] [4] Total errors found: 76 examples/NFD/include/decode.h:0: No #ifndef header guard found, suggested CPP variable is: _DECODEH [build/header_guard] [5] examples/NFD/include/decode.h:23: { should almost always be at the end of the previous line [whitespace/braces] [4] examples/NFD/include/decode.h:30: { should almost always be at the end of the previous line [whitespace/braces] [4] examples/NFD/include/decode.h:35: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] examples/NFD/include/decode.h:49: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] examples/NFD/include/decode.h:49: { should almost always be at the end of the previous line [whitespace/braces] [4] Total errors found: 6 examples/NFD/lib/basic_classes.cpp:19: Found C system header after C++ system header. Should be: basic_classes.h, c system, c++ system, other. [build/include_order] [4] examples/NFD/lib/basic_classes.cpp:20: Found C system header after C++ system header. Should be: basic_classes.h, c system, c++ system, other. [build/include_order] [4] examples/NFD/lib/basic_classes.cpp:22: Found C system header after C++ system header. Should be: basic_classes.h, c system, c++ system, other. [build/include_order] [4] examples/NFD/lib/basic_classes.cpp:25: Do not use namespace using-directives. Use using-declarations instead. [build/namespaces] [5] examples/NFD/lib/basic_classes.cpp:69: Missing space before { [whitespace/braces] [5] examples/NFD/lib/basic_classes.cpp:80: An else should appear on the same line as the preceding } [whitespace/newline] [4] examples/NFD/lib/basic_classes.cpp:80: If an else has a brace on one side, it should have it on both [readability/braces] [5] examples/NFD/lib/basic_classes.cpp:85: Missing space before { [whitespace/braces] [5] examples/NFD/lib/basic_classes.cpp:93: An else should appear on the same line as the preceding } [whitespace/newline] [4] examples/NFD/lib/basic_classes.cpp:93: If an else has a brace on one side, it should have it on both [readability/braces] [5] examples/NFD/lib/basic_classes.cpp:99: Missing space before { [whitespace/braces] [5] examples/NFD/lib/basic_classes.cpp:107: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] examples/NFD/lib/basic_classes.cpp:112: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] examples/NFD/lib/basic_classes.cpp:116: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] examples/NFD/lib/basic_classes.cpp:125: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] examples/NFD/lib/basic_classes.cpp:127: Should have a space between // and comment [whitespace/comments] [4] examples/NFD/lib/basic_classes.cpp:131: Lines should be <= 120 characters long [whitespace/line_length] [5] examples/NFD/lib/basic_classes.cpp:132: Should have a space between // and comment [whitespace/comments] [4] examples/NFD/lib/basic_classes.cpp:139: Add #include for string [build/include_what_you_use] [4] examples/NFD/lib/basic_classes.cpp:50: Add #include for vector<> [build/include_what_you_use] [4] Total errors found: 20 examples/NFD/lib/basic_methods.cpp:22: Add #include for string [build/include_what_you_use] [4] examples/NFD/lib/basic_methods.cpp:20: Add #include for vector<> [build/include_what_you_use] [4] Total errors found: 2 examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:64: Missing space before { [whitespace/braces] [5] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:65: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:78: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:78: Should have a space between // and comment [whitespace/comments] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:81: Found C system header after C++ system header. Should be: stateful_firewall.h, c system, c++ system, other. [build/include_order] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:83: Found C++ system header after other header. Should be: stateful_firewall.h, c system, c++ system, other. [build/include_order] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:84: Found C++ system header after other header. Should be: stateful_firewall.h, c system, c++ system, other. [build/include_order] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:85: Found C++ system header after other header. Should be: stateful_firewall.h, c system, c++ system, other. [build/include_order] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:86: Found C++ system header after other header. Should be: stateful_firewall.h, c system, c++ system, other. [build/include_order] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:87: Found C system header after other header. Should be: stateful_firewall.h, c system, c++ system, other. [build/include_order] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:88: Found C system header after other header. Should be: stateful_firewall.h, c system, c++ system, other. [build/include_order] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:89: Found C++ system header after other header. Should be: stateful_firewall.h, c system, c++ system, other. [build/include_order] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:90: Found C system header after other header. Should be: stateful_firewall.h, c system, c++ system, other. [build/include_order] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:92: Do not use namespace using-directives. Use using-declarations instead. [build/namespaces] [5] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:107: Missing spaces around = [whitespace/operators] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:108: Missing spaces around = [whitespace/operators] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:114: Should have a space between // and comment [whitespace/comments] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:118: Missing space before { [whitespace/braces] [5] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:127: Missing space before { [whitespace/braces] [5] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:129: Missing space before { [whitespace/braces] [5] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:130: Lines should be <= 120 characters long [whitespace/line_length] [5] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:130: Missing spaces around = [whitespace/operators] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:132: An else should appear on the same line as the preceding } [whitespace/newline] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:132: If an else has a brace on one side, it should have it on both [readability/braces] [5] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:132: Missing space before { [whitespace/braces] [5] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:134: An else should appear on the same line as the preceding } [whitespace/newline] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:134: If an else has a brace on one side, it should have it on both [readability/braces] [5] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:134: Missing space before { [whitespace/braces] [5] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:142: Missing spaces around = [whitespace/operators] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:147: { should almost always be at the end of the previous line [whitespace/braces] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:251: Should have a space between // and comment [whitespace/comments] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:258: Missing space before ( in if( [whitespace/parens] [5] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:258: Missing space before { [whitespace/braces] [5] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:262: An else should appear on the same line as the preceding } [whitespace/newline] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:262: If an else has a brace on one side, it should have it on both [readability/braces] [5] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:262: Missing space before { [whitespace/braces] [5] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:281: Should have a space between // and comment [whitespace/comments] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:283: Should have a space between // and comment [whitespace/comments] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:290: Should have a space between // and comment [whitespace/comments] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:292: Should have a space between // and comment [whitespace/comments] [4] examples/NFD/stateful_firewall_NFD/stateful_firewall.cpp:110: Add #include for string [build/include_what_you_use] [4] Total errors found: 41 examples/aes_decrypt/aes.h:176: #endif line should be "#endif // _AESH" [build/header_guard] [5] Total errors found: 1 examples/aes_encrypt/aes.h:185: #endif line should be "#endif // _AESH" [build/header_guard] [5] Total errors found: 1 examples/flow_table/flow_table.h:63: #endif line should be "#endif // _FLOW_TABLEH" [build/header_guard] [5] Total errors found: 1 examples/flow_table/msgbuf.h:71: #endif line should be "#endif // _MSGBUFH" [build/header_guard] [5] Total errors found: 1 examples/flow_table/openflow.h:969: #endif line should be "#endif // _OPENFLOWH" [build/header_guard] [5] examples/flow_table/openflow.h:50: Using deprecated casting style. Use static_cast(...) instead [readability/casting] [4] examples/flow_table/openflow.h:569: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:634: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:771: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:804: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:865: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:926: Extra space before ( in function call [whitespace/parens] [4] Total errors found: 8 examples/flow_table/sdn.c:334: { should almost always be at the end of the previous line [whitespace/braces] [4] examples/flow_table/sdn.c:365: Should have a space between // and comment [whitespace/comments] [4] Total errors found: 2 examples/flow_table/sdn.h:104: #endif line should be "#endif // _SDNH" [build/header_guard] [5] Total errors found: 1 examples/flow_table/sdn_pkt_list.h:122: #endif line should be "#endif // _SDN_PKT_LISTH" [build/header_guard] [5] Total errors found: 1 examples/flow_table/setupconn.h:53: #endif line should be "#endif // _SETUPCONNH" [build/header_guard] [5] Total errors found: 1 onvm/onvm_mgr/onvm_init.c:107: { should almost always be at the end of the previous line [whitespace/braces] [4] onvm/onvm_mgr/onvm_init.c:114: { should almost always be at the end of the previous line [whitespace/braces] [4] onvm/onvm_mgr/onvm_init.c:116: { should almost always be at the end of the previous line [whitespace/braces] [4] Total errors found: 3 onvm/onvm_mgr/onvm_pkt.c:68: Are you taking an address of a cast? This is dangerous: could be a temp var. Take the address before doing the cast, rather than after [runtime/casting] [4] Total errors found: 1 onvm/onvm_nflib/onvm_common.h:366: #endif line should be "#endif // _ONVM_COMMONH" [build/header_guard] [5] Total errors found: 1 onvm/onvm_nflib/onvm_config_common.h:208: #endif line should be "#endif // _ONVM_CONFIG_COMMONH" [build/header_guard] [5] Total errors found: 1 onvm/onvm_nflib/onvm_flow_table.h:201: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] Total errors found: 1 onvm/onvm_nflib/onvm_msg_common.h:61: #endif line should be "#endif // _ONVM_MSG_COMMONH" [build/header_guard] [5] Total errors found: 1 onvm/onvm_nflib/onvm_nflib.c:523: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] Total errors found: 1 onvm/onvm_nflib/onvm_pkt_common.c:98: Are you taking an address of a cast? This is dangerous: could be a temp var. Take the address before doing the cast, rather than after [runtime/casting] [4] Total errors found: 1 onvm/onvm_nflib/onvm_sc_common.h:70: #endif line should be "#endif // _ONVM_SC_COMMONH" [build/header_guard] [5] onvm/onvm_nflib/onvm_sc_common.h:70: Should have a space between // and comment [whitespace/comments] [4] Total errors found: 2 onvm/onvm_nflib/onvm_sc_mgr.h:79: #endif line should be "#endif // _ONVM_SC_MGRH" [build/header_guard] [5] Total errors found: 1

koolzz commented 5 years ago

Okay lint finally works, ignore the lint errors not related to your code, regarding your changes you can try running an automatic lint, we used clang-format see config here

evandengdbw commented 5 years ago

These days I have been working on the transformation tasks. And I have already added all NFs in NFD here. Maybe you can test them for correctness. But I don't know what the automatic lint is so I didn't make changes to my codes. If you have any suggestion, please tell me.

koolzz commented 5 years ago

@evandengdbw That is a lot of NFs! Thank you for adding those and providing a library, I'll add a better description on how to run clang-format on all these so you don't have to manually fix style changes. And then we'll start testing the NFs.

The one thing I've noticed is you're using an older version of onvm launch scripts, onvm now offers the flexibility of onvm_mgr core assignment and NFs no longer need to declare a core they will be running on. You would have to pull the latest onvm develop changes and replace your current go.sh scripts with a symlink similar to our current /examples/nf_name/go.sh

I'll provide more info about both later, thank you for editing this.

evandengdbw commented 5 years ago

Great! And I have replaced the go.sh file with the new-version one just now and add one line in start_nf.sh file to help get the real name of an NF in NFD/ directory. Please check whether it is suitable later. Thanks for your time ^_^.

koolzz commented 5 years ago

Can you run clang-format on the code(sudo apt-get install clang-3.8, or any other version should work)? Then just run this script for easy fromatting https://github.com/eklitzke/clang-format-all/blob/master/clang-format-all. If you run it from the openNetVM/ directory and give it examples/NFD it will automatically find the https://github.com/sdnfv/openNetVM-dev/blob/develop/.clang-format.

koolzz commented 5 years ago

I like your / removal solution in the start_nf.sh script, although I would prefer us to avoid having so many go.sh scripts. We should just have a go.sh script in the NFD directory and use a symlink from all the NFD sub directories.

Also I'm having trouble compiling these, lib compiles fine but the NFs fail, see

  ^                                                                                                                                                                                           [0/4356]
/local/onvm/openNetVM/examples/NFD/napt/../../../dpdk/build/include/rte_memcpy.h:590:98: error: ‘_mm_alignr_epi8’ was not declared in this scope
         _mm_storeu_si128((__m128i *)((uint8_t *)dst + 0 * 16), _mm_alignr_epi8(xmm1, xmm0, offset));        \
                                                                                                  ^
/local/onvm/openNetVM/examples/NFD/napt/../../../dpdk/build/include/rte_memcpy.h:652:16: note: in expansion of macro ‘MOVEUNALIGNED_LEFT47_IMM’
     case 0x0E: MOVEUNALIGNED_LEFT47_IMM(dst, src, n, 0x0E); break;    \
                ^
/local/onvm/openNetVM/examples/NFD/napt/../../../dpdk/build/include/rte_memcpy.h:793:2: note: in expansion of macro ‘MOVEUNALIGNED_LEFT47’
  MOVEUNALIGNED_LEFT47(dst, src, n, srcofs);
  ^
/local/onvm/openNetVM/examples/NFD/napt/../../../dpdk/build/include/rte_memcpy.h:612:102: error: ‘_mm_alignr_epi8’ was not declared in this scope
             _mm_storeu_si128((__m128i *)((uint8_t *)dst + 0 * 16), _mm_alignr_epi8(xmm1, xmm0, offset));    \
                                                                                                      ^
/local/onvm/openNetVM/examples/NFD/napt/../../../dpdk/build/include/rte_memcpy.h:652:16: note: in expansion of macro ‘MOVEUNALIGNED_LEFT47_IMM’
     case 0x0E: MOVEUNALIGNED_LEFT47_IMM(dst, src, n, 0x0E); break;    \
                ^
/local/onvm/openNetVM/examples/NFD/napt/../../../dpdk/build/include/rte_memcpy.h:793:2: note: in expansion of macro ‘MOVEUNALIGNED_LEFT47’
  MOVEUNALIGNED_LEFT47(dst, src, n, srcofs);
  ^
/local/onvm/openNetVM/examples/NFD/napt/../../../dpdk/build/include/rte_memcpy.h:590:98: error: ‘_mm_alignr_epi8’ was not declared in this scope
         _mm_storeu_si128((__m128i *)((uint8_t *)dst + 0 * 16), _mm_alignr_epi8(xmm1, xmm0, offset));        \
                                                                                                  ^
/local/onvm/openNetVM/examples/NFD/napt/../../../dpdk/build/include/rte_memcpy.h:653:16: note: in expansion of macro ‘MOVEUNALIGNED_LEFT47_IMM’
     case 0x0F: MOVEUNALIGNED_LEFT47_IMM(dst, src, n, 0x0F); break;    \
                ^
/local/onvm/openNetVM/examples/NFD/napt/../../../dpdk/build/include/rte_memcpy.h:793:2: note: in expansion of macro ‘MOVEUNALIGNED_LEFT47’
  MOVEUNALIGNED_LEFT47(dst, src, n, srcofs);
  ^
/local/onvm/openNetVM/examples/NFD/napt/../../../dpdk/build/include/rte_memcpy.h:612:102: error: ‘_mm_alignr_epi8’ was not declared in this scope
             _mm_storeu_si128((__m128i *)((uint8_t *)dst + 0 * 16), _mm_alignr_epi8(xmm1, xmm0, offset));    \
                                                                                                      ^
/local/onvm/openNetVM/examples/NFD/napt/../../../dpdk/build/include/rte_memcpy.h:653:16: note: in expansion of macro ‘MOVEUNALIGNED_LEFT47_IMM’
     case 0x0F: MOVEUNALIGNED_LEFT47_IMM(dst, src, n, 0x0F); break;    \
                ^
/local/onvm/openNetVM/examples/NFD/napt/../../../dpdk/build/include/rte_memcpy.h:793:2: note: in expansion of macro ‘MOVEUNALIGNED_LEFT47’
  MOVEUNALIGNED_LEFT47(dst, src, n, srcofs);
  ^
In file included from /local/onvm/openNetVM/examples/NFD/napt/../../../onvm/onvm_nflib/onvm_flow_table.h:48:0,
                 from /local/onvm/openNetVM/examples/NFD/napt/../../../onvm/onvm_nflib/onvm_flow_dir.h:45,
                 from /local/onvm/openNetVM/examples/NFD/napt/../../../onvm/onvm_nflib/onvm_pkt_common.h:55,
                 from /local/onvm/openNetVM/examples/NFD/napt/../../../onvm/onvm_nflib/onvm_nflib.h:55,
                 from /local/onvm/openNetVM/examples/NFD/napt/NAPT.cpp:71:
/local/onvm/openNetVM/examples/NFD/napt/../../../dpdk/build/include/rte_thash.h: In function ‘void rte_thash_load_v6_addrs(const ipv6_hdr*, rte_thash_tuple*)’:
/local/onvm/openNetVM/examples/NFD/napt/../../../dpdk/build/include/rte_thash.h:176:52: error: ‘_mm_shuffle_epi8’ was not declared in this scope
    _mm_shuffle_epi8(ipv6, rte_thash_ipv6_bswap_mask);
                                                    ^
make[1]: *** [NAPT.o] Error 1
make: *** [all] Error 2
koolzz commented 5 years ago

Also please provide a more in depth README in the NFD directory, it would be great if you provided some general info about what these NFs are, who developed them, your contact info just in case. stuff like that

evandengdbw commented 5 years ago

I am sorry about that I didn't find the solution of the error but I did compile the NFs correctly and run them smoothly in my local machine. It seems that I have failed to include some dpdk functions but I don't know where they are. Do you have any ideas about the compiling problem?

koolzz commented 5 years ago

Sorry I've been sick for a while I'll try to find out if there is some platform specific cause that might be leading to these errors.

Could you provide the os/compiler details for your system just so I have a point of reference? Thank you

evandengdbw commented 5 years ago

My system version is ubuntu16.04, and I use g++4.9.4 to compile them. But I use the master branch of openNetVM to build the NF, and get DPDK lib by following instructions provided by openNetVM.

evandengdbw commented 5 years ago

OK~

evandengdbw commented 5 years ago

OK.

koolzz commented 5 years ago

@onvm just curious what this reports for c++

onvm commented 5 years ago

@onvm just curious what this reports for c++

CI Message

Your results will arrive shortly

koolzz commented 5 years ago

We're aiming to include this into this month release. I'm still testing these, but the minor code nits I currently have is removing the commented out unused code in the cpp files and moving the architecture file into the README.md

Also don't worry about the recent API changes for ONVM I've already applied them to all NFD NFS on my branch so I'll submit a pr to your branch soon.

evandengdbw commented 5 years ago

OK! Thanks a lot for your help:):)

koolzz commented 5 years ago

@evandengdbw I've pinged you with some setup questions on slack, please respond when you have time

koolzz commented 5 years ago

Hey @evandengdbw can you please provide exact command line options for how you tester the NFs? I'm having trouble with testing the NF functionality, I can do Pktgen or pcap replay or anything else you tested with, I just need to know how to replicate the test environment.

If you could provide those in the form of command line snippets saying what to run or what to send that would be ideal. If you could also provide a short msg to each NF of whats supposed to happen when testing the NFs -> which should also be in READMEs. Ror example if the heavy_hitter_detection NF detection drops packets after X packets of certain type has been received add that to the heavy_hitter_detection NF README.md. This kind of simple explanation would fit great in the READMEs and this would make it a lot easier for people to understand how your NFs function before digging into the code for details.

evandengdbw commented 5 years ago

All right, I will write it soon :).

evandengdbw commented 5 years ago

@koolzz I just use speed_tester replay mode to test our NFs. I use the command to run speed_tester: ./go.sh 1 -d 2 -o ./pcap/64B_download.pcap and run our NF in this command: ./go.sh 2 -d 1

koolzz commented 5 years ago

@koolzz I just use speed_tester replay mode to test our NFs. I use the command to run speed_tester: ./go.sh 1 -d 2 -o ./pcap/64B_download.pcap and run our NF in this command: ./go.sh 2 -d 1

I've testing this with the heavy hitter detection NF. I'm seeing the NF just pass all the packets and nothing seems to change? I've waited until it received about 100Mil pkts. Can you tell me how to properly test it, to make it actually drop packets?

evandengdbw commented 5 years ago

When the number of SYN packets with the same source IP address reaches 100, the heavy hitter detection will drop the following SYN packets from the same source IP . I think we need more than 100 SYN packets with same source IP to trigger the drop process. I am sorry for being late to reply your message. Recently I am busy in preparing for the final exams.

koolzz commented 5 years ago

When the number of SYN packets with the same source IP address reaches 100, the heavy hitter detection will drop the following SYN packets from the same source IP . I think we need more than 100 SYN packets with same source IP to trigger the drop process. I am sorry for being late to reply your message. Recently I am busy in preparing for the final exams.

Yes that was my interpretation, does it count the same syn packet twice(when pcap replays again)? If not we might need other PCAP files. No problem take your time, good luck with finals

evandengdbw commented 5 years ago

Thanks. I think replaying the PCAP file with the same SYN packets for enough times is able to trigger the drop process.

koolzz commented 5 years ago

@evandengdbw let me know how to get it to drop packets because it didn't work for me, thank you.

evandengdbw commented 5 years ago

So sorry about the mistakes during parsing, I think now it can perform the functions smoothly using the pcap file 64B_download.pcap.

evandengdbw commented 5 years ago

It seems that the problem comes from the different sizes of u_long types of different operating systems. I replaced the corresponding types(u_long) with deterministic type(uint32_t).

koolzz commented 5 years ago

Thanks for the prompt fix I'll test soon

koolzz commented 5 years ago

@evandengdbw great stuff I've tested and it appears that SYN packets are all dropped and its only cycling other ACK and FIN packets. As a next step I want to verify that other NFs also work accordingly. Can you please add a short note to every NF README saying how to verify its functionality?

F.E something like this for the heavy hitter detection would be great ->

Testing

The Heavy Hitter Detection NF will start dropping SYN packets(from the same source IP address) after a certain threshold is reached, to verify this run these 2 NFs:

Run the Heavy Hitter Detection NF with:

./go.sh 1 -d 2

Run Speed Tester NF(to replay pcap file) with:

./go.sh 2 -d 1  -o pcap/64B_download.pcap
evandengdbw commented 5 years ago

Okkay~

koolzz commented 5 years ago

@evandengdbw I've tested a good number of NFs so far I only had an issue with the Super Spread Detection, I'm using an only SYN packets pcap but no packets are being dropped? Is there something I'm missing here about the pcap specifics? Thank you, and here is the list of NFs I've tested.

evandengdbw commented 5 years ago

There were some incomplete logic in Super Spreader Detection NF, I have complemented them already. And you can use similar method to verify the UDP Flood Detection NF by sending one UDP packet continuously.

evandengdbw commented 5 years ago

@koolzz The IP was lost and I added it already.