Consolidates All NF Structs

[koolzz]

This PR merges together onvm_nf_info and onvm_nf structs, into a single onvm_nf struct with internal variable groupings for readability, also renames key structs.

Summary:

• The old onvm_nf_info is re purposed as onvm_nf_init_cfg a struct used to initialize the NF with the manager. This struct is passed to onvm_mgr and is then freed when initialization is complete.
• In attempt to merge these structs the usage of onvm_nf_info has been removed from all internal and external API calls. Excluding the startup sequence.
• The struct onvm_nf_context was renamed to onvm_nf_local_ctx
• Added a nf_function_table struct that will hold all the NF defined callback functions.
• Changed all NF facing API calls to pass onvm_nf_local_ctx instead of onvm_nf as NFs might need access to the keep_running variable.

Usage:

This PR includes
Resolves issues
Breaking API changes	👍
Internal API changes	👍
Usability improvements	👍
Bug fixes
New functionality
New NF/onvm_mgr args
Changes to starting NFs
Dependency updates
Web stats updates

Merging notes:

• Dependencies: None

TODO before merging :

• [x] PR is ready for review

Test Plan:

Regular testing + pktgen

Review:

Please review, release is near

[onvm]

In response to PR creation

CI Message

Your results will arrive shortly

[onvm]

In response to PR creation

CI Message

Error: ERROR: Failed to fetch results from nimbnode30

Linter Output

examples/aes_decrypt/aesdecrypt.c:176: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/aes_encrypt/aesencrypt.c:176: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/arp_response/arp_response.c:268: Lines should be <= 120 characters long [whitespace/line_length] [5] examples/arp_response/arp_response.c:284: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 2 examples/bridge/bridge.c:145: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/flow_table/flow_table.c:207: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/flow_tracker/flow_tracker.c:294: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/load_balancer/load_balancer.c:491: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/load_generator/load_generator.c:236: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/nf_router/nf_router.c:173: Lines should be <= 120 characters long [whitespace/line_length] [5] examples/nf_router/nf_router.c:215: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 2 examples/payload_scan/payload_scan.c:95: Lines should be <= 120 characters long [whitespace/line_length] [5] examples/payload_scan/payload_scan.c:114: Almost always, snprintf is better than strcpy [runtime/printf] [4] examples/payload_scan/payload_scan.c:126: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 3 examples/scaling_example/scaling.c:240: Lines should be <= 120 characters long [whitespace/line_length] [5] examples/scaling_example/scaling.c:382: Should have a space between // and comment [whitespace/comments] [4] examples/scaling_example/scaling.c:430: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] Total errors found: 3 examples/simple_forward/forward.c:157: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/test_flow_dir/test_flow_dir.c:158: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 onvm/onvm_mgr/onvm_args.c:193: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] Total errors found: 1 onvm/onvm_mgr/onvm_nf.c:204: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 onvm/onvm_mgr/onvm_stats.c:306: If an else has a brace on one side, it should have it on both [readability/braces] [5] onvm/onvm_mgr/onvm_stats.c:409: Lines should be <= 120 characters long [whitespace/line_length] [5] onvm/onvm_mgr/onvm_stats.c:410: Lines should be <= 120 characters long [whitespace/line_length] [5] onvm/onvm_mgr/onvm_stats.c:412: Lines should be <= 120 characters long [whitespace/line_length] [5] onvm/onvm_mgr/onvm_stats.c:416: Lines should be <= 120 characters long [whitespace/line_length] [5] onvm/onvm_mgr/onvm_stats.c:417: Lines should be <= 120 characters long [whitespace/line_length] [5] onvm/onvm_mgr/onvm_stats.c:418: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 7 onvm/onvm_nflib/onvm_common.h:70: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 onvm/onvm_nflib/onvm_nflib.c:98: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] onvm/onvm_nflib/onvm_nflib.c:254: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] onvm/onvm_nflib/onvm_nflib.c:269: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] onvm/onvm_nflib/onvm_nflib.c:1180: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] onvm/onvm_nflib/onvm_nflib.c:1186: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] onvm/onvm_nflib/onvm_nflib.c:1190: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] Total errors found: 6 onvm/onvm_nflib/onvm_nflib.h:100: Lines should be <= 120 characters long [whitespace/line_length] [5] onvm/onvm_nflib/onvm_nflib.h:100: Should have a space between // and comment [whitespace/comments] [4] Total errors found: 2

[koolzz]

@onvm perf?

[onvm]

@onvm perf?

CI Message

Your results will arrive shortly

[koolzz]

Is this consistent @onvm

[onvm]

Is this consistent @onvm

CI Message

Your results will arrive shortly

[koolzz]

@onvm what about no packing

[onvm]

@onvm what about no packing

CI Message

Your results will arrive shortly

[onvm]

@onvm what about no packing

CI Message

Error: ERROR: Failed to fetch results from nimbnode30

Linter Output

examples/arp_response/arp_response.c:282: Lines should be <= 120 characters long [whitespace/line_length] [5] examples/arp_response/arp_response.c:284: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 2 examples/nf_router/nf_router.c:173: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/payload_scan/payload_scan.c:95: Lines should be <= 120 characters long [whitespace/line_length] [5] examples/payload_scan/payload_scan.c:114: Almost always, snprintf is better than strcpy [runtime/printf] [4] examples/payload_scan/payload_scan.c:126: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 3 examples/scaling_example/scaling.c:95: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] examples/scaling_example/scaling.c:375: Should have a space between // and comment [whitespace/comments] [4] examples/scaling_example/scaling.c:422: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] Total errors found: 3 onvm/onvm_mgr/onvm_args.c:193: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] Total errors found: 1 onvm/onvm_mgr/onvm_nf.c:204: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 onvm/onvm_mgr/onvm_stats.c:306: If an else has a brace on one side, it should have it on both [readability/braces] [5] onvm/onvm_mgr/onvm_stats.c:409: Lines should be <= 120 characters long [whitespace/line_length] [5] onvm/onvm_mgr/onvm_stats.c:410: Lines should be <= 120 characters long [whitespace/line_length] [5] onvm/onvm_mgr/onvm_stats.c:412: Lines should be <= 120 characters long [whitespace/line_length] [5] onvm/onvm_mgr/onvm_stats.c:416: Lines should be <= 120 characters long [whitespace/line_length] [5] onvm/onvm_mgr/onvm_stats.c:417: Lines should be <= 120 characters long [whitespace/line_length] [5] onvm/onvm_mgr/onvm_stats.c:418: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 7 onvm/onvm_nflib/onvm_common.h:70: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 onvm/onvm_nflib/onvm_nflib.c:98: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] onvm/onvm_nflib/onvm_nflib.c:257: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] onvm/onvm_nflib/onvm_nflib.c:270: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] onvm/onvm_nflib/onvm_nflib.c:1143: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] onvm/onvm_nflib/onvm_nflib.c:1149: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] onvm/onvm_nflib/onvm_nflib.c:1153: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] Total errors found: 6 onvm/onvm_nflib/onvm_nflib.h:100: Lines should be <= 120 characters long [whitespace/line_length] [5] onvm/onvm_nflib/onvm_nflib.h:100: Should have a space between // and comment [whitespace/comments] [4] Total errors found: 2

[koolzz]

@onvm my bad try again

[onvm]

@onvm my bad try again

CI Message

Your results will arrive shortly

[koolzz]

@onvm agaaaain

[onvm]

@onvm agaaaain

CI Message

Your results will arrive shortly

[onvm]

@onvm agaaaain

CI Message

Error: ERROR: Failed to fetch results from nimbnode30

Linter Output

examples/arp_response/arp_response.c:282: Lines should be <= 120 characters long [whitespace/line_length] [5] examples/arp_response/arp_response.c:284: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 2 examples/nf_router/nf_router.c:173: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/payload_scan/payload_scan.c:95: Lines should be <= 120 characters long [whitespace/line_length] [5] examples/payload_scan/payload_scan.c:114: Almost always, snprintf is better than strcpy [runtime/printf] [4] examples/payload_scan/payload_scan.c:126: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 3 examples/scaling_example/scaling.c:95: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] examples/scaling_example/scaling.c:375: Should have a space between // and comment [whitespace/comments] [4] examples/scaling_example/scaling.c:422: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] Total errors found: 3 onvm/onvm_mgr/onvm_args.c:193: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] Total errors found: 1 onvm/onvm_mgr/onvm_nf.c:204: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 onvm/onvm_mgr/onvm_stats.c:306: If an else has a brace on one side, it should have it on both [readability/braces] [5] onvm/onvm_mgr/onvm_stats.c:409: Lines should be <= 120 characters long [whitespace/line_length] [5] onvm/onvm_mgr/onvm_stats.c:410: Lines should be <= 120 characters long [whitespace/line_length] [5] onvm/onvm_mgr/onvm_stats.c:412: Lines should be <= 120 characters long [whitespace/line_length] [5] onvm/onvm_mgr/onvm_stats.c:416: Lines should be <= 120 characters long [whitespace/line_length] [5] onvm/onvm_mgr/onvm_stats.c:417: Lines should be <= 120 characters long [whitespace/line_length] [5] onvm/onvm_mgr/onvm_stats.c:418: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 7 onvm/onvm_nflib/onvm_common.h:70: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 onvm/onvm_nflib/onvm_nflib.c:98: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] onvm/onvm_nflib/onvm_nflib.c:257: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] onvm/onvm_nflib/onvm_nflib.c:270: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] onvm/onvm_nflib/onvm_nflib.c:1143: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] onvm/onvm_nflib/onvm_nflib.c:1149: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] onvm/onvm_nflib/onvm_nflib.c:1153: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] Total errors found: 6 onvm/onvm_nflib/onvm_nflib.h:100: Lines should be <= 120 characters long [whitespace/line_length] [5] onvm/onvm_nflib/onvm_nflib.h:100: Should have a space between // and comment [whitespace/comments] [4] Total errors found: 2

[koolzz]

@onvm more

[onvm]

@onvm more

CI Message

Your results will arrive shortly

[koolzz]

@onvm lint me

[onvm]

@onvm lint me

CI Message

Your results will arrive shortly

[koolzz]

@onvm oi m8

[onvm]

@onvm oi m8

CI Message

Your results will arrive shortly

[koolzz]

@onvm testy test

[onvm]

@onvm testy test

CI Message

Your results will arrive shortly

[koolzz]

@onvm more

[onvm]

@onvm more

CI Message

Your results will arrive shortly

[kevindweb]

Don't know if you're completely ready for review, but I had success with 2 speed_testers, basic_monitor, and Pktgen running with this pr. For the two speed_testers running with all this, I had above 49 million pps for both. Just as a note, I ran with the go script suggestion ./go.sh 0,1,2,3 3 0xF0 -a 0x7f000000000 -s stdout. Ran with my usual secondary test of adding and removing tons of speed_testers and received no performance degradation, which was awesome. Not really sure which recent pr contributed to this the most, but even with 3 speed_testers running, I received 49 million+ per NF. Very pleased with the new changes to all recent prs (side note).

[koolzz]

This is mostly just reorganizing & renaming but its a lot of changes. Pinging everyone to review as this is a bit high priority. @nks5295 @twood02 @kevindweb @dennisafa

[koolzz]

@onvm give me the linty lint

[onvm]

@onvm give me the linty lint

CI Message

Your results will arrive shortly

[koolzz]

@onvm fixed some linty lint errors

[onvm]

@onvm fixed some linty lint errors

CI Message

Your results will arrive shortly

[koolzz]

@onvm Okay but now though

[onvm]

@onvm Okay but now though

CI Message

Your results will arrive shortly

[koolzz]

@onvm only speed tester works will you work

[onvm]

@onvm only speed tester works will you work

CI Message

Your results will arrive shortly

[onvm]

@onvm only speed tester works will you work

CI Message

Error: ERROR: Failed to fetch results from nimbnode30

Linter Output

onvm/onvm_nflib/onvm_nflib.c:465: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1

[koolzz]

I guess thats a no

[koolzz]

@onvm do your thing

[onvm]

@onvm do your thing

CI Message

Your results will arrive shortly

[onvm]

@onvm do your thing

CI Message

Error: ERROR: Failed to analyze results from nimbnode30

Linter Output

examples/aes_decrypt/aesdecrypt.c:173: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/aes_encrypt/aesencrypt.c:173: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/arp_response/arp_response.c:266: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/basic_monitor/monitor.c:161: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/bridge/bridge.c:142: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/firewall/firewall.c:198: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/flow_table/flow_table.c:208: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/flow_tracker/flow_tracker.c:292: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/load_balancer/load_balancer.c:488: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/load_generator/load_generator.c:233: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/nf_router/nf_router.c:212: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/scaling_example/scaling.c:220: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/simple_forward/forward.c:154: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/speed_tester/speed_tester.c:279: Lines should be <= 120 characters long [whitespace/line_length] [5] examples/speed_tester/speed_tester.c:517: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] Total errors found: 2 examples/test_flow_dir/test_flow_dir.c:155: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 onvm/onvm_nflib/onvm_nflib.c:453: Lines should be <= 120 characters long [whitespace/line_length] [5] onvm/onvm_nflib/onvm_nflib.c:466: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 2

[koolzz]

@onvm you happy

[onvm]

@onvm you happy

CI Message

Your results will arrive shortly

[koolzz]

@onvm lets do 3 out of 3

[onvm]

@onvm lets do 3 out of 3

CI Message

Your results will arrive shortly