Core Page on Web Stats

[kevindweb]

Added a core page to view where all the threads are located.

Summary:

In the web stats view of onvm, seen through ./onvm/go.sh 0,1,2 1 0xF8 -s web, previously, the page was empty. Now, we have information about the cores, which can be seen in card format. Right now each core only runs one process, but with shared cpu in the future, will show multiple items on the same core.

Usage: Run onvm web stats as shown above, and click on the Core Mappings page. This will be in the localhost (port 8080) of the node.

This PR includes
Resolves issues
Breaking API changes
Internal API changes
Usability improvements
Bug fixes
New functionality	👍
New NF/onvm_mgr args
Changes to starting NFs
Dependency updates
Web stats updates	👍

Merging notes:

• Dependencies: None

TODO before merging :

• [ ] Verify the Rx, Tx, and master thread is display correctly
• [ ] Test many NFs
• [ ] PR is ready for review

Review:

• Sanity checks, assigned to @koolzz @dennisafa

  • Run make in /onvm and /examples directories
  • For extra testing, run ./build_web.sh in onvm_web (if needed)
  • Code style, assigned to @koolzz @dennisafa

• Run linter

  • Check everything style related
  • Code design, assigned to @koolzz @dennisafa

• Check for memory leaks

  • Check that code is reusable
  • Code is clean, functions are concise
  • Verify that edge cases properly handled

• Performance, assigned to @koolzz @dennisafa

  • Run Speed Tester NF, report performance.

• Documentation, assigned to @koolzz @dennisafa

  • Check if the new changes are well documented, in both code and READMEs

[onvm]

In response to PR creation

CI Message

Your results will arrive shortly

[onvm]

In response to PR creation

CI Message

Run successful see results: [Results from nimbnode30] Median TX pps for Speed Tester: 35191355

Linter Passed

[koolzz]

Everything is minified and web stuff is rebuilt, I'll test later today

[kevindweb]

Awesome thanks, I tried to limit the changes in c code, and there is no added code to any loops so performance shouldn't drop a result (most of the changes are initialization).

[twood02]

Semi related question - do we currently show any info about what core an NF is running on in the text console stats? If not, we should

[koolzz]

@twood2 We don't but that's definitely something we should include for this release

[kevindweb]

check it out @onvm

[onvm]

check it out @onvm

CI Message

Your results will arrive shortly

[kevindweb]

@twood02 @koolzz Should I update the stdout stats to reflect the nf core in this pr? Or I can do it in another one, just seems relevant to this one, especially since I'm already updating onvm_mgr/stats.c

[onvm]

check it out @onvm

CI Message

Run successful see results: [Results from nimbnode30] Median TX pps for Speed Tester: 35173967

Linter Failed

examples/aes_decrypt/aes.h:176: #endif line should be "#endif // _AESH" [build/header_guard] [5] Total errors found: 1 examples/aes_encrypt/aes.h:185: #endif line should be "#endif // _AESH" [build/header_guard] [5] Total errors found: 1 examples/flow_table/flow_table.h:63: #endif line should be "#endif // _FLOW_TABLEH" [build/header_guard] [5] Total errors found: 1 examples/flow_table/msgbuf.h:71: #endif line should be "#endif // _MSGBUFH" [build/header_guard] [5] Total errors found: 1 examples/flow_table/openflow.h:969: #endif line should be "#endif // _OPENFLOWH" [build/header_guard] [5] examples/flow_table/openflow.h:50: Using deprecated casting style. Use static_cast(...) instead [readability/casting] [4] examples/flow_table/openflow.h:569: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:634: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:771: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:804: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:865: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:926: Extra space before ( in function call [whitespace/parens] [4] Total errors found: 8 examples/flow_table/sdn.c:334: { should almost always be at the end of the previous line [whitespace/braces] [4] examples/flow_table/sdn.c:365: Should have a space between // and comment [whitespace/comments] [4] Total errors found: 2 examples/flow_table/sdn.h:104: #endif line should be "#endif // _SDNH" [build/header_guard] [5] Total errors found: 1 examples/flow_table/sdn_pkt_list.h:122: #endif line should be "#endif // _SDN_PKT_LISTH" [build/header_guard] [5] Total errors found: 1 examples/flow_table/setupconn.h:53: #endif line should be "#endif // _SETUPCONNH" [build/header_guard] [5] Total errors found: 1 examples/speed_tester/speed_tester.c:364: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 onvm/onvm_mgr/main.c:109: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 onvm/onvm_mgr/onvm_init.c:107: { should almost always be at the end of the previous line [whitespace/braces] [4] onvm/onvm_mgr/onvm_init.c:114: { should almost always be at the end of the previous line [whitespace/braces] [4] onvm/onvm_mgr/onvm_init.c:116: { should almost always be at the end of the previous line [whitespace/braces] [4] Total errors found: 3 onvm/onvm_mgr/onvm_pkt.c:68: Are you taking an address of a cast? This is dangerous: could be a temp var. Take the address before doing the cast, rather than after [runtime/casting] [4] Total errors found: 1 onvm/onvm_mgr/onvm_stats.c:240: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] onvm/onvm_mgr/onvm_stats.c:278: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] Total errors found: 2 onvm/onvm_nflib/onvm_common.h:366: #endif line should be "#endif // _ONVM_COMMONH" [build/header_guard] [5] Total errors found: 1 onvm/onvm_nflib/onvm_config_common.h:208: #endif line should be "#endif // _ONVM_CONFIG_COMMONH" [build/header_guard] [5] Total errors found: 1 onvm/onvm_nflib/onvm_msg_common.h:61: #endif line should be "#endif // _ONVM_MSG_COMMONH" [build/header_guard] [5] Total errors found: 1 onvm/onvm_nflib/onvm_nflib.c:523: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] Total errors found: 1 onvm/onvm_nflib/onvm_pkt_common.c:98: Are you taking an address of a cast? This is dangerous: could be a temp var. Take the address before doing the cast, rather than after [runtime/casting] [4] Total errors found: 1 onvm/onvm_nflib/onvm_sc_common.h:70: #endif line should be "#endif // _ONVM_SC_COMMONH" [build/header_guard] [5] onvm/onvm_nflib/onvm_sc_common.h:70: Should have a space between // and comment [whitespace/comments] [4] Total errors found: 2 onvm/onvm_nflib/onvm_sc_mgr.h:79: #endif line should be "#endif // _ONVM_SC_MGRH" [build/header_guard] [5] Total errors found: 1

[kevindweb]

Welp it seems nicer when there's no errors but whatever.

[koolzz]

@twood02 @koolzz Should I update the stdout stats to reflect the nf core in this pr? Or I can do it in another one, just seems relevant to this one, especially since I'm already updating onvm_mgr/stats.c

I prefer a separate PR, the reason being is we want to make the stats code prettier (currently its a bit ugly). I was thinking of changing the description and value lines to global const char * values and overall cleanup. That can probably wait tho until shared CPU so we don't have to deal with conflicts, besides review shared CPU as that is higher priority 😉.

[kevindweb]

@onvm make sure we're all good

[onvm]

@onvm make sure we're all good

CI Message

Your results will arrive shortly

[onvm]

@onvm make sure we're all good

CI Message

Run successful see results: [Results from nimbnode30] Median TX pps for Speed Tester: 35165829

Linter Failed

examples/aes_decrypt/aes.h:176: #endif line should be "#endif // _AESH" [build/header_guard] [5] Total errors found: 1 examples/aes_encrypt/aes.h:185: #endif line should be "#endif // _AESH" [build/header_guard] [5] Total errors found: 1 examples/flow_table/flow_table.h:63: #endif line should be "#endif // _FLOW_TABLEH" [build/header_guard] [5] Total errors found: 1 examples/flow_table/msgbuf.h:71: #endif line should be "#endif // _MSGBUFH" [build/header_guard] [5] Total errors found: 1 examples/flow_table/openflow.h:969: #endif line should be "#endif // _OPENFLOWH" [build/header_guard] [5] examples/flow_table/openflow.h:50: Using deprecated casting style. Use static_cast(...) instead [readability/casting] [4] examples/flow_table/openflow.h:569: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:634: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:771: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:804: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:865: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:926: Extra space before ( in function call [whitespace/parens] [4] Total errors found: 8 examples/flow_table/sdn.c:334: { should almost always be at the end of the previous line [whitespace/braces] [4] examples/flow_table/sdn.c:365: Should have a space between // and comment [whitespace/comments] [4] Total errors found: 2 examples/flow_table/sdn.h:104: #endif line should be "#endif // _SDNH" [build/header_guard] [5] Total errors found: 1 examples/flow_table/sdn_pkt_list.h:122: #endif line should be "#endif // _SDN_PKT_LISTH" [build/header_guard] [5] Total errors found: 1 examples/flow_table/setupconn.h:53: #endif line should be "#endif // _SETUPCONNH" [build/header_guard] [5] Total errors found: 1 onvm/onvm_mgr/onvm_init.c:107: { should almost always be at the end of the previous line [whitespace/braces] [4] onvm/onvm_mgr/onvm_init.c:114: { should almost always be at the end of the previous line [whitespace/braces] [4] onvm/onvm_mgr/onvm_init.c:116: { should almost always be at the end of the previous line [whitespace/braces] [4] Total errors found: 3 onvm/onvm_mgr/onvm_pkt.c:68: Are you taking an address of a cast? This is dangerous: could be a temp var. Take the address before doing the cast, rather than after [runtime/casting] [4] Total errors found: 1 onvm/onvm_mgr/onvm_stats.c:240: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] Total errors found: 1 onvm/onvm_nflib/onvm_common.h:367: #endif line should be "#endif // _ONVM_COMMONH" [build/header_guard] [5] Total errors found: 1 onvm/onvm_nflib/onvm_config_common.h:208: #endif line should be "#endif // _ONVM_CONFIG_COMMONH" [build/header_guard] [5] Total errors found: 1 onvm/onvm_nflib/onvm_msg_common.h:61: #endif line should be "#endif // _ONVM_MSG_COMMONH" [build/header_guard] [5] Total errors found: 1 onvm/onvm_nflib/onvm_nflib.c:523: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] Total errors found: 1 onvm/onvm_nflib/onvm_pkt_common.c:98: Are you taking an address of a cast? This is dangerous: could be a temp var. Take the address before doing the cast, rather than after [runtime/casting] [4] Total errors found: 1 onvm/onvm_nflib/onvm_sc_common.h:70: #endif line should be "#endif // _ONVM_SC_COMMONH" [build/header_guard] [5] onvm/onvm_nflib/onvm_sc_common.h:70: Should have a space between // and comment [whitespace/comments] [4] Total errors found: 2 onvm/onvm_nflib/onvm_sc_mgr.h:79: #endif line should be "#endif // _ONVM_SC_MGRH" [build/header_guard] [5] Total errors found: 1

[onvm]

@onvm make sure we're all good

CI Message

Run successful see results: [Results from nimbnode30] Median TX pps for Speed Tester: 35165829

Linter Failed

examples/aes_decrypt/aes.h:176: #endif line should be #endif

CI Message

Your results will arrive shortly

[onvm]

@onvm make sure we're all good

CI Message

Run successful see results: [Results from nimbnode30] Median TX pps for Speed Tester: 35165829

Linter Failed

examples/aes_decrypt/aes.h:176: #endif line should be #endif

CI Message

Run successful see results: [Results from nimbnode30] Median TX pps for Speed Tester: 35174025

Linter Failed

examples/aes_decrypt/aes.h:176: #endif line should be "#endif // _AESH" [build/header_guard] [5] Total errors found: 1 examples/aes_encrypt/aes.h:185: #endif line should be "#endif // _AESH" [build/header_guard] [5] Total errors found: 1 examples/flow_table/flow_table.h:63: #endif line should be "#endif // _FLOW_TABLEH" [build/header_guard] [5] Total errors found: 1 examples/flow_table/msgbuf.h:71: #endif line should be "#endif // _MSGBUFH" [build/header_guard] [5] Total errors found: 1 examples/flow_table/openflow.h:969: #endif line should be "#endif // _OPENFLOWH" [build/header_guard] [5] examples/flow_table/openflow.h:50: Using deprecated casting style. Use static_cast(...) instead [readability/casting] [4] examples/flow_table/openflow.h:569: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:634: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:771: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:804: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:865: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:926: Extra space before ( in function call [whitespace/parens] [4] Total errors found: 8 examples/flow_table/sdn.c:334: { should almost always be at the end of the previous line [whitespace/braces] [4] examples/flow_table/sdn.c:365: Should have a space between // and comment [whitespace/comments] [4] Total errors found: 2 examples/flow_table/sdn.h:104: #endif line should be "#endif // _SDNH" [build/header_guard] [5] Total errors found: 1 examples/flow_table/sdn_pkt_list.h:122: #endif line should be "#endif // _SDN_PKT_LISTH" [build/header_guard] [5] Total errors found: 1 examples/flow_table/setupconn.h:53: #endif line should be "#endif // _SETUPCONNH" [build/header_guard] [5] Total errors found: 1 onvm/onvm_mgr/onvm_init.c:107: { should almost always be at the end of the previous line [whitespace/braces] [4] onvm/onvm_mgr/onvm_init.c:114: { should almost always be at the end of the previous line [whitespace/braces] [4] onvm/onvm_mgr/onvm_init.c:116: { should almost always be at the end of the previous line [whitespace/braces] [4] Total errors found: 3 onvm/onvm_mgr/onvm_pkt.c:68: Are you taking an address of a cast? This is dangerous: could be a temp var. Take the address before doing the cast, rather than after [runtime/casting] [4] Total errors found: 1 onvm/onvm_mgr/onvm_stats.c:240: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] Total errors found: 1 onvm/onvm_nflib/onvm_common.h:367: #endif line should be "#endif // _ONVM_COMMONH" [build/header_guard] [5] Total errors found: 1 onvm/onvm_nflib/onvm_config_common.h:208: #endif line should be "#endif // _ONVM_CONFIG_COMMONH" [build/header_guard] [5] Total errors found: 1 onvm/onvm_nflib/onvm_msg_common.h:61: #endif line should be "#endif // _ONVM_MSG_COMMONH" [build/header_guard] [5] Total errors found: 1 onvm/onvm_nflib/onvm_nflib.c:523: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] Total errors found: 1 onvm/onvm_nflib/onvm_pkt_common.c:98: Are you taking an address of a cast? This is dangerous: could be a temp var. Take the address before doing the cast, rather than after [runtime/casting] [4] Total errors found: 1 onvm/onvm_nflib/onvm_sc_common.h:70: #endif line should be "#endif // _ONVM_SC_COMMONH" [build/header_guard] [5] onvm/onvm_nflib/onvm_sc_common.h:70: Should have a space between // and comment [whitespace/comments] [4] Total errors found: 2 onvm/onvm_nflib/onvm_sc_mgr.h:79: #endif line should be "#endif // _ONVM_SC_MGRH" [build/header_guard] [5] Total errors found: 1

[kevindweb]

@koolzz We should be good now. I made a fix in the javascript so we don't even need a nf stopping flag in onvm_stats.c. I updated the style nits you asked for as well.

[koolzz]

@kevindweb Tested, core page is nice. Everything seems to work as expected. MGR thread proly needs to be set to something like mgr aux(stats) or something. Might be cool to actually show cores allocated from the onvm_mgr and say that they are idle rn. Also we would want to know which cpu socket the core is on (basically a few future improvements, probably not in this pr)

Also noticed possibly a small js bug (the highlight under the 3 different tabs isn't under the right tab) See screenschot

(See some of my code comments above)

[kevindweb]

Thanks for reviewing, ya we can work on some of those functionality updates this summer. I've looked into the tab issue before when testing the old web stats. You need to click somewhere else and it will update. I'll see if there is an easy fix. Also, I'm making the updates you requested and pulling develop so I'll push again in a minute.

[koolzz]

@onvm Sanity check

[onvm]

@onvm Sanity check

CI Message

Your results will arrive shortly

[onvm]

@onvm Sanity check

CI Message

Run successful see results: [Results from nimbnode30] Median TX pps for Speed Tester: 35184823

Linter Failed

examples/arp_response/arp_response.c:284: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 examples/flow_table/openflow.h:50: Using deprecated casting style. Use static_cast(...) instead [readability/casting] [4] examples/flow_table/openflow.h:569: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:634: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:771: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:804: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:865: Extra space before ( in function call [whitespace/parens] [4] examples/flow_table/openflow.h:926: Extra space before ( in function call [whitespace/parens] [4] Total errors found: 7 examples/nf_router/nf_router.c:173: Lines should be <= 120 characters long [whitespace/line_length] [5] Total errors found: 1 onvm/onvm_mgr/onvm_pkt.c:68: Are you taking an address of a cast? This is dangerous: could be a temp var. Take the address before doing the cast, rather than after [runtime/casting] [4] Total errors found: 1 onvm/onvm_mgr/onvm_stats.c:306: If an else has a brace on one side, it should have it on both [readability/braces] [5] Total errors found: 1 onvm/onvm_nflib/onvm_pkt_common.c:98: Are you taking an address of a cast? This is dangerous: could be a temp var. Take the address before doing the cast, rather than after [runtime/casting] [4] Total errors found: 1