Dashboard page has 90+ % CPU utilization

[Rangak]

When on Dashboard page, CPU utilization on router is very high, The bytecount script is responsible for this. Figure out alternative (e.g no python or cython) for this script to bring CPU utilization down.

[Rangak]

Websockets would be an easy way to solve this. Instead of periodic XHR to fetch bytecount, open a websocket when user goes to Dashboard page. Then send back the bytecounts periodically over the websocket. One can also break up the response in the dashboard python script with this websocket to speed up the dashboard page load time. See also #274 comments. Such websockets implementation needs security review to avoid introduction of vulnerabilities.

[ThePletch]

I can work on this.

[Rangak]

Cool ! Feel free to ping me if I can answer any questions.

[ThePletch]

@Rangak Pretty much done with the client-side part of this, and the server-side part looks fairly simple (maintain a list of connected clients, send them updated bytecounts every n seconds, where n is a value that doesn't cripple the router), but the 'every n seconds' part is the challenge.

Is there a better way to regularly execute some code on lighttpd than just defining a cron job?

[Rangak]

Great ! Are you using https://github.com/nori0428/mod_websocket ? You could just modify the bytecount python script to wake up 'every n seconds' and send to mod_websocket. That would be the simplest thing to do, rather than start a cron job. There needs to be a way to stop the bytecount script when the user goes away from the dashboard page. Not sure how that would work.

[ThePletch]

I'm actually fairly unfamiliar with Lighttpd and am stuck on where/how to install mod_websocket.

Conceptually, this seems as simple as "run the bytecount script every n seconds, exit if there are no clients," just using mod_websocket's default 20-second ping to ensure that inactive clients/users not still on the page are disconnected. Given a typical router, I don't think that sending a few bytecount updates that aren't received will cause many issues.

[Rangak]

Were you able to progress with mod_websocket ? If not how are you testing the client side work you completed ? There are other lighttpd modules in use; looking at that will tell you how to install mod_websocket.

Regarding the exiting of bytecount script, your solution using mod_websocket's 20 second ping should be adequate.

[Rangak]

Can I help ?

[Rangak]

Hi, I am motivated to make progress on this and can help complete it. Perhaps we could discuss how to land this. Thanks.

[ThePletch]

Hey Ranga, Apologies for the lack of communication - I don't really have an excuse besides being somewhat nervous about this project.

My main obstacle is still confusion over installing the mod_websocket module, though I intend to give that a closer look this week - my plan is to have this completed by the end of the week. I will post here again if I have any more specific problems.

[ThePletch]

Javascript is complete and assumes the following interface:

• A websocket is open and listening on the server at the /cgi-bin/routerapi/bytecount URL.
• The websocket server sends data to all connected clients periodically in the previously-used JSON format.
• The websocket server accepts clients only from the same origin. This makes using CSRF headers unnecessary.
• The websocket server will consider them idle if a keepalive ping is not sent to the server every twenty seconds.

The bytecounts JS is, thankfully, quite cleanly written, so I can test that they will update properly if sent the correct data without wiring up the websockets server.

I intend to build the server side of this tomorrow evening and will, as before, post here if I have further difficulties with mod_websocket.

In the meantime, could you provide a link to an example of how to cause the bytecounts script to wake up every so often? I imagine sending the data to mod_websocket is fairly trivial, but I'm, as mentioned, quite unfamiliar with this server framework.

[Rangak]

Great. Look forward to it. Here is a bytecount script with changes so that it wakes up every 2 seconds. https://github.com/Rangak/OpenWireless/blob/master/routerapi/bytecount . Let me know if you have any questions or I can help further.

[ThePletch]

I'm deeply apologetic about this, but I think I lack the server expertise to actually finish this - I've done as much configuration as I can without actually installing/testing mod_websocket, but I can't find any information online about its functionality and I've hit a dead end with installing it after sinking about ten hours into it.

You can find the code I wrote thus far in #301.

[Rangak]

Not a problem. Thanks for bringing it this far. I think you are most of the way there so I will finish it from here on. This is an important improvement and your contribution is much appreciated.