Virtual Machine Statistical Reports

[GoogleCodeExporter]

Hi,

Is there any future plans to add some sort of statistical reporting for VMs in 
phpvirtualbox? Please advise.

Thanks,
Ant

Original issue reported on code.google.com by anthony....@powellitc.com on 28 Oct 2011 at 10:51

[GoogleCodeExporter]

Hi,

There are plans to do it at some point. But it is probably not going to happen 
any time in the near future.

Original comment by imooreya...@gmail.com on 30 Oct 2011 at 2:05

[GoogleCodeExporter]

I would like to see a memory usage breakdown per vm.... perhaps a bar graph 
(similar to system / in-use / reserved) or a pie chart

Original comment by brickey....@gmail.com on 8 Nov 2011 at 1:11

[GoogleCodeExporter]

to investigate the possibility of obtaining such information... I opened a 
ticket with  VirtualBox (https://www.virtualbox.org/ticket/9872).
They indicated that I should investigate the 'VBoxManage metrics' command.

Original comment by brickey....@gmail.com on 9 Nov 2011 at 11:14

[GoogleCodeExporter]

I found the VirtualBox API approach to do this... I am testing some code... 
will post when confirmed.

Original comment by brickey....@gmail.com on 11 Nov 2011 at 3:52

[GoogleCodeExporter]

changes to two files (pretty sure this was all i changed)...

todo:
- currently only shows *HOST* CPU (user+kernel) and memory usage (much more is 
available... but these three were MY primary interest)
- currently only shows when guest VM is selected... no aggregate views 
(charts/graphs) to indicate which VMs use the most
- currently uses the chipset icon... a pretty bar graph icon would be nice
- not the most efficient code (multiple queries, each call ensures that metrics 
are enabled, etc)... my days are in c#, not php... I just wanted it to work :)

###################################
###   panes/tabVMDetails.html   ###
###################################

                                        <table id='vboxDetailsTableBoxSystem' class='vboxDetailsTable vboxDetailsTableBox ui-corner-all'>
                        ...
                                        </table>
+                                       <table id='vboxDetailsTableBoxMetrics' 
class='vboxDetailsTable vboxDetailsTableBox ui-corner-all'>
+                                               <tr class='vboxDetailsHead'>
+                                               <th class='vboxDetailsSection' 
colspan='2'>
+                                                       <div>
+                                                               <img 
style='float:left;  margin-right: 3px;' src='images/vbox/chipset_16px.png' />
+                                                               <span 
style='float:left'>Performance Metrics</span>
+                                                               <span 
class='vboxDetailsUpDnImgSpan' style='float:right'>
+                                                                       <img 
src='images/arrow_grad_up.png' />
+                                                               </span>
+                                                       </div>
+                                               </th>
+                                               </tr>
+                                       </table>
                                </td>
                                <td id='vboxDetailsTableBoxPreviewSpacer'>
                                        <!-- Spacer -->
                                <td>

...

        // HW advanced config enabled?
        if(advancedView) {
                $(vboxDetailsTable).append(__vboxDetailRow(trans("VCPU"), (data['HWVirtExProperties'].Enabled ? trans('Enabled') : trans('Disabled')))).append(__vboxDetailRow(trans("Nested Paging"), (data['HWVirtExProperties'].NestedPaging ? trans('Enabled') : trans('Disabled')))).append(__vboxDetailRow(trans("Large Pages"), (data['HWVirtExProperties'].LargePages? trans('Enabled') : trans('Disabled'))));
                $(vboxDetailsTable).append(__vboxDetailRow(trans("Exclusive use of the hardware virtualization extensions"), (data['HWVirtExProperties'].Exclusive ? trans('Enabled') : trans('Disabled')))).append(__vboxDetailRow(trans("VT-x VPID (Intel only)"), (data['HWVirtExProperties'].VPID ? trans('Enabled') : trans('Disabled'))));
        }

+       // Performance Metrics
+       //////////////////////
+       if(boxes) {
+
+               // append blank footer row
+               $(vboxDetailsTable).append(__vboxDetailRow('',''));
+
+               var vboxDetailsTable = $('#vboxDetailsTableBoxMetrics');
+               $(vboxDetailsTable).find('tr.vboxDetailRow').empty().remove();
+
+               
if($('#vboxIndex').data('vboxCookies')["vboxSectionShowMetrics"] == 0) {
+                       $(vboxDetailsTable).hide();
+               } else {
+                       $(vboxDetailsTable).show();
+               }
+
+       } else {
+               
$(vboxDetailsTable).append(__vboxDetailHeader('chipset_16px.png',trans("Metrics"
)));
+       }
+
+       for($i = 0; $i < data['performance'].length; $i++) {
+               
$(vboxDetailsTable).append(__vboxDetailRow(data['performance'][$i]['name'], 
data['performance'][$i]['text']));
+       }

        // Preview box

#################################
###   lib/vboxconnector.php   ###
#################################

                // Shared Folders
                $data['sharedFolders'] = $this->__getCachedMachineData('__getSharedFolders',@$args['vm'],$machine,@$args['force_refresh']);

+               // Performance Data
+               $data['performance'] = 
$this->__getCachedMachineData('__getPerformanceMetrics',@$args['vm'],$machine,@$
args['force_refresh']);

                // USB Filters
                $data['USBController'] = $this->__getCachedMachineData('__getUSBController',@$args['vm'],$machine,@$args['force_refresh']);

/**
 * Return a list of performance metrics for machine $m
 *
 * @param IMachine $m virtual machine
 * @return array of performance metric information
 */
private function __getPerformanceMetrics(&$m) {

        /* IMachine -> Parent : iVirtualBox */
        $vb = $m->parent;

        /* iVirtualBox -> getPerformanceCollector() : IPerformanceCollector */
        $c = $vb->getPerformanceCollector();

        /* ensure that metrics are enabled (TODO: move this) */
        $affectedMetrics = array();
        $c->enableMetrics(null, null, $affectedMetrics);

        /* define the metrics of interest */
        $metricNames = array(
                          "CPU/Load/User",
                          "CPU/Load/Kernel",
                          "RAM/Usage/Used"
                          );

        /* define the object to query */
        $queryObjects = array( $m );

        /* not sure why I can't just query once, but this at least works reliably */
        $return = array();
        for($i = 0; $i < count($metricNames); $i++) {

                /* Query Metrics! */
                $pmds = $c->queryMetricsData($metricNames[$i], $queryObjects);

                $return[$i] = array(
                                        'name'   => $metricNames[$i],
                                        'value'  => $pmds[0][0],
                                        'scale'  => $pmds[4][0],
                                        'unit'   => $pmds[3][0],
                                        'text'   => ( $pmds[0][0] / $pmds[4][0] ) . " " . $pmds[3][0]
                                );
        }

        return $return;
}

Original comment by brickey....@gmail.com on 12 Nov 2011 at 4:39

[GoogleCodeExporter]

Do you have any screen shots after implementing the code? Thanks Ant

Original comment by anthony....@powellitc.com on 14 Nov 2011 at 11:39

[GoogleCodeExporter]

Screenshot taken after adding a few more metrics to $metricNames

guest CPU isn't as interesting as I thought it'd be... though I do feel guest 
RAM (primarily, the % used/free) will be useful... also interesting to see that 
ballooning is either off, or having no effect.

Original comment by brickey....@gmail.com on 14 Nov 2011 at 1:29

Attachments:

• [phpVB Metrics.png](https://storage.googleapis.com/google-code-attachments/phpvirtualbox/issue-433/comment-7/phpVB Metrics.png)

[GoogleCodeExporter]

i should note... screenshot was taken while running windows updates (thus the 
high CPU) :)

Original comment by brickey....@gmail.com on 14 Nov 2011 at 1:30

[GoogleCodeExporter]

just noticed I'm using > 100% of host CPU (99.7 + 0.4)... interesting... 
probably the dual cores.

I will have to see what happens to a VM which has more than 1 guest CPU (don't 
think any of mine are configured as such).

Original comment by brickey....@gmail.com on 14 Nov 2011 at 5:07

[GoogleCodeExporter]

I'm pretty sure the sample rate here is about 1 millisecond. So I think you're 
getting the CPU load in that one instant. From the API docs:

Data collection continues behind the scenes after call to queryMetricsData. The 
return data can be seen as the snapshot of the current state at the time of 
queryMetricsData call. The internally kept metric values are not cleared by the 
call. This makes possible querying different subsets of metrics or aggregates 
with subsequent calls. If periodic querying is needed it is highly suggested to 
query the values with interval*count period to avoid confusion. This way a 
completely new set of data values will be provided by each query. 

The whole thing seems a bit ambiguous to me. That's why I've put off going down 
the rabbit hole of VM performance metrics data for so long.

Original comment by imooreya...@gmail.com on 14 Nov 2011 at 8:32

[GoogleCodeExporter]

As I noted, there are a few opportunities to optimize the code.. As I'm not 
familiar with the language or debugging options, I simply chose the option that 
"got it working"... I would guess that putting them all in the same request 
(thus eliminating calls for each metric) would make them more "atomic".... that 
said, I could care less about tiny discrepancies.

What I think I would like most of all:
- automatically refresh the metrics (currently I have to switch to another VM, 
then back, to refresh the values)
- Visualizations... bar/pie graph of the host CPU/RAM usage, broken down by VM. 
I'm fine with this being a snapshot (no concerns about storing/retrieving 
history), as long as it can refresh easily.
- Visualize relations among memory... I'd like to better understand how the 
host RAM is being used between ballooning, guest total/shared/free, etc. This 
may also be applicable to CPUs (as I said, my guests all have one CPU) to see 
how multiple cores are being used across many VMs.

But... the code grabs the info very effectively... I was comparing the VBox 
metrics, guest's task manager, and hosts 'top' results... they all seemed 
pretty close (always hard to tell when dealing with delayed refresh rates)... 
as I said, nothing so far off as to warrant an investigation.

Original comment by brickey....@gmail.com on 14 Nov 2011 at 9:13

[GoogleCodeExporter]

I'm not knocking what you did or how you did it. I just don't know if doing it 
that way is accurate or not. If you're saying that it matches up with what the 
guest and host are reporting, I guess it must be accurate. When I said "The 
whole thing..etc.." I was referring to the VirtualBox API documentation, not 
your code.

Original comment by imooreya...@gmail.com on 16 Nov 2011 at 2:39

[GoogleCodeExporter]

So I wanted to try this out and added the code but I don't see the 
vboxDetailsTableBoxMetrics table. It's in the file but I think there's 
something missing that wasn't provided. Can you help me figure this out?

Original comment by toddk...@gmail.com on 6 Jan 2012 at 9:47

Attachments:

• tabVMDetails.html
• vboxconnector.php

[GoogleCodeExporter]

ToddK...@gmail:

I noticed that the vboxconnector.php looked a little different than mine... i 
should've noted that the changes were based on 4.1-4... that said, I think I 
found the reason it's not working.

in lib/vboxconnector.php, seems you added the function 
(__getPerformanceMetrics) fine... but added the performance data 
($data['performance'] = $this->__getCachedMachineData...) to the SaveVMRunning 
function (line 806) instead of the GetVMDetails function (line 2449)... try 
moving the code from 806 to 2449 and see how it goes.

I also found that I adjusted the line (since the earlier submission) in the 
panes/tabVMDetails.html to include a percentage...
from: 
$(vboxDetailsTable).append(__vboxDetailRow(data['performance'][$i]['name'], 
data['performance'][$i]['text']));
to  : 
$(vboxDetailsTable).append(__vboxDetailRow(data['performance'][$i]['name'], 
data['performance'][$i]['value'] . "%" ));

I also found that restarting apache and clearing cache was useful in seeing the 
changes I made to code.

Thanks,
-Scott

Original comment by brickey....@gmail.com on 8 Jan 2012 at 4:02

[GoogleCodeExporter]

brickey....@gmail

Ok I got it working but without your "%" modification. I get this error when 
making that adjustment.

missing name after . operator
[Break On This Error]   

...lRow(data['performance'][$i]['name'], data['performance'][$i]['value'] . "%" 
))

jquery...-min.js (line 445, col 113)

I'm not sure what the issue is there, but I got the original code working

Original comment by toddk...@gmail.com on 9 Jan 2012 at 3:16

[GoogleCodeExporter]

every time I stop my guest and restart it I have to go back to the command line 
and issue VBoxManage collect <guest_name>

Once I do that I start seeing data in the performance section. if I don't it's 
all zero's. I'm wondering if I have to add in vbox connector a call to 
setupMetrics to set the period and count? have you run into this also?

Original comment by toddk...@gmail.com on 13 Jan 2012 at 2:05

[GoogleCodeExporter]

I do not recall. I thought the getVMDetails included a line of code to enable 
stats just as you did. It wasn't ideal for performance but I wasn't really 
worried about that.

Original comment by brickey....@gmail.com on 13 Jan 2012 at 4:00

[GoogleCodeExporter]

Original comment by simonsmi...@gmail.com on 19 Apr 2012 at 11:01

• Added labels: Type-Enhancement
• Removed labels: Type-Support_Request

[GoogleCodeExporter]

This is not good for 4.1.8 

Original comment by closerw...@gmail.com on 18 May 2012 at 6:14

[GoogleCodeExporter]

i am in the process of putting in a Performance TAB for each VM with charts :)
i would like some advice wat charts people would like showing?
currently i have memory used and CPU used, any more?

Original comment by simonsmi...@gmail.com on 19 May 2012 at 2:00

[GoogleCodeExporter]

Hi!

Should we expect anything (CPU, memory usage etc.) in nearest feature releases?

Original comment by Boris.Ko...@gmail.com on 15 Oct 2012 at 11:15

[GoogleCodeExporter]

Hi,
i have been working on this for a while now and sadly i havnt moved any further 
due to work and no time...
its generates all charts using googles chart code, but i havnt pushed it yet
another issue i had was my developement machine died on me and i lost my code, 
so im having to start a fresh again
Simon

Original comment by simonsmi...@gmail.com on 15 Oct 2012 at 11:19

[GoogleCodeExporter]

That's sad :(

Original comment by Boris.Ko...@gmail.com on 15 Oct 2012 at 6:51

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Good to see efforts towards the use of the VirtualBox-Metrics-System being 
made! :)
On the topic of graph generation: Has the popular rrdtool[1] been considered 
for this? I've seen that there is a php-extension to facilitate the connection 
to rrdtool[2].
While the extension is not yet packaged with Debian it seems to be easy to 
build.

According to my use and experience rrdtool is quite usefull to generate 
monitoring and metrics graphs of any sort, especially timerow and compressed 
timerow graphs. The usage is, after you get used to it, only a smaller 
annoyance, anyhow compared to the effort neccessary to recreate similar 
functionality probably worth it.

[1]http://oss.oetiker.ch/rrdtool/
[2]http://www.php.net/manual/en/book.rrd.php

Original comment by johanne...@web.de on 16 Oct 2012 at 9:25

[GoogleCodeExporter]

Hi,
i will have alook into the rrdtool - i forgot about that php extension
I just simply used googles chart tools - https://developers.google.com/chart/
Simon

Original comment by simonsmi...@gmail.com on 16 Oct 2012 at 9:48

[GoogleCodeExporter]

Project moved to Sourceforge.net

https://sourceforge.net/projects/phpvirtualbox/

Original comment by imooreya...@gmail.com on 25 Jul 2013 at 4:54

• Changed state: Done