search

mirage / xen-arm-builder

Archived - the Xen and ARM support in MirageOS has been superseeded by our PVH support - Build an SDcard image for Xen/ARM, for a Cubieboard
ISC License
57 stars 36 forks source link

xcp-rrdd logging extensively to syslog #53

Closed gpskentzos closed 7 years ago

gpskentzos commented 9 years ago

I'm working on a cubietruck and cloned the master branch.

In addition to the basic setup from the scripts, I have installed mysql and apache.

Problem: When leaving the cubietruck on for an extended period of time, e.g. a full day of work and leaving it on overnight, the partition will completely fill up.I found out that the culprit is the syslog (var/log). It appears that xcp-rrdd is logging many, many times a second. An example of the syslog is here:

Apr 14 14:16:05 cubietruck xapi: [debug|cubietruck|69 ||xapi_stats] rrdd says next reading is overdue by inf seconds; not sleeping Apr 14 14:16:05 cubietruck xcp-rrdd: [debug|cubietruck|11 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } Apr 14 14:16:05 cubietruck xapi: [debug|cubietruck|65 |xapi events D:683eca15e37d|mscgen] xapi=>xapi [label="event.from"]; Apr 14 14:16:05 cubietruck xapi: [debug|cubietruck|69 ||xapi_stats] rrdd says next reading is overdue by inf seconds; not sleeping Apr 14 14:16:05 cubietruck xapi: [debug|cubietruck|72 UNIX /var/lib/xcp/xapi||dummytaskhelper] task dispatch:event.from D:3830d3e10016 created by task D:683eca15e37d Apr 14 14:16:05 cubietruck xcp-rrdd: [debug|cubietruck|12 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } Apr 14 14:16:05 cubietruck xapi: [debug|cubietruck|69 ||xapi_stats] rrdd says next reading is overdue by inf seconds; not sleeping Apr 14 14:16:05 cubietruck xcp-rrdd: [debug|cubietruck|13 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } Apr 14 14:16:05 cubietruck xcp-rrdd: [debug|cubietruck|14 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } Apr 14 14:16:05 cubietruck xcp-rrdd: [debug|cubietruck|15 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } Apr 14 14:16:05 cubietruck xcp-rrdd: [debug|cubietruck|16 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } Apr 14 14:16:05 cubietruck xcp-rrdd: [debug|cubietruck|17 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } Apr 14 14:16:05 cubietruck xcp-rrdd: [debug|cubietruck|18 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi }

If I stop the service: sudo service xcp-rrdd stop

the logging stops. When I restart the service, the logging commences at a more reasonable every ~5 seconds.

Before realizing this issue, I increased the size of the dom0 partition to 10GB since I was having this problem with the default xen-arm-builder setup and thought that it might have been the installation of mysql and apache that caused the space problem. This morning the syslog was 8GB. I removed the syslog and rebooted the cubietruck and the logging started again. Stopping and starting the service seems to fix the problem

I appreciate your time and appreciate the work with the scripts! They are extremly valuable.

Best regards, Paul Skentzos DornerWorks www.dornerworks.com

djs55 commented 9 years ago

If think this is caused by the clock on the cubie being set to Jan 1 1970. Iirc this interacts badly with a "please call me back later" message returned from xapi to xcp-rrdd. Try installing an ntpd to set the clock on boot and see if it still happens.

Thanks for the bug report BTW! On 14 Apr 2015 3:59 pm, "gpskentzos" notifications@github.com wrote:

I'm working on a cubietruck and cloned the master branch.

In addition to the basic setup from the scripts, I have installed mysql and apache.

Problem: When leaving the cubietruck on for an extended period of time, e.g. a full day of work and leaving it on overnight, the partition will completely fill up.I found out that the culprit is the syslog (var/log). It appears that xcp-rrdd is logging many, many times a second. An example of the syslog is here:

Apr 14 14:16:05 cubietruck xapi: [debug|cubietruck|69 ||xapi_stats] rrdd says next reading is overdue by inf seconds; not sleeping Apr 14 14:16:05 cubietruck xcp-rrdd: [debug|cubietruck|11 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } Apr 14 14:16:05 cubietruck xapi: [debug|cubietruck|65 |xapi events D:683eca15e37d|mscgen] xapi=>xapi [label="event.from"]; Apr 14 14:16:05 cubietruck xapi: [debug|cubietruck|69 ||xapi_stats] rrdd says next reading is overdue by inf seconds; not sleeping Apr 14 14:16:05 cubietruck xapi: [debug|cubietruck|72 UNIX /var/lib/xcp/xapi||dummytaskhelper] task dispatch:event.from D:3830d3e10016 created by task D:683eca15e37d Apr 14 14:16:05 cubietruck xcp-rrdd: [debug|cubietruck|12 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } Apr 14 14:16:05 cubietruck xapi: [debug|cubietruck|69 ||xapi_stats] rrdd says next reading is overdue by inf seconds; not sleeping Apr 14 14:16:05 cubietruck xcp-rrdd: [debug|cubietruck|13 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } Apr 14 14:16:05 cubietruck xcp-rrdd: [debug|cubietruck|14 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } Apr 14 14:16:05 cubietruck xcp-rrdd: [debug|cubietruck|15 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } Apr 14 14:16:05 cubietruck xcp-rrdd: [debug|cubietruck|16 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } Apr 14 14:16:05 cubietruck xcp-rrdd: [debug|cubietruck|17 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } Apr 14 14:16:05 cubietruck xcp-rrdd: [debug|cubietruck|18 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi }

If I stop the service: sudo service xcp-rrdd stop

the logging stops. When I restart the service, the logging commences at a more reasonable every ~5 seconds.

Before realizing this issue, I increased the size of the dom0 partition to 10GB since I was having this problem with the default xen-arm-builder setup and thought that it might have been the installation of mysql and apache that caused the space problem. This morning the syslog was 8GB. I removed the syslog and rebooted the cubietruck and the logging started again. Stopping and starting the service seems to fix the problem

I appreciate your time and appreciate the work with the scripts! They are extremly valuable.

Best regards, Paul Skentzos DornerWorks www.dornerworks.com

— Reply to this email directly or view it on GitHub https://github.com/mirage/xen-arm-builder/issues/53.

gpskentzos commented 9 years ago

Dave,

Thanks for the quick response. It looks like I already have the ntpd running. My syslog has the correct UTC time from bootup to where the multiple xcp-rrdd messages start. I'm still trying to think how stopping and starting "fixes" the problem.

I will try rebuilding another sd card with a clean install so that I might be able to eliminate some of the other software I installed after the build.

Here's a short example of the beginning of the syslog just to show that time seems to be set appropriately: Apr 14 14:15:16 cubietruck rsyslogd: [origin software="rsyslogd" swVersion="7.4.4" x-pid="753" x-info="http://www.rsyslog.com"] start Apr 14 14:15:16 cubietruck rsyslogd: rsyslogd's groupid changed to 106 Apr 14 14:15:16 cubietruck rsyslogd: rsyslogd's userid changed to 102 Apr 14 14:15:16 cubietruck rsyslogd-2039: Could no open output pipe '/dev/xconsole': No such file or directory [try http://www.rsyslog.com/e/2039 ] Apr 14 14:15:16 cubietruck kernel: [ 0.000000] Booting Linux on physical CPU 0x0 Apr 14 14:15:16 cubietruck kernel: [ 0.000000] Linux version 3.16.0-00124-g6b6dcc2-dirty (arlx@arlx-63) (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #1 SMP Mon Apr 6 09:18:54 EDT 2015 Apr 14 14:15:16 cubietruck kernel: [ 0.000000] CPU: ARMv7 Processor [410fc074] revision 4 (ARMv7), cr=10c5387d Apr 14 14:15:16 cubietruck kernel: [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache Apr 14 14:15:16 cubietruck kernel: [ 0.000000] Machine model: Cubietech Cubietruck Apr 14 14:15:16 cubietruck kernel: [ 0.000000] cma: CMA: reserved 64 MiB at 7c000000 Apr 14 14:15:16 cubietruck kernel: [ 0.000000] Memory policy: Data cache writealloc Apr 14 14:15:16 cubietruck kernel: [ 0.000000] On node 0 totalpages: 131072 Apr 14 14:15:16 cubietruck kernel: [ 0.000000] free_area_init_node: node 0, pgdat c0cd1ac0, node_mem_map dbbd9000 Apr 14 14:15:16 cubietruck kernel: [ 0.000000] DMA zone: 1024 pages used for memmap Apr 14 14:15:16 cubietruck kernel: [ 0.000000] DMA zone: 0 pages reserved Apr 14 14:15:16 cubietruck kernel: [ 0.000000] DMA zone: 131072 pages, LIFO batch:31 Apr 14 14:15:16 cubietruck kernel: [ 0.000000] psci: probing for conduit method from DT. Apr 14 14:15:16 cubietruck kernel: [ 0.000000] psci: Using PSCI v0.1 Function IDs from DT Apr 14 14:15:16 cubietruck kernel: [ 0.000000] PERCPU: Embedded 8 pages/cpu @dbbb7000 s8832 r8192 d15744 u32768 Apr 14 14:15:16 cubietruck kernel: [ 0.000000] pcpu-alloc: s8832 r8192 d15744 u32768 alloc=8*4096 Apr 14 14:15:16 cubietruck kernel: [ 0.000000] pcpu-alloc: [0] 0 [0] 1 Apr 14 14:15:16 cubietruck kernel: [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 130048 Apr 14 14:15:16 cubietruck kernel: [ 0.000000] Kernel command line: console=hvc0 ro root=/dev/mmcblk0p2 rootwait clk_ignore_unused Apr 14 14:15:16 cubietruck kernel: [ 0.000000] PID hash table entries: 2048 (order: 1, 8192 bytes) Apr 14 14:15:16 cubietruck kernel: [ 0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes) Apr 14 14:15:16 cubietruck kernel: [ 0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes) Apr 14 14:15:16 cubietruck kernel: [ 0.000000] Memory: 442568K/524288K available (6927K kernel code, 837K rwdata, 2700K rodata, 580K init, 317K bss, 81720K reserved, 0K highmem) Apr 14 14:15:16 cubietruck kernel: [ 0.000000] Virtual kernel memory layout:

Thanks for the time!

Paul Skentzos

djs55 commented 9 years ago

Hi,

When it reproduces, try capturing the traffic between xcp-rrdd and xapi by running something like

strace -s 10000 -p

The messages back and forth are json rpc and should be fairly obvious in the output -- you won't need to capture very much! I think once we can see those we'll understand the problem better.

Cheers, Dave

On Tue, Apr 14, 2015 at 5:04 PM, gpskentzos notifications@github.com wrote:

Dave,

Thanks for the quick response. It looks like I already have the ntpd running. My syslog has the correct UTC time from bootup to where the multiple xcp-rrdd messages start. I'm still trying to think how stopping and starting "fixes" the problem.

I will try rebuilding another sd card with a clean install so that I might be able to eliminate some of the other software I installed after the build.

Here's a short example of the beginning of the syslog just to show that time seems to be set appropriately: Apr 14 14:15:16 cubietruck rsyslogd: [origin software="rsyslogd" swVersion="7.4.4" x-pid="753" x-info="http://www.rsyslog.com"] start Apr 14 14:15:16 cubietruck rsyslogd: rsyslogd's groupid changed to 106 Apr 14 14:15:16 cubietruck rsyslogd: rsyslogd's userid changed to 102 Apr 14 14:15:16 cubietruck rsyslogd-2039: Could no open output pipe '/dev/xconsole': No such file or directory [try http://www.rsyslog.com/e/2039 ] Apr 14 14:15:16 cubietruck kernel: [ 0.000000] Booting Linux on physical CPU 0x0 Apr 14 14:15:16 cubietruck kernel: [ 0.000000] Linux version 3.16.0-00124-g6b6dcc2-dirty (arlx@arlx-63) (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #1 SMP Mon Apr 6 09:18:54 EDT 2015 Apr 14 14:15:16 cubietruck kernel: [ 0.000000] CPU: ARMv7 Processor [410fc074] revision 4 (ARMv7), cr=10c5387d Apr 14 14:15:16 cubietruck kernel: [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache Apr 14 14:15:16 cubietruck kernel: [ 0.000000] Machine model: Cubietech Cubietruck Apr 14 14:15:16 cubietruck kernel: [ 0.000000] cma: CMA: reserved 64 MiB at 7c000000 Apr 14 14:15:16 cubietruck kernel: [ 0.000000] Memory policy: Data cache writealloc Apr 14 14:15:16 cubietruck kernel: [ 0.000000] On node 0 totalpages: 131072 Apr 14 14:15:16 cubietruck kernel: [ 0.000000] free_area_init_node: node 0, pgdat c0cd1ac0, node_mem_map dbbd9000 Apr 14 14:15:16 cubietruck kernel: [ 0.000000] DMA zone: 1024 pages used for memmap Apr 14 14:15:16 cubietruck kernel: [ 0.000000] DMA zone: 0 pages reserved Apr 14 14:15:16 cubietruck kernel: [ 0.000000] DMA zone: 131072 pages, LIFO batch:31 Apr 14 14:15:16 cubietruck kernel: [ 0.000000] psci: probing for conduit method from DT. Apr 14 14:15:16 cubietruck kernel: [ 0.000000] psci: Using PSCI v0.1 Function IDs from DT Apr 14 14:15:16 cubietruck kernel: [ 0.000000] PERCPU: Embedded 8 pages/cpu @dbbb7000 s8832 r8192 d15744 u32768 Apr 14 14:15:16 cubietruck kernel: [ 0.000000] pcpu-alloc: s8832 r8192 d15744 u32768 alloc=8*4096 Apr 14 14:15:16 cubietruck kernel: [ 0.000000] pcpu-alloc: [0] 0 [0] 1 Apr 14 14:15:16 cubietruck kernel: [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 130048 Apr 14 14:15:16 cubietruck kernel: [ 0.000000] Kernel command line: console=hvc0 ro root=/dev/mmcblk0p2 rootwait clk_ignore_unused Apr 14 14:15:16 cubietruck kernel: [ 0.000000] PID hash table entries: 2048 (order: 1, 8192 bytes) Apr 14 14:15:16 cubietruck kernel: [ 0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes) Apr 14 14:15:16 cubietruck kernel: [ 0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes) Apr 14 14:15:16 cubietruck kernel: [ 0.000000] Memory: 442568K/524288K available (6927K kernel code, 837K rwdata, 2700K rodata, 580K init, 317K bss, 81720K reserved, 0K highmem) Apr 14 14:15:16 cubietruck kernel: [ 0.000000] Virtual kernel memory layout:

Thanks for the time!

Paul Skentzos

— Reply to this email directly or view it on GitHub https://github.com/mirage/xen-arm-builder/issues/53#issuecomment-92935310 .

Dave Scott

talex5 commented 9 years ago

I'm seeing endless messages like this in /var/log/syslog on a cubieboard2:

May 23 16:26:33 cubieboard2 xcp-rrdd: [debug|cubieboard2|991504 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [  ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi }
May 23 16:26:33 cubieboard2 xcp-rrdd: [debug|cubieboard2|991505 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [  ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi }
May 23 16:26:33 cubieboard2 xcp-rrdd: [debug|cubieboard2|991506 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [  ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi }
May 23 16:26:33 cubieboard2 xcp-rrdd: [debug|cubieboard2|991507 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [  ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi }

top shows these processes using all the CPU:

  953 root      20   0   59096   4116   1584 S  34.0  0.8  51:06.33 xcp-rrdd                                    
  886 root      20   0  376100  25628   4520 S  31.4  5.0  47:24.89 xapi                                        
  652 syslog    20   0   30476   2016   1292 S   7.3  0.4  10:38.12 rsyslogd

But strace shows nothing for any of them (just sitting in select)!

Is xcp-rrdd needed for anything, or can I just uninstall it?

avsm commented 9 years ago

It's part of xapi, so @djs55 or @jonludlam will know

On 23 May 2015, at 17:31, Thomas Leonard notifications@github.com wrote:

I'm seeing endless messages like this in /var/log/syslog on a cubieboard2:

May 23 16:26:33 cubieboard2 xcp-rrdd: [debug|cubieboard2|991504 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } May 23 16:26:33 cubieboard2 xcp-rrdd: [debug|cubieboard2|991505 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } May 23 16:26:33 cubieboard2 xcp-rrdd: [debug|cubieboard2|991506 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } May 23 16:26:33 cubieboard2 xcp-rrdd: [debug|cubieboard2|991507 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } top shows these processes using all the CPU:

953 root 20 0 59096 4116 1584 S 34.0 0.8 51:06.33 xcp-rrdd
886 root 20 0 376100 25628 4520 S 31.4 5.0 47:24.89 xapi
652 syslog 20 0 30476 2016 1292 S 7.3 0.4 10:38.12 rsyslogd
But strace shows nothing for any of them (just sitting in select)!

Is xcp-rrdd needed for anything, or can I just uninstall it?

— Reply to this email directly or view it on GitHub.

djs55 commented 9 years ago

I believe this is a bug in the protocol between xcp-rrdd and xapi triggered by the clock on the cubie being set to 1970. Normally I work around it by setting the clock with ntp and then restarting 'xcp-rrdd' and 'xapi'.

My strace tip earlier in the bug report was itself a bit buggy: you need to add -ff to see the RPC traffic coming from threads.

On Sat, May 23, 2015 at 5:34 PM, Anil Madhavapeddy <notifications@github.com

wrote:

It's part of xapi, so @djs55 or @jonludlam will know

On 23 May 2015, at 17:31, Thomas Leonard notifications@github.com wrote:

I'm seeing endless messages like this in /var/log/syslog on a cubieboard2:

May 23 16:26:33 cubieboard2 xcp-rrdd: [debug|cubieboard2|991504 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } May 23 16:26:33 cubieboard2 xcp-rrdd: [debug|cubieboard2|991505 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } May 23 16:26:33 cubieboard2 xcp-rrdd: [debug|cubieboard2|991506 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } May 23 16:26:33 cubieboard2 xcp-rrdd: [debug|cubieboard2|991507 unix_rpc||http] Request { frame = false; method = POST; uri = /var/lib/xcp/xcp-rrdd; query = [ ]; content_length = [ 330 ]; transfer encoding = ; version = 1.1; cookie = [ = ]; task = ; subtask_of = ; content-type = ; host = ; user_agent = /usr/sbin/xapi } top shows these processes using all the CPU:

953 root 20 0 59096 4116 1584 S 34.0 0.8 51:06.33 xcp-rrdd 886 root 20 0 376100 25628 4520 S 31.4 5.0 47:24.89 xapi 652 syslog 20 0 30476 2016 1292 S 7.3 0.4 10:38.12 rsyslogd But strace shows nothing for any of them (just sitting in select)!

Is xcp-rrdd needed for anything, or can I just uninstall it?

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/mirage/xen-arm-builder/issues/53#issuecomment-104922975 .

Dave Scott

mor1 commented 7 years ago

After the refresh, chronyd is installed and running so hopefully clocks are set correctly. I'll go ahead and close this, but please do reopen if it still occurs.