Trivial GB .js app quits (with no error) after a few days

[jsquyres]

I have a Raspberry Pi that I have connected to my GB. I've been connecting the GB to a Viper and a Hawk.

While (slowly) learning about the data that comes back from the appliances, I've been connecting the GB to either the Viper or Hawk and leaving a trivial "status.js" program running in a "screen" for a week or so (I capture all the data for later analysis) -- see source here: https://gist.github.com/jsquyres/feb45fe7b34ffe0a034d

status.js runs just fine and outputs all manner of wonderful data (I leave it connected to a single appliance during these runs -- e.g., I'm not connected to the washer for a while and then unplug/replug into the dryer during a single status.js run).

However, after a few days, status.js stops and returns to a shell prompt for no apparent reason. There's no error, and I don't know why the program would just stop (none of the node.js subscriptions should have exited, so the loop should keep firing AFAIK...?).

In some cases, status.js terminates during the middle of a Hawk/Viper cycle. In other cases, it terminates not during a cycle (i.e., a cycle completed, and I don't have a data point for exactly when status.js terminated).

Is this a known issue? If not, is there anything I can run on my end to further diagnose this issue?

[rudybrian]

It probably isn't related the the issue you reported, but do you actually get data from most of the subscribe() functions? When running on a fast machine (compared with a Pi) it didn't work consistently for me (it seems to be overloading something.) I ended up inserting a 250ms delay between each function to improve reliability.

What version of Node.js are you running on your Pi?

[jsquyres]

Yes, I get output from a bunch of the subscribe functions (to be honest, I just put trivial subscribe functions on most of the subscribe-able actions that I could find, just so that I could learn about what would be sent when).

Here's some sample output (I started status.js yesterday, and it hasn't terminated yet):

2014-11-10 10:47:05  time remaining (seconds) changed: 46
2014-11-10 10:47:06  time remaining (seconds) changed: 45
2014-11-10 10:47:07  time remaining (seconds) changed: 44
2014-11-10 10:47:08  time remaining (seconds) changed: 43
2014-11-10 10:47:09  time remaining (seconds) changed: 42
2014-11-10 10:47:10  time remaining (seconds) changed: 41
2014-11-10 10:47:11  time remaining (seconds) changed: 40
2014-11-10 10:47:12  time remaining (seconds) changed: 39
2014-11-10 10:47:13  time remaining (seconds) changed: 38
2014-11-10 10:47:14  machine status changed: 4
2014-11-10 10:47:14  machine sub-cycle changed: 0
2014-11-10 10:47:14  end of cycle changed: 1
2014-11-10 10:47:14  time remaining (seconds) changed: 0
2014-11-10 11:02:14  machine status changed: 0
2014-11-10 11:02:14  end of cycle changed: 0

Here's the version:

$ /home/pi/nodejs-local/bin/node --version
v0.10.32

Keep in mind that I'm also running the hidapi modification that I (initially) contributed upstream in signal11/hidapi#190. I.e., I'm running just the simple "make hid_write() be infinite" fix that was the original pull request -- it has since morphed into something a little bigger, and sadly, is not yet resolved.

[rudybrian]

I just picked up a couple of Pi's myself for my laundry pair and once I get them wired up I will see if I see something similar (with the same version of Node)

[jsquyres]

Cool. If I need to switch to another version of Node, I'm happy to do so. I just don't know where the failure is coming from -- Node, the GEA APIs, ...etc.

On Tue, Nov 11, 2014 at 1:34 AM, Brian Rudy notifications@github.com wrote:

I just picked up a couple of Pi's myself for my laundry pair and once I get them wired up I will see if I see something similar (with the same version of Node)

— Reply to this email directly or view it on GitHub https://github.com/GEMakers/green-bean/issues/2#issuecomment-62508065.

{+} Jeff Squyres

[rudybrian]

I'm testing with the same Node version on my MacBook and just installed the same on my Pi's, so hopefully any inconsistencies will show up for me as well. I think I have a newer Node version on my Ubuntu 14.04.1 LTS based netbook, but haven't let any of them run for that long yet to see if there are crashes, but will report on my findings.

[jsquyres]

Ok. Let me know if there's any additional data I can provide (I'm a C / systems-level programmer, so although I have little experience with node, I don't mind digging under the covers for additional debugging information).

On Tue, Nov 11, 2014 at 4:44 PM, Brian Rudy notifications@github.com wrote:

I'm testing with the same Node version on my MacBook and just installed the same on my Pi's, so hopefully any inconsistencies will show up for me as well. I think I have a newer Node version on my Ubuntu 14.04.1 LTS based netbook, but haven't let any of them run for that long yet to see if there are crashes, but will report on my findings.

— Reply to this email directly or view it on GitHub https://github.com/GEMakers/green-bean/issues/2#issuecomment-62627413.

{+} Jeff Squyres

[rudybrian]

FYI: I finally got my Pis and green beans hooked up to our laundry pair last night and ran GreenBeanify on the washing machine, and your test app status.js running on the dryer. My wife ran loads in both early the next morning before heading to bed, and both apps crashed after a few minutes once the loads were started for what appears to be the timeout issue you reported against the gea-sdk GEMakers/gea-sdk#2 and signal11/hidapi#190.

I'll update with your patch and try again.

[jsquyres]

​Great; thanks for the update (I'll be unfortunately a bit swamped until December, but if there's any additional details I can supply, let me know).​

On Wed, Nov 19, 2014 at 2:16 PM, Brian Rudy notifications@github.com wrote:

FYI: I finally got my Pis and green beans hooked up to our laundry pair last night and ran GreenBeanify on the washing machine, and your test app status.js running on the dryer. My wife ran loads in both early the next morning before heading to bed, and both apps crashed after a few minutes once the loads were started for what appears to be the timeout issue you reported against the gea-sdk GEMakers/gea-sdk#2 https://github.com/GEMakers/gea-sdk/issues/2 and signal11/hidapi#190 https://github.com/signal11/hidapi/pull/190.

I'll update with your patch and try again.

— Reply to this email directly or view it on GitHub https://github.com/GEMakers/green-bean/issues/2#issuecomment-63705985.

{+} Jeff Squyres

[rudybrian]

Thanks @jsquyres,

FYI: I applied your patch to the hidapi being used on both my Pis yesterday and the the apps on both are still running after running 2 loads with no crashes. I will continue to monitor.

Something (unrelated) that I want to investigate is why my dryer is only firing the callback from a few subscribe() functions, yet the read() function seems to be working. On the washing machine (which I used for GreenBeanify development), just doing a subscribe() seems to work for everything.

For your Hawk dryer with status.js you were seeing:

machine status
machine sub-cycle
time remaining
selected cycle
operating mode
end of cycle

For mine, I am currently only getting:

delay time remaining
time remaining
selected cycle
operating mode

I seem to recall seeing more from the dryer when doing testing earlier with my MacBook/Netbook so I'll have to confirm and see if this is an issue with the Pi after letting this test complete.

[bakerface]

Just a note. Each appliance supports a predefined number of subscriptions. For most appliances, if there are no slots available for subscription, the subscription will be denied. This may be the reason for the inconsistent subscription responses. Reads, unlike subscriptions, have no internal limit.

[rudybrian]

Thanks @bakerface,

Is there a way to know which subscription requests fail and those that succeed? How long to the subscription requests hang around? (i.e. do I need to invalidate any stale subscriptions if my app crashes and I re-launch)

[bakerface]

The appliance will respond to a subscription request with the number of successful subscriptions. The node.js code currently ignores the response. Code would need to be added to check the response and return an error to the user. Subscriptions are cached on the appliance in RAM, so power-cycling will free up the subscriptions. Also, some appliances support a special ERD to clear all subscriptions.

[jsquyres]

Ah, so the code should be something like this? (I'm away from the RPi/GB and can't check it right now)

if (1 != laundry.machineStatus.subscribe(function (value) {
    logme("machine status changed:", value);
})) {
    logme("laundry.machineStatus.subscribe failed");
}

[rudybrian]

I had to restart the apps on the Pis due to a brief (self-induced) network issue and took the opportunity to power-cycle the dryer. Now it gives me a slightly different set of responses than before. This seems to be the case with both status.js and GreenBeanify:

machine status
machine sub-cycle
end of cycle
dryer service error codes
time remaining
selected cycle
operating mode

I also tested out @jsquyres code snippet, and it doesn't appear to work (always shows failure even when a subscription callback is firing and returning data). I may not be reading it correctly, but the subscribe() function doesn't appear to be blocking, so a result would not instantly be available. I imagine we would need to get the error in a callback in order to do something with it.

[rudybrian]

FYI: I finally caught a crash of @jsquyres status.js, but it doesn't reveal much. (Note: I am running node supervisor which automatically re-starts the app after a crash.)

.
.
.
2014-12-05 19:44:45  time remaining (seconds) changed: 55
2014-12-05 19:44:46  time remaining (seconds) changed: 54
2014-12-05 19:44:47  time remaining (seconds) changed: 53
2014-12-05 19:44:48  time remaining (seconds) changed: 52
2014-12-05 19:44:49  time remaining (seconds) changed: 0
Program node /home/pi/node_modules/GreenBeanify/status.js exited with code null

Starting child process with 'node /home/pi/node_modules/GreenBeanify/status.js'
2014-12-06 08:08:10  delay time remaining (minutes) changed: 0
2014-12-06 08:08:10  operating mode changed: 0
2014-12-06 08:08:10  selected cycle changed: 13
2014-12-06 08:08:10  time remaining (seconds) changed: 0
2014-12-06 08:08:10  dryer service error codes changed: 0