Spine does not receive full script output

[eschoeller]

I am experiencing an issue with spine cutting off the output from a script being used by a data input method.

The best way to begin illustrating this issue is with a code snippet:

if ($query=="current") {
   $return_arr = cacti_snmp_walk($hostname, $snmp_community,
        $oids["PhaseStatusCurrent"], $snmp_version, $snmp_auth_username,
        $snmp_auth_password, $snmp_auth_protocol, $snmp_priv_passphrase, $snmp_priv_protocol,
        $snmp_context, $snmp_port, $snmp_timeout, $snmp_retries, $max_oids, SNMP_POLLER);
   for ($i=0;($i<sizeof($return_arr));$i++) {
        $index = $i+1;
        print "current$index:" . chop($return_arr[$i]["value"]);
        if ($i < (sizeof($return_arr)-1)) {
           print " ";
        }
   }
}

This should produce the following output: current1:186 current2:162 current3:178 current4:167 current5:204 current6:160 current7:153 current8:193 current9:163

When I use spine, I'm getting very inconsistent results, example: 2017-04-23 21:08:47 - SPINE: Poller[1] Device[1] TH[1] DS[2] SCRIPT: php -q /cacti/scripts/query_device.php current '192.168.1.3' 'public' '2' '161' '500' "''" "''" "''" "''" "''", output: current1:185 current2:162 current3:178

Switching to cmd.php resolves the problem.

Now, I tried a slightly different approach, see this code snippet:

if ($query=="current") {
   $output = '';
   $return_arr = cacti_snmp_walk($hostname, $snmp_community,
        $oids["PhaseStatusCurrent"], $snmp_version, $snmp_auth_username,
        $snmp_auth_password, $snmp_auth_protocol, $snmp_priv_passphrase, $snmp_priv_protocol,
        $snmp_context, $snmp_port, $snmp_timeout, $snmp_retries, $max_oids, SNMP_POLLER);
   for ($i=0;($i<sizeof($return_arr));$i++) {
        $index = $i+1;
        $output .= "current$index:" . chop($return_arr[$i]["value"]);
        if ($i < (sizeof($return_arr)-1)) {
           $output .= " ";
        }
   }
   print $output;
}

By storing the complete command output and printing it just once, instead of through a for loop spine now consistently captures the full command output. My spine command timeout is 25 seconds. I'm running spine 1.1.0. There are literally just microseconds between each value being returned from this script when being run through a for() loop ... I'm only looping through an array and doing very little processing in the interim. But it seems like this extremely small difference is enough to trip up spine. I can't tell you just how crazy this made me today ... !

[cigamit]

This is a known limitation of spine. You have to output all of your data in one fell swoop. Spine intentionally does not handle buffering due to possible race conditions. It's actually documented, but people who develop templates know this 'eventually'.

[eschoeller]

That makes a lot of sense, and I think I've actually been bitten by this before while developing other data input methods in the past ... it seemed very familiar. And I'm sure that any changes to the code to support buffering would have a negative impact on performance, which is certainly important for spine. I doubt there would even be a way to toss a warning out from spine in this scenario? Maybe indicate that spine never received a program completion return code or something?