Periodically recurring high CPU spikes (PHP 5.5/5.6)

[lsrio]

Rather than a bug in Phirehose, this issue is informative in nature: after upgrading to PHP 5.6.8, the consuming scripts caused high CPU spikes. Debugging the source I found these originating here (Line 443 ironically):

https://github.com/fennb/phirehose/blob/master/lib/Phirehose.php#L443

$chunk_info=trim(fgets($this->conn)); //First line is hex digits giving us the length

It's caused by a bug in PHP:

https://bugs.php.net/bug.php?id=69402 / https://bugs.php.net/bug.php?id=69428

and fixed in PHP 5.6.9RC1:

https://github.com/php/php-src/blob/php-5.6.9RC1/NEWS

[fennb]

Wow, this is incredibly helpful, thank you!

Do you mind posting this same information to the phirehose-users mailing list? (http://groups.google.com/group/phirehose-users)

[vishymuku]

Thanks @lsrx for the info. I'm using aws ec2 t2.micro instance with php5.6.8. The moment i started connecting, I saw spikes of about 75-85%. Then over a period of 5-6hrs, it came down to 10%. Basically, t2.micro instances have a provisioned 10% cpu usage. Anymore than that, utilization credits have to be spent. So, the twitter stream connection exhausted the credits. Now also the instance is running at 10% utilization, negating all chances of earning back the credits for any burst of performance needed.

Question: What would be the immediate solution? roll-back to older version of php?

[vishymuku]

Thanks to a detailed response @fennb on #84, and a few links (listed at the end) I have been able to find a probable working solution.

I added this line to make the blocking true for the stream:

 stream_set_blocking($this->conn, TRUE);

Now CPU usage is down to <0.4%!

I tested it and I am able to get the tweets into database. But I don't know what the downside of blocking will be. As of now, I'll go ahead with this.

Here are the links which helped me: an old bug(with funny comments at the end): https://bugs.php.net/bug.php?id=40566 from php manual: http://php.net/manual/en/function.stream-set-blocking.php

[lsrio]

Ubunutu users out there, save on electricity costs: PPAs for PHP 5.6.9 are available now. https://launchpad.net/~ondrej/+archive/ubuntu/php5-5.6/+index?batch=75&memo=75&start=75

[compwright]

Noted in the readme. Thanks!