niemanlab / openfuego

Watching Twitter all day—so you don’t have to.
http://www.niemanlab.org/fuego
MIT License
175 stars 53 forks source link

Empty queue #9

Open ohofmann opened 10 years ago

ohofmann commented 10 years ago

Hi!

Trying to set up Fuego and seem to have gotten fairly far along -- fetch.php starts up without errors, the MySQL tables get created and populated, at least the citizens table; all other tables remain empty.

In verbose mode fetch regularly reports back:

[2013-09-12 12:22:32]: Found 0 queue files to process...

The tmp directory remains empty as well, other than the token 'nothing' file and the process IDs. Plenty of links posted by the authorities while fetch.php was running.

Is there any way to troubleshoot this further? Any log files to peek at?

Thanks! Oliver

ohofmann commented 10 years ago

Seems I found the culprit:

PHP Fatal error: Uncaught exception 'PhirehoseConnectLimitExceeded' with message 'TCP failure limit exceeded with 21 failures. Last error: ' in /home/openfuego-master/lib/Phirehose/Phirehose.class.php:611 Stack trace:

0 /home/openfuego-master/lib/Phirehose/Phirehose.class.php(788): Phirehose->connect()

1 /home/openfuego-master/lib/Phirehose/Phirehose.class.php(386): Phirehose->reconnect()

2 /home/openfuego-master/collect.php(65): Phirehose->consume()

3 /home/openfuego-master/fetch.php(46): include_once('/home/openfuego...')

4 {main}

thrown in /home/openfuego-master/lib/Phirehose/Phirehose.class.php on line 611

So it seems that Phirehose cannot get the message it is expected to fetch. A bit of googling indicates missing SSL support in PHP as a typical source of this error, but that doesn't seem to be the case here:

root@11f143a94423:/home/openfuego-master# echo '<?php phpinfo(); ?>' | php 2>&1 |grep -i ssl Registered Stream Socket Transports => tcp, udp, unix, udg, ssl, sslv3, tls SSL => Yes SSL Version => OpenSSL/1.0.1 openssl OpenSSL support => enabled OpenSSL Library Version => OpenSSL 1.0.1 14 Mar 2012 OpenSSL Header Version => OpenSSL 1.0.1 14 Mar 2012 Native OpenSSL support => enabled

alanpeart commented 10 years ago

Did you ever find a solution to this? I'm pulling my hair out at the moment. Everything seems fine, Fuego runs and the openfuego_citizens table gets populated...and nothing else happens. All other tables remain empty and the only thing reported in verbose mode is "Found 0 queue files to process... Sleeping..."

I had it working before, so I'm completely at a loss to know what is going wrong now. Any hints would be very welcome.

andreapernici commented 9 years ago

I have exactly the same issue.

kerryrm commented 9 years ago

Same issue.

komete4 commented 9 years ago

it's very important * Only MySQL is supported, hipsters. */

This needs to disabled InnoDB

andreapernici commented 9 years ago

Disable InnoDB?

komete4 commented 9 years ago

Yes, http://dev.mysql.com/doc/refman/5.6/en/innodb-turning-off.html

andreapernici commented 9 years ago

Ok, but how this can be related to the OpenSSL issue?

jeffehobbs commented 8 years ago

Hello from the FUTURE!

Has anyone found a solution to this issue? I'm still seeing it on a completely stock version of Ubuntu with mySQL version 5.6.2.

miguelcarvajal commented 8 years ago

Ey!

Has anyone found a solution for this issue?

[2015-12-09 10:21:53]: Sleeping...
[2015-12-09 10:22:03]: Found 0 queue files to process...

Looping that message for hours. Tables got populated and all the set up is correct.

ictdcapilot commented 8 years ago

Same issue.

ictdcapilot commented 8 years ago

UPDATE: I think I've got a fix.

It's not a PHP issue, it's not a MySQL issue... it's a system level one.

My system clock was showing the wrong time, therefore OAuth failed, and no new tweets were being read. I synced my clock with this:

sudo ntpdate 0.north-america.pool.ntp.org

And now fetch.php is all good. Hope this helps.

miguelcarvajal commented 8 years ago

Thanks, @ictdcapilot. I don't know if it was your fix or what, but after upgrading MySQL and doing:

sudo ntpdate 0.north-america.pool.ntp.org

OpenFuego is working perfectly! Thanks!

ictdcapilot commented 8 years ago

It's definitely not a MySQL issue... it has to do with the system clock drifting (? can't believe this is a thing) and as a result, screwing up openSSL. Glad the fix worked for you!

nickkycom commented 8 years ago

Same issue here :( ntpdate doesn't seem to help, SSL is installed properly. Any other ideas?