search

niemanlab / openfuego

Watching Twitter all day—so you don’t have to.
http://www.niemanlab.org/fuego
MIT License
176 stars 53 forks source link

Collected tweets are not stored in table openfuego_tweets_cache #1

Closed kleinmind closed 11 years ago

kleinmind commented 11 years ago

We have installed and run openfuego locally. After more than an hour the openfuego_tweets_cache table is still empty. I quickly scanned the code in the Collector and Consumer classes and couldn't find any SQL statement inserting the tweets into the DB. Am I missing sth?

phelps commented 11 years ago

Hello:

The most common cause of problems with OpenFuego is database misconfiguration. It may sound silly, but are you positive your DB credentials are entered correctly?

I would run fetch.php in verbose mode (fetch.php -v) and monitor the output on screen. Please report any errors or warnings you see there.

The SQL that inserts links into the DB can be found in the processUrls() method of the Consumer class.

kleinmind commented 11 years ago

Thanks for the response. I checked the credentials and they seem to be ok. The three tables of the database (openfuego_links, _short_links, _citizens) are progressively filled with data. The one that is not filled is the openfuego_tweets_cache, which I would expect to be filled with the actual tweets. But I couldn't see this happen in the Consumer class.

phelps commented 11 years ago

Oh I gotcha. The tweets_cache isn't populated until you query the database with the Getter, which fetches the first tweet associated with each link. So if you query the DB for 20 items, Fuego will request and store 20 tweets. If you request those same 20 items again, Fuego will fetch the tweets from cache. Hope that makes sense.

phelps commented 11 years ago

To be clear, Fuego is not storing tweets, just the URLs that people share.

kleinmind commented 11 years ago

I still cannot spot where the caching takes place in the Getter. Actually, I couldn't find any place in the code where the table openfuego_tweets_cache is used. Wouldn't it be better if that particular table is not created at all? [sorry for me insisting so much; maybe I'm missing sth obvious]

phelps commented 11 years ago

That table is read and written to by the getTweet() and updateTweet() methods of lib/Metadata.

kleinmind commented 11 years ago

Now I get it! Many thanks for your patience! I'll now close the issue.

phelps commented 11 years ago

No problem. If it wasn't apparent to you, it may mean the application could be better organized. I'm still trying to figure it out. Suggestions welcomed.