digitalmethodsinitiative / dmi-tcat

Digital Methods Initiative - Twitter Capture and Analysis Toolset
Apache License 2.0
366 stars 114 forks source link

Export/download all tweets to csv from query empty after deleting trying to solve PHP Fatal error: Uncaught exception 'PDOException' with message 'SQLSTATE[42S02]: Base table or view not found: 1146 Table 'twittercapture #387

Closed snosko closed 4 years ago

snosko commented 4 years ago

After a server update and restart (yes I know) A couple of query databases went corrupted. myisamchk, mysqlchk didn't save the day. and myDMITCATserver/capture webpage was blank.

Apache error log showed: [:error] [pid 25999] [client 158.227.44.45:50152] PHP Fatal error: Uncaught exception 'PDOException' with message 'SQLSTATE[42S02]: Base table or view not found: 1146 Table 'twittercapture.Conquis_EiTB_tweets' doesn't exist' in /var/www/dmi-tcat/capture/query_manager.php:788\nStack trace:\n#0 /var/www/dmi-tcat/capture/query_manager.php(788): PDOStatement->execute()\n#1 /var/www/dmi-tcat/capture/index.php(19): getBins()\n#2 {main}\n thrown in /var/www/dmi-tcat/capture/query_manager.php on line 788 Checking the databases I proceed to remove the references to the troublesome queries:

mysql> delete from tcat_query_bins where querybin = Conquis_EiTB;
  Query OK, 1 row affected (0.00 sec)                                                      |
| 95 | conquis_etb                                   | track |      1 |      0 |                                                         |
| 96 | iglesias                                      | track |      1 |      0 |                                                         |
+----+-----------------------------------------------+-------+--------+--------+---------------------------------------------------------+
89 rows in set (0.00 sec)

mysql> delete from tcat_query_bins where querybin = "conquis_etb";
Query OK, 1 row affected (0.00 sec)

myDMITCATserver/capture webpage was fixed and everything looked fine. Until I discovered that some tweet exports had only the header in the csv or tsv file:

id,time,created_at,from_user_name,text,filter_level,possibly_sensitive,withheld_copyright,withheld_scope,truncated,retweet_count,favorite_count,lang,to_user_name,in_reply_to_status_id,quoted_status_id,source,location,lat,lng,....

Is there any way to fix the export problem or have I messed around too much and I should restart from zero? I am considering trying to use the export php script

niczem commented 4 years ago

Is there any other error appearing in he error log, after trying to export the csv?

snosko commented 4 years ago

No errors, no clues

niczem commented 4 years ago

is there any more information, like which csvs only contain the header information and which do not?

snosko commented 4 years ago

I can only know which ones fail when I download them. I haven't found any pattern.

niczem commented 4 years ago

Okey based on that information we cant help, sorry.

We an need error or debug logs or any hint which exports are working and which ones aren't