fossasia / loklak_webclient

loklak web application
http://loklak-web.herokuapp.com
MIT License
1.56k stars 49 forks source link

Wall does not utilize data after Sep 24 #537

Closed mariobehling closed 9 years ago

mariobehling commented 9 years ago

Even though I enter a data scale after Sep. 24, the wall does not show the data e.g. for the hashtag Mozilla. Data actually is in the system as the search shows. screenshot from 2015-10-13 14 33 06

Orbiter commented 9 years ago

due to most recent additions for a detailed query logging I can see exactly what happend:

{"start":"2015-10-13T14:52:30.560Z","host":"178.19.212.228","class":"SearchServlet","local":false,"idle":20001,"dosb":false,"dosr":false,"query":{"count":"0","source":"cache","q":"#mozilla include:retweets near:Berlin since:2015-09-21_13:00 until:2015-10-30_17:00","callback":"angular.callbacks._2h","timezoneOffset":"-120","fields":"created_at,screen_name,mentions,hashtags"},"event_cache_time":2247,"event_postprocessing_time":1,"busy":2248,"finish":"2015-10-13T14:52:32.808Z"}
{"start":"2015-10-13T14:52:40.705Z","host":"178.19.212.228","class":"SearchServlet","local":false,"idle":10145,"dosb":false,"dosr":false,"query":{"count":"10","q":"#mozilla include:retweets near:Berlin since:2015-09-21_13:00 until:2015-10-30_17:00","callback":"angular.callbacks._2i","fromWall":"true"},"event_backend_time":1,"event_cache_time":23,"event_backend_time_join":0,"event_remote_scraper_on_http://kaskelix.de:9000":2021,"event_local_scraper_after_unsuccessful_remote":554,"event_twitterscraper_time_join":8000,"event_postprocessing_time":6,"busy":8029,"finish":"2015-10-13T14:52:48.734Z"}
{"start":"2015-10-13T14:52:50.595Z","host":"178.19.212.228","class":"SearchServlet","local":false,"idle":9890,"dosb":false,"dosr":false,"query":{"count":"0","source":"cache","q":"#mozilla include:retweets near:Berlin since:2015-09-21_13:00 until:2015-10-30_17:00","callback":"angular.callbacks._2j","timezoneOffset":"-120","fields":"created_at,screen_name,mentions,hashtags"},"event_cache_time":2313,"event_postprocessing_time":1,"busy":2315,"finish":"2015-10-13T14:52:52.910Z"}
{"start":"2015-10-13T14:53:10.588Z","host":"178.19.212.228","class":"SearchServlet","local":false,"idle":19993,"dosb":false,"dosr":false,"query":{"count":"0","source":"cache","q":"#mozilla include:retweets near:Berlin since:2015-09-21_13:00 until:2015-10-30_17:00","callback":"angular.callbacks._2k","timezoneOffset":"-120","fields":"created_at,screen_name,mentions,hashtags"},"event_cache_time":83,"event_postprocessing_time":0,"busy":84,"finish":"2015-10-13T14:53:10.672Z"}
{"start":"2015-10-13T14:53:18.917Z","host":"178.19.212.228","class":"SearchServlet","local":false,"idle":8329,"dosb":false,"dosr":false,"query":{"count":"10","q":"#mozilla include:retweets near:Berlin since:2015-09-21_13:00 until:2015-10-30_17:00","callback":"angular.callbacks._2l","fromWall":"true"},"event_backend_time":1,"event_remote_scraper_on_http://kaskelix.de:9000":17,"event_cache_time":26,"event_backend_time_join":0,"event_local_scraper_after_unsuccessful_remote":524,"event_twitterscraper_time_join":8000,"event_postprocessing_time":7,"busy":8033,"finish":"2015-10-13T14:53:26.950Z","event_local_scraper_wait_ready":8003,"event_query_recorder":0,"event_twitterscraper_time":8545}
{"start":"2015-10-13T14:53:30.571Z","host":"178.19.212.228","class":"SearchServlet","local":false,"idle":11654,"dosb":false,"dosr":false,"query":{"count":"0","source":"cache","q":"#mozilla include:retweets near:Berlin since:2015-09-21_13:00 until:2015-10-30_17:00","callback":"angular.callbacks._2m","timezoneOffset":"-120","fields":"created_at,screen_name,mentions,hashtags"},"event_cache_time":3823,"event_postprocessing_time":1,"busy":3824,"finish":"2015-10-13T14:53:34.395Z"}

The date is limited to 2015-10-30_17:00 which means most recent results cannot be in the results.

mariobehling commented 9 years ago

The date is limited to 2015-10-30_17:00 which means most recent results cannot be in the results.

From a user perspective the dates have been set to a) start on Sep 21 and b) end on 30 Oct. 2015 (which is in the future)

All tweets in this timeframe should show up.

screenshot from 2015-10-13 17 32 47

Orbiter commented 9 years ago

arr, sorry, yes the date modifer is correct and should work. There may be othe reasons that this does not work but it includes at least the at this time missing include:retweets modifier in loklak_server, see https://github.com/loklak/loklak_server/issues/81 and https://github.com/loklak/loklak_server/issues/100

Orbiter commented 9 years ago

hm, that was also not the cause of the problem. I added more test attributes which you can evaluate now. Please re-test, then open http://loklak.org/api/access.json and locate the query result for the wall. Look for the value "event_result_count": this should appear with value 0 (for aggregation-results) and with value 100 (for tweet results). If you can locate a value > 0 here, then this is probably a bug in the loklak_webclient.

Orbiter commented 9 years ago

fixed with https://github.com/loklak/loklak_webclient/commit/783924decd4df9e9538cae73d2749bc9e5f1379b and following commits replacing === with ==

mariobehling commented 9 years ago

Great!