search

transist / echidna-spider

0 stars 0 forks source link

"Failed to gather user: check sign error" after restart #32

Open rngadam opened 11 years ago

rngadam commented 11 years ago

Restarting the spider service and I'm seeing this:

Apr  2 14:57:25 echidna1 [echidna:spider][5994]: Tencent Weibo agent flyerhzm: Failed to gather user: check sign error

full log:

Apr  2 14:57:25 echidna1 [echidna:spider][5994]: Tencent Weibo agent flyerhzm: Gathering users...
Apr  2 14:57:25 echidna1 [echidna:spider][5994]: Tencent Weibo agent flyerhzm: Gathering first user from tweets of keyword "额首称庆"...
Apr  2 14:57:25 echidna1 [echidna:spider][5994]: Tencent Weibo agent flyerhzm: Cache hit: GET api/search/t {:keyword=>"额首称庆", :pagesize=>30}
Apr  2 14:57:25 echidna1 [echidna:spider][5994]: Tencent Weibo agent flyerhzm: Failed to gather user: check sign error
Apr  2 14:57:25 echidna1 [echidna:spider][5994]: Tencent Weibo agent flyerhzm: Finished users gathering
Apr  2 14:57:25 echidna1 [echidna:spider][5994]: Tencent Weibo agent flyerhzm: Tracking users...
Apr  2 14:57:25 echidna1 [echidna:spider][5994]: Tencent Weibo agent flyerhzm: Aborted users tracking: Tencent Weibo API calls limitation reached
Apr  2 14:57:25 echidna1 [echidna:spider][5994]: Tencent Weibo agent flyerhzm: Gathering tweets...
Apr  2 14:57:25 echidna1 [echidna:spider][5994]: Tencent Weibo agent flyerhzm: Aborted tweets gathering: Tencent Weibo API calls limitation reached
rainux commented 11 years ago

I did a backup of the redis database, then run the following script to overrite the data under echidna account with my version. This action fixed this access token expired issue, and ensure the spider under echidna account runs with a dedicated Tencent Weibo account agent0xff8.

#!/usr/bin/env ruby
                                                                                                                                                                            require 'redis'

Redis.current.keys('e:echidna:p:*').each do |key|
  Redis.current.del(key)
end

Redis.current.keys('e:rainux:p:*').each do |key|
  Redis.current.rename key, key.gsub(/^e:rainux:/, 'e:echidna:')
end
rainux commented 11 years ago

Since the master branch of echidna-streaming has a bug that prevent us get correct data, I've redid the migration with the backup data, and running z-score branch of echidna-streaming under echidna account. But the pull requests for both the z-score and rainux branches are not reviewed and merged to master.

flyerhzm commented 11 years ago

I have merged your branch to echidna-streaming, but we need to add tests to you code