search

WizardFactory / BlogSyncer

one write posting all of the world
Apache License 2.0
3 stars 10 forks source link

twitter에서 작성한 글이 kakao story에 두번 올라가는 문제. #335

Closed kimalec closed 9 years ago

kimalec commented 9 years ago

새 글 정보를 가지고 올때 아래와 같은 에러가 발생했지만, 증상과는 상관 없는듯

 » 27 Jul 2015 11:45:28.843 2015-07-27 02:45:28.557800+00:00 app web.1 - - error: { [Error: getaddrinfo ENOTFOUND www.kbench.com] Fatal
» 27 Jul 2015 11:45:28.844 2015-07-27 02:45:28.557805+00:00 app web.1 - - code: 'ENOTFOUND',
» 27 Jul 2015 11:45:28.844 2015-07-27 02:45:28.557807+00:00 app web.1 - - errno: 'ENOTFOUND',
» 27 Jul 2015 11:45:28.844 2015-07-27 02:45:28.557809+00:00 app web.1 - - syscall: 'getaddrinfo',
» 27 Jul 2015 11:45:28.844 2015-07-27 02:45:28.557811+00:00 app web.1 - - hostname: 'www.kbench.com' } cName=twitter, userId=558c10590528f4ac83c0f061, url=/bot_posts/2378038464/625497005133934592?userid=558c10590528f4ac83c0f061

facebook이 postInfo에 없는 것을 확인.

 » 27 Jul 2015 11:45:28.844 2015-07-27 02:45:28.578305+00:00 app web.1 - - verbose: Fail to get postInfo
» 27 Jul 2015 11:45:28.844 2015-07-27 02:45:28.578379+00:00 app web.1 - - verbose: providerName=facebook blogId=666944600038182
» 27 Jul 2015 11:45:28.844 2015-07-27 02:45:28.579047+00:00 app web.1 - - verbose: { provider_name: 'twitter',
» 27 Jul 2015 11:45:28.844 2015-07-27 02:45:28.579049+00:00 app web.1 - - blog_id: '2378038464',
» 27 Jul 2015 11:45:28.844 2015-07-27 02:45:28.579051+00:00 app web.1 - - post_id: '625497005133934592',
» 27 Jul 2015 11:45:28.844 2015-07-27 02:45:28.579052+00:00 app web.1 - - url: 'https://twitter.com/2378038464/status/625497005133934592',
» 27 Jul 2015 11:45:28.845 2015-07-27 02:45:28.579054+00:00 app web.1 - - modified: Mon Jul 27 2015 02:44:41 GMT+0000 (UTC),
» 27 Jul 2015 11:45:28.845 2015-07-27 02:45:28.579055+00:00 app web.1 - - _id: 55b59b46594042110013662f }

facebook에 새글 등록

» 27 Jul 2015 11:45:28.845 2015-07-27 02:45:28.579178+00:00 app web.1 - - info: postId=625497005133934592 to provider=facebook blog=666944600038182 cName=BlogBot, fName=_cbSendPostToBlogs, userId=558c10590528f4ac83c0f061
» 27 Jul 2015 11:45:28.885 2015-07-27 02:45:28.599803+00:00 app web.1 - - info: + cName=facebook, userId=558c10590528f4ac83c0f061, url=/bot_posts/new/666944600038182?userid=558c10590528f4ac83c0f061
 » 27 Jul 2015 11:45:28.919 2015-07-27 02:45:28.581729+00:00 app web.1 - - verbose: Post url=http://www.justwapps.com/facebook/bot_posts/new/666944600038182?userid=558c10590528f4ac83c0f061 cName=BlogBot, fName=_requestPostContent, userId=558c10590528f4ac83c0f061 
 » 27 Jul 2015 11:45:29.803 2015-07-27 02:45:29.511645+00:00 heroku router - - at=info method=POST path="/facebook/bot_posts/new/666944600038182?userid=558c10590528f4ac83c0f061" host=www.justwapps.com request_id=224fec76-f3b8-4e9e-a46a-3a1760bcd7a5 fwd="54.82.83.239/ec2-54-82-83-239.compute-1.amazonaws.com" dyno=web.1 connect=1ms service=929ms status=200 bytes=497
 » 27 Jul 2015 11:45:29.812 2015-07-27 02:45:29.525930+00:00 app web.1 - - info: !!! cName=BlogBot, fName=_cbAddPostInfoToDb, userId=558c10590528f4ac83c0f061

kakao stroy에 postInfo확인

 » 27 Jul 2015 11:45:28.845 2015-07-27 02:45:28.580322+00:00 app web.1 - - verbose: Fail to get postInfo
» 27 Jul 2015 11:45:28.845 2015-07-27 02:45:28.580390+00:00 app web.1 - - verbose: providerName=kakao blogId=4514421
» 27 Jul 2015 11:45:28.845 2015-07-27 02:45:28.580748+00:00 app web.1 - - verbose: { provider_name: 'twitter',
» 27 Jul 2015 11:45:28.845 2015-07-27 02:45:28.580750+00:00 app web.1 - - blog_id: '2378038464',
» 27 Jul 2015 11:45:28.845 2015-07-27 02:45:28.580751+00:00 app web.1 - - post_id: '625497005133934592',
» 27 Jul 2015 11:45:28.845 2015-07-27 02:45:28.580753+00:00 app web.1 - - url: 'https://twitter.com/2378038464/status/625497005133934592',
 » 27 Jul 2015 11:45:28.919 2015-07-27 02:45:28.580754+00:00 app web.1 - - modified: Mon Jul 27 2015 02:44:41 GMT+0000 (UTC),
» 27 Jul 2015 11:45:28.919 2015-07-27 02:45:28.580756+00:00 app web.1 - - _id: 55b59b46594042110013662f }

kakao stroy에 posting

» 27 Jul 2015 11:45:28.919 2015-07-27 02:45:28.580881+00:00 app web.1 - - info: postId=625497005133934592 to provider=kakao blog=4514421 cName=BlogBot, fName=_cbSendPostToBlogs, userId=558c10590528f4ac83c0f061
 » 27 Jul 2015 11:45:29.812 2015-07-27 02:45:29.526416+00:00 app web.1 - - verbose: Post url=http://www.justwapps.com/kakao/bot_posts/new/4514421?userid=558c10590528f4ac83c0f061 cName=BlogBot, fName=_requestPostContent, userId=558c10590528f4ac83c0f061

kakao stroy에 posting 중에 timeout이 발생함.

 » 27 Jul 2015 11:45:29.888 2015-07-27 02:45:29.666679+00:00 app web.1 - - info: + cName=kakao, userId=558c10590528f4ac83c0f061, url=/bot_posts/new/4514421?userid=558c10590528f4ac83c0f061
» 27 Jul 2015 11:45:40.735 2015-07-27 02:45:40.455923+00:00 heroku router - - at=info method=POST path="/kakao/bot_posts/new/4514421?userid=558c10590528f4ac83c0f061" host=www.justwapps.com request_id=d9bbd744-9d01-4b3e-94a0-1a92544178a5 fwd="54.82.83.239/ec2-54-82-83-239.compute-1.amazonaws.com" dyno=web.1 connect=1ms service=10798ms status=400 bytes=500 High Response Time
» 27 Jul 2015 11:45:40.742 2015-07-27 02:45:40.455562+00:00 app web.1 - - error: { statusCode: 400, Fatal
» 27 Jul 2015 11:45:40.742 2015-07-27 02:45:40.455567+00:00 app web.1 - - body: '{"code":-603,"msg":"timed out for waiting a response from http://www.kbench.com/?q=node/153416"}',
» 27 Jul 2015 11:45:40.742 2015-07-27 02:45:40.455570+00:00 app web.1 - - url: 'https://kapi.kakao.com/v1/api/story/linkinfo?url=http://www.kbench.com/?q=node/153416' } cName=kakao, fName=_makeLinkPost
» 27 Jul 2015 11:45:40.742 2015-07-27 02:45:40.456055+00:00 app web.1 - - error: { statusCode: 400, Fatal
» 27 Jul 2015 11:45:40.742 2015-07-27 02:45:40.456058+00:00 app web.1 - - body: '{"code":-603,"msg":"timed out for waiting a response from http://www.kbench.com/?q=node/153416"}',
» 27 Jul 2015 11:45:40.742 2015-07-27 02:45:40.456060+00:00 app web.1 - - url: 'https://kapi.kakao.com/v1/api/story/linkinfo?url=http://www.kbench.com/?q=node/153416' } cName=kakao, userId=558c10590528f4ac83c0f061, url=/bot_posts/new/4514421?userid=558c10590528f4ac83c0f061
» 27 Jul 2015 11:45:40.742 2015-07-27 02:45:40.460181+00:00 app web.1 - - warn: { statusCode: 400,
» 27 Jul 2015 11:45:40.743 2015-07-27 02:45:40.460185+00:00 app web.1 - - body: '"{\\"statusCode\\":400,\\"body\\":\\"{\\\\\\"code\\\\\\":-603,\\\\\\"msg\\\\\\":\\\\\\"timed out for waiting a response from http://www.kbench.com/?q=node/153416\\\\\\"}\\",\\"url\\":\\"https://kapi.kakao.com/v1/api/story/linkinfo?url=http://www.kbench.com/?q=node/153416\\"}"',
» 27 Jul 2015 11:45:40.743 2015-07-27 02:45:40.460187+00:00 app web.1 - - url: 'http://www.justwapps.com/kakao/bot_posts/new/4514421?userid=558c10590528f4ac83c0f061' } cName=BlogBot, fName=_requestPostContent, userId=558c10590528f4ac83c0f061
» 27 Jul 2015 11:45:40.743 2015-07-27 02:45:40.504865+00:00 app web.1 - - warn: Fail to post content cName=BlogBot, fName=_cbSendPostToBlogs, userId=558c10590528f4ac83c0f061
» 27 Jul 2015 11:45:40.743 2015-07-27 02:45:40.505563+00:00 app web.1 - - warn: undefined cName=BlogBot, fName=_cbPushPostsToBlogs, userId=558c10590528f4ac83c0f061
» 27 Jul 2015 11:45:40.743 2015-07-27 02:45:40.506773+00:00 app web.1 - - info: - cName=BlogBot, fName=_getAndPush, userId=558c10590528f4ac83c0f061

timed out error로 인하여, _retryPostings에서 한번더 posting하지만 timed out 발생함.

 » 27 Jul 2015 11:45:40.743 2015-07-27 02:45:40.506929+00:00 app web.1 - - info: + cName=BlogBot, fName=_retryPostings
» 27 Jul 2015 11:45:40.743 2015-07-27 02:45:40.507512+00:00 app web.1 - - verbose: Post url=http://www.justwapps.com/kakao/bot_posts/new/4514421?userid=558c10590528f4ac83c0f061 cName=BlogBot, fName=_requestPostContent, userId=558c10590528f4ac83c0f061
» 27 Jul 2015 11:45:40.791 2015-07-27 02:45:40.509434+00:00 app web.1 - - info: - cName=BlogBot, fName=_retryPostings

 » 27 Jul 2015 11:45:51.627 2015-07-27 02:45:51.382568+00:00 heroku router - - at=info method=POST path="/kakao/bot_posts/new/4514421?userid=558c10590528f4ac83c0f061" host=www.justwapps.com request_id=2e52f106-57ce-448f-bc03-e631de90485e fwd="54.82.83.239/ec2-54-82-83-239.compute-1.amazonaws.com" dyno=web.1 connect=2ms service=10803ms status=400 bytes=506 High Response Time
» 27 Jul 2015 11:45:51.665 2015-07-27 02:45:51.378342+00:00 app web.1 - - error: { statusCode: 400, Fatal
» 27 Jul 2015 11:45:51.666 2015-07-27 02:45:51.378347+00:00 app web.1 - - body: '{"code":-603,"msg":"timed out for waiting a response from http://www.kbench.com/?q=node/153416"}',
» 27 Jul 2015 11:45:51.666 2015-07-27 02:45:51.378350+00:00 app web.1 - - url: 'https://kapi.kakao.com/v1/api/story/linkinfo?url=http://www.kbench.com/?q=node/153416' } cName=kakao, fName=_makeLinkPost
» 27 Jul 2015 11:45:51.666 2015-07-27 02:45:51.378513+00:00 app web.1 - - error: { statusCode: 400, Fatal
» 27 Jul 2015 11:45:51.666 2015-07-27 02:45:51.378515+00:00 app web.1 - - body: '{"code":-603,"msg":"timed out for waiting a response from http://www.kbench.com/?q=node/153416"}',
» 27 Jul 2015 11:45:51.666 2015-07-27 02:45:51.378517+00:00 app web.1 - - url: 'https://kapi.kakao.com/v1/api/story/linkinfo?url=http://www.kbench.com/?q=node/153416' } cName=kakao, userId=558c10590528f4ac83c0f061, url=/bot_posts/new/4514421?userid=558c10590528f4ac83c0f061
» 27 Jul 2015 11:45:51.666 2015-07-27 02:45:51.382287+00:00 app web.1 - - warn: { statusCode: 400,
» 27 Jul 2015 11:45:51.741 2015-07-27 02:45:51.382290+00:00 app web.1 - - body: '"{\\"statusCode\\":400,\\"body\\":\\"{\\\\\\"code\\\\\\":-603,\\\\\\"msg\\\\\\":\\\\\\"timed out for waiting a response from http://www.kbench.com/?q=node/153416\\\\\\"}\\",\\"url\\":\\"https://kapi.kakao.com/v1/api/story/linkinfo?url=http://www.kbench.com/?q=node/153416\\"}"',
» 27 Jul 2015 11:45:51.741 2015-07-27 02:45:51.382292+00:00 app web.1 - - url: 'http://www.justwapps.com/kakao/bot_posts/new/4514421?userid=558c10590528f4ac83c0f061' } cName=BlogBot, fName=_requestPostContent, userId=558c10590528f4ac83c0f061
» 27 Jul 2015 11:45:51.741 2015-07-27 02:45:51.430810+00:00 app web.1 - - warn: Fail to post content
» 27 Jul 2015 11:45:51.741 2015-07-27 02:45:51.430946+00:00 app web.1 - - warn: [object Object]

다음 task에서 facebook에서 새로운 post가 발견됨.(실제로는 twitter를 통해 등록된 글)

 » 27 Jul 2015 11:46:42.614 2015-07-27 02:46:42.329636+00:00 app web.1 - - info: + cName=facebook, userId=558c10590528f4ac83c0f061, url=/bot_posts/666944600038182/666944600038182_949146518484654?userid=558c10590528f4ac83c0f061 
 » 27 Jul 2015 11:46:44.695 2015-07-27 02:46:44.433124+00:00 heroku router - - at=info method=GET path="/facebook/bot_posts/666944600038182/666944600038182_949146518484654?userid=558c10590528f4ac83c0f061" host=www.justwapps.com request_id=a62fd133-5e93-491f-a683-50897d8c5410 fwd="54.82.83.239/ec2-54-82-83-239.compute-1.amazonaws.com" dyno=web.1 connect=1ms service=2104ms status=200 bytes=1554
» 27 Jul 2015 11:46:44.724 2015-07-27 02:46:44.438504+00:00 app web.1 - - verbose: Fail to get postInfo
» 27 Jul 2015 11:46:44.724 2015-07-27 02:46:44.438548+00:00 app web.1 - - verbose: providerName=kakao blogId=4514421
» 27 Jul 2015 11:46:44.724 2015-07-27 02:46:44.439064+00:00 app web.1 - - verbose: { provider_name: 'twitter',
» 27 Jul 2015 11:46:44.724 2015-07-27 02:46:44.439067+00:00 app web.1 - - blog_id: '2378038464',
» 27 Jul 2015 11:46:44.724 2015-07-27 02:46:44.439068+00:00 app web.1 - - post_id: '625497005133934592',
» 27 Jul 2015 11:46:44.724 2015-07-27 02:46:44.439070+00:00 app web.1 - - url: 'https://twitter.com/2378038464/status/625497005133934592',
» 27 Jul 2015 11:46:44.724 2015-07-27 02:46:44.439072+00:00 app web.1 - - modified: Mon Jul 27 2015 02:44:41 GMT+0000 (UTC),
» 27 Jul 2015 11:46:44.724 2015-07-27 02:46:44.439073+00:00 app web.1 - - _id: 55b59b46594042110013662f },{ provider_name: 'facebook',
» 27 Jul 2015 11:46:44.724 2015-07-27 02:46:44.439075+00:00 app web.1 - - blog_id: '666944600038182',
» 27 Jul 2015 11:46:44.724 2015-07-27 02:46:44.439076+00:00 app web.1 - - post_id: '666944600038182_949146518484654',
» 27 Jul 2015 11:46:44.770 2015-07-27 02:46:44.439077+00:00 app web.1 - - url: 'http://',
» 27 Jul 2015 11:46:44.770 2015-07-27 02:46:44.439079+00:00 app web.1 - - modified: Thu Jan 01 1970 00:00:00 GMT+0000 (UTC),
» 27 Jul 2015 11:46:44.770 2015-07-27 02:46:44.439080+00:00 app web.1 - - _id: 55b59b495940421100136630 }

새글 등록?

 » 27 Jul 2015 11:46:44.761 2015-07-27 02:46:44.476275+00:00 app web.1 - - info: + cName=kakao, userId=558c10590528f4ac83c0f061, url=/bot_posts/new/4514421?userid=558c10590528f4ac83c0f061
» 27 Jul 2015 11:46:44.770 2015-07-27 02:46:44.439201+00:00 app web.1 - - info: postId=666944600038182_949146518484654 to provider=kakao blog=4514421 cName=BlogBot, fName=_cbSendPostToBlogs, userId=558c10590528f4ac83c0f061
 » 27 Jul 2015 11:46:44.770 2015-07-27 02:46:44.439683+00:00 app web.1 - - verbose: Post url=http://www.justwapps.com/kakao/bot_posts/new/4514421?userid=558c10590528f4ac83c0f061 cName=BlogBot, fName=_requestPostContent, userId=558c10590528f4ac83c0f061
» 27 Jul 2015 11:46:46.625 2015-07-27 02:46:46.339465+00:00 app web.1 - - info: !!! cName=BlogBot, fName=_cbAddPostInfoToDb, userId=558c10590528f4ac83c0f061
 » 27 Jul 2015 11:46:46.651 2015-07-27 02:46:46.327037+00:00 heroku router - - at=info method=POST path="/kakao/bot_posts/new/4514421?userid=558c10590528f4ac83c0f061" host=www.justwapps.com request_id=b5bd34d8-8b75-49a9-a840-343b30735e22 fwd="54.82.83.239/ec2-54-82-83-239.compute-1.amazonaws.com" dyno=web.1 connect=3ms service=1872ms status=200 bytes=1480

_retryPostings에서 kakao에 새글 등록

 » 27 Jul 2015 11:46:52.490 2015-07-27 02:46:52.330837+00:00 app web.1 - - info: + cName=BlogBot, fName=_retryPostings
» 27 Jul 2015 11:46:52.490 2015-07-27 02:46:52.331162+00:00 app web.1 - - verbose: Post url=http://www.justwapps.com/kakao/bot_posts/new/4514421?userid=558c10590528f4ac83c0f061 cName=BlogBot, fName=_requestPostContent, userId=558c10590528f4ac83c0f061
» 27 Jul 2015 11:46:52.490 2015-07-27 02:46:52.331761+00:00 app web.1 - - info: - cName=BlogBot, fName=_retryPostings
kimalec commented 9 years ago

원인 : posting이 실패하는 경우에, retry queue에 존재하게 되고, 새글을 찾아서 갱신하는 경우에도 실패한 post을 찾아내서 posting을 시도하게 되면서, 두 번 들어가게 된다.

개선이 필요한 포인트 _retryPostings에서도 callback을 연결하여 순차처리되도록 개선 필요 _retryPosting에도 userId표시 _cbAddPostInfoToDb 에서 DB에 추가될 postinfo log 출력

차라리 retryPosting을 없애고, 들어가지 않는 글을 찾아서 추가 하는 부분을 사용하는 것이 좋을 것으로 판단됨.

kimalec commented 9 years ago

316 에서 추가한 retry 제거