search

jcustenborder / kafka-connect-twitter

Kafka Connect connector to stream data in real time from Twitter.
Apache License 2.0
126 stars 80 forks source link

Request Refused: Please use V2 filtered and sample volume stream as alternatives #31

Open ajilk opened 2 years ago

ajilk commented 2 years ago

How can I update the twitter API to point to v2?

[2022-05-13 00:46:49,143] INFO 403:The request is understood, but it has been refused. An accompanying error message will explain why. This code is used when requests are being denied due to update limits (https://support.twitter.com/articles/15364-about-twitter-limits-update-api-dm-and-following).
<html>\n<head>\n<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>\n<title>Error 403 
Please use V2 filtered and sample volume stream as alternatives
</title>
</head>
<body>
<h2>HTTP ERROR: 403</h2>
<p>Problem accessing '/1.1/statuses/filter.json'. Reason:
<pre>    
Please use V2 filtered and sample volume stream as alternatives
</pre>
</body>
</html>
 (twitter4j.TwitterStreamImpl:62)
thebetauser commented 2 years ago

I get the same error which unfortunately breaks this connector.

I'm using Twitter API v2 with Elevated access since v1.1 has been depreciated as of May 2022

https://twittercommunity.com/t/deprecation-announcement-removing-compliance-messages-from-statuses-filter-and-retiring-statuses-sample-from-the-twitter-api-v1-1/170500

Issue is with twitter4j which hasn't been updated since 2021 so seems this issue will not be fixed until twitter4j pushes a fix

2022-06-13T23:30:35.199779923Z [2022-06-13 23:30:35,199] INFO 403:The request is understood, but it has been refused. An accompanying error message will explain why. This code is used when requests are being denied due to update limits (https://support.twitter.com/articles/15364-about-twitter-limits-update-api-dm-and-following).
2022-06-13T23:30:35.199828633Z <html>\n<head>\n<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>\n<title>Error 403 
2022-06-13T23:30:35.199836473Z Please use V2 filtered and sample volume stream as alternatives
2022-06-13T23:30:35.199839413Z </title>
2022-06-13T23:30:35.199842003Z </head>
2022-06-13T23:30:35.199844593Z <body>
2022-06-13T23:30:35.199847123Z <h2>HTTP ERROR: 403</h2>
2022-06-13T23:30:35.199850903Z <p>Problem accessing '/1.1/statuses/filter.json'. Reason:
2022-06-13T23:30:35.199854883Z <pre>    
2022-06-13T23:30:35.199859343Z Please use V2 filtered and sample volume stream as alternatives
2022-06-13T23:30:35.199864463Z </pre>
2022-06-13T23:30:35.199867973Z </body>
2022-06-13T23:30:35.199871783Z </html>
2022-06-13T23:30:35.199875473Z  (twitter4j.TwitterStreamImpl)
2022-06-13T23:30:35.199879453Z [2022-06-13 23:30:35,199] WARN This account is not in required role. 403:The request is understood, but it has been refused. An accompanying error message will explain why. This code is used when requests are being denied due to update limits (https://support.twitter.com/articles/15364-about-twitter-limits-update-api-dm-and-following).
2022-06-13T23:30:35.199897943Z <html>\n<head>\n<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>\n<title>Error 403 
2022-06-13T23:30:35.199902723Z Please use V2 filtered and sample volume stream as alternatives
2022-06-13T23:30:35.199906453Z </title>
2022-06-13T23:30:35.199910433Z </head>
2022-06-13T23:30:35.199914243Z <body>
2022-06-13T23:30:35.199917983Z <h2>HTTP ERROR: 403</h2>
2022-06-13T23:30:35.199921893Z <p>Problem accessing '/1.1/statuses/filter.json'. Reason:
2022-06-13T23:30:35.199925963Z <pre>    
2022-06-13T23:30:35.199929353Z Please use V2 filtered and sample volume stream as alternatives
2022-06-13T23:30:35.199932813Z </pre>
2022-06-13T23:30:35.199936113Z </body>
2022-06-13T23:30:35.199939553Z </html>
2022-06-13T23:30:35.199942843Z  (twitter4j.TwitterStreamImpl)
2022-06-13T23:30:35.200338983Z [2022-06-13 23:30:35,199] ERROR onException (com.github.jcustenborder.kafka.connect.twitter.TwitterSourceTask)
2022-06-13T23:30:35.200355113Z 403:The request is understood, but it has been refused. An accompanying error message will explain why. This code is used when requests are being denied due to update limits (https://support.twitter.com/articles/15364-about-twitter-limits-update-api-dm-and-following).
2022-06-13T23:30:35.200360853Z <html>\n<head>\n<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>\n<title>Error 403 
2022-06-13T23:30:35.200371653Z Please use V2 filtered and sample volume stream as alternatives
2022-06-13T23:30:35.200377273Z </title>
2022-06-13T23:30:35.200398473Z </head>
2022-06-13T23:30:35.200403743Z <body>
2022-06-13T23:30:35.200407433Z <h2>HTTP ERROR: 403</h2>
2022-06-13T23:30:35.200411273Z <p>Problem accessing '/1.1/statuses/filter.json'. Reason:
2022-06-13T23:30:35.200415273Z <pre>    
2022-06-13T23:30:35.200418793Z Please use V2 filtered and sample volume stream as alternatives
2022-06-13T23:30:35.200422143Z </pre>
2022-06-13T23:30:35.200425813Z </body>
2022-06-13T23:30:35.200429653Z </html>
2022-06-13T23:30:35.200433213Z 
2022-06-13T23:30:35.200436923Z Relevant discussions can be found on the Internet at:
2022-06-13T23:30:35.200440453Z  http://www.google.co.jp/search?q=1169356e or
2022-06-13T23:30:35.200444313Z  http://www.google.co.jp/search?q=c3c376ff
2022-06-13T23:30:35.200448213Z TwitterException{exceptionCode=[1169356e-c3c376ff], statusCode=403, message=null, code=-1, retryAfter=-1, rateLimitStatus=null, version=4.0.6}
2022-06-13T23:30:35.200452393Z  at twitter4j.HttpClientImpl.handleRequest(HttpClientImpl.java:164)
2022-06-13T23:30:35.200463653Z  at twitter4j.HttpClientBase.request(HttpClientBase.java:57)
2022-06-13T23:30:35.200467803Z  at twitter4j.HttpClientBase.post(HttpClientBase.java:86)
2022-06-13T23:30:35.200471493Z  at twitter4j.TwitterStreamImpl.getFilterStream(TwitterStreamImpl.java:347)
2022-06-13T23:30:35.200475203Z  at twitter4j.TwitterStreamImpl$8.getStream(TwitterStreamImpl.java:323)
2022-06-13T23:30:35.200478853Z  at twitter4j.TwitterStreamImpl$TwitterStreamConsumer.run(TwitterStreamImpl.java:554)
valerioditinno commented 2 years ago

Hi @thebetauser and @ajilk, did you find any work around or alternative kafka-connect source?

conradwt commented 2 years ago

Hey, @thebetauser and @ajilk were you able to find a solution to this issue?

seregamazur commented 2 years ago

Faced the same issue. Any updates on this one?