pathwaycom / pathway

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
https://pathway.com
Other
2.84k stars 98 forks source link

[Bug]: Reproduction of Twitter's Custom Python Connector code encours Auth issues #43

Closed TaufeeqNoamaan closed 2 months ago

TaufeeqNoamaan commented 2 months ago

Steps to reproduce

I've used the code directly from the Python Custom Connector example as give below: Code

There's a constant Authentication issue when I use the BEARER_TOKEN generated directly from Twitter's API dashboard.

I'm using the default project and the default app generated by twitter upon signing-up for free tier

Relevant log output

When use BEARER TOKEN following is the log-output on dashboard:

Stream encountered HTTP error: 403                                                                                                                    
                      ERROR    HTTP error response text: {"client_id":"28689517","detail":"When authenticating requests to the Twitter API v2 endpoints, you must use keys and       
                               tokens from a Twitter developer App that is attached to a Project. You can create a project via the developer                                         
                               portal.","registration_url":"https://developer.twitter.com/en/docs/projects/overview","title":"Client Forbidden","required_enrollment":"Appropriate   
                               Level of API Access","reason":"client-not-enrolled","type":"https://api.twitter.com/2/problems/client-forbidden"}

What did you expect to happen?

Expected to run without any AUTH issues

Version

0.9.0

Docker Versions (if used)

No response

OS

Linux

On which CPU architecture did you run Pathway?

x86-64

voodoo11 commented 2 months ago

Thanks for reporting. This is not really an issue with Pathway. It seems that Twitter API has changed, and streaming endpoints are no longer available, at least in the free tier.