search

n0madic / twitter-scraper

Scrape the Twitter frontend API without authentication with Golang.
MIT License
899 stars 187 forks source link

unexpected end of JSON input #96

Closed clownfish2023 closed 1 year ago

clownfish2023 commented 1 year ago

I use the latest version to search, but reported this error "unexpected end of JSON input". github.com/n0madic/twitter-scraper v0.0.0-20230510192940-b65855f49dc

scraper.SetSearchMode(twitterscraper.SearchLatest)
for tweet := range scraper.SearchTweets(context.Background(), tweetQuery, 100) {
                if tweet.Error != nil {
                        log.Println("search error:",tweet.Error)
                        continue
                }
}
n0madic commented 1 year ago

did you log in before using? search now does not work without authorization ..

clownfish2023 commented 1 year ago

did you log in before using? search now does not work without authorization ..

Yes,i'm already logged in.

n0madic commented 1 year ago

I can not reproduce - I have a search without problems .. scraper.IsLoggedIn() == true ?

clownfish2023 commented 1 year ago

It's false.Do I need to log in with a developer account? I made a mistake earlier and used my personal account to log in

n0madic commented 1 year ago

You must log in as user, like on Twitter site

clownfish2023 commented 1 year ago

You must log in as user, like on Twitter site

At the beginning, when I log in with my personal account, I will report this error “37 Your credentials do not allow access to this resource”，then I set my account to remove 2fa authentication，now report "auth error (399): Incorrect. Please try again"

n0madic commented 1 year ago

in browser you log in without problem and email verification?

clownfish2023 commented 1 year ago

in browser you log in without problem and email verification?

There is no problem logging in on chrome, only fill in the user name, password, and email address to log in successfully

n0madic commented 1 year ago

Both logins on same computer? If you have email confirmation, use email address in addition:

err := scraper.Login("username", "password", "email")
clownfish2023 commented 1 year ago

Both logins on same computer? If you have email confirmation, use email address in addition:

err := scraper.Login("username", "password", "email")

The error occurred is running on the ubuntu server.Can this still be supported? I already provide the email

n0madic commented 1 year ago

It looks like Twitter is blocking requests from strange places like datacenters, so I can't set up CI in GitHub Action...

clownfish2023 commented 1 year ago

It looks like Twitter is blocking requests from strange places like datacenters, so I can't set up CI in GitHub Action...

Is there any way to solve it?

n0madic commented 1 year ago

maybe try using a proxy..

clownfish2023 commented 1 year ago

maybe try using a proxy..

I changed a cloud host and it works now