markowanga / stweet

Advanced python library to scrap Twitter (tweets, users) from unofficial API
MIT License
580 stars 67 forks source link

class SearchTweetsTask, since、until、tweets_limit not work #92

Closed pylidou closed 1 year ago

pylidou commented 1 year ago
import arrow
import stweet as st

since = arrow.get('2022-1-11')
until = arrow.get('2022-12-13')

def try_search():
    search_tweets_task = st.SearchTweetsTask(from_username='@elonmusk',since=since, until=until,tweets_limit=1000)
    output_jl_tweets = st.JsonLineFileRawOutput('output_raw_search_tweets2.jl')
    output_jl_users = st.JsonLineFileRawOutput('output_raw_search_users2.jl')
    output_print = st.PrintRawOutput()
    st.TweetSearchRunner(search_tweets_task=search_tweets_task,
                         tweet_raw_data_outputs=[output_print, output_jl_tweets],
                         user_raw_data_outputs=[output_print, output_jl_users]).run()

when i run the code above, I only got 23 pieces of data, data date from 2022-12-2 to 2022-12-12, I tried many times, The result is the same

1689665417 commented 1 year ago

Have you solved your problem? I have the same problem.

928095762 commented 1 year ago

Are you solved? I have the same question.

markowanga commented 1 year ago

I'm working on it

markowanga commented 1 year ago

First thing -- Twitter bloc send since and until as timestamp, now works only date in format YYYY-MM-DD Second thing -- Twitter block in guest API many scrolls (I have fixed one error -- now is 51 tweets, fix will be in next version). To better performance use scrapping by day range.