Closed rjsu26 closed 4 years ago
@pielco11 can u please help?
@rjsu26 quite busy these days
New code is being pushed. clean_follow_list
was used to clean (objects) used by Pandas' DataFrames.
I renamed that function to a "private" one, _clean_follow_list
. If you are not doing an advanced usage of Twint and so using core functions, you should not use it. To clean lists you may want to do twint.output.users_list = []
, I added a function called twint.output.clean_lists
to reset users_list
,tweets_list
and follows_list
.
So, we now need to use both of the new features of any one of them?
No, if you want to clean the lists, each of them, just call twint.output.clean_lists()
That worked like a charm. Thanks for the immediate correction @pielco11
Also, can you reason out why even though the lists must be getting fetched in batches of 20, my resume files are of many many lines for even small number of followers/followings.
Example: for following count of 20, there are about 25 lines in resume.raw file being generated. While for 700 800 followings, its done in 30 lines max.
Probably Twint retries the requests
Yes, It was somewhat that you pointed about. Thanks for your help @pielco11 .
pip3 install --user --upgrade -e git+https://github.com/twintproject/twint.git@origin/master#egg=twint
;Command Ran
i = 0 dic = {} c = twint.Config() while i<2: i+= 1 c.Username = "rajsahuofficial" c.Hide_output = True c.Store_object = True
while True: try: twint.run.Lookup(c) break except Exception as e: print(e) print("Retrying in 1 sec") time.sleep(1) fol = twint.output.users_list[:] dic["followers"] = fol[0].followers dic["following"] = fol[0].following while True: try: twint.run.Followers(c) break except Exception as e: print(e) print("Retrying in 1 sec") time.sleep(1)
lst = twint.output.follows_list[:] dic["followers_list"] = lst twint.output.clean_follow_list()
while True: try: twint.run.Following(c) break except Exception as e: print(e) print("Retrying in 1 sec") time.sleep(1)
lst1 = twint.output.follows_list[:] dic["following_list"] = lst1 twint.output.clean_follow_list() print(dic)