twintproject / twint

An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.
MIT License
15.82k stars 2.73k forks source link

some wiki/documentation feedback #1035

Open CouldBeThis opened 3 years ago

CouldBeThis commented 3 years ago

Relating to #1033 and while I am thinking about the documentation: I'm brand new which I guess is the best time to provide feedback about this because it's the time of maximum confusion.

This is long but I hope no one minds because it is completely out of a desire to use this very cool piece of software and contribute to it in the only way I'm able.

I have been trying to learn this for the past few days but was totally pulling my hair out because of gaps in existing knowledge. Obviously the developers are very familiar with this stuff and probably lots of users as well.

setup page

order of methods

The preferred install method (pip3) should be listed first. People who know how to use git will not need the coddling.

Of course all end users should read the whole page, thoroughly, before proceeding, but just in case someone didn't.

I am not sure if there is a reason to have 2 different methods listed to install via pip3 but I do notice that this very issue submission box seems to favor one of them. That method should be listed either exclusively or at least first.

start at the beginning

same feedback for readme.md

The instructions should start with opening a terminal and installing pip3. Just link to something like this for Mac/Win/*nix.

provide instructions on how to create and run .py file

I do not even want to tell you how many hours it took for me to figure this out. Most tutorials just have text but they don't tell you what to do with it.

I started writing out actual instructions but a) I am not sure I am doing everything properly and in a way that will universally work, b) I only know my own OS, and c) surely someone has already written clear cross platform instructions about how to create and run a python script? Is there something like a library of public domain documentation snippets?

Even just articulating that it's a python script that needs to be run would be an improvement (once I figured that out it didn't take me very long)... You wouldn't believe the strange and fruitless paths I went down trying to figure out exactly what I was meant to be doing. Let's just say putting import twint directly into the terminal wakes up ImageMagick and I spent quite a while trying to correct that misconfiguration in ??bash?? or something.

Basically every tutorial I found referred to "modules" as though that was supposed to explain things.

This is what should be at the top of the wiki page labeled as "Basic usage"

python basics

Obviously it would help to have had previous experience with python. But seeing as I'd like to learn as I go, it would be helpful to be able to follow what is going on. The python documentation is... thorough. I don't know what I am looking for though. Just enough to be able to see what each thing is meant to be. Like I can see in the highlighting that certain things are orange, red, blue and black, but I don't know why.

I don't expect coding lessons. But, If someone could tell me what I am looking for maybe I can find some resource that would be helpful to a person in my position and that can be added. (Unless such a resource is readily available, of course.)

Although I am very curious about why the letter c is consistently used in all the examples (e.g. c = twint.Config()) and if it has some special significance.

Being able to follow the code would help me to troubleshoot problems. as it is I am kind of changing things randomly which is only even soft of effective because I have lots of time on my hands at the moment. Normally I'd have give up on this ages ago.

segregate features related to optional packages

For example, in options, configurations and parameters and atrributes, things that are available only if you have extra stuff installed (elastic search etc) should be separated.

test.py

I just found this file test.py; which is great cause I was totally wishing such a thing existed. But what is the expected output? Should it be run as is or edited first?


thank you for your time. hopefully something here is helpful. I am willing to work on fixing this if I am capable.

gtaswin commented 3 years ago

I'm trying to get trending [https://twitter.com/explore/tabs/trending] that happening for particular country since last 2 -3 days but no result. Looking for good documentation. Pls, anyone reply with config to get trending.

rachmadaniHaryono commented 3 years ago

I just found this file test.py; which is great cause I was totally wishing such a thing existed. But what is the expected output? Should it be run as is or edited first?

basically just run it with python3 test.py and see if there is any error when running it.

but imo it is not enough because the test result https://github.com/twintproject/twint/blob/a45a8ac719d5710e7da92f77d9801b351e505c1f/test.py#L84 is not tested


For example, in options, configurations and parameters and atrributes, things that are available only if you have extra stuff installed (elastic search etc) should be separated.

i actually do hope this is merged

so user / can see on single page which parameter needed for their case

other solution is to create github page that will get the documentation from the code instead using github wiki

for example there is this pr on command line parameter https://github.com/twintproject/twint/pull/1052

currently it only change the faq section. it should also change the multiple wiki page, code, and also test