search

DocNow / twarc

A command line tool (and Python library) for archiving Twitter JSON
https://twarc-project.readthedocs.io
MIT License
1.37k stars 255 forks source link

Add /1.1/users/search endpoint and twarc1 command #631

Open igorbrigadir opened 2 years ago

igorbrigadir commented 2 years ago

There is a v1.1 endpoint for searching for users https://developer.twitter.com/en/docs/twitter-api/v1/accounts-and-users/follow-search-get-users/api-reference/get-users-search that's not yet implemented for v1.1.

It's not a very reliable search, and can only return 1000 users max, however, it could be useful for some people. Since this is for the v1.1 API which will go away eventually, it shouldn't be a big priority, but it would be great if someone new could implement this endpoint. I think it makes for a great opportunity for a first contribution for someone.

In v2, the streaming API provides some operators for filtering on user details but there is no equivalent search. https://developer.twitter.com/en/docs/twitter-api/tweets/filtered-stream/integrate/build-a-rule#list

igorbrigadir commented 2 years ago

cc @DavidHernandez because you were interested in this earlier and i didn't think of this endpoint at all at the time.

DavidHernandez commented 2 years ago

Thanks, @igorbrigadir I will try to find someone to do it.

mariosanz92 commented 2 years ago

Hi @igorbrigadir I am working on this issue and I have a question, how can I prove the new command locally before push my changes? i don't find the way to do it. Thanks

igorbrigadir commented 2 years ago

Sure thing!

To test your new endpoint you're going to add to https://github.com/DocNow/twarc/blob/main/twarc/client.py, add a new test case to https://github.com/DocNow/twarc/blob/main/test_twarc.py maybe called test_user_search (even a really simple one will do, like the https://github.com/DocNow/twarc/blob/main/test_twarc.py#L49 one) that way you can test your new function using:

pytest -k test_user_search

Something like that.

Then when the client library is updated, to add a new command line option to the tool, edit https://github.com/DocNow/twarc/blob/main/twarc/command.py#L35 to add a new command, same as the others.

To make sure you're working with the version of the code you're modifying, it helps to be in a new python environment, and to install twarc from the code you're working on, not from pypi using:

pip install -e .

when inside the twarc folder.

Now any edits you make to the command line or client should be updated immediately in your environment.

Hope that helps!

mariosanz92 commented 2 years ago

Thanks for the help igorbrigadir, I will try it.