search

DocNow / twarc

A command line tool (and Python library) for archiving Twitter JSON
https://twarc-project.readthedocs.io
MIT License
1.37k stars 255 forks source link

I could not generate Twitter data by using Twarc #415

Closed Shaimaai closed 3 years ago

Shaimaai commented 3 years ago

Hi everyone!

I hope you are doing well!

I am trying to use Twarc to generate Twitter data, but I could not! once I enter the hashtag (as shown below), I did not get any data?

Does anyone know how I can solve it?

Many thanks!

Screen Shot 2021-03-25 at 9 35 45 AM
SamHames commented 3 years ago

Unfortunately the free Twitter search API only covers the last seven days. It looks like their aren't many recent tweets to retrieve which is why you aren't getting many results.

Options for you to consider:

On Thu, 25 Mar 2021, 15:46 Shaimaai, @.***> wrote:

Hi everyone!

I hope you are doing well!

I am trying to use Twarc to generate Twitter data, but I could not! once I enter the hashtag (as shown below), I did not get any data?

Does anyone know how I can solve it?

Many thanks!

[image: Screen Shot 2021-03-25 at 9 35 45 AM] https://user-images.githubusercontent.com/75612098/112424880-9adbe580-8d4e-11eb-85a3-5d143e41c4d6.png

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/DocNow/twarc/issues/415, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACADAUJ2U7NL47PKIRL6Z7TTFLE4PANCNFSM4ZYVHVKQ .

Shaimaai commented 3 years ago

Unfortunately the free Twitter search API only covers the last seven days. It looks like their aren't many recent tweets to retrieve which is why you aren't getting many results. Options for you to consider: - find and rehydrate an archive of #blacklivesmatter tweets, then filter down - get access to the Twitter academic program and use the full archive search (we will soon be releasing support in Twarc for this) - Use the Twitter premium search (not free :( but supported in Twarc) …

Hi Sam, Thanks for your reply!

I am using API for academic research but once I enter the #hastage and pressing ENTER, nothing happen! as shown in the image that I uploaded before!

SamHames commented 3 years ago

Ah, I see.

What happens if you remove the "> tweets.json" from the end? Do you see tweets on the screen or an error?

If anything is happening you should also see a twarc.log file, is there anything in that?

The twarc command you are using won't use the academic access (but you'll be able to use that soon).

On Thu, 25 Mar 2021, 18:10 Shaimaai, @.***> wrote:

Unfortunately the free Twitter search API only covers the last seven days. It looks like their aren't many recent tweets to retrieve which is why you aren't getting many results. Options for you to consider: - find and rehydrate an archive of #blacklivesmatter tweets, then filter down - get access to the Twitter academic program and use the full archive search (we will soon be releasing support in Twarc for this) - Use the Twitter premium search (not free :( but supported in Twarc) … <#m_8706950287963085262_m1538071797700458402>

Hi Sam, Thanks for your reply!

I am using API for academic research but once I enter the #hastage and pressing ENTER, nothing happen! as shown in the image that I uploaded before!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/DocNow/twarc/issues/415#issuecomment-806452088, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACADAUP547XYVIVYXXFT5D3TFLVXVANCNFSM4ZYVHVKQ .

Shaimaai commented 3 years ago

Ah, I see. What happens if you remove the "> tweets.json" from the end? Do you see tweets on the screen or an error? If anything is happening you should also see a twarc.log file, is there anything in that? The twarc command you are using won't use the academic access (but you'll be able to use that soon). …

Thanks a lot, Sam!

I removed (tweets.json) and I got data (as shown in the attached image) but I do not know where it saved! Do you have any idea where should I find it? because it did not ask me where should I save it!

Do you know, how can I use API for academic? I mean what kind of tool/software can I use to extract historical data.

Thanks,

images.githubusercontent.com/75612098/112444725-c9b28580-8d67-11eb-92d1-d5c859021558.png">

Screen Shot 2021-03-25 at 12 38 00 PM
SamHames commented 3 years ago

The first command you ran with the > tweets.json saved your original search to the file tweets.json. If you run the command "pwd" it will print the directory the file is saved in.

The "twarc" command you ran only supports the original v1.1 Twitter API. The academic API is the new V2 API, and will soon be supported with a new "twarc2" command, but we haven't released it yet. The V2 API requires different code inside twarc, and returns data in a different format.

Cheers, Sam

On Thu, 25 Mar 2021, 18:45 Shaimaai, @.***> wrote:

Ah, I see. What happens if you remove the "> tweets.json" from the end? Do you see tweets on the screen or an error? If anything is happening you should also see a twarc.log file, is there anything in that? The twarc command you are using won't use the academic access (but you'll be able to use that soon). … <#m_162664445905173352_m8209090726082682424>

Thanks a lot, Sam!

I removed (tweets.json) and I got data (as shown in the attached image) but I do not know where it saved! Do you have any idea where should I find it? because it did not ask me where should I save it!

For API academic, I do not know what do you mean! because I have already applied and they confirmed it! (see the image)

[image: Screen Shot 2021-03-25 at 12 41 51 PM] https://user-images.githubusercontent.com/75612098/112444725-c9b28580-8d67-11eb-92d1-d5c859021558.png

[image: Screen Shot 2021-03-25 at 12 38 00 PM] https://user-images.githubusercontent.com/75612098/112444226-42651200-8d67-11eb-88f8-35a395d5c75f.png

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/DocNow/twarc/issues/415#issuecomment-806471761, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACADAUJUOFYGMLPCZIU6ZODTFLZZ7ANCNFSM4ZYVHVKQ .

Shaimaai commented 3 years ago

The first command you ran with the > tweets.json saved your original search to the file tweets.json. If you run the command "pwd" it will print the directory the file is saved in. The "twarc" command you ran only supports the original v1.1 Twitter API. The academic API is the new V2 API, and will soon be supported with a new "twarc2" command, but we haven't released it yet. The V2 API requires different code inside twarc, and returns data in a different format. Cheers, Sam …

Thanks a lot, Sam! I really appreciate your help and time :)

Stay well and Safe!

edsu commented 3 years ago

Hopefully it's ok to have closed this ticket. Keep an eye out for the twarc v2 release next week (hopefully).