Client forbidden

[aborruso]

Hi, I have configured twarc2

Your keys have been written to /home/aborruso/.config/twarc/config

✨ ✨ ✨  Happy twarcing! ✨ ✨ ✨

But when I run twarc2 conversation 1406852944784412675 I have

⚡ Client Forbidden

My log

2021-06-21 09:41:00,283 INFO using config /home/aborruso/.config/twarc/config
2021-06-21 09:41:00,283 INFO creating HTTP session headers for app auth.
2021-06-21 09:41:00,283 INFO getting ('https://api.twitter.com/2/tweets/search/recent',) {'params': {'expansions': 'author_id,in_reply_to_user_id,referenced_tweets.id,referenced_tweets.id.author_id,entities.mentions.username,attachments.poll_ids,attachments.media_keys,geo.place_id', 'user.fields': 'created_at,description,entities,id,location,name,pinned_tweet_id,profile_image_url,protected,public_metrics,url,username,verified,withheld', 'tweet.fields': 'attachments,author_id,context_annotations,conversation_id,created_at,entities,geo,id,in_reply_to_user_id,lang,public_metrics,text,possibly_sensitive,referenced_tweets,reply_settings,source,withheld', 'media.fields': 'duration_ms,height,media_key,preview_image_url,type,url,width,public_metrics', 'poll.fields': 'duration_minutes,end_datetime,id,options,voting_status', 'place.fields': 'contained_within,country,country_code,full_name,geo,id,name,place_type', 'max_results': 100, 'query': 'conversation_id:1406852944784412675'}}

If I run twarc filter covid I have no error. I'm using twarc v2.2.0.

Thank you

[edsu]

Interesting! You must have also run twarc configure for the twarc filter covid command to succeed. twarc and twarc2 both use separate configuration files at the moment.

Can you open the twarc2 config file using a text editor and confirm your keys are correct?

/home/aborruso/.config/twarc/config

[igorbrigadir]

This can also come up if the bearer token / api keys you used to configure are not attached to a Project in your dashboard https://developer.twitter.com/en/portal/projects-and-apps

[aborruso]

@edsu my two config files

[edsu]

Ok good. As @igorbrigadir was saying do you see the app for those keys attached to project in the developer portal? The app should be nested under the Projects & Apps section in the menu on the left?

[aborruso]

@edsu yes I have it :(

[edsu]

If you run twarc2 --verbose conversation 1406852944784412675 do you see a line in your twarc.log file that looks like this:

2021-06-21 11:16:07,050 DEBUG authorization: Bearer ... 

and does the bearer token listed there match the one you see in the developer portal?

[aborruso]

@edsu I have in the log, what I have in /home/aborruso/.config/twarc/config

and does the bearer token listed there match the one you see in the developer portal?

In the portal I cannot see the token

[edsu]

Ah yes, that's true. You only get to see it once (when it is generated) for security reasons. Could you try regenerating the bearer token and configuring twarc2 with it again?

twarc2 configure

I've done a partial copy/paste before, and doing it again might catch this kind of problem.

[aborruso]

Could you try regenerating the bearer token and configuring twarc2 with it again?

If I regenerate it, do I lose the ability to use other apps based on the same keys and tokens?

[edsu]

Yes, anywhere you are currently using the BEARER TOKEN for that app will need to be updated. Are you successfully using the BEARER TOKEN in other applications?

[aborruso]

Yes, anywhere you are currently using the BEARER TOKEN for that app will need to be updated

@edsu before doing this without knowing if it will affect other things, I have a question for you: does this configuration method - generate access keys by visiting Twitter -then not work (I have done a screen recording, below)?

Thank you

https://www.youtube.com/watch?v=HJDeqLgIIVM

[edsu]

Thanks for the video! It demonstrates that you have copied the API Key and API Secret correctly, or else the authorization step you performed at twitter.com would have failed. However it doesn't demonstrate that you have copied the Bearer Token correctly.

If you don't want to reset your app's Bearer Token you could also try configuring twarc2 without the Bearer Token. The Bearer Token is really only needed when you don't configure twarc with your API Key and API Secret.

To try this press ENTER when prompted for the Bearer Token to leave it blank and then enter your API Key and API Secret as you did in the video you made. Then try the conversation command to test.

[aborruso]

Thanks for the video! It demonstrates that you have copied the API Key and API Secret correctly, or else the authorization step you performed at twitter.com would have failed. However it doesn't demonstrate that you have copied the Bearer Token correctly.

@edsu thank you. There is something I don't understand. I launch the config procedure, I receive a positive result and in the end the app does not work.

If you don't want to reset your app's Bearer Token you could also try configuring twarc2 without the Bearer Token. The Bearer Token is really only needed when you don't configure twarc with your API Key and API Secret.

Please don't hate me, I'm dumb and I don't understand: I launched the config procedure without using the token. Am I wrong?

To try this press ENTER when prompted for the Bearer Token to leave it blank and then enter your API Key and API Secret as you did in the video you made. Then try the conversation command to test.

I already did. If you watch the video I left it blank and I tried conversation command and it does not work

[edsu]

Oh I'm sorry, you're not dumb, actually I am since I somehow missed that you skipped the Bearer Token entry in the video! :laughing:

I think @igorbrigadir was on the right track above. I think this behavior you are seeing is the result of your App not being in a Project and is instead in the in Standalone Apps section. Can you confirm the name of your App that you are using keys from and share what you see in the left hand menu of your Twitter Developer Portal? For example here's mine:

Of course Twitter don't let you see your keys once you've initially created them so you might be mistaken about what App keys you are actually using unless you keep a separate record of them. I've often confused mine and now keep a record of them in my password manager to keep myself from getting confused.

[aborruso]

I think @igorbrigadir was on the right track above. I think this behavior you are seeing is the result of your App not being in a Project and is instead in the in Standalone Apps section.

Ok, now it works, thank you @igorbrigadir and @edsu

I think it should be better to add a note in documentation and in the config steps via shell.

Solved

[edsu]

Yeah, it is pretty tedious what Twitter have done here. We could have the twarc error message mention something about ensuring that the keys are for an app in a project. But hopefully this conversation here will help people as well. Thanks for bearing with us to get to the bottom of it!

[barripdmx]

Hi, I have the same problem and I can't figure out how to fix it. I'll give you the screenshots of how I have it configured. Typing this gives me the error: Client Forbidden

This is how I have configured the project:

twarc2 search --archive blacklivesmatter tweets.jsonl IT DOES NOT WORK twarc2 search blacklivesmatter tweets.json IT WORKS

Sorry for my english translated

Best regards, jb

[igorbrigadir]

@barripdmx

twarc2 search --archive blacklivesmatter tweets.jsonl IT DOES NOT WORK twarc2 search blacklivesmatter tweets.json IT WORKS

Judging by your screenshot you have Elevated access, and you can search for tweets in the last 7 days. To use --archive you have to have an "Academic Research" project after applying for academic access. You will then have 2 projects, 1 elevated and 1 academic.