xdevplatform / Twitter-API-v2-sample-code

Sample code for the Twitter API v2 endpoints
https://developer.twitter.com/en/docs/twitter-api/getting-started/about-twitter-api
Apache License 2.0
2.61k stars 967 forks source link

Truncated tweet texts returned, even tho explicitly filtering out retweets #127

Open pannadf opened 1 year ago

pannadf commented 1 year ago

Describe the bug I'm seeing truncated tweet texts, even though I've explicitly filtered out retweets in my query.

To Reproduce

import requests
import os
import json

# To set your environment variables in your terminal run the following line:
# export 'BEARER_TOKEN'='<your_bearer_token>'
bearer_token = os.environ.get("BEARER_TOKEN")

search_url = "https://api.twitter.com/2/tweets/search/recent"

# Optional params: start_time,end_time,since_id,until_id,max_results,next_token,
# expansions,tweet.fields,media.fields,poll.fields,place.fields,user.fields
query_params = {'query': '(from:lightningai -is:retweet)',
                'tweet.fields': 'author_id,created_at,public_metrics', 
                'max_results':10}

def bearer_oauth(r):
    """
    Method required by bearer token authentication.
    """

    r.headers["Authorization"] = f"Bearer {bearer_token}"
    r.headers["User-Agent"] = "v2RecentSearchPython"
    return r

def connect_to_endpoint(url, params):
    response = requests.get(url, auth=bearer_oauth, params=params)
    print(response.status_code)
    if response.status_code != 200:
        raise Exception(response.status_code, response.text)
    return response.json()

def main():
    json_response = connect_to_endpoint(search_url, query_params)
    print(json.dumps(json_response, indent=4, sort_keys=True))

if __name__ == "__main__":
    main()

Expected behavior I would expect all the tweet texts returned to be the full text.

Additional context An example of what I'm seeing, where it's clear that the tweet is truncated at StableLM, Pythia,...:

{'author_id': '1157283331509235713', 'created_at': '2023-05-09T15:13:09.000Z', 'edit_history_tweet_ids': ['1655954064474279938'], 'id': '1655954064474279938', 'public_metrics': {'impression_count': 14351, 'like_count': 113, 'quote_count': 3, 'reply_count': 3, 'retweet_count': 29}, 'text': '⚡Lit-Parrot 🦜 - the simplest implementation of the best open-source language models out there is now available!\n\nFresh out of the cage👉 https://t.co/TWSVqU6izY\n\n💡“nanoGPT” minimal\n🛠️hackable\n📜comes with pre-training + finetuning scripts\n\nShips with StableLM, Pythia,… https://t.co/RypG8NLUgy https://t.co/nW2W80Cwup'}

Any guidance would be greatly appreciated. Thank you!

naveedamir488569 commented 8 months ago

add note_tweet in expansion,

take refrence from this, i am getting full text data for origin tweet,

user.fields=created_at,description,entities,id,location,name,pinned_tweet_id,profile_image_url,protected,url,username,verified,withheld,public_metrics&expansions=attachments.media_keys,author_id,geo.place_id&tweet.fields=attachments,author_id,entities,public_metrics,lang,created_at,note_tweet,text&start_time=2023-09-14T23:35:16.000Z&max_results=10