js-robinson
commented
2 years ago I think we need a little explainer for the main page/description: Something like:
Welcome,
This Twitter API search tool was developed to provide a simplified way of extracting data from the Twitter API. We set out to make social media research accessible and inclusive of a wider group of users, who may not have sufficient developer experience to immediately use the Twitter Developer Platform. Before using the Web Data Research Assistant users will need to apply for access to the Twitter API {https://developer.twitter.com/en/docs/twitter-api/getting-started/getting-access-to-the-twitter-api} and receive credentials for access from Twitter. Once you have logged in to the Web Data Research Assistant using those credentials you can use the research tool. You can change your credentials if necessary and appropriate due to a change in your activity or position.
The WDRA develops completely reproducible results in line with best practices in Open Science. Once you have created a search you can save and replicate it. [say something here about sustainable software and link to Git?]
We are committed to ethical research. Extensions to the tool were built under ethics approval by the University of Southampton Faculty of Social Sciences Ethics and Research Governance Committee 62643.A2 (Nov 2021). Users should be aware of best practices in ethics of social media research and the need to comply with Twitter's policies [https://developer.twitter.com/en/developer-terms/agreement-and-policy]
The tool's development was supported by the Participedia project [Participedia.net], the University of Southampton, and the UKRI-funded Rebooting Democracy Project {Rebootingdemocracy.ac.uk}, grant reference MR/S032711/1.
If you encounter problems Feedback can be sent to (???) - should we include?
Some slight suggested changes for text on the 'Pick Tweet Return Method' page - Actually can we change that to 'How do you want to search for tweets and data?'
There are currently two extract methods, Search and ID.
Search has two forms, the basic and advanced. Both will search for terms in tweets and return information about those tweets. The basic form offers 'include' and 'exclude' terms for the search as well as total number of tweets to return. The basic form will return the tweet ID and the tweet text of tweets, as is the default for the Twitter APIv2. It is also limited in scope to searching withing the last 7 days.
The advanced form has 'include' and 'exclude' terms, and total number of tweets to return with batch size control [can we add a one-line explanation of what this is?]. The advanced form allows selection of date and time range and/or selection of tweet ID range (within the limitation of your API credentials - non-academic users are still limited to the last 7 days so if you want more you will need to apply for academic research access). The advanced form allows selection of the return information giving access to the full suite of API return fields which you can browse and select.
ID is a tweet hydrating feature. That is it will use the IDs provided to find and reproduce complete details of all fields of a tweet. By supplying a list of tweet IDs the API is used to obtain all available fields from the tweets.
Then on search hints
This form provides a simple compatible software translating and providing interoperability (wrapping around) the twitter recent search API endpoint: https://developer.twitter.com/en/docs/twitter-api/tweets/search/integrate/build-a-rule
To return more objects or to use timeframe searches beyond the 7-day default you will need to use the advanced search and have applied for the requisite credentials from Twitter.
It is important to separate search terms by commas as the words given in the 'Search terms' form will be split on commas. The words will be added to the search using AND logic by default., i.e. the tweets returned must include all the search parameters. To implement OR logic, i.e. include tweets that contain either search term, simply add OR with a space 'cat, OR dog, ....' [is this correct?] before your search term. In addition we can search for a phrase using quotation marks, i.e. the search Black AND Lives AND Matter may be better implemented as "Black Lives Matter". Finally several search terms can be grouped using parentheses. Using the AND and OR logic along with parentheses, search terms of high complexity can be built. e.g. (United States, OR USA), AND (Germany OR Deutschland) will search for only for tweets containing [-can we check if this is correct and correct use of comma for search logic as I am not clear?].
The words in the 'Exclude terms' form will be implemented with NOT logic. Tweets with these terms will not be returned in the search. Words in this list should be alone and not grouped in parentheses.
An operator can be used to increase the power of your search. We have discussed using quotes to switch from keyword to phrase matching. Other operators include the hashtag # to match exact hashtags, and the @ to match usernames. Other operators can assert that tweets contain elements for example 'has:media' asserts that tweets searched for will contain a media object. One of the most commonly used operators is '-is:retweet' that excludes retweets from the search. We implement the removal of retweets as a default suggestion in the 'exclude' terms.
Furter information on operators for filtering in the Twitter API search can be found by following the link [https://developer.twitter.com/en/docs/twitter-api/tweets/filtered-stream/integrate/build-a-rule#availability].
For full examples see... [can we add some examples later?] I think Pip had some with cats and dogs or something...
/wdrax/extract/nethod/search/TrueMatt Ryan will supply the tweaked wording. Chase this.
Link to Twitter API docs is broken