search

Leibniz-HBI / Social-Media-Observatory

This repository is the central communication and project management interface for the Social Media Observatory hosted by the Leibniz Insitute for Media Research | Hans-Bredow-Institute
https://leibniz-hbi.github.io/SMO/
Creative Commons Attribution 4.0 International
27 stars 1 forks source link

Read and summarise new Twitter for academics page #26

Closed FlxVctr closed 4 years ago

FlxVctr commented 4 years ago

https://developer.twitter.com/en/use-cases/academic-researchers

For example we can use their tool recommendations as a start for our Twitter tool list (even though it can definitely be improved, e.g. Tweepy is badly maintained for ages while twarc is imho the go to tool for Python now, and still Twitter recommends Tweepy first)

manilevian commented 4 years ago

Hi there,

I can edit our Merged Scraper List according to their recommendation. We already have a huge list of Twitter Tools. You can check those up in the merged scraper list. I haven't made a complete Twitter List on our wiki, because i am not done testing them all to know which will actually make the list :)!

I think a lot of those tools are already in our tool list. I will compare them and edit our merged list!

manilevian commented 4 years ago

So i Just went through their list and like you said, its not really updated or in a specific order. On the other hand they have a handful of applications I haven't had heard of at all.

DocNow.io seems really promising. The project is also behind twarc and another App called "DocNow" which seems to be a general API wrapper of some sorts.

They also provide a Tweet Catalog with DB's of different past events. There is also a possibility of uploading your own DB and share it with them. They also have a extremely limited set of ethical guidelines

EDIT: They are also behind Hydrator (Client), which is a application to turn Tweets into Tweet ID's (as far as I understood tbh).

I also added a set of Java, PHP applications, which I will add into a "Advanced non supported“ category. Got some Java and PHP friends, who will probably have enough time to dig into those apps to at least communicate with us, if they function or not :).

The List fo Apps also provide two API Clients. One is Postman, which is basically free, but also has a premium service of some sort, this is a general API Wrapper. It can help building API Wrappers for any site. The premium service provides also 250.000 API wrappers to choose from. Therefore it could be a helpful resource if things have to „go fast“. The second application is Insomnia. This is a REST client, which is completely free for Mac, Linux and Windump.

I would add the Clients to „General News Scrapers“ under a „Advanced non supported“ category if that’s okay with you ! Since they are not directly bind to Twitter, but to general API’s.

so far!

FlxVctr commented 4 years ago

I wouldn't add Postman or Insomnia at all as they seem rather targeted at API developers. Feel free to disagree, but I'd need an example where they'd be handy in an academic data collection.

FlxVctr commented 4 years ago

I'd add Hydrator to the Twitter tools though if it works well. Very handy then

manilevian commented 4 years ago

Well since they are API clients and still need programming skills, i guess we can cut them. I wrote this as I had a "Advanced Tools" list in mind. Since we cut that, we can also cut the two tools.

Hydrator will be added, which brings me to the following question: Since hydrator is not really a scraper, do we want to make a "Useful tools" page, or just add them to the platforms tool site. They just don't fit into the table at all :D!

FlxVctr commented 4 years ago

Twitter tools would be the place I'd look for such a tool if I'd be a SMO user. You?

manilevian commented 4 years ago

Well imo id put them also wherever they are being used. I would probably add a "Other Usefull Tools" to the bottom of the page. I would also not make a table for them because it wouldnt make sence :D

FlxVctr commented 4 years ago

manilevian commented 4 years ago

Added hydrator to "Other useful tools" within the Twitter wiki. Ignored the other tools after evaluating again. Other tools were already in out merged list and now on the twitter wiki.

We cut the advanced tool list from every social network, since we only will add supported tools. Therefore no advanced tool list for twitter :D!

Tweet catalogue has been added to our DB list on google.

Other Issues with the wiki page can be addressed here!

Closing this issue :)!