twitter / hbc

A Java HTTP client for consuming Twitter's realtime Streaming API
https://developer.twitter.com/en/docs/tweets/filter-realtime/overview
Apache License 2.0
962 stars 373 forks source link

Enterprise volume and compliance streams for the GNIP v2.0 API #174

Closed paulrizzo closed 2 years ago

paulrizzo commented 8 years ago

This adds supports for the v2 GNIP volume and compliance streams. I also added in the new host URLs for convenience.

It may be worth considering deprecating the old Twitter endpoints so we can clean up this project a bit. I'd propose releasing a final version of 2.x.x, bumping master to 3.x.x and removing all the legacy Twitter code. This will allow us to clean up the interfaces/implementations for the GNIP API's.

coveralls commented 8 years ago

Coverage Status

Changes Unknown when pulling 4f49677c78b9f66e226df8eedd910aa1ce4b4477 on paulrizzo:gnip-v2-streaming-api-update into \ on twitter:master**.

bobber205 commented 8 years ago

@paulrizzo Have you published a third party of this change I might be able to use? Twitter no longer actively maintains this project.

paulrizzo commented 8 years ago

@bobber205 I have not published this to a repository so right now you would need to pull this change into a local version of master and built a snapshot from that. If others are interested I would like to split off of this project and clean up bunch of the legacy Twitter pieces that won't be valid after the end of this year anyways. This will allow us to clean up the interfaces and code quite a bit so its more maintainable moving forward.

bobber205 commented 8 years ago

@paulrizzo That's what I figured. no problem :)

Having a hell of a time publishing a local version, if you have any tutorials/reference guides to share I'd love to see them! <3

I'm a mostly FE/ruby guy, I don't know jack about Java and Maven.

bobber205 commented 8 years ago

@paulrizzo Are you using this library along with https://github.com/gnip/sample-kinesis-connector? Got any examples of how to use your changes with that library? Thanks! <3

paulrizzo commented 8 years ago

@bobber205 If you pull this change into you local version you can run mvn clean install to build the jar file. From there you just use as needed. I am not using the kinesis-connector but rather a custom implementation for managing connections and processing messages to the appropriate location. Unfortunately I cannot post this code because it's owned by the company I work for. If you are using PowerTrack kinesis seems like a good place to start, if you are consuming the firehose/compliance streams you'll need to write something more involved to manage the partitions and failover cases. Hope this helps

bobber205 commented 8 years ago

Not sure if you saw this @paulrizzo http://support.gnip.com/articles/updating-hbc-for-ptv2.html

I like your changes alot. I may end up using them! I will let you know if I do <3

bobber205 commented 8 years ago

Thanks for doing the work twitter doesn't want to do!

coveralls commented 8 years ago

Coverage Status

Changes Unknown when pulling 4ecde211d08ba639adcf6d6b607bfa5c64f25efa on paulrizzo:gnip-v2-streaming-api-update into \ on twitter:master**.

bobber205 commented 8 years ago

Good morning @paulrizzo Have you had any issue with HttpUriRequest trying to connect to the non secure port 80 version of the stream?

The url says https://gnip-stream.com/blah/blah but the IOException says connection to "http://gnip-stream.twitter.com" was refused.

Just putting it out there, in case you've seen it <3

coveralls commented 8 years ago

Coverage Status

Changes Unknown when pulling 47e5ba45035f744b74464b1d94e7d3a487af9904 on paulrizzo:gnip-v2-streaming-api-update into \ on twitter:master**.

CLAassistant commented 5 years ago

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Paul Rizzo seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.