htm-community / flink-htm

Distributed, streaming anomaly detection and prediction with HTM in Apache Flink
GNU Affero General Public License v3.0
136 stars 37 forks source link

Develop a Socrata connector #12

Open EronWright opened 8 years ago

EronWright commented 8 years ago

Let's develop a connector to read Socrata streams.

Consider using the soda-java library, but be aware that its dependencies are fairly old (jackson 1, jersey 1). We would probably need to shade the dependencies and/or update them.

Alternatively, we could write something from scratch (consumer-side only, to begin with). The River source is written with Spray, which is easy to use. There are quite a few soda model classes (for geo data, timestamps, etc) and query classes that would need to be re-implemented.