search

twitter / scalding

A Scala API for Cascading
http://twitter.com/scalding
Apache License 2.0
3.48k stars 703 forks source link

remove scalding-jbdc #1969

Closed daniel-sudz closed 2 years ago

daniel-sudz commented 2 years ago

based on https://github.com/twitter/scalding/issues/1968. We are not aware of any popular use of this module. Even if someone is using it it will likely need to be re-written in future scalding releases because:

  1. it is hard-wired to cascading
  2. it depends on cascading ecosystem packages that have not been maintained since cascading 2.X release

I would prefer to reduce support burden on things that do not provide value to most consumers. This is also something that can implemented on the consumer side if it is really required.

daniel-sudz commented 2 years ago

I found com.twitter.scalding.db and looks like in the past there were plans to move jbdc in there but there was never any incentive to do so after a while.

daniel-sudz commented 2 years ago

based on https://github.com/twitter/scalding/pull/1382 twitter used/uses an internal implementation of jbdc for scalding

tlazaro commented 2 years ago

I don't remember, if jdbc was used at Twitter. scalding-db for sure was. If @germanschnyder or @nownikhil could chime in.

johnynek commented 2 years ago

we can revert this if Twitter corrects us later. Let's keep scalding-db for now.

nownikhil commented 2 years ago

Sorry for the delay, was out for few days. Verified that its not being used at Twitter. Thanks @daniel-sudz