search

pingcap / tispark

TiSpark is built for running Apache Spark on top of TiDB/TiKV
Apache License 2.0
884 stars 244 forks source link

Plan to add 2pc for tikv-client? #323

Closed nanne007 closed 3 years ago

nanne007 commented 6 years ago

Is there any plan to support 2pc for tikv-client ? It will open many opportunities for tikv in Java world, like a underlining storage for janusgraph.

ilovesoup commented 6 years ago

For short term (within this year), there is no plan for it. For now, most users dump data into TiDB via Spark JDBC connector. The performance overhead is on transaction related stuff and manually 2PC on Java side has no improvement on it. But we do have plan to allow Spark dump directly to rocksDB, but without transaction support.

nanne007 commented 6 years ago

Thanks for the replay. Get your points.

What's I mean is a full tikv client(without tidb stuff) with transaction support, which I think can be modeled similar to the implementation in tidb.

If there's not plan for it, I'm going to start this.

Thank you again.

ilovesoup commented 6 years ago

There is no near plan for it and you are welcome to take it :)

ilovesoup commented 6 years ago

@lerencao We are planning on related features. Do you have any progress on 2PC part? If so, we can cooperate on it.

nanne007 commented 6 years ago

Nope... The 2PC impl in tidb repo was coupled with tidb specific codes, and I had some issues on mapping Go Context concept to Java world.

After writing some trivial classes in my branch, I left it decay... :)