search

apache / incubator-seata

:fire: Seata is an easy-to-use, high-performance, open source distributed transaction solution.
https://seata.apache.org/
Apache License 2.0
25.1k stars 8.73k forks source link

[Discussion] Seata and SkyWalking integration proposal #714

Open wu-sheng opened 5 years ago

wu-sheng commented 5 years ago

Hi Seata team,

I am glad to see the project has more contributors to join. As Apache SkyWalking creator, I am wondering, does Seata have interested to provides SkyWalking plugins, in order to observe how seata works with applications.

This is your architecture image Right now, SkyWalking could trace and provide metric for most RPC and DB accesses. Now if Seata could provide tracing based on SkyWalking agent plugin, which works between application and your TC, then user could see the status of seata, and have metrics(successful rate, latency, even topology) related to TC and whole distributed transaction.

I don't know whether this idea makes sense to seata team, since we already work with Dubbo and SOFA team tightly, I hope we could integrate on this point too.

If you have more ideas to discuss, and what we should do, we could discuss here.

slievrly commented 5 years ago

@wu-sheng welcome ，whether it is convenient to use dingTalk communication with you？

wu-sheng commented 5 years ago

I am in San Francisco this whole week, and back to Beijing at the weekend, so next week?

slievrly commented 5 years ago

I am in San Francisco this whole week, and back to Beijing at the weekend, so next week?

ok，metrics designing，https://github.com/seata/seata/pull/658

wu-sheng commented 5 years ago

Yes. I noticed that. In my mind, inside a highly distributed environment, topology related metrics make more sense to the user. Imaging we could show seata TC in this topology. image Right now, the next 6.1.0 release has better topology than this.

Also, SkyWalking will provide other traditional metrics based on the topology, too, such as recognizing TC as a service in a distributed environment.

Prometheus metrics make sense, if you want to skywalking to receive that, but rather than read id like prometheus. SkyWalking supported Envoy metric service in this way. But, anyway, if we have topology, should be cooler and benifit for end users.

You could invite other people in the community to join the meeting, such as set an open zoom/hangout meeting.

wu-sheng commented 5 years ago

@kezhenxu94 How quickly you noticed I submit this discussion :) Welcome to join.

kezhenxu94 commented 5 years ago

@kezhenxu94 How quickly you noticed I submit this discussion :) Welcome to join.

I've been watching this repo for a long time :stuck_out_tongue_winking_eye:

jovany-wang commented 5 years ago

@zhengyangyong cc

jovany-wang commented 5 years ago

Thanks. I agree that integrating SkyWalking into Seata.

But there are 2 optional scenarios in my idea: (1). I don't know if SkyWalking support Prometheus protocol, if SkyWalking support Prometheus protocol, we can implement current metric scenario and make SkyWalking server to show the result of metric or the topology that you mentioned above. (2). Add a metric/trace module into Seata, make SkyWalking client and Prometheus client as the implementation of plugins.

jovany-wang commented 5 years ago

Btw, if it's necessary, let's talk in slack. @slievrly @wu-sheng @zhengyangyong

wu-sheng commented 5 years ago

(1). I don't know if SkyWalking support Prometheus protocol, if SkyWalking support Prometheus protocol, we can implement current metric scenario and make SkyWalking server to show the result of metric or the topology that you mentioned above.

SkyWalking supports Prometheus metric family formats, but don't support Prometheus wire protocol. Because SkyWalking don't support pull mode.

(2). Add a metric/trace module into Seata, make SkyWalking client and Prometheus client as the implementation of plugins.

From my understanding and GitHub repo shows, seata is 100% wroten by Java. SkyWalking supports commercial APM level auto instrumentation. So, if you could provide seata plugin(client/server sides), SkyWalking even doesn't require you to change seata codes, which is also why we have wide adoption across nearly all popular libraries.

Definitely, I prefer the latter. If Seata has more metrics to visualization, we could support it in agents and new protocols too.

wu-sheng commented 5 years ago

SkyWalking slack is here with ml, as an Apache project, prefer more on GitHub discussion, and mail list discussion.

wu-sheng commented 5 years ago

SkyWalking has a very long supported framework list, https://github.com/apache/incubator-skywalking/blob/master/docs/en/setup/service-agent/java-agent/Supported-list.md

I hope seata could be a part of it.

zhengyangyong commented 5 years ago

Great, It is a good news that SkyWalking could support Seata. I think embeded metrics is still needed for users do not use any APM but only simple monitor system, right?

slievrly commented 5 years ago

@wu-sheng When Seata releases 0.5.0，I hope to have a plugin that supports Seata。https://github.com/apache/incubator-skywalking/tree/master/apm-sniffer/apm-sdk-plugin/seata-0.5.x-plugin

wu-sheng commented 5 years ago

Great, It is a good news that SkyWalking could support Seata. I think embeded metrics is still needed for users do not use any APM but only simple monitor system, right?

Yes. I agree. That is why I am just proposing a seata plugin and maybe some document guides in seata doc website.

wu-sheng commented 5 years ago

Do we still need a discussion? If so, please set up a google hangout or zoom in next week, except Wednesday.

slievrly commented 5 years ago

I think a real-time multi-person communication tool is necessary to deal with the problem.

wu-sheng commented 5 years ago

Let's have one on Tuesday afternoon 2pm-3pm. I could join.

wu-sheng commented 5 years ago

Lost update in half month, anything you want to talk?

zhangthen commented 5 years ago

@wu-sheng Seata needs SkyWalking to provide tracing and metrics capabilities, but now Seata's protocol does not support tracing and metrics, which need to be modified;

And we also intend to support OpenTracing 2.0 in the future.

wu-sheng commented 5 years ago

OpenTracing is not our high priority work, no matter where it goes. Especially, OT/OC are merging, API thing goes to chaotic.

In my mind, SkyWalking native plugin is always a better solution. Better for the end user.

zhangthen commented 5 years ago

OpenTracing is not our high priority work, no matter where it goes. Especially, OT/OC are merging, API thing goes to chaotic.

In my mind, SkyWalking native plugin is always a better solution. Better for the end user.

Seata need support both OpenTracing and Sky Walking. The work will begin after modifying seata's protocol, which need help of community.

slievrly commented 5 years ago

We will hold an online meeting to discuss about Seata and SkyWalking integration proposal , the meeting information:

季敏 is inviting you to a scheduled Zoom meeting.

Topic: Seata and SkyWalking integration proposal Time: Apr 25, 2019 3:00 PM Beijing, Shanghai

Join Zoom Meeting https://zoom.us/j/151664056

One tap mobile +14086380968,,151664056# US (San Jose) +16465588656,,151664056# US (New York)

Dial by your location +1 408 638 0968 US (San Jose) +1 646 558 8656 US (New York) Meeting ID: 151 664 056 Find your local number: https://zoom.us/u/adkIXVPs82

slievrly commented 5 years ago

@wu-sheng

wu-sheng commented 5 years ago

Got it.

wu-sheng commented 5 years ago

@kezhenxu94 If you want to contribute this feature, welcome to join this meeting. I think this is a good chance for you to take part in an important feature.

zhaoyuguang commented 5 years ago

Look forward to Seata and SkyWalking~

wu-sheng commented 5 years ago

Hi I am here to brief today's meeting result.

SkyWalking will integrate with Seata. We will do the following things.

  1. Trace Seata transaction as part of trace, including TC.
  2. Show TC as an independent service in SkyWalking topology.
  3. Show metric(cpm, success rate, etc.) of TC server.

Above is step 1.

Step 2, we will try to work more close with Seata team, try to make trace continue, even seata TC reschedule the transaction by timer mechanism. Need some persistence mechanism for tracing context in TC side.

SkyWalking community will have two contributors work on this.

wu-sheng commented 5 years ago

After step integration finished, and tests passed. SkyWalking will deliver some binary package, one stable version agent, which should be included in seata TC server binary. Users could use command argument to open agent(default maybe off?)

slievrly commented 5 years ago

According to the results of today's meeting, Seata team will work closely with SkyWalking. The Seata community will work with the SkyWalking community to complete the above development work.

Seata community will have two contributors work on this.

@CoffeeLatte007 Seata committer @zhengyangyong Seata contributor

I look forward to a smooth cooperation.

jyizheng commented 5 years ago

Is this issue still open?

wu-sheng commented 5 years ago

Yes. SkyWalking community @kezhenxu94 is waiting for seata stable version. Seata internal API were changing in these several months. We can't deliver a good plugin yet.