ray-project / mobius

Mobius is an AI infrastructure platform for distributed online learning, including online sample processing, training and serving.
https://ray-project.github.io/mobius/
Apache License 2.0
85 stars 13 forks source link

[Docs] compile and setup instructions #53

Closed wilsonwang371 closed 2 years ago

wilsonwang371 commented 2 years ago

can we have document about how to compile and setup ray streaming?

I am trying to do some mobius evaluation. I tried to do this manually, but I am not sure if I am doing it correctly.

We may need to run mobius in the following environments.

ashione commented 2 years ago

can we have document about how to compile and setup ray streaming?

I am trying to do some mobius evaluation. I tried to do this manually, but I am not sure if I am doing it correctly.

We may need to run mobius in the following environments.

  • container
  • linux environment.
  • macosx environment

Install instruction doc : https://github.com/ray-project/mobius/blob/master/docs/how-to-install.rst The lastest mobius need some patches these cannot fetch from ray-1.12, so we need build ray manully(ray whl pakcage with python-3.8 on linux has been uploaded on aliyun oss bucket).

BTW, could you provide us more details about that requirements? Is only python or java used for evalution?

BalaBalaYi commented 2 years ago

We haven't done testing in a variety of environments, so there might be some problems. Please let us know if u have any problems for installing.

BTW, many implementations are outdated by current. The repo will have huge changes in next few month.

wilsonwang371 commented 2 years ago

I am planning to do some evaluation on the network communication overhead between Mobius and hazelcast. I did some deep-dive on Hazelcast and I am thinking comparing Mobius with it can help us get a better understanding of Ray core and where it needs to improve.

BalaBalaYi commented 2 years ago

I am planning to do some evaluation on the network communication overhead between Mobius and hazelcast. I did some deep-dive on Hazelcast and I am thinking comparing Mobius with it can help us get a better understanding of Ray core and where it needs to improve.

@ashione I think we used to have performance test, is there any data we can provide to him?

ashione commented 2 years ago

I am planning to do some evaluation on the network communication overhead between Mobius and hazelcast. I did some deep-dive on Hazelcast and I am thinking comparing Mobius with it can help us get a better understanding of Ray core and where it needs to improve.

@ashione I think we used to have performance test, is there any data we can provide to him?

We do some perf test in two years ago(Mar, 2020), and get brief result : WordCount 1X1 Job gets 450K TPS within 200bytes each word, Yahoobenchmark gets 1250K totally and P99 10ms lantency between upstream and downstream pair(internal production version). cc @wilsonwang371

wilsonwang371 commented 2 years ago

Thanks for the info. But in order to compare it with other alternatives, I need to setup a common setup for them. Let me do that and if I need you guys' help, I will open another issue to track this.