search

xlab-uiuc / slooo

Slooo: A Fail-slow Fault Injection Testing Framework
11 stars 1 forks source link

Feature requests #8

Closed tianyin closed 3 years ago

tianyin commented 3 years ago

IMPORTANT: Only works with Microsoft Azure (so far).

I accidentally saw this. This is no good, especially for a tool paper.

  1. Not every developer has resources to test stuff in cloud (cloud credits are expensive) -- if it is Azure only, it's hard for you to claim usefulness
  2. Even developers test on cloud, not every developers use Azure -- your tool can't be used for AWS and GCP?

So, this needs to be addressed and (1) is more important than (2).

We need to provide a local mode to allow developers to do testing on a local machine or VM. This can make people's life much easier. Certainly, local testing can't be as versatile as cloud testing, but we should provide whatever needed for the best local testing.

For (2), we already prove that it works for Azure. We don't have to implement AWS and GCP support, but need to convince people that it's easy to extend and how to extend if later someone wants to extend.

varshith15 commented 3 years ago

We need to provide a local mode to allow developers to do testing on a local machine or VM. This can make people's life much easier. Certainly, local testing can't be as versatile as cloud testing, but we should provide whatever needed for the best local testing.

The code currently doesn't have support for local setup. I think local setup support implementation needs some thought and some time, because the a lot of the current code is written assuming that this framework is used only in a cloud setup.

We will basically have to replicate the whole current code to support local mode, it shouldn't take as long as it took the first time though that's one positive. And also testing can be done fast.

For (2), we already prove that it works for Azure. We don't have to implement AWS and GCP support, but need to convince people that it's easy to extend and how to extend if later someone wants to extend.

Yes, the tool can be used on any cloud service, I believe the support would just need a few lines of code (start and stop the VMs) which shouldn't be inconvenient at all in my opinion.

tianyin commented 3 years ago

Yes, the tool can be used on any cloud service, I believe the support would just need a few lines of code (start and stop the VMs) which shouldn't be inconvenient at all in my opinion.

Agree. It's more an abstraction problem -- what's needed to abstract out the platform-specific implementations (for Azure, AWS, and GCP) from platform-generic code.

tianyin commented 3 years ago

The code currently doesn't have support for local setup. I think local setup support implementation needs some thought and some time, because the a lot of the current code is written assuming that this framework is used only in a cloud setup.

That's surprising. How is the assumption coupled with the implementation?

Ideally, you are not even aware of that, as what's different is merely a local loophole versus a remote IP address.

The only difference is that some faults injected does not apply to local setup.

varshith15 commented 3 years ago

Ideally, you are not even aware of that, as what's different is merely a local loophole versus a remote IP address.

Yes, I've just realized its actually very similar to the cloud setup, just need to make the code more general, which I think isn't a big change like I thought it was yesterday. This generalization should also help with some of the troubles with TiDB porting!

This generalization is actually a very good thing for our framework. Thanks for pointing it out Prof. @tianyin I will add local setup support to action plan tasks.

tianyin commented 3 years ago

Very good, @varshith15 !!