Kubernetes support

[elgalu]

Kubernetes support in the roadmap?

There is an official Go client: https://github.com/kubernetes/client-go

[vania-pooh]

Currently we consider Kubernetes mostly an enterprise solution. Because of that we are developing a closed-source commercial solution based on Kubernetes and using just the same browser images as Selenoid. Now it is possible to launch Selenoid in Kubernetes (we have already tried it) without any code modification but it will launch browsers only on the same host machine. With such limitation it makes no sense to do that - only if all the rest of the infrastructure is already there.

[SrinivasanTarget]

Nowadays most of us maintain our infra is in cloud for dev, QA & prod, since we have good infra world i see GGR is good enough to handle any load to run tests across several machines/VMs/AWS instances/any infra.. Selenoid handles everything very stable and more powerful solution too thus far. Apart from self healing (which selenoid does anyway to an extent to maintain max limit given) i don't see any other advantages in using kubernetes for test. Do you @elgalu really see other added advantage in kubernetes when one maintains their test infra using powerful AWS instances and runs test using Selenoid & GGR? Would like to know if any.

[elgalu]

@vania-pooh

we are developing a closed-source commercial solution based on Kubernetes

Glad to hear that, I think is a smart way to indirectly sponsor your open source efforts and to ensure the open source parts continue to be maintained and evolved.

Have you guys decided your pricing model? The one thing I don't like about Sauce Labs and Browser Stack (and probably the rest of the cloud providers) is that they force you go to enterprise through a fixed amount of parallel tests, instead of offering a pay per use approach, like AWS does. I hope you guys figure out a pricing model that doesn't limit parallel testing, that would be great.

@SrinivasanTarget

The advantage of using k8s is automated horizontal scaling. When we only scale vertically the 1 big machine becomes a bottleneck and a single point of failure, this is critical if you are serving >1000 developers for example. On smaller teams it should work well I agree with you.

Using GGR + Selenoid instances also helps, but the scaling is manually made on your side? if so that would waste unused resources and require manual intervention to add more Selenoid instances when the hardware capacity is reached by the required # of tests. Or maybe I missed something.

[SrinivasanTarget]

@elgalu

automated horizontal scaling

AWS does horizontal scaling as per my knowledge in our production. It spins up additional instances to balance load automatically as they call it as scale in protection but not tested with our test model though or not sure how does it fit in or help in my automated test setup. Great point though.

Using GGR + Selenoid instances also helps, but the scaling is manually made on your side?

We set the limit initially based on our prior knowledge with Selenoid and GGR. For say, 8 Gig Ubuntu with 8 Cores handles 23 instances of selenoid without any issues. We fix the limit based on it. Yes accepted manual intervention and few though process required. 👍 Haven't leveraged AWS Scale in Protection caps here, may be worth trying it.

[vania-pooh]

Glad to hear that, I think is a smart way to indirectly sponsor your open source efforts and to ensure the open source parts continue to be maintained and evolved.

Yes, you correctly understood the reason. Good open source projects are either sponsored by commercial companies or have commercial versions to exist.

Have you guys decided your pricing model?

We have no resources to deliver good SaaS. So we plan to propose "boxes" - a ready to use software package to be installed on Kubernetes on top of own hardware or private cloud (e.g. Google Container Engine). This would prevent potential users to launch ugly tunnels over websockets. We are still thinking about final pricing model, so we would be interested in any comments regarding this.

[evanburrows]

From an Enterprise perspective it all depends on what our organizations use. Some have made investments in orchestration tools like kubernetes, mesos, rancher, etc. to handle larger scale container deployments. Could you do this natively on AWS without those types of services? Perhaps, but it depends on the use case. Heck, even AWS just joined the CNCP earlier this month.

I have tried to get selenoid running in mesos/marathon to meet a use case similar to the kubernetes request. I have not been successful in getting it to work though. Long story short I think this makes sense for those that have existing clusters in their Enterprise environments. For those that don't, the previously suggested kube closed source selenoid offering may welcomed. Thanks for putting so much hard work into this effort

On Aug 26, 2017 2:10 PM, "Srinivasan Sekar" notifications@github.com wrote:

Nowadays most of us maintain our infra is in cloud for dev, QA & prod, since we have good infra world i see GGR is good enough to handle any load to run tests across several machines/VMs/AWS instances/any infra.. Selenoid handles everything very stable and more powerful solution too thus far. Apart from self healing (which selenoid does anyway to an extent to maintain max limit given) i don't see any other advantages in using kubernetes for test. Do you @elgalu https://github.com/elgalu really see other added advantage in kubernetes when one maintains their test infra using powerful AWS instances and runs test using Selenoid & GGR? Would like to know if any.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/aerokube/selenoid/issues/200#issuecomment-325153025, or mute the thread https://github.com/notifications/unsubscribe-auth/ADnUTI3w5KLGh95V1zPiEZP9zW_RSrwrks5scF-UgaJpZM4PDYXr .

[elgalu]

@vania-pooh I think is good to avoid offering a SaaS because, as you mention, that implies tunneling for local testing and that's something many companies just don't want to do, for security reasons.

Regarding the enterprise pricing model.

So your team would probably provide a set of selenoid-k8s closed-source docker images. You would also provide k8s template config files, but due to the diversity of k8s setups you will probably need to start selling enterprise support hours. Eventually when you have deployed all the k8s setups (AWS, GCP, OpenShitf, Data Centers, etc...), after many projects, you will probably have bullet-prof templates and the enterprise for every use case, and then, consulting hours will not be that necessary anymore. So in that scenario I would start charging a low price for the binaries but good price for the consulting. Maybe a yearly access to updated binaries. Later you can charge per year for example. The k8s templates you could open source so you can leverage your customers to send improvements. Or maybe give private access to your customers but then they can still collaborate. The only part that would remains closed source in this proposition is basically hitting the k8s api from Go. But you could also add more enterprise features like video recording, dashboards and so on. For the selling stage, dashboards and video play an important role, even though is the value is overrated it will ease to sell the product IMHO.

[vania-pooh]

@elgalu: I agree. We are moving towards an enterprise features like per session logs, enterprise-level ACL features like LDAP support for username\password keypairs, video storage and recording and so on. We also plan to provide a rich UI for all this stuff similar to commercial SaaS services. Let us work on the stuff during a month or so and we'll show more stuff. Hope I will have some Kubernetes demo video on SeleniumConference to show during the break.

[vania-pooh]

Here is a short demo video showing basic browser startup functionality already working on Kubernetes. Just click on the picture below:

It is not shown on the video but VNC functionality is already implemented and working. We are currently clarifying requirements about authentication. If anybody wants to take a look at precompiled prototype (it is closed-source) - then contact me. My email is visible in commit history.

[vania-pooh]

I am closing this. More Moon-related links to follow:

• http://github.com/aerokube/moon/ - issues and milestones
• http://aerokube.com/moon/latest/ - latest documentation