search

eclipse-archived / codewind

The official repository of the Eclipse Codewind project
https://codewind.dev
Eclipse Public License 2.0
113 stars 45 forks source link

Long-term stress testing of Codewind #1164

Open johnmcollier opened 4 years ago

johnmcollier commented 4 years ago

We're frequently testing Codewind with new / fresh installs, and aren't doing much long-term testing (or stress testing for that matter) on older installs.

We need to be doing long-term stress testing of Codewind to see how well it holds up over time (and with heavy load over time). A month-old install (or even longer) of Codewind should be just as functional as a fresh install.

jgwest commented 4 years ago

A great way to do stress testing like this is to keep running your existing automated tests over and over again, against a single PFE instance/cluster, without restarting it, until a failure occurs (or if failures occur too often, then run it X number of times). This has the advantage of not requiring a month to setup and reproduce.

tobespc commented 4 years ago

How does this tie in with SVT testing work? They might have something already

ssh24 commented 4 years ago

This is definetely do-able. We can test previous Codewind releases but we also need to make sure our test source is also pointing to the same release tag.

johnmcollier commented 4 years ago

@jagraj Is this something your test squad would be able to look at and / or help with at some point?

jagraj commented 4 years ago

@johnmcollier Yes, we are going to work on this in future releases. Now we have selenium integration tests working for Che and we need to work on load testing with multiple che users. I also see Eclipse Che using selenium as their automation test framework and I need to investigate how they are running their tests for multi users feature.

ssh24 commented 4 years ago

This is nice to have on org-wide integration tests but does not stop tribe based tests to be run based on release flags as well. For example, turbine and portal can also enhance their tests so it can be ran based on an older release.

elsony commented 4 years ago

@ssh24 I think those are two separate items. I agree there should be an easy way for us to be able to run our test on specific release. However, for the performance/stress testing, those are typical SVT level of testing and if SVT already have plan for it, there is no need to duplicate the effort. Maybe @jagraj can share the scenario that they planned to cover and we can review that to see if there is any scenario that we find is not covered and request SVT to add those.

ssh24 commented 4 years ago

That sounds good to me.

rwalle61 commented 4 years ago

linking this to testing epic https://github.com/eclipse/codewind/issues/2291