CRITICAL - UiLicious tests are running in the DEV/Staging instead of the DEMO

[franck-boullier]

The problem:

It appears that some of the UiLicious Automated tests that are running every night are done in the DEV/Staging environment.

Why is this critical:

The DEV/Staging environement is an unstable environment and this might create some issues, false positives and other problems

The Solution:

Make sure that we do NOT run nightly automated tests in the DEV/Staging environment. All nightly automated test should only be run in the DEMO environment

[jmtiong]

Hi @franck-boullier, Eugene is currently away for a conference. The issues can be directed to me :) I'm from Uilicious too!

In regards to this, would you say that all the dev and staging environment test cases should be rescheduled to day time instead? or do you have a slack channel that you want the DEV/Staging tests to direct their notifications to?

[franck-boullier]

@jmtiong OK noted I'll keep you in the loop too from now on (I just invited you as a collaborator to this repo so you can be assigned to issues there).

would you say that all the dev and staging environment test cases should be rescheduled to day time instead?

Nope. We should NOT have Uilicious tests that are automatically running in the DEV/Staging environment. The Uilicious tests in the DEV/Staging environment should only be manually triggered on an ad hoc basis.

In other words:

• Uilicious tests that are automatically running every night should ONLY happen in the DEMO environment.
• DEV/Staging environment should NOT have any Uilicious tests that are automatically running there at all.

do you have a slack channel that you want the DEV/Staging tests to direct their notifications to?

No need for a dedicated slack channel for that at this point.

[jmtiong]

@franck-boullier, understood, in that case, I will be removing all the DEV/Staging environment scheduled jobs and only the DEMO jobs will be left.

[franck-boullier]

in that case, I will be removing all the DEV/Staging environment scheduled jobs and only the DEMO jobs will be left.

Perfect, thanks! Let me know when this is done so I can close this issue.

[jmtiong]

@franck-boullier alright! I have removed all DEV/staging environment jobs :)

[kaihendry]

I emailed about this already, but if you removed the jobs, how is one supposed to trigger staging jobs?

I.e. not automatic, but opt in?

[jmtiong]

@franck-boullier @kaihendry What I would suggest is that we can set up DEV/staging schedules that will run perhaps once a week or DEV/staging will only run in the daytime?

And also we could have a channel that will notify the status of the DEV/staging jobs and another channel for the DEMO so that it will not be confusing and trigger false negatives.

[franck-boullier]

I have no issue with the above suggestion if this makes our life easier