search

ambianic / ambianic-subscriptions

Repo for ambianic premium subscription management
Apache License 2.0
3 stars 2 forks source link

Add CI test run against a Netflify staging instance #5

Open ivelin opened 3 years ago

ivelin commented 3 years ago

Is your feature request related to a problem? Please describe. Currently the CI runs local tests, but not live tests against a Netlify functions pre-deployment environment.

Describe the solution you'd like Levelarge the CI best practices from ambianic-ui. We run two separate CI workflow scripts of lighthouse tests against local and live netlify staging version of the new code in a PR or commit push.

vickywane commented 3 years ago

@ivelin To understand this better. Are you requesting tests are run against the local Netlify instance created using netlify-dev?

Currently, tests are run against deployed functions using their API endpoints from Netlify.

ivelin commented 3 years ago

@vickywane good point. I overlooked that the tests are running against the actual deployed ambianic-functions on Netflify. The issue with this approach is that we deploy first (potentially buggy new code) which instantly affects users and then we run tests against it.

The tests must be focused on the new code in the specific PR.

I will correct my request. Ideally we would have two separate workflows:

  1. One that runs against a local CI copy of functions via netlify dev and catches any issues before deployment.
  2. Another one that runs against a preview deployment of the code changed in the given PR. See how our LH flow waits for a preview deployment and tests against it. The goal of point 2 is to catch any issues before these functions are deployed in production where all users are instantly affected.
vickywane commented 3 years ago

This is quite an interesting edge case. It’s is important to note that we are using newman which run the postman scripts. For the scripts to be created they have to be given a live endpoint, i dont think this might be possible with a local endpoint.

Also netlify-dev changes the PORT where the functions are run each time it’s started.

I would give this a deeper look-into

ivelin commented 3 years ago

Postman with Netflify dev for local testing

https://blog.postman.com/serverless-functions-the-fast-way/

Deploy previews https://docs.netlify.com/configure-builds/get-started/

Split testing and staged deployment https://docs.netlify.com/site-deploys/split-testing/

Deploy front end and functions without a maintenance window : https://docs.netlify.com/configure-builds/get-started/

On Sun, Mar 7, 2021 at 4:12 PM Nwani Victory notifications@github.com wrote:

This is quite an interesting edge case. It’s is important to note that we are using newman which run the postman scripts. For the scripts to be created they have to be given a live endpoint, i dont think this might be possible with a local endpoint.

Also netlify-dev changes the PORT where the functions are run each time it’s started.

I would give this a deeper look-into

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ambianic/ambianic-subscriptions/issues/5#issuecomment-792362799, or unsubscribe https://github.com/notifications/unsubscribe-auth/AARBUFP23GWIHHPHKHZZP33TCP25VANCNFSM4YYBJJ3Q .

-- Ivelin Ivanov https://www.linkedin.com/in/ivelinatanasoffivanov/ founder, Ambianic http://ambianic.ai Fall Detector ® Instantly know if your loved one has fallen down™