mint-metrics / mojito-js-delivery

🧪 Source-controlled JS split testing framework for building and launching A/B tests.
https://mojito.mx/docs/js-delivery-intro
Other
16 stars 30 forks source link

Print the IDs of tests included in the container when the build script runs #6

Closed kingo55 closed 5 years ago

kingo55 commented 5 years ago

For large containers with lots of experiments which are in various states (live, staging and inactive), it could be helpful to know which tests are being written to the container. By showing the user what's inside the container helps them to reduce the contents to reduce and page speed impacts.

E.g. Instead of this:

$ gulp scripts && gulp test
[21:40:40] Using gulpfile ~/Documents/github/mojito-js-delivery/gulpfile.js
[21:40:40] Starting 'scripts'...
[21:40:41] Finished 'scripts' after 469 ms

We could list what's active (inactive tests, or archived tests, may just add clutter):

$ gulp scripts && gulp test
[21:40:40] Using gulpfile ~/Documents/github/mojito-js-delivery/gulpfile.js
[21:40:40] Starting 'scripts'...
Mojito container building with these tests objects:
  Staging - w124 123 122
  Live - w120 w119 w118
[21:40:41] Finished 'scripts' after 469 ms

What do you guys think about this? Is there anything else I'm missing? Other data we should output to the console?

allmywant commented 5 years ago

@kingo55 Good idea. Maybe we can display the count of tests, e.g. Total 20 tests, 5 tests in staging, 15 tests in live.

kingo55 commented 5 years ago

Oh yeah, that sounds sensible David. Sam suggested we should add a count for inactive tests too.

Come to think of it, a container weight in kilobytes could be handy to know too.

On Tue, 24 Sep 2019 at 23:41, David Lee notifications@github.com wrote:

@kingo55 https://github.com/kingo55 Good idea. Maybe we can display the count of tests, e.g. Total 20 tests, 5 tests in staging, 15 tests in live.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mint-metrics/mojito-js-delivery/issues/6?email_source=notifications&email_token=AASAQLERP4S5NLK3IMD7PELQLIKKJA5CNFSM4IZ6Z6F2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7OMXQA#issuecomment-534563776, or mute the thread https://github.com/notifications/unsubscribe-auth/AASAQLC4P4ZABIE5PG4HP4LQLIKKJANCNFSM4IZ6Z6FQ .

kingo55 commented 5 years ago

OK, so the full implementation needs:

  1. List of tests by live and staging / NULL
  2. Count of tests by live, staging / NULL and inactive
  3. Optional If it's possible, show the library weight in KBs minified & gzipped.

e.g.:

$ gulp scripts
[21:40:40] Using gulpfile ~/Documents/github/mojito-js-delivery/gulpfile.js
[21:40:40] Starting 'scripts'...
[21:40:41] Finished 'scripts' after 469 ms
Mojito container built with 7 tests (45.12 KB):
  Live (4) - w120 w119 w118 w121
  Staging (3) - w124 w123 w122
  Inactive (28)

Edge cases:

I can't think of anything else for now though.