elastic / synthetics

Synthetic Monitoring with Real Browsers
MIT License
66 stars 41 forks source link

journey-level tags are ignored by push command #927

Open dmlemeshko opened 6 months ago

dmlemeshko commented 6 months ago

The goal is to define a single journey file and run it on different serverless test envs (qa, staging, prod). Since some journeys maybe qa-only, I rely on tagging mechanism to run/push only required journeys.

Journey example:

journey(
  { name: '[AppEx] Sample data: flights dashboard', tags: ['qa', 'prod'] },
  ({ page, params, request, context }) => {
    monitor.use({
      id: 'appex-kibana-dasboard-monitor',
      schedule: DEFAULT_SCHEDULE,
      tags: ['appex', 'serverless'],
    });
    step('open flights dashboard', async () => {
      const dashboard = new DashboardPage(page);
      await page.goto(`${params.base_url}/app/dashboards`);
      await page.getByTestId('dashboardListingTitleLink-[Flights]-Global-Flight-Dashboard').click();
      await dashboard.waitToBeLoaded({ itemsCount: 15 });
      await page.waitForLoadState('domcontentloaded');
    });
  }
);
synthetics.config.ts ```ts import type { SyntheticsConfig } from '@elastic/synthetics'; export default (env) => { const configQA: SyntheticsConfig = { params: { base_url: process.env.BASE_URL, es_username: process.env.ES_USERNAME, es_password: process.env.ES_PASSWORD, }, playwrightOptions: { ignoreHTTPSErrors: false, testIdAttribute: 'data-test-subj', timezoneId: 'UTC', locale: 'en-US', }, /** * Configure global monitor settings */ monitor: { // @ts-ignore locations: ['us_central_qa'], privateLocations: [], schedule: 60, }, /** * Project monitors settings */ project: { id: 'serverless-qa', url: , space: 'default', }, }; const configStaging: SyntheticsConfig = { params: { base_url: process.env.BASE_URL, es_username: process.env.ES_USERNAME, es_password: process.env.ES_PASSWORD, }, playwrightOptions: { ignoreHTTPSErrors: false, testIdAttribute: 'data-test-subj', timezoneId: 'UTC', locale: 'en-US', }, monitor: { locations: ['us_east'], privateLocations: [], schedule: 60, }, project: { id: 'serverless-staging', url: , space: 'default', }, }; const configProd: SyntheticsConfig = { params: { base_url: process.env.BASE_URL, es_username: process.env.ES_USERNAME, es_password: process.env.ES_PASSWORD, }, playwrightOptions: { ignoreHTTPSErrors: false, testIdAttribute: 'data-test-subj', timezoneId: 'UTC', locale: 'en-US', }, monitor: { locations: ['us_east'], privateLocations: [], schedule: 60, }, project: { id: 'serverless-prod', url: , space: 'default', }, }; if (env === 'prod') { return configProd; } else if (env === 'staging') { return configStaging; } else { return configQA; } }; ```

run command works as expected: TEST_ENV=prod npx @elastic/synthetics journeys --tags=prod

Journey: [AppEx] Sample data: flights dashboard
waitForRender: 0 out of 15 are loaded...
Retrying..
waitForRender: 1 out of 15 are loaded...
Retrying..
waitForRender: 1 out of 15 are rendered...
Retrying..
waitForRender: 8 out of 15 are rendered...
Retrying..
waitForRender: 14 out of 15 are rendered...
Retrying..
waitForRender: 14 out of 15 are rendered...
Retrying..
waitForRender: 14 out of 15 are rendered...
Retrying..
waitForRender: 14 out of 15 are rendered...
Retrying..
waitForRender: 14 out of 15 are rendered...
Retrying..
waitForRender: 15 out of 15 are rendered...
   ✓  Step: 'open flights dashboard' succeeded (9228 ms)

push command completely ignores journey tags:

NODE_ENV=prod npx @elastic/synthetics push --tags "prod" --auth <REDACTED>
> Pushing monitors for 'serverless-prod' project in kibana 'default' space
> bundling 0 monitors
> Monitor Diff: Added(0) Updated(0) Removed(1) Unchanged(0)

It only works if I add 'prod' tag in config-level, but this way tags in Synthetics UI will look confusing: 'appex', 'serverless', 'qa', 'prod';

I believe that ideally tagging mechanism should be identical for both commands, could be possible to use Journey context while filtering monitors for push?

vigneshshanmugam commented 6 months ago

Okay here is the gist of the issue

Look at this test which clearly explains the process - https://github.com/vigneshshanmugam/synthetics/blob/8fd2b339f4961f1b4f8e78d76f7704975455047a/__tests__/core/runner.test.ts#L846

We could fix this by appending the monitor.use tags during push instead of replacing them.