code-pushup / cli

A CLI to run all kinds of code quality measurements to align your team with company goals
https://code-pushup.dev
MIT License
257 stars 14 forks source link

New Source Type for URLs #677

Open BioPhoton opened 6 months ago

BioPhoton commented 6 months ago

User story

At the moment the Issue type only accepts SourceFileLocation like follows:

const sourceFileLocationSchema = z.object(
  {
    file: filePathSchema.describe('Relative path to source file in Git repo'),
    position: z
      .object(
        {
          startLine: positiveIntSchema.describe('Start line'),
          startColumn: positiveIntSchema.describe('Start column').optional(),
          endLine: positiveIntSchema.describe('End line').optional(),
          endColumn: positiveIntSchema.describe('End column').optional(),
        },
        { description: 'Location in file' },
      )
      .optional(),
  });

export const issueSchema = z.object(
  {
    // ...
    source: sourceFileLocationSchema.optional(),
  }
);

As some of problems could be HTML under a rendered URL we could consider this format as additional issue location.

A example for the above described case could be CLS elements within a page measured by the cumulative-layout-shifts audit.

Acceptance criteria

Implementation details

const sourceUrlLocationSchema = z.object(
  {
    url: urlSchema.describe('Url in the browser'),
    snippet: z.string({ description: 'HTMl. snippet in rendered URL' })
  });

export const issueSchema = z.object(
  {
    // ...
    source: z.union([sourceFileLocationSchema, sourceUrlLocationSchema]).optional(),
  }
);
vmasek commented 5 months ago

TODO: @matejchalk create ticket for portal

hanna-skryl commented 1 month ago

I’ve been looking into this, and it seems like the CLS audit (and possibly some others) doesn’t actually include information about the relevant URL or HTML snippet. Instead, it’s more about performance metrics like cumulativeLayoutShiftMainFrame, which are unrelated to the issue source or rendered HTML elements.

Could you share more details on when and where we’d expect to use the new sourceUrlLocationSchema? Just trying to get a better understanding of what specific cases we’d need it for.

Raw CLS audit output
{
  id: 'cumulative-layout-shift',
  title: 'Cumulative Layout Shift',
  description:
    'Cumulative Layout Shift measures the movement of visible elements within the viewport. [Learn more about the Cumulative Layout Shift metric](https://web.dev/articles/cls).',
  score: 1,
  scoreDisplayMode: 'numeric',
  numericValue: 0,
  numericUnit: 'unitless',
  displayValue: '0',
  explanation: undefined,
  errorMessage: undefined,
  errorStack: undefined,
  warnings: undefined,
  scoringOptions: { p10: 0.1, median: 0.25 },
  metricSavings: undefined,
  details: {
    type: 'debugdata',
    items: [
      {
        cumulativeLayoutShiftMainFrame: 0.00019666053497657064,
        newEngineResult: undefined,
        newEngineResultDiffered: false,
      },
    ],
  },
  guidanceLevel: undefined,
};