search

lowlighter / metrics

📊 An infographics generator with 30+ plugins and 300+ options to display stats about your GitHub account and render them as SVG, Markdown, PDF or JSON!
https://metrics.lecoq.io
MIT License
13.46k stars 1.67k forks source link

refactor: v4 #1542

Closed lowlighter closed 3 days ago

lowlighter commented 11 months ago

See https://github.com/lowlighter/metrics/discussions/1533

This PR introduces a major refactor to the codebase to offer more possibilities, make the code cleaner, safer and more maintainable while keeping a compatibility layer to make the migration smoothly for users

📝 Todo-list before pre-release

lowlighter commented 11 months ago

✈️ Migration guide will be made at the same time that cli/compat.ts is implemented

lowlighter commented 10 months ago

This is an unfinished draft that will be publied eventually in announcement when this pr somewhat ready. Subject to change

4️⃣ Metrics v4 alpha preview

[!NOTE] While not available yet, metrics v4 is expected to offer a pre-release in the coming weeks. You can track the progression in this pull request #1542, stay tuned !

🧩 Plugins changes and new features

🏛️ Legacy plugins .legacy

It will take some time before all plugins and options are eventually migrated to v4. Fortunately, the new .legacy plugin lets you call plugins directly from v3.x versions!

It will pull the corresponding docker image, pass your inputs, generate the render within v3 context and retrieve it inside the v4 engine.

Here's a simple example of how to achieve this:

plugins:
  - .legacy:
      version: v3.34
      inputs:
        plugin_isocalendar: yes
        plugin_isocalendar_duration: full-year

📰 Recent activity activity

This plugin has been completly redesigned, to offer more data, code highlighting, and better event filtering.

📆 Commit calendar calendar

Current isocalendar and calendar have been merged together to ensuring features consistency between both. It means that you can now generate isometric calendars over multiples years, while the top-down calendar now have additional statistics along each year. You can also change colors more easily.

💻 Lines of code changed lines

This plugin has been redesigned to offer more data granularity, where you'll be able to show diff per week and by contributor. Additionally, the issue with the stats sometimes being empty have been fixed in this release!

📦 Miscellaneous

All plugins are now totally independant (actually the base plugin has not even be migrated yet).

Each migration also take a look at current opened issues, so if you've opened one and haven't receive a response yet it may actually be fixed in v4 ! Note that it also means that development of v3 is currently "frozen"

The following plugins have been migrated but don't really have any notable changes:

🪄 Processors and new features

The core plugin is being decommissioned entirely as it's being split into different "processors", which will offer more flexibility and power.

You can apply processors on a per-plugin basis, on even on a group basis using the .await plugin (which await for all previous plugins execution). Thanks to this you can now do neat tricks, such as using different templates per plugin, publishing your renders under different formats to various destinations, etc.

plugins:
  - introduction:
    template: terminal 
  - gists:
  - .await: 
    processors:
    - inject.style:
        style: .title { color: red; }
    - render:
        format: svg
    - optimize.css:
    - render.twemojis:
    - transform.base64:
    - publish.git:
        repository: my/repository-A
          commit:
            to: metrics-render-branch 
  - processors: # ".await" plugin is actually implicit :)
    - render:
        format: json
    - publish.gist:
        gist: ...

In the future, it is planned to offer more option to aggregate data together and also "rehydrate" data from cached data.

Below is a list of currently supported processors:

🧼 Templates simplification

The classic template (the default one) is being reworked heavily. The aim is to use simpler styling rules (without compromising design) to improve performances, but also to make it easily overridable by users through the new injection processors.

They're also designed to have better class conventions to make it easily manipulable with document.querySelector(). Additionnaly, styling related to single plugins are being moved out of the global stylesheet to avoid loading useless rules when a plugin is unused.

<section class="introduction">
  <div class="title">
    <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16" width="16" height="16">...</svg>
    About ...
  </div>
  <div class="body">
    <!-- ... -->
  </div>
  <style>
    .introduction .text {
      white-space: normal;
      margin: 0 13px 2px;
    }
  </style>
</section>

🚂 An entirely rewritten engine focusing on...

🔐 Security

Metrics now support GitHub fine-grained tokens and deno permissions, which means you are guaranteed to have full control on which data this action has access to and what it can do with it.

Since specifying all permissions may be tedious, metrics is relying on deno_make and describe the "minimal permissions subset" in the deno.jsonc configuration file, but you are free to specify your own permissions !

🍱 Portability

Configuration syntax has been simplified and unified across the different modes of execution which are officially supported. Same configuration file can now be reused across the different modes.

☁️ Deployability

Currently the web version is hosted on a server that I rent. Since it's managed by myself I sometimes forget to do it (who said "invalid ssl certificate" ?).

The plan with this new engine is to make it compatible with serverless services, such as deno deploy and vercel (with deno), and also a docker image that is designed to be used outside of the metrics GitHub Action.

To achieve this, all features relying on prerequirements (browser, filesystem, etc.) are being abstracted so they can be replaced (for example the browser can use a remote service such as browserless.io or just being made optional. It probably won't be possible to achieve features parity but it'll still be close to it.

This will also be used to test and preview new pull requests changes.

🗿 Stability

Fully rewritten in TypeScript, configuration, inputs and outputs are now fully typed and strictly checked directly in the code with Zod, making it easier to work with Metrics.

/** Inputs */
readonly inputs = is.object({
  feed: is.string().url().default("https://news.ycombinator.com/rss").describe("RSS feed (e.g. `https://news.ycombinator.com/rss`)"),
  limit: is.number().int().min(1).nullable().default(4).describe("Display limit. Set to `null` to disable"),
})

/** Outputs */
readonly outputs = is.object({
  name: is.string().describe("Feed name"),
  description: is.string().describe("Feed description"),
  entries: is.array(is.object({
    title: is.string().describe("Entry title"),
    link: is.string().url().describe("Entry link"),
    date: is.date().nullable().describe("Entry date"),
  })).describe("Feed entries"),
})

🔄 Versatility

While it is not entirely ready yet, in the future you will be able to import metrics plugin and processors directly in external programs. The new inputs/outputs validations schemas ensure a stable API with consistant results.

So you can built new apps upon metrics:

import { Plugin } from "https://deno.land/x/metrics/plugins/mod.ts";
const {result} = await Plugin.run({
  context: {
    id: "introduction",
    entity: "user",
    handle: "octocat",
    retries: {},
    mock: true
  }
})

🧪 Reliability

The code is now tested more thoroughly with better tests and less hacky workarounds. It also now features code coverage. The aim is to provide a fully tested codebase with a coverage close to 100%.

⚡ Rapidity

The boot-time is now significantly lower as code has been refactored and optimized to decrease bottlenecks (no more dependencies on base and core plugins, and no need to load up all plugins at startup).

Most of queries have been optimized to consume less requests, and templates have been refactored to reduce the number of nodes created to make simpler outputs.

Unlike node_modules, ESM modules offers easier tree-shaking which also means that running a non-cached version of the code will be faster as only the used code will be fetched to run it

♻️ Reusability

As mentionned previously, same plugin can now be called multiple times using differents scopes and contexts (e.g. different users, tokens, timezones, api endpoints, templates, etc.).

The end goal is to offer an easier version of the existing presets system by integrating it directly in the configuration file. It means that you will be able to reuse a configuration from another user just by overriding the variables you want to.

plugins:
  - calendar:
      handle: octocat
      token: ${{ secrets.TOKEN_OCTOCAT }}
      timezone: Europe/Paris
  - calendar:
      handle: octodog
      token: ${{ secrets.TOKEN_OCTODOG }}
      timezone: America/Montreal

🎴 Extensibility

Using the power of ESM modules, you can easily include remote versions of plugins and templates (perfect to rollback a given plugin on a specific version or to use a custom version of it). Less patches to apply and more flexibility! 

plugins:
  - https://raw.githubusercontent.com/lowlighter/metrics/v4/source/plugins/calendar/mod.ts:
      handle: octocat
    template: https://raw.githubusercontent.com/lowlighter/metrics/v4/source/plugins/calendar/templates/classic.ejs

🏛️ Compatibility

Configurations from v3 and earlier versions will still work thanks to the compatibility layer which will automatically convert old inputs into the new v4 format, and offering suggestions and tips for migration. Apart from really niche options and design changes, migrating will be almost transparent.

🧰 Ergonomy

The web UI has been entierly redesigned with Alpine.js and Tailwind CSS to make it simpler and friendlier. The inputs are now auto-generated through the inputs JSON schemas. Thanks to the improved test data, mocked previews are now more close to what will eventually be generated

🤝 Community

While the old codebase survived more than 50 versions, it wasn't really friendly for new contributions. This rewrite hopes to make it easier to contribute by:

💪 Interested in contributing ? Check out issues with the 🆕 v4 label Contributions for v4 will be accepted once #1542 is landed Whether it's to migrate old plugins from v3 to v4, or create new plugins or options, all contributions are welcomed !

💕 Liking (the future of) metrics ? You can sponsor me to support development ! Even one-time contributions are welcomed 🥰

github-actions[bot] commented 5 months ago

This pull request seems to be inactive and will be automatically closed in a few days if it remains without any activity.

Andre601 commented 5 months ago

This pull request seems to be inactive and will be automatically closed in a few days if it remains without any activity.

lmao.

Andre601 commented 5 months ago

I just received a PR from dependabot that updates metrics to v4..... while it still isn't released here yet... How does that even work?!

github-actions[bot] commented 1 week ago

This pull request seems to be inactive and will be automatically closed in a few days if it remains without any activity.