openwebdocs / mdn-bcd-collector

Data collection service for MDN's browser-compat-data
https://mdn-bcd-collector.gooborg.com/
Apache License 2.0
23 stars 17 forks source link

Scrape more ECMAScript specifications #893

Open Elchi3 opened 10 months ago

Elchi3 commented 10 months ago

We currently only get to know about new ECMAScript builtins when they appear in the main ECMAScript spec. We also need to take into account Intl, proposals, and more.

The web-specs package can be helpful to find all of them:

const tc39specs = [
  ...web-specs
    .filter((spec) => spec.standing == 'good' && spec.url.includes('tc39'))
    .map((spec) => spec.url),
];
console.log(tc39specs);

Currently outputs:

[
  'https://tc39.es/ecma262/multipage/',
  'https://tc39.es/ecma402/',
  'https://tc39.es/proposal-array-from-async/',
  'https://tc39.es/proposal-array-grouping/',
  'https://tc39.es/proposal-arraybuffer-transfer/',
  'https://tc39.es/proposal-async-explicit-resource-management/',
  'https://tc39.es/proposal-decorators/',
  'https://tc39.es/proposal-explicit-resource-management/',
  'https://tc39.es/proposal-float16array/',
  'https://tc39.es/proposal-import-attributes/',
  'https://tc39.es/proposal-intl-duration-format/',
  'https://tc39.es/proposal-intl-locale-info/',
  'https://tc39.es/proposal-iterator-helpers/',
  'https://tc39.es/proposal-json-modules/',
  'https://tc39.es/proposal-json-parse-with-source/',
  'https://tc39.es/proposal-promise-with-resolvers/',
  'https://tc39.es/proposal-regexp-modifiers/',
  'https://tc39.es/proposal-set-methods/',
  'https://tc39.es/proposal-shadowrealm/',
  'https://tc39.es/proposal-source-phase-imports/',
  'https://tc39.es/proposal-temporal/',
  'https://tc39.es/source-map-spec/'
]
queengooborg commented 10 months ago

@Josh-Cena, would you have any interest in implementing this upstream in the es-scraper package?

Josh-Cena commented 10 months ago

Yes, I'll try. This is very, very hard, because many proposals are outdated and don't use the standard spec format (for example, they may use a diff format).

queengooborg commented 10 months ago

Different formats can definitely be annoying -- I see that one of them even uses an API spec format (https://tc39.es/source-map-spec/)!

Thank you very much, though, @Josh-Cena! Let us know if we can help out, I'd be more than happy to submit PRs to assist with this!

Elchi3 commented 10 months ago

Thanks sooo much @Josh-Cena! My list above might be a bit verbose. For the moment, we are really only interested in specs that add additional builtins.

Josh-Cena commented 10 months ago

FWIW, I also manually manage a list of stage-3 specs: https://github.com/jc-verse/mdn-checker/blob/master/packages/mdn-checker/src/data/stage-3.ts