Meta: ES Modules for documentation / distributed artifacts

[cdata]

TL;DR Please consider distributing the DAT SDK as an ES Module, and prefer documentation that demonstrates how to use it as an ES Module.

The frontier of the JavaScript library-building community is moving towards expressing distributed libraries as ES Modules. Tools like Rollup.js and Node.js¹ are creating cowpaths for a package ecosystem where modules can be more easily/consistently imported in multiple environments by adopting the standard JavaScript module syntax.

On the one hand, I'm sure that a full-conversion to ES Modules would be a lot of work, and is likely not feasible given the sheer number of dependencies involved. On the other hand, it seems like it would be feasible to distribute an artifact that uses ES Modules so that others can consume the SDK that way, and to reflect that mode of usage in documentation.

In the long run, preferring ES Module-based packages will make it easier for dependent projects to apply optimizations such as sharing dependency packages with the SDK, tree shaking and code splitting. These optimizations may be possible today, but in order to apply them effectively we would need a comparatively intimate knowledge of the DAT SDK's dependency graph and all the errata to correctly bundle it.

The DAT SDK is a massive body of JavaScript. The more of it that can be consumed in a standard, consistent module format, the easier it will be to accept the byte cost it brings as a dependency.

Thank you for your consideration, and I look forward to hearing your thoughts on this issue.

¹ Node v12.17.0 "--experimental-modules flag is no longer necessary to use ECMAScript modules (ESM)"

[RangerMauve]

Yeah, I'm super excited to use ESM more in the field.

However, I don't think it'd be possible to rewrite all the dependencies to use ESM since it's a pretty huge undertaking. A good place to start would be to ping the hypercore-protocol module authors about the rewrite first to see if they'd be into it.

I'd be happy to accept a PR to add some sort of ESM support in the meantime. 😁

[okdistribute]

@cdata Thanks. Agree with @RangerMauve, this would be a massive undertaking and there are no funds for much work on dat beyond basic maintenance and apps built on top of it.

[cdata]

@okdistribute @RangerMauve thanks for your enthusiasm and prompt and constructive feedback. As suggested above, I agree that rewriting the whole dependency graph would be an unrealistic goal.

That said, it should be feasible to start from the outside and incrementally work our way through the graph. If the dat-sdk package could be published with an ESM bundle, it would be one small step down the road of progress.

One approach could be to continue publishing as you are today, but also take advantage of the "exports" field in package.json to publish CommonJS and ESM in a way that advertises a different entrypoint depending on the context (legacy Node.js, modern Node.js or bundling tool).

I'll ping the upstream dependencies and try to read the temperature in the room for this kind of change. Thanks again!

[RangerMauve]

Ah cool. A PR with the "exports" field and some docs in the README would be more than welcome. 💜

[cdata]

Thanks @RangerMauve . I have filed a few upstream issues on this topic, and I will rely on the maintainer's best judgement for when it is most appropriate to pursue publishing ESM artifacts in the various Hypercore packages.

In the mean time, I'm just getting to know the Dat SDK so I have some exploring to do. I will look for an opportunity to suggest a PR like the one you described, and we can continue the discussion at that time.

Kind regards,

Chris

[RangerMauve]

Gonna close this for now since it's mostly dependent on upstream. Once all the Hypercore stuff supports ESM, I'll be more than happy to migrate the SDK over.