Better browser builds and ES6 module support

[carsonfarmer]

Ideally, we'd take a page from the https://github.com/multiformats/js-multiformats book and support cjs, esm, etc builds via rollup and friends to support multiple platforms/use-cases. Since this is designed to be a light-weight alternative to other ipds-based libraries, it should have a really low barrier to usage out of the box. In fact, moving forward, this should be a pretty critical requirement. Further (planned) dependency reduction will help here significantly, but we need a sustainable path to ESM builds that makes it really easy to consume this lib.

See also #73 and #74

[Gozala]

Hey @carsonfarmer what exactly do you have in mind ? One thing that I've been wondering is trade-offs between:

1. Authoring all in TS, then generating ESM, and then generating CJS.
2. Authoring in TS via JSDoc in ESM, and then generating typedefs & CJS as in https://github.com/Gozala/web-blob

[carsonfarmer]

Yes great question. I have been exploring the TS -> ESM -> CJS route via rollup, and this seems to be pretty nice. Couple that with package.json exports and things are finally feeling pretty useful for all three "worlds". On the same token, I've recently started playing with TS via JSDocs in pure ESM, and while I'm not 100% convinced this is going to be nearly as "safe" as writing in TS to begin with, if it increases the likelihood that external folks will contribute, I guess I'm for it.

One thing I've noticed in my experiments with TS via JSDocs, is that if you have the right IDE (VSCode is a good example), it'll take advantage of d.ts files and try to infer types, and it does a good job. But it benefits from having those d.ts files. So we'd have to ensure that we are enforcing correct JSDocs there. Incorrect types are almost worse than no types at all IMO.

[carsonfarmer]

Note that in my experiments with TS -> ESM -> CJS, I'm actually not using tsc at all. Just the typescript IDE integration, and then rollup to actually compile all the outputs in parallel.

[Gozala]

On the same token, I've recently started playing with TS via JSDocs in pure ESM, and while I'm not 100% convinced this is going to be nearly as "safe" as writing in TS to begin with

What do you mean ? It can be just as "safe" as authoring in TS, it just purely a syntax change. Granted it requires some extra tsconfig settings.

So we'd have to ensure that we are enforcing correct JSDocs there. Incorrect types are almost worse than no types at all IMO.

Absolutely! I think we should continue running type-cheker on CI regardless of .ts / .js settings. BTW you have very rad setup with a bot reporting type checking inline!

[carsonfarmer]

What do you mean ? It can be just as "safe" as authoring in TS, it just purely a syntax change. Granted it requires some extra tsconfig settings.

I just mean in terms of code updates. There's always the possibility that someone won't update the JSDocs, whereas, if the code itself is written in TS, you can't not update it correctly. But as I said, I'm willing to live with this if it increases the likelihood of external contribs and makes others happy :)

Absolutely! I think we should continue running type-cheker on CI regardless of .ts / .js settings. BTW you have very rad setup with a bot reporting type checking inline!

Yeh pretty cool right?

[carsonfarmer]

Also, I guess keeping all the code linting and type-checking in place does mitigate mostly.

[Gozala]

Personally only benefit I personally see with using JSDoc over just doing it in TS are:

1. No need to compile to run code. Specifically being able to copy & paste code from source to repl or browser console and eval end up helping more than I've anticipated. However this might be very individual to how I use tooling.
2. Contributors discover type-checker only after they submit pull. So they are sort of invested to get it to the finish line. Code in TS might deter some. That said, I don't think people deterred by TS will stick around regardless of TS or JSDocs.

In terms of downsides to use of JSDoc, here's my experience:

1. It is way, way, way more verbose. Annoyingly so...
2. Not everything can be typed in JSDocs, so sometimes I do resort to additional .ts files that just contain interface & type definitions.

[Gozala]

If it is trivial to get TS -> ESM -> CJS going, I'd say there is little benefit to rewriting the code base into JSDocs and we should leave it as is unless faced with a compelling reason to make a change.

[carsonfarmer]

I actually agree re: point 1 above, and I have also found that it increases my efficiency when "testing" things locally. But I also agree with this last point, so let's keep it in TS for now, and then if we find that it is getting in the way of things, I'm happy to revisit. Its easier to drop TS support than to add it back in later :P

[Gozala]

Hey @carsonfarmer do you have something in the works here ? If not I think I'd try to take it off of your plate.

[Gozala]

One thing I did run into the issues with trying to do something like this with TS in the past was that ESM wants you to include file extension while TS does not. If you do have some solution for this would be great if you could share.

[carsonfarmer]

I don't have anything in the works yet for this, no. So please feel free to claim it 😎. I didn't find a solution for the file extension issue, though did explore this to some degree. I think there are hacks possible by including the file extension and somehow telling ts to ignore those, but not sure if that was something I dreamed up. I'm afk at the moment, so unable to confirm just now 🤷‍♂️.

[mikeal]

ok, so, we have some better tooling for this now ;)

I have js-multiformats and all of its dependencies updated to use ipjs for build and publish, which resolves a lot of issues with using a native ESM flow and still supporting require() and older browser compilers.

The block API is also upgraded now, and I’m working through the things that depend on @ipld/block as well. Now that almost 10 libraries are using the toolchain I think I’ve found most of the bugs. However, the best practices for how to write a native ESM module that works well given the toolchain and the limitations in different platforms is not entirely documented, so whenever you take this on we may want to sync up about that ahead of time.

[carsonfarmer]

This is great @mikeal! Note also @Gozala that I am starting to think that maybe TS via JSDocs is going to be the easier way to handle things. Right now, we depend on on a lot of libs that don't have typings yet, so there is a folder full of types that are difficult to maintain. It might be better to just avoid this whole mess, do our own types via JSDocs, create the typings as part of the build process, and be done with it?

[Gozala]

Right now, we depend on on a lot of libs that don't have typings yet, so there is a folder full of types that are difficult to maintain. It might be better to just avoid this whole mess, do our own types via JSDocs, create the typings as part of the build process, and be done with it?

I don't think JSDocs is going to make a difference in regards to needing typings for external libraries. TS will still need those just the same.