matthewp / haunted

React's Hooks API implemented for web components 👻
BSD 2-Clause "Simplified" License
2.59k stars 92 forks source link

How to import haunted — in the future! #98

Closed chase-moskal closed 5 years ago

chase-moskal commented 5 years ago

hello friends, i absolutely love the way the javascript ecosystem is evolving, and there are a lot of emerging ideas — so please pitch in with your thoughts about these best practices and changes to haunted that i'm proposing

this proposal calls for a major version bump, as it involves some important changes to the way haunted is packaged and imported

let's imagine using haunted, but in the future 🕶️

in the future, the coolest way to include haunted (or anything really) in your app, will be:

direct imports in the browser, empowered by import maps

  1. configure an import map to resolve haunted and lit-html from unpkg or anywhere else
  2. import {component} from "haunted", and get cracking! it's that easy!
    • haunted and any other libraries will use the lit-html version you specify in your import map
    • everything can work together in harmony :beers:

bundling
also in the future, we'll be able to run that same import map through a bundler to prepare a nice compact bundle for our app, to save on http-requests in production

we can use this pattern today
the best part is that we can use this pattern today via es-module-shims!
another alternative is unpkg's really cool ?module feature.
well, we can't use it with haunted just yet — only after we all agree to and implement the changes in this proposal :)

what's the difference compared to how we import haunted right now?
if you're using a traditional bundler, like with npm, i think there'll be no change for you.
otherwise for es module importers, you'll have to tell haunted which version of lit-html to use, and where to find it.
if you don't, the browser will throw an error.
this is already how it works for other packages like lit-element.
"isn't that just annoying?": no, you see, we actually need our apps to be in control of which versions of dependencies are being used across multiple libraries, so that things like lit-element and haunted can coexist in the same app while sharing a single copy of lit-html (or different versions if you want, the point is that the app can control and configure this)

how haunted must change to get with the program

take a peek at haunted's current surface area

  1. the bundle has got to go

    • in the future, it's the default for packages to present individual es modules, and not to bundle them up
    • es module users usually don't want a bundle, they only want to load what's necessary
    • it's up to each consumer to decide whether or not they want to use bundling in their application (just like minification)
    • consider how lit-element is used: like haunted, it also uses lit-html as a dependency, however lit-element imports lit-html with a bare-specifier, expecting you to resolve this with an import map or a bundler

    proposed changes to haunted:
    haunted's build should emit individual modules, and a new main module should be elected (probably such that import "haunted" will resolve to "haunted/component.js").
    perhaps a legacy bundle could be included.

  2. remove the hard-baked lit-html dependency in lit.js

    • currently, haunted imports lit-html in lit.js directly from unpkg
    • this is a problem, because it locks haunted to a specific version of lit-html
      • that's not cool, because it makes haunted the "dictator king" of the lit-html version being used in your entire app — and if you don't obey, you risk loading multiple copies of lit-html
      • in response, apps will need to configure their own import map to, across the app, resolve "lit-html" to the exact same unpkg link as haunted
      • apps are also forced to load lit-html from unpkg, they can't load it locally or from another cloud vendor like jsdelivr
    • if we don't remove the hard-baked dependency, the recommended practice will end up being to use import {haunted} from "haunted/core.js" and have to configure your own component function, even if you're just using lit-html like default

    proposed changes to haunted:
    we should remove lit.js, and replace all of haunted's internal references to it with a bare-specifier import, like import {render} from "lit-html", and expect the user to resolve the lit-html dependency with an import map or their own bundling step

some of these ideas were discussed with @matthewp and @stevenvachon in pull request #81 (making haunted agnostic with respect to lit-html/lighterhtml) which is at least somewhat related

let me know what you think :wave:

matthewp commented 5 years ago

@chase-moskal Thanks for posting this! Glad to continue the discussion.

The reason src/lit.js uses unpkg is so that the tests can be run in the browser without bundling. I'm fully in favor of non-bundling solutions which is why I made Haunted work this way. It doesn't depend on any future feature like import maps. It works today.

I understand what you're wanting but am a little confused because I think it already exists. Haunted comes with a build, haunted/index.js that uses the bare specifier of lit-html. So it will use the version you selected to download. I don't think this is any different from how lit-element works.

Can you explain what it is about haunted/index.js that doesn't meet your needs?

stevenvachon commented 5 years ago

I'm not familiar with import maps, but if lit-html were removed from this project's source code, could it not then be imported into tests?:

import haunted from 'haunted';
import {html, render} from 'lit-html';

haunted.config({html, render}); // or similar

it('works', () => {});

I'd think this removes the need for unpkg URLs.

chase-moskal commented 5 years ago

@matthewp

Haunted comes with a build, haunted/index.js that uses the bare specifier of lit-html

oh hey that's great, i wasn't aware that the main module for haunted is using bare specifiers -- i was just confused

even still, i think it would be best to remove lit.js entirely, and refactor the tests something like @stevenvachon is indicating, if even to simply to avoid confusion

as for the bundle/modules:
i think we all agree that we want to expose individual es modules (it's a prerequisite for #81), but, there might be two ways to do it

  1. individual modules are the new default
    • import "haunted" gives you component.js
    • import "haunted/bundle" is an alternative pathway to the bundle
  2. the bundle remains the default
    • import "haunted" gives you the bundle
    • import "haunted/component.js" becomes the recommended pathway

i favor option 1, because i see bundling in the same boat as minification -- package authors should be leaving those concerns to the package users -- however i see no reason not to continue including the bundle via haunted/bundle

i think the only point in favor of option 2, is to avoid any breaking changes, and sure, but i do see this as a wart, and ultimately we should just rip this bandaid off sooner than later

what do you think?

matthewp commented 5 years ago

@stevenvachon No, I don't want to require configuration to use Haunted. I personally do not enjoy frameworks / tools where you have to install a dozen plugins and configure them together to do anything useful. Haunted should remain lit-html by default.

There may be some confusion in your example though, Haunted doesn't depend on the html function. It only needs render. This is why it works fine with hyperHTML. It doesn't require that you be a lit-html user.

Now, I don't think people not using lit-html should get the added bytes that they don't need. So that's what this discussion has always been about. The solution is to create a bundle that does not include lit-html.

matthewp commented 5 years ago

@chase-moskal I think I agree with you. I think we can change it so that the main and module fields points to src/index.js. Then we can create this new haunted/component (or whatever we decide to call it) that doesn't have the lit-html dependency and you can use that to configure Haunted to use lighterhtml or hyperhtml instead.

At this point I think I need to get to work and implement a version of what we've been discussing and submit a PR for you to review. I'll try to do that some time this week if possible.

chase-moskal commented 5 years ago

@matthewp

@stevenvachon No, I don't want to require configuration to use Haunted. I personally do not enjoy frameworks / tools where you have to install a dozen plugins and configure them together to do anything useful. Haunted should remain lit-html by default.

i worry you've misread what @stevenvachon is trying to say, and i do agree with him

Now, I don't think people not using lit-html should get the added bytes that they don't need. So that's what this discussion has always been about. The solution is to create a bundle that does not include lit-html.

i'm just not really sure why you want to provide bundles at all?

anyways, neither the lit.js concern nor the bundle concern are really dealbreakers, they are just ergonomic nice-to-have's that i'd like to sell you on ;)

the most important thing that we all agree on, is that we need to expose individual es modules as a prerequisite to #81 in order to use haunted without loading lit-html -- since haunted isn't using any transpiler like typescript or babel, i think this should be implemented with a simple build step that literally copies every file from src/ straight into the package root -- i want to stress that i think including a new bundle for every use-case is a bad pattern, and that the modules should be exposed directly unbundled (even if the current bundle remains the default entry point)

At this point I think I need to get to work and implement a version of what we've been discussing and submit a PR for you to review.

i enjoy discussing best practices and the finer points of javascript package ergonomics, so i'm open to further discussion and i look forward to participating in that review

cheers friend! :wave:

matthewp commented 5 years ago

this is not related to lit-html being default and ready out-of-the-box -- we agree with keeping things that way

I don't follow, how? If I remove lit.js then Haunted has no dependency on the render function that it needs.

i'm just not really sure why you want to provide bundles at all?

I want it to run directly in the browser with no bundler required on the part of the end user. That's why there is the haunted.js and web.js bundles. Once I change the src/ to use the bare specifier for lit-html then the src/ folder won't be loaded in the browser. So the bundles are necessary.

cheers friend! 👋

Cheers to you! Thanks for being willing to participate here, I appreciate it a lot.

chase-moskal commented 5 years ago

@matthewp

I don't follow, how? If I remove lit.js then Haunted has no dependency on the render function that it needs.

haunted doesn't need lit.js to exist in order to import lit-html in any case -- neither for haunted's source code nor the tests -- its removal won't have any meaningful effect except to clean up the codebase

separately from the lit.js situation, i think haunted's component.js should become the main module (instead of the bundle) -- component.js should be importing haunted internals and also lit-html, and then exporting the component function that is configured for lit-html

I want it to run directly in the browser with no bundler required on the part of the end user.

i don't want to take that away, i'd just like to see the bundle move to an entry point like haunted/bundle.js

consider how we use modern packages like lit-element -- they don't provide a bundle:

to clarify what i mean

looking now further into the current haunted bundles:

i strongly suggest, let's make one bundle to rule them all: the bundle should be fully inclusive, including all of the dependencies like lit-html within the bundle -- the ultimate in convenience

i'm in support of keeping the bundle as a convenience, however keep in mind that lit-element has chosen not to provide that convenience

matthewp commented 5 years ago

i think haunted should follow this pattern for which lit-element is setting a good example

I disagree that this is a good example. Web modules are available today. Lit-element requires you either bundle first or use a special server or special CDN. Using modules in the web should not wait on future features. So I think moving the bundle to some place like haunted/bundle.js kind of makes web modules a second class citizen and I don't want to do that. But I don't see why we need to rename the bundles to achieve what you're wanting (a way to load the src code directly from your bundler).

As far as lit.js, it was done that way because it currently has the unpkg urls and it was more convenient to only have to update those in one file rather than in multiple. When I switch the src/ to use the bare specifiers then indeed this file likely won't be needed any more.

chase-moskal commented 5 years ago

So I think moving the bundle to some place like haunted/bundle.js kind of makes web modules a second class citizen and I don't want to do that.

i think maybe that's backwards? i think electing the es modules as the main entrypoint would indicate them as the first class citizen, and so it's my preference to see the bundle demoted to the second-class citizen position

chase-moskal commented 5 years ago

plot twist!

woah! @matthewp, @stevenvachon — check this out — the new pika cdn!

pika just launched this new cdn a few days ago, and it's really, really cool!

this is exactly what we've needed, and, my friends — i do believe — we are indeed witnessing a glimpse of the future of javascript

tech like this won't just go away

packages should be exposing an es module as their main module entrypoint, and the responsibility for bundling, minification, optimization, convenience and likewise concerns, should be left to dedicated tooling like build-time bundlers, and cdn's like this new one by pika

@matthewp — given this new development, i strongly suggest that we engineer haunted to fit into this new ecosystem that pika and others are spearheading — make the es modules the main entrypoint, forget providing in-package bundles, and let users and emerging tooling like pika handle this

while it is possible to support both the old entrypoint for the bundle and then also es modules via the package.json "module" field, i think it's a confusing antipattern to supply different code in the "main" and the "module" fields

keep in mind, the whole motto of pika is basically abandoning build tools like webpack and others, hoping to create the ultimate in convenient web module usage: i think this is in-line with haunted's goals in terms of providing convenient usage patterns

super cool super modern haunted

i think these should become the recommended ways to use haunted

  1. es modules, and do your own bare specifier resolution
    • import {component} from "haunted"entrypoint is "component.js"
    • you can resolve the bare specifiers with import maps (like via es-module-shims)
    • or resolve with a bundling tool (like rollup)
    • or resolve via unpkg ?module mode:
      import {component} from "https://unpkg.com/haunted?module"
      (how cool is that?)
    • also, all of haunted's modules are available, eg, after #81:
      import {haunted} from "haunted/core.js"
  2. use a conveniently bundled version, like from pika's cdn
    • import {component} from "https://cdn.pika.dev/haunted"entrypoint is "component.js"
    • you might choose to use an import map to resolve "haunted" to the above pika url
    • of course you don't have to use the pika cdn: you could just download that bundle and serve it from your own server (or use a bundler tool and you're back above at option 1)
    • of course this is for dev/convenience/learning purposes, because bundled libraries are bad for peer dependency situations

you can include legacy/deprecated stuff too if you want, but i wouldn't recommend their usage

i'd like to know if anybody can think of a use-case that is not well served under this proposal

cheers! :wave:

chase-moskal commented 5 years ago

i'm all-inning on es modules!

you know, this new pika development is so revelatory, and i'm so confident in the advice to make es modules the main entrypoint for all packages moving into the future, that i hereby retract my old-timey proposal for haunted to include funky bundles or to allow a bundle to take the main entrypoint position

and this isn't about pika per se — pika is just one tool implementing these emerging best practices

also of note, haunted is technically pika compatible, because it provides a module field in the package.json — however this is currently an awkward hack, because now pika is bundling the code that is already a bundle, which is weird on its own, but also the bundles won't work architecturally for agnostic #81... it's confusing and i think it'd be simpler to just supply es modules

i'm no longer interested in my other proposals before the plot twist: i just want this done the modern simple cool way, or the highway! i've gotta stand for what i believe in ¯\_(ツ)_/¯

so that's my twenty-two cents, and i hope i can entice you down this yellow brick road towards a harmonious es module ecosystem where packages can be lean and minimalistic, and yet still provide everything everybody wants in terms of flexibility and convenience :rose:

this investigation has proved exceptionally useful for at least me personally, i feel much closer to javascript package best practices and this will improve my own work considerably

cheers! :wave:

edit: i hope i'm not coming off too strongly, and i'm more than happy to continue any discussion — i'm just trying to express excitement for my discovery of this pika cdn bundling feature, which has triggered something inside me cognitively and there appears to be no going back

matthewp commented 5 years ago

@chase-moskal I appreciate your enthusiasm 😀 . I hope to have time to work on this in the next few days. I'll definitely design it with pika in mind. Will let you know when the PR is up and you can let me know if it fits with what you want.

matthewp commented 5 years ago

So if we get rid of the index.js build, how will that impact typescript? There is the index.d.ts file which I think describes index.js. If the new main is src/haunted.js does the typescript file need to move to the src/ folder and become src/haunted.d.ts?

chase-moskal commented 5 years ago

@matthewp

for each module, say component.ts, typescript should be configured to generate the following

i think the haunted package.json should have the main and module fields set to component.js

the package.json types field should be set to component.d.ts

when a user import {component} from "haunted", typescript will refer to component.js and component.d.ts thanks to the package.json directives

typescript is also smart enough to use the relevant .d.ts for other modules too, eg
import {haunted} from "haunted/core.js" should automatically use core.d.ts

i prefer names like "component" and "core" over "haunted" — it makes sense to me that haunted might contain a "component" or a "core", but it's redundant for haunted to contain a "haunted"

for small packages which have zero dependencies, i sometimes opt to use barrelsby to automatically generate an index.ts file, and make that the main module and set package.json types to index.d.ts — this is sort of like an "unbundled" bundle, which can be a convenience, and works well with tree-shaking, however i think this would be a bad practice for haunted because of the dependencies like lit-html (it would be too simple for people to accidentally and unknowingly import configureComponent through the index file and accidentally bring lit-html along for the ride)

matthewp commented 5 years ago

@chase-moskal We're not using TypeScript in this project. The index.d.ts file is written by hand. But since I'm going to get rid of the index.js bundle I'm wondering where the typings should be moved to. I'm guessing src/haunted.d.ts but I'm inexperienced in this.

chase-moskal commented 5 years ago

We're not using TypeScript in this project. The index.d.ts file is written by hand.

oh yeah, right

i think you just place .d.ts files alongside each module, the same as if you had used typescript to generate them

however, if you are planning to provide and maintain typescript definition files, i strongly, strongly, strongly suggest that you actually use typescript — otherwise you have no means of verifying whether or not the code actually matches the type definitions that you are shipping — so your typings are likely to be error-prone, and as a typescript user, i might rather have no typings at all than allowing the possibility that they could be incorrect — i think typescript should be entirely embraced, or completely discarded

converting the haunted codebase to typescript sounds like it would be a big scary project, but it's really not a big deal — somebody already well-versed in typescript could enhance this codebase before lunch

matthewp commented 5 years ago

Yeah, I'm sure its not hard. I'm not a TypeScript user myself. If we had more regular maintainers and they all wanted to convert then I'd be open to doing that.

matthewp commented 5 years ago

Notes to self

Required

Open questions

Progress

chase-moskal commented 5 years ago

package.json main and module fields point to src/ folder.

actually i think that you mean the dist area instead of src

in haunted's case, we're just dumping compiled output into the root directory, it's a tradeoff which calls for more stringent maintenance of the .gitignore, but gives us prettier looking package paths (eg "haunted/core.js" instead of "haunted/dist/core.js") -- personally, i'm a dist/ directory guy -- but that's cool, tomato tomato

of course, the reason for not pointing to src/, is because the sourcecode can be a moving target and be refactored, without any breaking changes to the compiled output (eg, no breaking changes for a typescript conversion in src)

so i think the relevant package.json bit should look like

{
  "type": "module",
  "main": "component.js",
  "module": "component.js"
}

haunted/core.js should point to src/haunted/core.js I guess?

given the above considerations, the following should hold true

import {component} from "haunted"
  // resolves to "haunted/component.js" thanks to main/module fields
  // lit-html will come along for this ride, via bare specifier

import {Container} from "haunted/core.js"
  // directly gets "haunted/core.js", real path in final published package
  // no lit-html

Should there be a "core bundle"?

nah, just make deprecated equivalents to the existing bundles available in the same locations, remove their recommendation from the readme, and drop a line about how users looking for a bundle should grab a fresh one from the pika cdn otherwise use rollup or their next favorite tool

tell you what, you go expose the individual modules like core.js -- and i'll make you a bundle for it faster than you can say "pika slash haunted slash core js" ;)

  • [ ] Update tests to use the haunted.js bundle.

why do you want the tests to use a bundle? is this just an artifact of a workflow that you are familiar with? why not use es modules in your tests?

matthewp commented 5 years ago

The haunted.js bundle is an ES module. There are no non-modules in this codebase. The haunted.js bundle is because bare module specifiers do not work in the browser. The haunted.js bundle instead uses unpkg.com to load lit-html.

Good point on src/, could possibly rename to lib/.

chase-moskal commented 5 years ago

The haunted.js bundle is an ES module. There are no non-modules in this codebase. The haunted.js bundle is because bare module specifiers do not work in the browser. The haunted.js bundle instead uses unpkg.com to load lit-html.

sure sure, but that's the deprecated way -- what happens with the old bundle isn't interesting, what's interesting is how haunted will interplay with the modern ecosystem -- for that we need our individualized module outputs, and that's all that matters to me (and is an architectural prerequisite for #81)

we'll tell haunted users to try a pika bundle, otherwise run a bundler for their app

the old haunted.js bundle being deprecated, doesn't need to be mentioned or promoted in the readme

Good point on src/, could possibly rename to lib/.

if lib is where the sourcecode lives -- then of course we should never point to lib, because we want to be able to refactor the sourcecode safely without affecting the emitted output

if you want to call the consumable output lib, then there's this minor ambiguity problem

so it seems like the hip thing to do is to keep it simple and unambiguous -- src for sourcecode, dist for what gets distributed -- and we just never use lib, in order to avoid the confusion altogether

haunted should take one of two options here

  1. haunted/src/ outputs to haunted/dist/
  2. haunted/src/ outputs to haunted/

personally i think number 1 is cleaner and simpler from a maintenance perspective, but number 2 provides more attractive-looking import paths for your consumers (at the expense of a grossly maintained .gitignore) -- it's a toss-up, but it's subjective

chase-moskal commented 5 years ago

i wanted to reiterate -- the tests shouldn't be using a bundle

each test suite should import the modules that it is testing, like any modern code

:wave: Chase

matthewp commented 5 years ago

To clarify, haunted.js is not deprecated and is still going to be mentioned in the readme. I disagree that using browser modules is not "modern" enough. I don't want to require that people use special tools like pika to use Haunted. They should be able import it directly from the browser; no tools required.

i wanted to reiterate -- the tests shouldn't be using a bundle each test suite should import the modules that it is testing, like any modern code

They can't, the browser doesn't support bare module specifiers.

stevenvachon commented 5 years ago

I like being able to npm install and work locally. I don't know, but I think using URLs in import statements will complicate that. I also use webpack/parcel and babel for everything.

matthewp commented 5 years ago

@stevenvachon If you use a bundler (or any other tool like pika) the package.json main and module fields will work. The haunted.js is only for browser usage.

chase-moskal commented 5 years ago

@matthewp

To clarify, haunted.js is not deprecated and is still going to be mentioned in the readme.

okay

i'd also like it if we could include a link to the pika bundles and encourage their usage too

it would be wise to make the individualized component.js the main/module for haunted

if unfortunately haunted.js was the take the top main/module spot, the above would go vice-versa, and the cool future people will have to specifiy "haunted/component.js" instead of just "haunted" — minor aesthetic wart is all

I disagree that using browser modules is not "modern" enough.

i'm not sure if this statement is confusing because it's a misunderstanding — of course in every case we both agree that we want people to be using es modules — i just don't want the main module to be the bundle — it's the bundles that aren't "modern" enough

They can't, the browser doesn't support bare module specifiers.

that's what the es-module-shims are for ;)

well, if you're going to keep the haunted.js bundle around, i suppose you might as well continue to use it in the tests..

@stevenvachon

I like being able to npm install and work locally. I don't know, but I think using URLs in import statements will complicate that. I also use webpack/parcel and babel for everything.

after the work related to this discussion is merged, insofar as we're on the same page, @stevenvachon, for your workflow you're describing, you'll want to use haunted this way

import {component} from "haunted"

this module will have a bare-specifier dependency on lit-html, which your bundler can resolve from your local node_modules — no URLs

if you want an optimized tree-shaken bundle, you'd then be able to grab one from pika

otherwise you can use the ol' bundle import {component} from "haunted/haunted.js"

@matthewp

i think we're mostly on the same page, let me know if anything i've said makes you queasy to indicate otherwise ;)

i can respect your decision to not deprecated haunted.js, it's nice to preserve familiar workflows for people — i hope that we can agree to make the component.js module the main/module — but let me know if we still disagree with that, in which case i'll update my above example for @stevenvachon's use-case to import {component} from "haunted/component.js"

cheers! :wave: Chase

matthewp commented 5 years ago

i just don't want the main module to be the bundle — it's the bundles that aren't "modern" enough

The main module is not the haunted.js bundle. Maybe that's the confusion. the package.json main will point to the unbundled source. If you're using a bundler, pika, or anything else that parses the package.json, you'll never be using any bundles. The only people who will use a bundle is people who use the browser <script type="module"> and manually type in the url https://unpkg.com/haunted/haunted.js. Does that make you feel better?

chase-moskal commented 5 years ago

@matthewp — yes it does make me feel much better!

cheers!

:wave: Chase