Publish sub-projects on npm, add them to package.json.

[timoxley]

Problem: The build process for polymer is clunky.

Solution: Publish individual components on npm, set up dependencies through each component's package.json, let npm manage gathering and building dependencies through npm's prepublish scripts.

This would get rid of this pull-all.sh hackery, allow pieces to be used in isolation, consolidate dependency management (you're already using npm to manage dev dependencies) as well as make it a more idomatic js project.

[tilgovi]

I think you should just publish them and let the community help discover and work out these issues when they appear in practice, rather than trying to predict them up front.

[robdodson]

@oleersoy it may be the case that there needs to be a dependency conflict resolution flow, similar to what bower does now.

[oleersoy]

So what would bower do when x-bar depends on paper-dialog 1.0 and x-foo depends on paper-dialog 2.0?

[robdodson]

It asks the developer to resolve the issue:

[oleersoy]

What happens when both of the jquery dependencies are mutually exclusive WRT the corresponding consumer APIs?

[robdodson]

Then you can't use both components at the same time in the browser. That's what I'm getting at. It's really important to know this up front otherwise it could lead to weird bugs.

[oleersoy]

So does polymer expect developers to update components, look for a different component, or an entirely different solution each time there is a collision?

[robdodson]

My understanding is it's not just Polymer, it's any situation in which you want to load competing versions of the same thing in the browser. The above example is taken from a jQuery project, for instance.

[oleersoy]

IIUC browserify has solved this by generating an IIFE for each component and injecting the corresponding dependencies.

[oleersoy]

There's also Atomify: https://www.npmjs.com/package/atomify

[robdodson]

I don't think that'll work for custom elements though. You can't have two different x-foo tags registered in the browser at the same time.

[oleersoy]

OK - I think I see what you are saying. So if we have two different paper-dialog versions pulled in as transitive dependencies, Polymer will get confused when attempting to find the implementation for the paper-dialog?

Any chance that Polymer might support version tags on the elements? Say: <paper-dialog:2.0.3>?

[robdodson]

So if we have two different paper-dialog versions pulled in as transitive dependencies, Polymer will get confused when attempting to find the implementation for the paper-dialog?

Not quite, it'll register whichever one loads first, then you'll get an error when the second one loads and it attempts to register it.

Any chance that Polymer might support version tags on the elements? Say: paper-dialog:2.0.3?

That's not really up to Polymer, the underlying Custom Elements spec would need to change for something like that.

[ghost]

If a page depends on two components that register the same element, could a warning or error be generated? It seems to me a runtime error would be sufficient. I would gladly give up install-time reporting of dependency issues for better mom support.

On Aug 31, 2015, at 8:12 PM, Ole Ersoy notifications@github.com wrote:

OK - I think I see what you are saying. So if we have two different paper-dialog versions pulled in as transitive dependencies, Polymer will get confused when attempting to find the implementation for the paper-dialog?

Any chance that Polymer might support version tags on the elements? Say: paper-dialog:2.0.3?

— Reply to this email directly or view it on GitHub.

[robdodson]

@wprl The page itself will throw an error saying something like "attempt to register 'x-foo' but 'x-foo' already exists".

[oleersoy]

It would be great to see @wprl recommendation included, so that we can at least get things up on NPM and start the party. However if we get warnings and have to deal with conflict resolutions frequently when combining components from The wild then a lot of us are going to run away screaming, call our mom, etc.

There are a number of solutions that would allow Polymer to look up the corresponding component implementation via the tag element and component version: (And I'm sure others will come up with more elegant ones, especially if the packages are on NPM): 1) Use a data-polymer-version attribute on the element 2) Include the version number in the element name (I'm sure there's a perfectly legal way to do this) 3) Have polymer read version meta data for each component (Could be generated into a manifest from package.json), in which case both the parent element and the version of the child would have to be used as keys for the lookup. In other words Polymer would have to understand the elements context, such that it could lookup the right version for the implementation.

Number 3 works for transitive dependency conflicts only though.

[adamreisnz]

I'm late to the party, but found out about Polymer and wanted to try it, only to find that it's (still) only installable via Bower or by hand. Having just recently closed the book on Bower and moved to npm for all front-end package management, I really don't want to be forced to go back to Bower.

@robdodson any update as to the status of npm support?

[Nevraeka]

@adambuczynski not yet (please correct me if I am wrong @robdodson). As Rob stated earlier - NPM 3 gets us half way there. We prob won't see official packages until at least NPM 4 (see the notes here. It might be possible to load these with WebPack or StealJS but I have not seen solid examples to solidify that theory yet.

[adamreisnz]

Thanks @Nevraeka

[ericelliott]

Looks like the reasoning boils down to the idea that "Polymer elements and Web Component packages tend to ship code that is only going to be used in the browser". This was basically the founding philosophy of Bower, which has, IMO been proven wrong, which is the primary reason that Bower has been abandoned by so many.

Granted, front-end code does have some concerns that don't belong on the server, but in practice, the real number of those concerns is tiny compared to concerns that can (and should) be shared across client and server.

That front-end-only approach is a non-starter for those of us writing universal apps (which I'd expect to see a LOT more of in 2016).

It's a little sad that the best known and supported framework with a foundation in standards-based web components is currently unusable for universal apps.

[jokeyrhyme]

Has dropping link[rel=import] in favour of ES2015 modules or something JavaScript-powered been discussed? It seems like it'll never be implemented outside of Chrome, and it seems to be a major blocker for using npm 3.

[addyosmani]

Hey folks. Getting official packages for Polymer published to npm is still a priority for us this quarter and we've been investigating it for some time. There's no intention to drop HTML imports as they're a core part of our developer experience.

Some background to previous comments: Polymer requires flat packages more than other JavaScript libraries due to the global Custom Element registry. Browsers have a global environment and not all libraries can be safely loaded in duplicate. This can be tricky in the npm ecosystem because it's common to have multiple versions of X with irresolvable conflicts. We also want to avoid having Polymer apps out where bloated due to multiple versions being pulled in without deduping being factored in.

To work around some of these issues, we're exploring additional tooling that would smooth over some of the rough edges we're unlikely to see resolved until future versions of npm. It's unfortunately not as simple as just running npm publish on all our library and elements, otherwise we'd have done that a long time ago :sweat_smile:.

[ericelliott]

Thanks for the update, @addyosmani -- I'm happy to hear that this effort is still a focus.

[jokeyrhyme]

framework	npm	bower
react	9346	128
angular	5574	5160
ember	2880	262
polymer	361	556

@addyosmani I'm naively assuming cause-and-effect here, but it seems like JS-import frameworks are easier for the community to develop for, compared to Polymer which seems to be the lone HTML-import framework. Even lodging CSS in npm is easier because url(...) in CSS is relative to the current file. I love the amount of 1st-party work that is going into Polymer, but it seems like JS-import frameworks have an easier time developing communities.

I'm not trying to turn this into some kind of meaningless big numbers contest, but surely this is something to investigate and consider. If you can link to mailing lists, etc where link[rel=import] has been discussed as a no-compromise feature, that would help me catch up with your thinking. Cheers.

[oleersoy]

Hi Addy,

I've asked about this before on the list, but maybe I did not detail the scenario enough or too much :). Suppose I find some components in the wild. Component A and Component B.

Both of these require Component C, except Component A requires version 1.2 and Component B requires version 2.3. Does this mean that effectively I either have to update Component A to use version 2.3 or Component B to use version 1.2 or just forget the whole thing?

WDYT?

Cheers, Ole

[robdodson]

I've asked about this before on the list, but maybe I did not detail the scenario enough or too much :). Suppose I find some components in the wild. Component A and Component B.

Both of these require Component C, except Component A requires version 1.2 and Component B requires version 2.3. Does this mean that effectively I either have to update Component A to use version 2.3 or Component B to use version 1.2 or just forget the whole thing?

You'll have to resolve the conflict. the reason is because the browser can't have multiple versions of the same component registered on the page.

[adamreisnz]

As @robdodson said, this is a choice that the developer has to make, and they would have to decide what package and version to use in the end.

Personally I don't think that situation should occur too often. If Component A requires 1.2 while version 2.3 is already out, then it would be an indication to me that Component A is either outdated and hasn't been maintained properly, or at least hasn't had its dependencies updated for a long time. In that case, i would probably steer away from that package and look to see if there's an alternative, or fork it and update it myself if I'd really want to use that particular component.

If it's just a difference in minor or patch version, it should be safe to use the last version of Component C according to semantic versioning.

[oleersoy]

What both of you are saying is true, but only because Polymer is designed to make it true. Polymer could support a versioned component registry (The same way Node supports different versions of transitive dependencies) and then the need for a custom registry goes away.

Also what if the assumption that conflict resolution is going to be required infrequently is off (Although it may be a self fulfilling prophecy assuming this is not fixed)?

The thing is we don't know that Component A has to be updated. And it's not necessarily a simple change. There could be several other components that need the version 1 API of component C and now all these have to be patched as well.

Polymer could print a notification on the console warning about it, and let the dev do whatever she wants to.

If you have looked at a versioned component registry and said that it's impossible for reason A, B, C, then fine, but if that's not the case then you are potentially sending the entire community in the wrong direction.

[robdodson]

What both of you are saying is true, but only because Polymer is designed to make it true. Polymer could support a versioned component registry (The same way Node supports different versions of transitive dependencies) and then the need for a custom registry goes away.

It's not a Polymer decision, this is how the underlying standard for custom elements currently works. The folks working on the specs realize there's more than can be done to give developers additional control over the registry but my understanding is they want to get at least a v1 agreed upon and implemented by all the browsers so we have a foundation to build upon and add new features to.

Polymer could print a notification on the console warning about it, and let the dev do whatever she wants to.

This is what bower does today

If you have looked at a versioned component registry and said that it's impossible for reason A, B, C, then fine, but if that's not the case then you are potentially sending the entire community in the wrong direction.

Not impossible, just not part of the existing spec as I understand it. Again, we're trying to get all the browsers on board and implementing the parts of web components that they can all agree on, then we can continue to iterate.

[oleersoy]

Have you looked at the number of packages in the Node ecosystem ATM? Did you look at the rate at which it has grown? Does Polymer intend / desire this level of success? If every time someone wanted to install a package using NPM they received a ton of warnings about transitive dependencies then we would not even be discussing NPM right now.

Polymer is pushing boundaries either way. Why not just make something that devs love and then worry about getting other browsers on board?

[tilgovi]

Tooling exists already for developers worried about redundant dependencies, transitive out-of-date dependency checking, etc.

This is not a scenario where I expect everything will be awful and broken all the time. This is a scenario where some people, playing a little toward the bleeding edge, may hit some snags and have to work it out as a community.

I feel like not shipping the modules is blocking the whole user community from helping to bake and test this path, though.

Use npm and if the community experiences pain the community will deliver solutions.

On Thu, Jan 28, 2016, 15:17 Ole Ersoy notifications@github.com wrote:

Have you looked at the number of packages in the Node ecosystem ATM? Did you look at the rate at which it has grown? Does Polymer intend / desire this level of success? If every time someone wanted to install a package using NPM they received a ton of warnings about transitive dependencies then we would not even be discussing NPM right now.

Polymer is pushing boundaries either way. Why not just make something that devs love and then worry about getting other browsers on board?

— Reply to this email directly or view it on GitHub https://github.com/Polymer/polymer/issues/326#issuecomment-176476190.

[adamreisnz]

I feel like not shipping the modules is blocking the whole user community from helping to bake and test this path, though.

I agree, and the lack of npm support is the only reason I am not using Polymer yet.

[ericelliott]

Use npm and if the community experiences pain the community will deliver solutions.

I'm inclined to agree with this. I think that if the packages had been published in 2013, solutions would exist already.

[addyosmani]

I've put together an (interim, experimental) npm package containing all official Polymer elements here: https://www.npmjs.com/package/npm-polymer-elements. This is just an idea, so please test.

Notes:

• This is an uber-package covering the current releases for Iron, Paper, Gold, Neon, Platinum elements.
• This package preserves the directory structure needed to get everything working (npm3 currently can't require a completely flat structure as we need). You can just npm install the package and start including elements. @samccone helped me verify it works as expected.
• This package isn't perfect by any means but does work. There are directories included that I'd otherwise be including as completely separate npm dependencies, but this would require changes to Polymer element source paths.
• This package is not 'official official'. I'd like to see if the current proposed interim solution works for folks. If it does, I'll pitch this as something we include in our element release process so it actually gets updated accordingly. We still want to eventually properly support publishing atomic packages to npm rather than this hack. It should only be viewed as a short term solution.
• README is minimal atm. I'll flesh out further depending on where we take this.

[sjmiles]

I re-opened this because there is a lot of discussion here that was in zombie form. Sorry we got stalled on this issue.

[tilgovi]

\o/ Thanks, @addyosmani.

[sjmiles]

After asking around, the information I have is that npm currently cannot require a flat dependency structure and will begin nesting modules when it encounters a conflict, and that this basic issue prevents us from simply switching to npm, as we would probably otherwise just do.

[chuckh]

@addyosmani what about including Google Web Components and Modules too? (did notice marked-element got installed anyway)

[addyosmani]

@chuckh we could try it out. I'd like to see how well this current set works and if we don't find too many thorns, will add in GWC, Molecules and maybe Firebase elements to the next test :)

[jokeyrhyme]

npm currently cannot require a flat dependency structure and will begin nesting modules when it encounters a conflict

@sjmiles does Polymer itself support multiple versions of the same component? if not, then why is it such a big deal if npm nests when it encounters version conflicts? it seems to me that you'd want the developer to intervene in any case, to get back to 1 version of each component

[addyosmani]

In case anyone would like to test the experimental package I published, here's a Material Design sampler page (based on the one by @ebidel) that just uses npm: https://github.com/addyosmani/npm-and-polymer-demo.

Note: This all gets a lot more complex once you start wanting to pull in other third-party elements.

[sjmiles]

@jokeyrhyme My understanding is that if npm nests, it's considered a successful install, but the packages from the web application's perspective would be broken. Yes, you are right we want to developer to intervene at this point, but it's not clear how to alert them to the problem.

[nwwells]

How about a postinstall script that checks for the condition and then reports an error? Haven't really thought this through all the way and can't test it out just now...

On Jan 28, 2016, 21:23 -0500, Scott J. Milesnotifications@github.com, wrote:

@jokeyrhyme(https://github.com/jokeyrhyme)My understanding is that if npm nests, it's considered a successful install, but the packages from the web application's perspective would be broken. Yes, you are right we want to developer to intervene at this point, but it's not clear how to alert them to the problem.

— Reply to this email directly orview it on GitHub(https://github.com/Polymer/polymer/issues/326#issuecomment-176529076).

[justinfagnani]

We don't just need to check for nesting errors, we need to copy files around to match our package naming. npm very unfortunately doesn't have namespacing and several of our package names are already taken, including polymer (I've tried to contact the owner, all dependents seem like they want our project), polymer-js (a fork/bundle of Polymer, maybe we could get it), and a bunch of paper- elements.

This means we either have to remap package names as a post-install task, or use scopes, which are unfortunately not very user friendly and still would require some cleanup to remove the scope name from the path.

Remapping package names would require steps like:

> npm install polymer-polymer --save
> polypackage polymer-polymer polymer // maybe re read config out of package.json

and then some folder like browser_modules/ will contain polymer/

Otherwise we could try to use scopes, but:

• Scoped packages don't show up in npm search :(
• Scoped packages are installed in subfolders of a scope folder. We'd have to either import them as ../@polymer/polymer, or run a similar remapping step as above

This situation is just bad. npm on its own is nowhere near ready for frontend packages.

[nwwells]

None of the problems you highlighted have anything to do with the frontend. They have to do with the fact that the polymer project didn't grab all the package names on npm because bower was selected as the package manager of choice.

Not judging anyone. Just pointing out that this isn't a problem with the design of npm's dependency resolution.

[justinfagnani]

We don't have a closed set of packages, we add elements to paper-, iron-, gold-*, etc. There is no possible way to reserve all the names. The polymer package name was taken over 3 years ago. Having organizations and namespacing on Github means that Bower has avoided these problems. Bower also lets a package remap its dependencies, making it much more likely that you can have a usable packages folder without additional tools.

[justinfagnani]

Right, but that changes the path the packages are available on disk, and so requires changing how things are imported, or a post-install remapping tool.

[ericelliott]

npm on its own is nowhere near ready for frontend packages.

I've been using npm for front-end packages for years, with zero problems. I use a project build step to put files where they're expected to be for execution. Since I'm already compiling from ES6 with Babel, this extra "remapping" step is a non-issue.

[justinfagnani]

We try to keep things as simple as possible so that we don't require much tooling, and so far Bower has had a much simpler workflow: install, import. That's why our focus has been on Bower - for most of our users, it's easier.

Many of our users are either not so advanced in the node/npm ecosystem, or are reluctant to take on the extra burden of build steps and additional tools, and some are already unhappy with even needing node and Bower.

IMO, you can use Bower and npm together well enough, especially if you're already maintaining build steps to convert npm dependencies for use in the browser. I do this for my projects (including exposing the JS as HTML imports). Considering that this is a fairly incremental burden on npm-centric projects, as you point out, it's probably not been an unreasonable tradeoff to let those most used to and capable of dealing with build configurations figure out how to integrate Bower into their setup.

By the way, my comment about npm not being ready for frontend is really tied to the part "npm on it's own". We both have build steps that we pair with npm to make it work. With Bower you don't need that. Not that Bower doesn't have it's own issues.

The equation has been changing over the last few years, but I still don't see that we're close to having an npm workflow that's as easy to understand and use as the Bower workflow. Maybe we need to not let that be a blocker. One obvious short-term option is that we publish everything under the @polymer scope to get things started and let our npm-using community help figure out how to smooth over the rough edges. Scopes in npm are still rough themselves, including the killer that scoped packages don't show up in npmjs.com search, so we probably wouldn't advertise this option yet. We also need our own tooling to help manage releases with well over 100 packages to publish regularly.

It's on our short list to figure this out, so we'll have movement very soon.

[nwwells]

Justin, I know you mean well, and I love polymer. The team has put together something amazing. But it's comments like "It's on our short list to figure this out, so we'll have movement very soon" that force me to conclude that the team behind polymer doesn't want to build an community around it.

You people (regretfully I have to say it that way) are neglecting the user you don't see. You assume that your users need hand holding and explicitly don't want to support those who don't. This has nothing to do with Bower vs npm.

I got excited when 1.0 came out because I thought it meant that I'd be able to champion polymer in my projects. Nope. Turns out that you just plain don't think I'm worth supporting.

Sorry guys, I'm out. I'll check back with you in a couple of years, maybe.

On Jan 29, 2016, 17:29 -0500, Justin Fagnaninotifications@github.com, wrote:

We try to keep things as simple as possible so that we don't require much tooling, and so far Bower has had a much simpler workflow: install, import. That's why our focus has been on Bower - for most of our users, it's easier.

Many of our users are either not so advanced in the node/npm ecosystem, or are reluctant to take on the extra burden of build steps and additional tools, and some are already unhappy with even needing node and Bower.

IMO, you can use Bower and npm together well enough, especially if you're already maintaining build steps to convert npm dependencies for use in the browser. I do this for my projects (including exposing the JS as HTML imports). Considering that this is a fairly incremental burden on npm-centric projects, as you point out, it's probably not been an unreasonable tradeoff to let those most used to and capable of dealing with build configurations figure out how to integrate Bower into their setup.

By the way, my comment about npm not being ready for frontend is really tied to the part "npm on it's own". We both have build steps that we pair with npm to make it work. With Bower you don't need that. Not that Bower doesn't have it's own issues.

The equation has been changing over the last few years, but I still don't see that we're close to having an npm workflow that's as easy to understand and use as the Bower workflow. Maybe we need to not let that be a blocker. One obvious short-term option is that we publish everything under the@polymer(https://github.com/polymer)scope to get things started and let our npm-using community help figure out how to smooth over the rough edges. Scopes in npm are still rough themselves, including the killer that scoped packages don't show up in npmjs.com search, so we probably wouldn't advertise this option yet. We also need our own tooling to help manage releases with well over 100 packages to publish regularly.

It's on our short list to figure this out, so we'll have movement very soon.

— Reply to this email directly orview it on GitHub(https://github.com/Polymer/polymer/issues/326#issuecomment-176998098).