search

dart-archive / stagehand

Dart project generator - web apps, console apps, servers, and more.
https://pub.dev/packages/stagehand
BSD 3-Clause "New" or "Revised" License
652 stars 118 forks source link

enable third-party generators? (was: create chromeapp generator) #192

Open damondouglas opened 9 years ago

damondouglas commented 9 years ago

Is this an acceptable addition?

After completing #167, I envision the following:

$ stagehand client chrome
...
Available commands:
  simple    Generate simple chrome application.
  polymer   Generate polymer chrome application.

I would like to accomplish this via extending existing web app and polymer generators so that as these change, the chrome app generators change.

sethladd commented 9 years ago

Thanks for the suggestion!

@devoncarew I think it's time to moving to PHASE TWO... how to enable third-party generators. Curious if you have any thoughts on how we might do this.

Andersmholmgren commented 9 years ago

\o/ On Wed, 14 Jan 2015 at 11:41 am, Seth Ladd notifications@github.com wrote:

Thanks for the suggestion!

@devoncarew https://github.com/devoncarew I think it's time to moving to PHASE TWO... how to enable third-party generators. Curious if you have any thoughts on how we might do this.

— Reply to this email directly or view it on GitHub https://github.com/google/stagehand/issues/192#issuecomment-69977842.

damondouglas commented 9 years ago

@sethladd does enabling third-party generators mean using generators that live outside this repo? If so, would it be acceptable to pass a git url via a command-line argument? Then, stagehand would clone the repo into the target directory and remove the .git directory. If this works, then I wonder if we could possibly move all the generators into separate git repos for ease of maintenance.

sethladd commented 9 years ago

Yes, third-party generators means generators that live outside this repo.

I was wondering if we could scan installed pub packages and find locally installed packages that contain templates. On Wed Jan 14 2015 at 3:07:09 PM Damon Douglas notifications@github.com wrote:

@sethladd https://github.com/sethladd does enabling third-party generators mean using generators that live outside this repo? If so, would it be acceptable to pass a git url via a command-line argument? Then, stagehand would clone the repo into the target directory and remove the .git directory. If this works, then I wonder if we could possibly move all the generators into separate git repos for ease of maintenance.

— Reply to this email directly or view it on GitHub https://github.com/google/stagehand/issues/192#issuecomment-70011092.

devoncarew commented 9 years ago

OK, sounds like we want to think about 3rd party generators. I think we're discussing two options:

templates live on github Essentially what @damondouglas suggested. A template is a github repo, and the template id is the github repo name. So doing something like stagehand damondouglas/chrome-app would look at http://github.com/damondouglas/chrome-app for a repo, and if it found one, download and instantiate it. The repo itself would probably be a fully formed project (with a pubspec, lib/, tests/, ... directories), and probably some mustaches for stagehand to inject some user varaibles ({{projectname}}, {{username}}, ...).

templates live in pub packages What @sethladd suggested. We introspect on packages installed on the system (harder than it sounds...), looking for packages that contain template data in some well known location. I'm not sure what the format of the templates looks like in the packages. We've had to do some unfortunate build steps to copy our template resources into source code, so they're visible to us at runtime. I wouldn't want other template authors to have to also do this.

The first option isn't dissimilar to how bower works - an explicit dependency on github paths. It would make it very easy to de-centralize how stagehand works - anybody could add a new template without the stagehand author's permissions or review. But it would still let us ship a small set of curated samples built-in. So, the 5-7 samples we want to have available, which would just be a list of 5-7 repos containing those templates.

sethladd commented 9 years ago

A couple of questions with templates from GitHub:

How do we run the tool if the developer is offline?

How do we list all available templates?

Andersmholmgren commented 9 years ago

Heat about using pub global activate as a way to install Templates.

Pub itself supports path, hosted, git as three ways to located packages. Would be nice if stagehand was consistent and supported similar methods

trentgrover-wf commented 9 years ago

+100 on 3rd party generators

For my 2 cents, I like the idea of the package source being a fully formed project (as @devoncarew noted in his first option), but I'd like to see path, hosted, and git package locations supported as the initial source.

From that point, I'd suggest following a pattern something like this:

Of course, once that first bit is in place, the very next ask would be for a simple way to prompt user for some more fine grained config options (a la yeoman's askFor methods).

sethladd commented 9 years ago

After chatting with @devoncarew, we'd like to open the request for a design doc and ultimately an implementation. We both would like to see stagehand facilitate third-party templates. If you're interested in helping make this happen, please feel free to jump in!

Also, what kinds of questions would we have to ask the user? What would the template need to know from the user?

damondouglas commented 9 years ago

@sethladd , @devoncarew , first extremely rough draft finished. I confess I had no idea what a design doc really was and googled it. https://docs.google.com/document/d/1eu6Acdjhwrt5LhyyrE9W0AAZhTKy0R5SqM3I7Wd5fcM/edit#

sethladd commented 9 years ago

Thanks! How can we do this without modifying any code?

Have you looked into Isolates as a way to dynamically load code?

kunaldeo commented 9 years ago

@sethladd thanks a lot. Using isolates here makes a lot more sense than anything else. I will be pushing a implementation which doesn't change the source of stagehand or third party template :) .

kunaldeo commented 9 years ago

To tackle Issue # 1 We can add stagehand as the dependency in the pubspec.yaml of the third party package.Fixed please see https://github.com/kunaldeo/stagehand/tree/third-party-templates .

Regarding Issue # 3 I am trying to use LibraryMirror to import the new types. If we don't import the new types mentioned in the sender isolate we get SegFault from VM. Please see this bug https://code.google.com/p/dart/issues/detail?id=15983 .

~~New Issue: When running as global package we get the following error: Uncaught Error: Failure getting http://localhost:50393/stagehandData/isloader.dart: 404 Not Found Looks like installed/compiled packages have limited file permissions :(. It works fine when it is run from source. This basically means isolates are not available from global packages.~~ All these issues are fixed. plugins package solve this problem by using isolates on the template side.

damondouglas commented 9 years ago

@sethladd , thank you for your recent suggestions to use grinder in the design document. Do you think the document is ready to post maybe on Google+ Dart community for feedback?

sethladd commented 9 years ago

Sorry for the delay. I'm a bit confused, though. What's the relationship between @damondouglas and @kunaldeo 's work?

kunaldeo commented 9 years ago

@sethladd No direct relationship. I just wanted to do a simple implementation of it. While it works I am not able to load isolates when running as package.

sethladd commented 9 years ago

@kunaldeo please leave a comment on https://code.google.com/p/dart/issues/detail?id=15983 so our engineers can know who this bug affects.

You won't be able to share a Type between two isolates loaded via spawnUri. You'll instead have to create a message-passing API and communicate with the other isolate-loaded template.

damondouglas commented 9 years ago

@sethladd sorry for the confusion. @kunaldeo , I gave you edit access to the design document. Maybe you and I can polish it up a bit and query feedback from the Dart community. Then, maybe if you still like me after that :), we can work together on a repo to push request.

kunaldeo commented 9 years ago

Sure @damondouglas. But we have to find what is technically possible with dart. I have filed a bug regarding not able to load isolate using Isolate.spawnUri when running from a globally installed pub package, https://code.google.com/p/dart/issues/detail?id=22436

kunaldeo commented 9 years ago

Thanks @sethladd for asking http://stackoverflow.com/questions/28549364/how-can-i-load-code-from-another-dart-package-known-only-at-runtime?stw=2 . plugins looks like the answer https://github.com/samrg472/plugins.dart . I am modifying my branch to use this.

kunaldeo commented 9 years ago

https://github.com/google/stagehand/pull/202 Includes my implementation of third party templates/plugins . @damondouglas nice work on the document :+1: My implementation does not support all the stuff mentioned in the document. Current implementation clones the repo to a directory called stagehandData then based on the plugins installed we add it to the generators . After that point stagehand takes care of it all. I like the idea of a grinder step which can build compatible third party templates. Or better yet provide a stagehand command line option which provides an option to generate third party template package. Sorry I couldn't modify the document. I am bad at editing docs.

Thanks @trentgrover-wf, implementation works exactly how you are proposing. Update command is still in works.

sethladd commented 9 years ago

w00t! Thanks very much for the patch. We'll take a look.

devoncarew commented 9 years ago

After noodling it over with a Seth, re-visiting the design ideas in this issue, and looking over @kunaldeo's PR (#204, thanks!), here's the direction we're thinking of going:

I think this will be a fairly small delta from the current version, will mean that stagehand can continue to ship with a small list of curated dart templates, but can easily support additional third-party templates.

Andersmholmgren commented 9 years ago

Makes sense to me. Leverage pub, make the simple things simple and leave the door open to fancier stuff in the future if there is demand

On Sat, 14 Mar 2015 at 10:55 Devon Carew notifications@github.com wrote:

After noodling it over with a Seth, re-visiting the design ideas in this issue, and looking over @kunaldeo https://github.com/kunaldeo's PR (#204 https://github.com/google/stagehand/pull/204, thanks!), here's the direction we're thinking of going:

  • Stagehand templates are just plain pub packages. They're easy to maintain; when under development they can be run like regular Dart apps. They're easy to version; they can be pushed up to pub once a new version is ready.
  • They contain a small amount of additional metadata, either in the pubspec file or in a sidecar metadata file. This metadata will contain things like required user vars, optional user vars, and things to print out after the template has been created. So, the templates are entirely declarative.
  • Users will install templates via 'pub global activate', just like they do for regular Dart CLI applications.
  • Stagehand will scan the pubcache to know what templates are available. Stagehand's built in templates are just packages that it depends on and that get installed with the main tool via 'pub global activate stagehand'. i.e., they don't need any special treatment.
  • We can do away with the generate-the-template-into-dart-source-code thing.

I think this will be a fairly small delta from the current version, will mean that stagehand can continue to ship with a small list of curated dart templates, but can easily support additional third-party templates.

— Reply to this email directly or view it on GitHub https://github.com/google/stagehand/issues/192#issuecomment-79539159.

kunaldeo commented 9 years ago

Thanks a lot for looking at the PR. I think git based approach will have the following advantages :

If this is an idea we are still toying with then I can build a template that demonstrate why loading code is useful.

sethladd commented 9 years ago

You bring up an interesting point... would we ever need to run pre or post template hooks?

On Fri, Mar 13, 2015 at 6:39 PM Kunal Deo notifications@github.com wrote:

Thanks a lot for looking at the PR. I think git based approach will have the following advantages :

  • As the dart ecosystem grows (like Python/Ruby) having templates as regular cli app on pub will uneven the pub repository. If number of templates are small its ok, but as the number grows you may find more templates in search results than anything else. Using git based approach we can keep the templates away from pub.
  • Storing all templates at one location gives us easy maintenance as they are just git packages. For example they can all be updated at once.
  • Since we are using isolates/plugins, templates can themselves execute some code based on the instructions from stagehand. While I have not demonstrated in current samples (See #204 https://github.com/google/stagehand/pull/204), it cenrtainly opens some interesting possiblities. For example, I can pass on some arguments and data via stagehand to the template to only build the main menu and use the so and so firebase url. The stagehand template can then just generate the main menu, verify if the firebase url is valid and use it while building the project.

If this is an idea we are still toying with then I can build a template that demonstrate why loading code is useful.

— Reply to this email directly or view it on GitHub https://github.com/google/stagehand/issues/192#issuecomment-79663776.

kunaldeo commented 9 years ago

We may need to run it, for example I have a polymer template which scaffolds a basic CRUD app using a firebase URL hooks can help template setup few stuff:

What I meant is having template hooks will allow us do things that can help the developer get started quickly. In Visual Studio 2013/Django you can create a new MVC project point it to a DB and it can generate a fairly good looking app quickly for you. I want something like this. Templates without intelligence need no integration to anything, but when you add intelligence to it it can be really useful.

Adding the hooks will be the template developer's responsibility. So if someone just wants to build a plain old template they can go with it, but if a really creative developer comes along he will have the opportunity to build a really dynamic stagehand template.

sethladd commented 9 years ago

Can we use pub to distribute and update templates, and still use plugins to allow for dynamic templates? That is, it doesn't look like Devon's ideas prohibit more dynamic templates.

sethladd commented 9 years ago

One of the first customers for this can be polymer.dart, to generate new polymer elements.

sethladd commented 9 years ago

Agreed there are cases where a template might have to do something dynamically.

I'd like to suggest that we separate the feature requests into two:

  1. Install, update, and run a third-party template.
  2. Templates that can execute logic when then run.

I think we can and should do both.

On Fri, Mar 13, 2015 at 10:46 PM Kunal Deo notifications@github.com wrote:

We may need to run it, for example I have a polymer template which scaffolds a basic CRUD app using a firebase URL hooks can help template setup few stuff:

  • Validate the firebase/db resource
  • Build necessary resources/code/configuration for the CRUD app
  • Pull external resources in the project

What I meant is having template hooks will allow us do things that can help the developer get started quickly. In Visual Studio 2013 you can create a new MVC project point it to a DB and it can generate a fairly good looking app quickly for you. I want something like this. Templates without intelligence need no integration to anything, but when you add intelligence to it it can be really useful.

Adding the hooks will be the template developer's responsibility. So if someone just wants to build a plain old template they can go with it, but if a really creative developer comes along he will have the opportunity to build a really dynamic stagehand template.

— Reply to this email directly or view it on GitHub https://github.com/google/stagehand/issues/192#issuecomment-79944130.

Andersmholmgren commented 9 years ago

I agree. If only templates that need run code incur the complexity associated with it then that is a good thing imo On Sun, 15 Mar 2015 at 10:44 am, Seth Ladd notifications@github.com wrote:

Agreed there are cases where a template might have to do something dynamically.

I'd like to suggest that we separate the feature requests into two:

  1. Install, update, and run a third-party template.
  2. Templates that can execute logic when then run.

I think we can and should do both.

On Fri, Mar 13, 2015 at 10:46 PM Kunal Deo notifications@github.com wrote:

We may need to run it, for example I have a polymer template which scaffolds a basic CRUD app using a firebase URL hooks can help template setup few stuff:

  • Validate the firebase/db resource
  • Build necessary resources/code/configuration for the CRUD app
  • Pull external resources in the project

What I meant is having template hooks will allow us do things that can help the developer get started quickly. In Visual Studio 2013 you can create a new MVC project point it to a DB and it can generate a fairly good looking app quickly for you. I want something like this. Templates without intelligence need no integration to anything, but when you add intelligence to it it can be really useful.

Adding the hooks will be the template developer's responsibility. So if someone just wants to build a plain old template they can go with it, but if a really creative developer comes along he will have the opportunity to build a really dynamic stagehand template.

— Reply to this email directly or view it on GitHub https://github.com/google/stagehand/issues/192#issuecomment-79944130.

— Reply to this email directly or view it on GitHub https://github.com/google/stagehand/issues/192#issuecomment-80762497.

kunaldeo commented 9 years ago

About distributing templates using pub.dartlang.org . Templates itself are pub packages but in my opinion pub may not be the correct way to distribute templates. If you look at other package distribution system like PIP (Python), gem (Ruby), CPAN (perl) or nuget (dot net), none of them hosts templates in general. You can find some templates here and there but they dont serve as the mechanism to provide project templates.

Again, I am saying this betting on the fact that stagehand templates ecosystem will grow exponentially (because of developer productivity) having templates on pub will hinder the discoverability. Besides pub is better suited to host code that we can add to our projects or can be run stand alone. If we list the templates separately we can have our own categories like polymer/dynamic/simple/firebase/crud and have various filter system like polymer and firebase and show people what they are looking for without even knowing the template name.

Furthermore, an approach like this can help projects like DartVoid (by @andreasolund) and IDEs where they will not have to reinvent the wheel again and again.

kasperpeulen commented 8 years ago

I very satisfied with cookiecutter. With is a python command line tool, but also works for dart projects, and can generate templates from github repos. For example: https://github.com/kasperpeulen/ng2-starter

But I also use it to generate specific boilerplate inside a project: https://github.com/kasperpeulen/ng2-component

Maybe the third-party generator for stagehand can lend some inspiration from cookiecutter?

sethladd commented 8 years ago

Maybe cookiecutter can be ported to Dart? :)

kasperpeulen commented 8 years ago

@sethladd I started working on a port: https://github.com/kasperpeulen/cookiecutter (not finished)

I'm not sure how you envision this. In theory, after I ported it, I could maybe customize it a bit, so that stagehand could use it as a 3th party generator (as a dependency).

Or, it could also just be an alternative to stagehand, for generating dart projects.

sethladd commented 8 years ago

@kasperpeulen we could swap out the guts of stagehand for cookiecutter.

stagehand has two nice properties:

If we can keep stagehand's machine-readable interface, then cookiecutter could become stagehand v2

How does that sound?

kasperpeulen commented 8 years ago

sounds great @sethladd

I will first try to finish the port of cookiecutter. I have not much time this week, but I expect to finish it next week.

I'm not totally sure what you mean with machine readable, and if I would need to add much to cookiecutter for that. But maybe better to come back for that once the port is finished.

luisvt commented 8 years ago

my 2 cents:

to add a yeoman generator you use npm, for example:

npm install -g generator-angular

furthermore all generators fallow the naming convention of adding generator- at the beginning of the name.

So basically what we need is just add something to stagehand to execute other global packages that begins with generator-. In other words, what we would need would be:

pub global activate generator-angular

and to run it, we would do:

stagehand angular

then internally stagehand would only need to do:

Process.start('generator-$name', [args]);

or something similar.

felangel commented 3 years ago

fwiw I’ve been working on mason which allows you to create your own custom templates using mustache which can be hosted either locally or on git.