Yii2 extension skeleton

[schmunk42]

Continued from: http://www.yiiframework.com/forum/index.php/topic/43047-guidelines-for-extension-authoring/

Template or command to generate a Yii2 extension, should include

• composer.json, see also composer init
• autoloading support (PSR-0)
• migrations, see also https://github.com/yiisoft/yii2/issues/384
• guidelines for
  • documentation
  • versioning, see also http://semver.org
  • packagist(?)
  • testing

[philippfrenzel]

close #396

Hi, don't know if its the right place to ask, but I'll do it ;)

So if I write an extension, based upon the yii/bootstrap rules.. and I want to register the asset bundle within the Widget -> so I can register Plugins later by calling view->regis...(/extension/core)... How can I let yii2 know, that all the information of the bundle is within the assets.php thats inside the extensions root folder?

Thank you very much for your support!

[cebe]

@philippfrenzel take a look at AssetManager and AssetBundle classes.

[qiangxue]

It is based on the definition of the so-called root path aliases. A root path alias is an alias that is registered via Yii::setAlias().

If you have an asset bundle foo/bar or foo/bar/something for your extension, there should be a root alias @foor/bar pointing to the directory containing the assets.php file. The AssetManager will use this to load the assets.php file and then identify the asset bundle definition.

When your extension is distributed via Composer, you usually will give it a namespace which is foo\bar. In this case, the root alias definition is automatically done via the mechanism of namespace mapping (Yii::importNamespaces()).

[schmunk42]

Here's a first example about the status of working with extensions, composer and Yii 2. This is closely related to this issue.

While the blog module is not a skeleton, I decided to use it for my first steps, because it provided an excellent use-case.

It's working really great so far!

Regarding the steps mentioned in the README:

Bootstrap project php composer.phar create-project and ./init

I tried to add init to composer script events, but there's a problem with the interactive mode on the shell. But in general I'd like to trigger this on create-project.

Adjust your database connection in ./common/config/params.php

Would be very nice, if we could ask for that in the init script. Further discussion should be handled in a separate issue.

Run migrations (2 steps): ./yii migrate ./yii migrate --migrationPath=@vendor/schmunk42/yii2-blog/schmunk42/blog/migrations

We could let packages register migrations via composer extras.

        "extra": {
        "yii-migrations": [
            "@schmunk42/blog/migrations",
            "@foo/bar/migrations/data"
        ]
    }

And run them post-install and post-update.

I was also thinking about a convention for accessing eg. the user model from the module.

[schmunk42]

We should create a Gii template for this.

[emilcondrea]

@schmunk42 agree

[schmunk42]

Any ideas about a convention for package names in the documentation? I added a few screenshots to illustrate the problem with creating a project documentation including extension code.

The following style are currently used in Phundament packages:

• vendor.vendorname.packagename.yii-type
• vendorname.packagename.yii-type
• packagename.yii-type
• Packagename
• yii-type
• system.yii-type (like Yii Framework)
• None

Is extending the system namespace a good practice? Pros: all behaviors, commands, etc... would be grouped together. Cons: Name conflicts.

Which role play namespaces here?

[qiangxue]

I'm working on yii-extension composer installer. All Yii extensions distributed via Composer should use this as their types. The installer will generate a file describing the installed extensions, e.g.,

<?php
return array (
  'yiisoft/yii2-jui-9999999-dev' =>
  array (
    'name' => 'yiisoft/yii2-jui',
    'version' => '9999999-dev',
    'bootstrap' => 'yii\\jui\\Extension',
  ),
  'yiisoft/yii2-twig-9999999-dev' =>
  array (
    'name' => 'yiisoft/yii2-twig',
    'version' => '9999999-dev',
  ),
);

An extension can specify a bootstrap class. Specific static methods (e.g. init()) in this class will be invoked by application during initialization. This will give the extension an opportunity to do various preparation work, such as registering event handlers, defining aliases, etc.

This array can be accessed via Application::extensions. This would allow features such as collecting migrations/assets required by extensions, displaying installed extensions, etc.

The call of Yii::importNamespaces() in application bootstrap file will also be removed due to this enhancement because an extension can define the needed aliases in its own bootstrap class.

@yiisoft/core-developers Any suggestions on what static methods of the extension bootstrap class we should support? And at which places should these methods be called?

[samdark]

Why do we need extra meta-file? A list of extensions should be stored in composer itself.

Out of the box I personally would like:

• A metod to set permissions / create directories.
• A method to apply migrations.
• A method to update config (not sure about it).

[klimov-paul]

I am concerned about VCS usage here.

Packages installation and composition always reminds me WordPress. It is good system, which is split to different independent modules, which can be installed at will without extra efforts. It is very nice and flexible if you running it at the single same (!) machine without extra code editing. However, if you attempt to make major changes inside the code and/or put the project under the version control or run different debug and production servers – it all becomes a large headache.

Now, to be more specific:

1) A method to apply migrations If I run different debug and production servers, which I always do for my project, I also have 2 (at least) different database. If I install a package, which should apply some DB migrations, I need these migrations to be applied for both databases and any new one I may be need in the future. Another problem here: what if I install some package even before I configured my DB connection? The solution I can see here is allowing several DB migration paths to be registered in the application. While installing package it can add its migrations path to the overall list, while applying of the migration should be left to “someone else”.

2) set permissions / create directories Well it would be nice of course, but what I will gain from it if I using VCS and different debug and production servers? I suppose I will run installer on my debug server, but to transfer the changes I will need to run it again on the production one. I doubt I can use the VCS to store some directories, which are created by package, cause they are probably should store temporary files, or files, which should not be tracked by VCS.

3) A method to update config What about “local” config parameters? For example let’s assume I have installed some ‘Blog’ package, which relies on MongoDb to store its posts. Such package should add a MongoDb connection component into the application config. But the actual configuration of the Mongo connection is not universal – it will require IP address, login, password and so on. And once again at debug server and production server these parameters probably will be different. I do not like the idea my config files (which are stored in VCS by the way) will be edited and recomposed by automated script, cause it may strip such things as comments, ‘if’ blocks and so on. Still we can extract some specific config file for this purposes, warning developers do not edit it by hand. But still there are configurations, which are independent and so can be stored in VCS, and there are configurations, which are “local” and should NOT be stored in VCS.

[klimov-paul]

While using Yii1, I have solved (not in general, just for my working style) the problem of the project deployment creating specific console command. The general idea was taken from the Phing Project, which my company was using some time and of course from ‘webapp’ command of Yii1.

I have created an independent console command (instance of ‘CConsoleCommand’), which performs the basic project deployment/update operations: – creating system and temporary directories, such as ‘assets’, ‘runtime’ and any other if needed. – clear temporary directories (such as ‘assets’) – compose the local configuration files from their templates, which are stored in the VCS. – apply DB migrations – setup cron jobs – etc.

This console command was stripped from the application itself, which allows it to be run just after checkout or update from VCS. It also has ability to use the configuration files, which can specify for example values for the placeholders inside the local config file templates.

I assume we can have something similar for Yii2. An installation script, which can use the configuration files, which can be composed on the package installation, but will be actually applied only on the script run. Such approach allows creation of the system, which can be stored under the VCS and can be deployed at any environment at any time.

[iJackUA]

agree with @klimov-paul (though I am not yiisoft/core-developers, sorry for my 2 cents :) ) in our company we also use custom Phing deploy scripts - it ensure all tmp directories exist, has correct access rights, clean all cache, apply environment config, run migrations etc.

As I can see you have crated init script in apps/advanced and it really nice idea to have such script out of the box that handle most Yii specific things, so I can decide myself when to run this script (inside my custom Phing deploy process).

So it seems that better approach would be if extension could define operations it need to run on deploy and to have some one general init script, that will run through all installed ext. and "re-initialize" each. Also migrations should be triggered separately.

As a general idea there could be init method that is called automatically by composer right after ext. install, but there definitely should be a straightforward way to call init method for all extensions manually during custom deploy process. Framework should simplify deployment, but do not dictate it. If I do not want to run something or want to make all preparations myself manually - I should be able to.

As for config - I would not prefer that any extension will have separate config, it seems right for me when all configs placed in one config folder and there is simple and clear way for me to apply different Env. config changes, and I do not need to iterate through a lot of places where "this thing could be configured". So maybe there could be some default config inside each ext. but there will be a way to re-configure each through some "extensions" section of main config (similar to "components") ?

[qiangxue]

I think there's some misunderstanding here. yii-extension is a new composer package type (enabled by the installer plugin I'm working on). When you write a new extension, you should specify its type as yii-extension in the composer.json file. Then when the extension is installed by composer in your application, the extension will be automatically listed in the array I described above (the array is saved by composer in the file vendor/yii-extensions.php).

What I'm describing above is NOT a console command or whatever. It is totally different thing from the installer that @klimov-paul and @iJackUA are describing. It actually makes developing such installers easier and more flexible.

Why is this needed

Currently, Yii doesn't have a generic way knowing which extensions are installed in an application. Extensions also have no way to automatically bind to important application events or do some preparation work before the extension is actually instantiated. The approach I described above make these possible.

@samdark composer by itself won't treat Yii extensions specially. What we are currently doing is to make use of the namespace map generated by composer. However, this map includes also non-Yii extensions and is mainly used for autoloading purpose.

[samdark]

@qiangxue I meant vendor/composer/installed.json. There it's specified which packages are installed along with their types. If you want to directly use it during framework runtime it makes sense to create PHP file though.

[cebe]

Another problem here: what if I install some package even before I configured my DB connection?

The migrations should not run on composer install, it should be placed for the migrate command to run it at the right time.

though I am not yiisoft/core-developers, sorry for my 2 cents :)

@iJackUA the mentioning is just to make sure all core developers get an email notifiaction about the ticket, it is not limiting request for comments to core devs. Your input is always welcome ;)

Now what I am thinking about this:

There are two different types of things an extension has to do.

1. on installation: apply migrations, set permissions / create directories. update configs...
2. on each app run: when extension is used it has to register event handlers, set aliases, ...

I think Application::extensions could be a configuration array that acts like AssetManager bundles and is there to make specific configurations for an extension and holds information about all extensions installed.

As far as I see the bootstrap class has to provide migrations and an init method for things that fit point 2. We can add other options later when we see the need for it.

For migrations we also need to think about the fact that they may need to be adjustable somehow. either for different dbms or table names etc. also the timestamp in an extension migration might not always fit the order it has to be applied.

Why does an extension need offer a list of assets somehow?

[alpharder]

I don't belong to core developers, but I hope that my opinion can be useful.

• There should be console command (or Gii template) that generates Yii extension skeleton. The generated extension files should be available for use in application right after command (Gii template) creates them. This allows to develop an extension within the context of an application, also you may use git init from the extension directory and git push.
• The boostrap file should contain class without static methods, but with implementation of an interface. So instance of this class should be created on every request, i.e. every time the framework instantiates. The one of probably methods isinit(), which should be probably called at the end of yii\base\Application::__construct().

A lot of design concepts can be found here: http://laravel.com/docs/packages

Hope that changes which are being described at this discussion will make Yii2 really extensible (remember "easy, efficient, extensible" - Yii1 wasn't easily extensible at all).

[klimov-paul]

At the moment I am feeling lost here… What exactly we are considering at this issue?

yii-extension is a new composer package type (enabled by the installer plugin I'm working on). When you write a new extension, you should specify its type as yii-extension in the composer.json file. Then when the extension is installed by composer in your application, the extension will be automatically listed in the array I described above (the array is saved by composer in the file vendor/yii-extensions.php).

I don’t get it. What the sense of it? File ‘vendor/composer/installed.json’ already store all necessary data, does not it? We have nice helper for JSON format parsing, so I can see no obstacles for anyone to track down any installed Yii extensions.

As I understand this issue and most of the inquiries here, we are taking of creating some ‘installation’ process for the extension, so they may adjust the application config, apply DB migrations and so on. The modules installation is main feature of such systems as WordPress and Joomla. I have not heard they are using Composer for such purposes (at least as self-sufficient solution). If we are about creating the full scale package installer, I have already spoken my concerns and fears (see previous comments).

[qiangxue]

@klimov-paul Please take a look at this new property: https://github.com/yiisoft/yii2/blob/master/framework/yii/base/Application.php#L119

This property will be initialized with the array return from yii-extensions.php. Yes, installed.json contains all information, but it is not directly usable and will require precious bootstrap time to parse it.

As @cebe wrote,

There are two different types of things an extension has to do.

1. on installation: apply migrations, set permissions / create directories. update configs...
2. on each app run: when extension is used it has to register event handlers, set aliases, ...

Task 1 is currently handled by post-create-project-cmd (https://github.com/yiisoft/yii2/blob/master/apps/basic/composer.json#L22). We may further improve it with the enhancements you have described.

Task 2 is handled by this new extensions property.

[iJackUA]

@qiangxue ok, so the main idea of current work is to create a common interface for extension init, and on each request to Yii app, it will loop through all extensions and call init method - where events, aliases, modules etc will be registered in App scope (and config) ? And right after ext. install composer will only add it to yii-extensions.php (and all other operations like setting right and migrations will be made by Composer scripts, so it could be called on demand by "composer install/update" or manually ) ?

btw. could there be any situations when an order on ext. declaration in yii-extensions.php will have some importance ? (For example if extensions override the same functionality or config setting, or Extension A is extension for Extension B :) so Extension B should be initialized before Extension A and there should be some way to make explicit declaration of order, or maybe smart generation based on dependency information)

[qiangxue]

@iJackUA Yes, that's the idea.

could there be any situations when an order on ext. declaration in yii-extensions.php will have some importance ?

This is a good question. This can probably be solved by enhancing the yii-extension composer installer so that it will adjust the order of extensions listed in yii-extensions.php according to the extension dependencies. I'd like to leave this for future work when there's such a request.

[schmunk42]

Some input from my side:

@qiangxue First of all, I like the idea of the features you've described here, I think it makes perfect sense to go the way you've described.

Some caveats:

yii-extension

I'd suggest to use yii2-extension since yii-extension is already used by some Yii 1.1 extensions and there was also an installer for yii-extension which was removed. IMHO, yii2 should be used as prefix over yii.

Another problem here: what if I install some package even before I configured my DB connection? The solution I can see here is allowing several DB migration paths to be registered in the application. While installing package it can add its migrations path to the overall list, while applying of the migration should be left to “someone else”.

Applying migrations has to be done after all packages are downloaded. And the chronological order is very important. Eg. I have a media-mangager extension which uses a specific database schema and some of my application components create FKs matching the current schema. Now let's say my media-manager db-schema gets updated and my application components also. I have to make sure that all the migrations are executed in the same order to make this work. btw: @cebe's EMigrateCommand for Yii 1.1 is already doing a wonderful job here!

1. on installation: apply migrations, set permissions / create directories. update configs...
2. on each app run: when extension is used it has to register event handlers, set aliases, ...

I'd like to see a system, where we can ask to user/developer to provide some data, eg. database credentials, enable mod_rewrite, environment settings ... but maybe the init script is sufficient.

[kartik-v]

So how do we set extension namespaces now with the new update. I see that the new implementation has eliminated Yii::importNamespaces (that used to pull entries from autoload_namespaces.php). How are autoloads done now from yii-extensions.php?

[philippfrenzel]

Hi, you need to add a Extension.php to your extension directory, then composer.json you add the path under the bootstrap item. After this you need to ensure, that the composer.json type is yii-extension and then the rest is done "automatically" for me good source to understand was the jui extension -> and if you wanna see, check out my github repos and you'll find a lot of examples;)

/**
 * @link http://www.yiiframework.com/
 * @copyright Copyright (c) 2008 Yii Software LLC
 * @license http://www.yiiframework.com/license/
 */

namespace yii2fullcalendar;

use Yii;

/**
 * This is the bootstrap class for the Yii JUI extension.
 *
 * @author Qiang Xue <qiang.xue@gmail.com>
 * @since 2.0
 */
class Extension extends \yii\base\Extension
{
  /**
   * @inheritdoc
   */
  public static function init()
  {
    Yii::setAlias('@yii2fullcalendar', __DIR__);
  }
}

{
    "name": "philippfrenzel/yii2fullcalendar",
    "description": "Yii2 fullcalendar Widgets",
    "keywords": ["yii", "framework","yii2 fullcalendar"],
    "homepage": "http://www.frenzel.net/",
    "type": "yii2-extension",
    "license": "MIT License",
    "authors": [
        {
            "name": "Philipp Frenzel",
            "email": "philipp@frenzel.net",
            "homepage": "http://www.frenzel.net/",
            "role": "Author Version Yii2"
        }
    ],
    "support": {
        "source": "https://github.com/philippfrenzel/yii2fullcalendar"
    },
    "minimum-stability": "dev",
    "require": {
        "yiisoft/yii2": "*"
    },
    "autoload": {
        "psr-0": { "yii2fullcalendar\\": "/" }
    },
    "extra": {
          "bootstrap": "yii2fullcalendar\\Extension"
  }
}

[kartik-v]

@philippfrenzel Thanks and appreciate the detailed update. As you mentioned its clear with the JUI extension as to how it should be implemented.

[schmunk42]

As a reference, here are the guidelines for Symfony Bundles, they have many reasonable rules.

[cebe]

Here is one extension I created: https://github.com/cebe/yii2-gravatar Did that quick without thinking about any guidelines. Namespace used is not optimal, should better be cebe\gravatar as other extensions can not use the widget subnamespace anymore. for composer this works but not for yii aliases.

[leandrogehlen]

I think that vendor namespace for all extension should be yiiext example: https://github.com/leandrogehlen/yii2-brvalidation

[samdark]

No, it should not.

[schmunk42]

Definitely not! Please see the extension docs.

[philippfrenzel]

can someone pls make an helloworld extension sceleton with a asset alert hello world and an echo hello world and psr-4 json...? @cebe yours is still psr-0

[schmunk42]

Should that go into a folder extensions/skeleton with a subtree split?

[philippfrenzel]

it's up to you... as I would like to have it as an best practice to clean all my "grown" yii2 extensions...;)

[qiangxue]

This should be part of Gii as it requires some inputs (e.g. name, namespace, location).

[schmunk42]

@qiangxue I am working on a proposal, will be available for review this week.

[schmunk42]

@qiangxue Have a look at my first draft: https://github.com/yiisoft/yii2/pull/2411