tdt / core

Transform any dataset into an HTTP API with The DataTank
http://thedatatank.com
83 stars 31 forks source link

Installed resource: use annotations for discovery? #330

Open netsensei opened 9 years ago

netsensei commented 9 years ago

So, I would like to add an custom created installed resource. But I was wondering how discovery of the plugin happens. The documentation says that this can be done in 2 ways:

1/ Use the UI to manually configure the location of the plugin in the installed folder and such. 2/ Create a so called "discovery document".

=> The latter options is a bit unclear: Where do I need to store this document? Config folder? Or do I need to add it to package it with the plugin itself?

=> I wondered: has using annotations to do auto discovery of plugins ever been suggested?

coreation commented 9 years ago

Hi @netsensei! Our UI is entirely build on our discovery document, meaning that the UI will automatically adapt when certain things in that discovery document change. This means that 3rd parties can use our API, if they create REST calls that are being outlined by the discovery document.

If you look at the UI of an installed resource you'll find all the parameters that are in there, matches all of the necessary parameters in the discovery document. So the "discovery document" is not a functional thing, but a template thing, namely the template of our entire REST API :).

Bottom-line: use the UI to configure your plug-in, if you feel like hacking, create a CURL request in the CLI with a JSON document that provides all of the necessary parameters (not recommended ;))

Edit: no annotations have never been suggested :) Are there ways in PHP to discover annotated classes?

netsensei commented 9 years ago

Interesting! I've toyed with in the past, but I'm now at a point where I'm ready to look at this a bit more in depth.

Are there ways in PHP to discover annotated classes?

Yes. Drupal 8 does this: https://drupalize.me/blog/201409/unravelling-drupal-8-plugin-system

I've been thinking about this a bit. Drupal heavily relies on the DI container to manage/load plugins and classes. The tdt doesn't use the DI container to load installed resources at all:

From INSTALLEDcontroller.php:

        // Include the class
        $class_file = app_path() . '/../installed/' .  $source_definition['path'];

        if (file_exists($class_file)) {
            require_once $class_file;

            $class_name = $source_definition['class'];

The other thing is that installed resources are not installed via a package manager like packagist / composer. You need to download them and move them into installed/ manually. Then you have to commit your entire tdt project in your own git repo. The problem is that this setup is hard to maintain: there is now code from two different sources in your project, and there is no way to track if an installed resource was updated.

My suggestion was:

(1) Turn installed resources into first class citizens by adding a composer.json file to them. (2) Use a annotation based discovery manager to dynamically scour for plugins in the vendor/ folder, register them (ie. in the database table) and load them on runtime. (3) Perhaps leverage the Laravel's DI container to manage their classes.

Then again, maybe this is an over engineered solution, given that installed resources are very application / case specific?

coreation commented 9 years ago

Hi @netsensei, they could be packages indeed! Haven't looked into annotation based discovery yet, but seems like a more developer friendly way to do things. It might not be over-engineered if it makes adding an installed resource easier though. For now, I'm not gonna include it in the 6.0 release, need to check up on it first, but thanks a bunch for the suggestion!

netsensei commented 9 years ago

Thanks for the feedback! Yes, this could be quite an architectural change.

I've been toying with Laravel packages (see: packalyst.com!). The idea was to:

The latter is a problem because that would still mean copying code from the package into an app specific location which is hard to maintain.

I did find out that each package gets its' own serviceProvider and you are still required to add that manually to app.php. I don't think this is an issue: it's common practice with Laravel. Maybe this would be a good place too hook TDT specific logic? The provider allows for wiring up classes in the container...

netsensei commented 9 years ago

FWIIW: We should keep an eye out on Pulli:

http://docs.puli.io/en/latest/ https://github.com/puli

"Puli (pronounced “poo-lee”) is a universal package system for PHP. Puli aims to replace “bundles”, “plugins”, “modules” and similar specialized packages of different frameworks with one generic, framework independent solution."

Instead of reinventing the wheel here through a custom plugin discovery system. We could leverage a generic/universal solution instead.

Note: the focus of the documentation emphasizes reuse of non-PHP resources, but the introduction mentions that a pulli package does consists of both PHP and non-PHP files.

coreation commented 9 years ago

Sounds very interesting, added to my todo list ;).