Provide web-api support

[Ragazzo]

I think it can be useful if Yii will provide some web-api support, for example REST, with catching event when request starts, also if Yii will provide some CRestAction < CAction, maybe other workaround can be here to fit REST, what do u think? Situation is common enough so if Yii will give developers some basis it can be good.

Features to implement

• [x] API Testing Framework (RC)
• [ ] File upload support (RC) moved to #4477
• [ ] Batch queries with transactions support and error handling. (GA or post GA) moved to #4478
• [ ] Searching and filtering (GA or post GA) moved to #4479
• [ ] Automatic API Documentation Generation (e.g. via https://developers.helloreverb.com/swagger/) (GA or post GA) moved to #2684
• [x] Ability to serve in multiple formats: JSON, JSONP, XML.
  • already nicely done by Response class: https://github.com/yiisoft/yii2/issues/158#issuecomment-19480770
• [x] Pagination for collections.
• [x] Documentation about which HTTP codes to return when.
• [x] Documentation on how to implement authentication.

[samdark]

I'm for implementing a base layer for rest APIs. Do you have any good ideas about requirements?

[Ragazzo]

yes, will post them later today here, with maybe some links to resources that also can be useful.

[bwoester]

More support for RESTful apps would be definitely a good idea. But if something like this is implemented, imo it should be a really solid foundation.

I certainly don't wanna start to flame here, but just to give a negative example: SOAP-support in 1.x. The idea was really cool, and providing a SOAP service basically by simply annotating your methods could have been a killer feature. But the lack of interoperability with other technologies like java or .net really limited its usability. Also, it didn't provide some enhanced features like the ws-* extensions (never checked if they could have been implemented as extensions though, which would have been okay).

For support of RESTful apps, I think some of the most important requirements would be:

• authentication - pops up in the forum again and again. Can't be provided out of the box, but definitely should be documented in more detail
• paged collections - unpaged just won't scale
• for sub-collections, there should be a ways to switch between including them directly and referencing them
• include/ exclude resource attributes by default, with the possibility for the client to override these defaults
• service discoverability
• possibility to use custom output renderers - while I think json would be a good default, xml remains an alternative
• obviously, it should be easy to re-use your existing models as REST resources

I liked the REST with spring series, it explains a lot of the concepts around REST, so it's worth reading even if you don't use spring.

[samdark]

I'm for never including sub-collection data directly. A reference in a good API should be URL.

Can you elaborate about the following?

1. include/ exclude resource attributes by default, with the possibility for the client to override these defaults
2. service discoverability

[Ragazzo]

@samdark i think we need to create new topic on forum and discuss all there to avoid big comments here, what do u think? can u than create topic and link it here in comments?

[Ragazzo]

also this book covers some common features to implement with comparison of other API implementation, worth to read this https://blog.apigee.com/detail/is_your_api_naked_10_api_roadmap_considerations_part_1_visibility

[bwoester]

Agree, references should be (absolute) URLs.

But simply never including sub-collections won't be the right solution imo. It might be a default, but there are enough use cases where the data you get is really useful and saves you from doing a lot of additional requests. As an example, think about a gallery that has a collection of categories, each category has a collection of images. If you include the sub-collection of images (maybe with a small page size like 5), you have enough data at hand to display some fancy category preview. And it saves you from introducing a special preview-image attribute in your category (which is purely presentation related and thus shouldn't be part of the resource).

I'll wait for the suggested thread for elaborating on the other stuff. Or if you prefer having it all in one place, I'll comment here later.

[samdark]

@bwoester batch queries will solve it.

[samdark]

@Ragazzo thanks for the book, will read it.

[bwoester]

@samdark you mean like "get image-collections for categories 1-N"?

[bwoester]

About "include/ exclude resource attributes by default, with the possibility for the client to override these defaults": "Web API design" refers to it as "partial response" (p. 16). At least, this is the "possibility for the client to override" portion of what I meant.

[cebe]

I implemented REST actions some time ago for a project with yii 1.1. They implement the basic CRUD actions and can handle normal CRUD via HTML but also REST requests by returning differen formats like JSON, csv or ical(used for date objects in this project).

Here is a gist with the actions: https://gist.github.com/cebe/5604091

And this is how a controller uses them: https://gist.github.com/cebe/5604091#file-myrestcontroller-php

Hope this is good for inspiration and helps finding a good solution :)

[bwoester]

"Some time ago" made me remember. :wink:

Here's my try. It's really work in progress, I did this "some time ago" and then stopped working on it, so it's a bit messy, not completed at all, but hopefully with a few ideas that might the helpful: https://github.com/bwoester/yii-rest-extension

I read through the source and tried to remember how things were meant to work, put that all in the README.md and pushed it.

[tonydspaniard]

I experiment on Yiinitializr-advanced boilerplate, and created https://github.com/tonydspaniard/yiinitializr-advanced/tree/master/api/extensions, allowing UrlManager to handle the routing (POST|GET|PUT|DELETE) hope it helps

ps: It only supports Json format

[tjconcept]

"Native" rest support would be amazing. I think it would be best handled by a new REST application extending \yii\web\Application.

[cebe]

FYI: Created yii\web\VerbFilter yesterday which can be used together with these actions.

[cebe]

Here is a proposal for RESTful routing: https://gist.github.com/cebe/5674918 Will implement changes to UrlManager soon.

[bwoester]

Does this proposal for restful routing imply you're concentrating on a solution where each controller implements rest behavior for its own model?

[cebe]

Does this proposal for restful routing imply you're concentrating on a solution where each controller implements rest behavior for its own model?

Why should it? Can't see why other approaches should not work then?

[bwoester]

Other approaches would still work, sure. The proposal only made me think about pros and cons. For example, activating rest behavior on a per controller level might be easier to get started with.

[cebe]

you mean not using <controller> but explicit name? This would of course work. but for the example I think it is better to show what is possible. More documentation will of course come to explain all the details and best practices.

[bwoester]

Nope, I didn't mean using <controller> or an explicit controller name.

What I mean is: Most earlier approaches concentrated on a solution with one single controller that is responsible for all REST actions. Only the resource/ model on which the controller operated changed.

Your new proposal seems to suggest handling REST actions in multiple controllers, one for each resource/ model.

Thinking about it, I see both pros and cons. The nice thing is, it is closer to the normal yii request life cycle. People are used to it and thus it will be easy for them to start writing REST backends.

On the other hand, you want have a uniform API for all your REST resources. You want the API to behave equally, regardless of the resource type you're operating on. Examples for this include the ability to accept and respond with different message formats like json or xml. Or to respond with appropriate http headers.

[cebe]

Yeah, now I see your point. Both variants are good dependent on the situation. I do not concentrate on one of them. Both benefit from this new syntax and this is of course not the most important thing to implement for this issue.

[bobonov]

Issue 303 and 489 are both closed referencing to each other, is it an error?

[samdark]

303 isn't closed.

[bobonov]

Sorry my error, I had bot issue open on 2 different tabs, I looked wrongly and posted same message on both.

[yjeroen]

Some nice reading materialy about Restful services:

• very nice blogpost about OPTIONS with restfull services: The HTTP OPTIONS method and potential for self-describing RESTful APIs
• www.restdoc.org

[bwoester]

Maybe also interesting: http://restdesc.org/

[bwoester]

Since there seems to be no standardized way of describing/ documenting a REST API, I think there should be a way to plugin and switch API desc generators.

However, whatever generator you use, the necessary data needs to be defined somewhere. Yii 1.x SOAP implementation did it using annotations in the controller and in used models. Having that data alongside the code helps maintaining it as a project grows and changes.

[bwoester]

I was unsure whether it is really a good idea to include human readable documentation and examples in an OPTIONS request. After all, the result is meant to be interpreted by a program, isn't it? Nothing a human being would read.

Thinking about what a program can do without human interaction, the first thing I think of is client-side validation. If the OPTIONS request describes the resource and associated messages, this description can be used to validate messages being sent to the server, input errors might be detected before sending them, saving the time needed for the round-trip to the server.

Maybe it could also be used for API versioning, detecting the availability of newer versions or deprecated resources. This could in turn be used to check if newer clients are available and to give the user a hint accordingly.

I also remember an approach that would generate a form for creating and modifying resources given the description of the resource and its attributes. Basically a client side equivalent of yii's form builder.

---

To make a long story short, I looked at WSDL. I think it's similar to a REST OPTIONS request, since it also describes an available API. IN WSDL, nearly every element supports a documentation element, containing human readable text describing the element. Be it a service, an operation or an error being thrown. [1]

Such information might be used to generate an API description out of the OPTIONS result, or at least to link to an existing documentation/ reference. But I'm still not sure if this is the right way to go.

Does anyone have experience with this?

[lubosdz]

@bwoester Generating human-readable documentation via parsing Docvar blocks is difficult and error-prone. But I dont know other better way:-)

BTW - It's a pity that Yii2 will drop support for SOAP/WSDL because it worked fine. Implementing comments/annotation parser for REST would be actually re-implementing CWsdlGenerator. Would it not be wise to have common annotation parser and use it for JSON/WSDL documentation generator?

[iJackUA]

We have build a lot of APIs with Yii framework and had some evolution in it. Definitely SOAP/WSDL is too complicated and harder to maintain. So most of our current API are REST-like with custom routing.

If it could help to push current thread - our biggest concerns was how to make versioned web service ( /v1/action ... ) while keeping native routing nad features of Yii. Currently we are using foloving concept :

• API is a separate web app
• controllers are used as Version routers
• actions are CAction classes (Controller-Version is defining list of supported actions and for example V1 can inherit actions from V2 but overwrite only some of them)
• little magic with dummy Model and override runWithParams in base action class and we can use native rules() from controller in Action (so each Action has validation rules and can vaildate input parameters with native validators).
• pagination of collections is made "manually" - so here is a room to automate, but it depends a lot of internal project Core architecture, just a simple use of ActiveRecors will not work everywhere, as in most big projects wit do not use ActiveRecords applciation-wide.

as for Auto Docs - we thought a lot about it also and implemented the idea of some Reflection usage

• get all controllers - list of available versions
• iterate by all actions registered in controller - list of supported actions
• parser rules() per each action - can display in human readable format list of request params , options state, limits, defaults etc. (everything is taken from standard Yii validators).

it is also possible to make auto-generated test GUI for web service that will prepare fields etc - and tester could play with API via GUI interface

What is still left problematic from my point of view - is authorisation and sessions handling, that is still require good thinking to make it really customizable.

I know - all these concepts I have described are quite home-grown and maybe will not work for all, but it could give some new ideas to everyone here.

[alexvasilchenko]

Regarding iJackUA's comment:

This engine also supports xml and json formats based on overriding getActionParams() method of CController.

BaseApiController has property

public $formats = array( 'xml' => 'XmlFormatter', 'json' => 'JsonFormatter', );

that allows to set which class is responsible for each supported format

Each formatting class should implement simple interface

interface IFormatter { public function encode($output); public function decode($input); public function sendHttpHeaders(); }

This allows us to select one of formats in autodocumentation where we have query emulator.

Would be nice to have something like this in Yii2 natively :)

[fernandezekiel]

thanks, looking forward to this feature.... i think we can make Gii to generate web api code... the extension http://www.yiiframework.com/extension/restfullyii provides a basic set of functions that can look at

[bwoester]

Maybe having a look at other frameworks also helps:

• http://grails.org/doc/2.3.x/guide/webServices.html#REST

Some thoughts:

• Why use vendor mime types including a serialization hint (application/vnd.books.org.book+json;v=1.0) if you support content negotiation?
• Why not include the links they talk about in 8.1.7 Hypermedia as the Engine of Application State in every REST response using HTTP Link headers?
• Didn't find anything about support for OPTIONS-requests (OPTIONS /resource HTTP/1.1 --> what can be done with "resource"?)
• Nothing about authentication?
• What about API documentation? Support for something like swagger would be sweet.

[cebe]

What about API documentation? Support for something like swagger would be sweet.

added it to the list

[minkbear]

http://waveframework.com/ http://www.apigility.org/

What's do you think?

[qiangxue]

I drafted an initial implementation of REST support here: https://gist.github.com/qiangxue/aba36ccad24c0787a7a1 Could you take a look and let me know your thought about it?

Note that this implementation doesn't cover all planned features. It only does the following:

• support serializing resource which allows end users to specify which fields to return and which objects to embed (expand).
• provide basic CRUD implementation that can be easily used by a REST controller.

The core code is the Api class (may choose a better name) which provides the above two features. For each resource, you should create a subclass of Api. In the Gist, I showed an example of UserApi.

Your comments are welcome.

[Ragazzo]

Well, if to me, then i was thinking of something completely another :D based on extending actions and separated stand-alone actions for each type of rest request. Anyway with brief look i think that RestController can be introduces because of the only difference will be in getApi method. Will look closer next few days. I also dont like that user need to spoil his model with api scenario, i think that some workaround can be here like for widgets that can be used with/or without models.

[iJackUA]

@qiangxue maybe it is better to have basic RestController where all functionality of Api component will be encapsulated ? As I see it - like Blueprint controllers that are very common nowadays in Node.js world. You create a blank Controller and specify AR Model that should be related to this controller (like $modelClass you have now). And in general actions should be called automatically considering request method (GET, POST,PUT,DELETE) - that is what not covered in your draft at all. I think respect of request methods - is one of the most common things that should have API to call itself "REST". What is not good as for me that for very basic usage for all models that are exposed via APi I will need to duplicate

public function actionCreate()
    {
        return $this->getApi()->create(Yii::$app->request->post);
    }

etc. I think it should have bacis implementation in parent Controller and then I should be able to override some methods for some Controller if I need, but not to copy paste this boilerplate each time from the very beginning.

[klimov-paul]

@qiangxue, at the moment it does not looks like an REST at all. REST protocol requires usage of request type as an action specification:

• GET – return item or items (actionIndex, actionView)
• POST – create new item (actionCreate)
• PUT, PATCH – update existing item (actionUpdate)
• DELETE – delete existing item (actionDelete)

The actions should be mapped automatically and action call with invalid request type should be restricted.

The URL endpoint for all these request type should stay the same, for example: http://mysite.com/user, http://mysite.com/item

Also for the API version tracking is important.

I am unsure creating an API component is an actually correct way. “export*” methods make sense, but I have feeling they should be composed in something separated as such feature may be used not only for the REST. Introduction of such methods as “create”, “delete” and so on seems to be inappropriate at this level. Their logic should be bound to the controller.

[qiangxue]

Thank you all for your comments!

As I said, this implementation only covers the data serialization and basic CRUD implementation. Features like routing, authentication will be covered in different components.

It seems everyone wants to have a RestController to contain the base CRUD implementation. I have a question, should we provide the CRUD implementation in terms of actions in the base class? If so, how should we "disable" or "hide" an action if we don't want to expose it? For example, the base class provides actionCreate, but I don't want to expose this.

[samdark]

Using a filter?

[samdark]

Or perharps actions will be in separate Action classes and connected to base contoller via actions().

[klimov-paul]

should we provide the CRUD implementation in terms of actions in the base class? If so, how should we "disable" or "hide" an action if we don't want to expose it?

Back in Yii1 I solved this problem in following way. I have created a filter (CFilter descendant), which compared request type with the current action id. If they are not matched it throws 405 HttpException. Then I have applied this filter to controller in following way:

public function filters() {
    return array(
        'defaultRequestType' => array(
            'FilterRequestType + get,post,put,delete,patch,head,options'
        ),
    );
}

So the filter applied automatically to rest actions like “actionGet”, “actionPut” if they are declared in controller. If you do not want these actions you simply do not declare it in controller.

[iJackUA]

+1 to filter. In any case you always can explicitly "mute" some function by throwing 405 http exception inside it or any other dummy response.

[klimov-paul]

For the RestController the most important part is mapping HTTP request type with the actual action. In Yii1 I have created a specific URL rule, which determined the action name for REST controllers as request type name. So HTTP request “GET” becomes “actionGet()”, “PUT” – “actionPut()” and so on. Then developer decides on his own if he wants to support “PUT” request type he should declare corresponding method in his controller, if he do not so “PUT” will not be supported.

[qiangxue]

@klimov-paul We already have support on how to map a HTTP method to an action via URL rules. We also have VerbFilter to determine which actions can be accessed via which methods. We just need some better encapsulation of these features so that they are easier to use. This will be covered via a different component to be addressed later.

I don't think actions should be named like actionGet, actionPut, etc. This is because a single method may be mapped to different actions for different purposes.

It seems we have the following options regarding the rest controllers:

• Define base rest controller with a default set of action methods. Child classes should use filter or override the action methods to disable some of them.
• Provide a set of action classes implementing a common set of rest actions. Controllers use actions() to plug in the needed actions. The drawback of this approach is that action classes cannot share common data (e.g. model class).
• Define a base rest controller with a default set of protected action methods. Child classes need to override an action method to make it public if they want to expose this action.

[iJackUA]

Define a base rest controller with a default set of protected action methods. Child classes need to override an action method to make it public if they want to expose this action.

Hm... as for me it seems to complicated as for very basic implementation. I think that "very bacis" implementation should work out of the box. I think It should work by default to be able to do something like "Super easy Angular/Backbone app and Yii REST api in 5 minutes" (you know like all those Node.js "magic"). If you say that "serious" api module/component is for later and current implementation should be very simple - I think this simple implementation is over-complicated now and would just misguide people if it will be "something not serious, but also not simple". I propose to have "prohibitive" default way rather than "restrictive". At my point of view all actions should work by default in connection wit AR models, and developer should be able to tunr them on/off via filter. And for sure it should automatically map request method to action name - it should work out of the box for Angular/Backbone REST modules without modification (for basic CRUD operations).

I don't think actions should be named like actionGet, actionPut, etc. This is because a single method may be mapped to different actions for different purposes.

I don't really get what do you mean here.

As I understand non-CRUD (verb) actions are out of the scope here (e.g. search, approveFriendshipRequest etc.) as it could be implemented as a simple controller/action with json serialization of response.

[Ragazzo]

@qiangxue can you please not merge it yet into core because of this discussion and suggestions?)) i will also look closer on it in next few days, have some suggestions. Thanks)