search

yiisoft / yii2

Yii 2: The Fast, Secure and Professional PHP Framework
http://www.yiiframework.com
BSD 3-Clause "New" or "Revised" License
14.24k stars 6.91k forks source link

Provide web-api support #303

Closed Ragazzo closed 10 years ago

Ragazzo commented 11 years ago

I think it can be useful if Yii will provide some web-api support, for example REST, with catching event when request starts, also if Yii will provide some CRestAction < CAction, maybe other workaround can be here to fit REST, what do u think? Situation is common enough so if Yii will give developers some basis it can be good.

Features to implement

qiangxue commented 10 years ago

I won't. I'm splitting it into RestController and ModelSerializer. Will ask for another review later.

Ragazzo commented 10 years ago

good, i also think that when it will be in core it should be under yii\api\rest ns and not just yii\api.

klimov-paul commented 10 years ago

We already have support on how to map a HTTP method to an action via URL rules

I remember that. But if I want to setup the rules to map HTTP request type with actual controller action, how many rules I will need to add? And most important: why should I do it? It should be something “out of the box”. Also, what should I do, if I do not use “friendly” URLs at all?

I don't think actions should be named like actionGet, actionPut, etc. This is because a single method may be mapped to different actions for different purposes.

Indeed, it could be (and probably will be) several actions in the same controller, which will have same HTTP request type. I have never told this should be restricted. But in this case their names should be explicitly passed to URL or mapped via URL rule. “actionGet”, “actionPut” and so on are just default ones, which will be applied magically, if defined, and no explicit action id is passed with URL. Same thing happens for “actionIndex” in normal web controller: just define such method and you have default page for controller, remove it and you will have 404 if no action specified.

Define base rest controller with a default set of action methods. Child classes should use filter or override the action methods to disable some of them.

Well, I don’t get it. Compare: there is a common set of operation for web controller named CRUD. It is a common start for most admin panels. However none is talking about creating base CrudController with a “default set of action methods”. Instead we have a Gii generator, which creates some basic code, which should be modified later. REST is not so strict as SOAP, I do not think we should framework should provide its implementation like “just add your model” style. REST support should be flexible.

fernandezekiel commented 10 years ago

the typical CRUD actions works well with me..... it's just up to the users how to map the Verbs to those actions

qiangxue commented 10 years ago

But if I want to setup the rules to map HTTP request type with actual controller action, how many rules I will need to add? And most important: why should I do it? It should be something “out of the box”. Also, what should I do, if I do not use “friendly” URLs at all?

URL rules are only part of the story. You need them if you want friendly URLs. I don't think we should do hidden magic here (how would you do this anyway?). What we can do is to provide something like a macro rule which expands into smaller ones mapping to each REST method and resource.

Indeed, it could be (and probably will be) several actions in the same controller, which will have same HTTP request type. I have never told this should be restricted. But in this case their names should be explicitly passed to URL or mapped via URL rule. “actionGet”, “actionPut” and so on are just default ones, which will be applied magically, if defined, and no explicit action id is passed with URL. Same thing happens for “actionIndex” in normal web controller: just define such method and you have default page for controller, remove it and you will have 404 if no action specified.

The other part of the story is about verb filtering. So you are suggesting using action names to imply which verb an action can accept. Again, I think we shouldn't do such magic because it's trivial to declare the verb filtering, which is more explicit and has less constraint. Another point is that the same verb can support multiple actions and the same action can support multiple verbs (e.g. actionUpdate can support GET, PUT and PATCH where GET when responding to HTML format will display a form).

Define base rest controller with a default set of action methods. Child classes should use filter or override the action methods to disable some of them.

I was mainly proposing some alternatives. So you are proposing we rely on Gii to generate the skeleton rest controller code, right? That's certainly fine too. While we never promote CrudController, in practice I believe many people still develop and use it because it avoids a lot of duplicated code, even if they are generated by Gii. Personally, I'm not against either approach.

samdark commented 10 years ago

What we can do is to provide something like a macro rule which expands into smaller ones mapping to each REST method and resource.

Good idea.

(e.g. actionUpdate can support GET, PUT and PATCH where GET when responding to HTML format will display a form).

Example should sayactionComment, not actionUpdate i.e. action should be a resource endpoint.

danschmidt5189 commented 10 years ago

"Native" rest support would be amazing. I think it would be best handled by a new REST application extending \yii\web\Application.

@tjconcept makes a lot of sense to me. Did you pursue this at all?

What is still left problematic from my point of view - is authorisation and sessions handling, that is still require good thinking to make it really customizable.

Why is this the hard part? HTTP Basic over SSL or OAuth2 using a library. Authentication filter comes first and throws 401, authorization comes pre-action and throws 403. REST is stateless so there shouldn't be a session - the client authenticates on every request. API-specific RBAC component handles specific authorizations.

App-wide management of error codes is an issue, however. Maybe Yii could provide direction on that? (App-specific codes, not HTTP codes.)

good, i also think that when it will be in core it should be under yii\api\rest ns and not just yii\api.

@Ragazzo Strongly agree that the namespace should be rest. But what else is in api?

Define base rest controller with a default set of action methods. Child classes should use filter or override the action methods to disable some of them.

What does the RestController do that ordinary controllers don't?

What is the scope of this component? I'd like to contribute but it's not clear to me what the goal of this library is.

Resources/Linking Resources and linking is by far more important and difficult than routing and controllers, so I think that should be our focus.

I strongly suggest we define a resource interface, e.g. rest\ResourceInterface to clarify our direction. With a good interface it should be pretty straightforward for developers to start contributing to this issue.

Minimum support:

The Api example is a start but IMO has too many responsibilities. (Expansion, property mapping, collection handling, error handling, ...) It also doesn't seem to address links.

Routing Put and Patch mean different things, so I would prefer splitting them up. (You can, for example, put a new record to a particular ID, but you can't Patch a new record.) HTML forms don't support either, so I don't see the harm in doing things this way.

GET /resource => list
POST /resource => create
GET /resource/:id => show
PUT /resource/:id => replace
PATCH /resource/:id => update
DELETE /resource/:id => delete
qiangxue commented 10 years ago

@danschmidt5189 Thank you for your insightful comments and suggestions. You mentioned ResourceInterface - who should implement this interface? And how should a resource link be created?

Below is the rough structure of my latest design:

Resource serialization is probably the most complicated one (which might include resource links as you suggested).

Ragazzo commented 10 years ago

Dont think that we should support PUT as we are not "RESTiarians".

samdark commented 10 years ago

@Ragazzo why not?

iJackUA commented 10 years ago

@qiangxue great ! So you have decided to go with "extended" api support out of the box (but not with a very bacis as you have described some time ago) ?

Also I vote for "resource/collections links", I think it is very important to have to make APi and client more separated to make less assumptions on how one or another resource should be called.

Automatic API doc generation

Do you mean just docs accroding to PHPDoc ? Or based on real request params and validation to them ? Btw - request params validation should be also considered. As for example for CRUD actions corresponding Model rules could be used for validation. But if it will be not a CRUD, but Verb action like search or confrimFriendship - it would be good to have ability for similar to Model declarative validation rules specification (and it appears that such validation should be made on Action level, as there could be no Model for Verb action).

@Ragazzo it will be very strange to support everything, but not PUT :)

Ragazzo commented 10 years ago

@samdark its senseless if we use PATCH. POST - create new instance, PATCH - update. Sometimes users make PUT work like POST, but as you see there is no good difference here.

samdark commented 10 years ago

"In a PUT request, the enclosed entity is considered to be a modified version of the resource stored on the origin server, and the client is requesting that the stored version be replaced. With PATCH, however, the enclosed entity contains a set of instructions describing how a resource currently residing on the origin server should be modified to produce a new version."

So these are basically different.

Ragazzo commented 10 years ago

@samdark yes, but its in theory, in practice its like: PUT or POST, PATCH or PUT.

dizews commented 10 years ago

@Ragazzo PUT like POST can make an object but: We use PUT if ID of object generate on a client. We use POST if we expect it from a server.

Ragazzo commented 10 years ago

We use PUT if ID of object generate on a client.

true, so put is like total substituation of object.

We use POST if we expect it from a server.

what does that mean?

Anyway there will be 4 verbs (and three already known: GET, POST, DELETE), not 5 :) I think we should go the rails way here.

klimov-paul commented 10 years ago

I assume the main goal allowing usage of any request method no matter PUT, PATCH or anything else. Nowadays people are talking about OPTIONS request type, which should return the help information about REST resource usage. There should not be any restrictions in this case.

samdark commented 10 years ago

Agree. Restricting it doesn't make sense. We may put together a doc with recommended way though.

Ragazzo commented 10 years ago

Fine by me since rails support also both PUT/PATCH in this way.

We may put together a doc with recommended way though.

yes, this should be done i think, so we will avoid mess in the routes.

dizews commented 10 years ago

We use POST if we expect it from a server.

what does that mean?

if we want make an object with specific id we use PUT if we don't want we use POST which generate id on a server.

I read it from http://restcookbook.com/HTTP%20Methods/put-vs-post/

samdark commented 10 years ago

Yes, both are valid.

Ragazzo commented 10 years ago

@dizews yeah, right. thanks for the docs :)

danschmidt5189 commented 10 years ago

@qiangxue Some feedback:

Automatic API doc generation

Existing libraries like Swagger can handle doc generation, so maybe this is a low high priority? Could just mention them in the guide as a heads-up for developers.

Versioning (this probably will be done with subdirectory support for controllers)

Hm... I will need to think about this. I'm worried this encourages too-rapid API changes and makes it harder to version control. Good post by Phil Sturgeon on the subject.

Resource serialization, including selective property mapping and object embedding/expansion, collection handling.

ModelSerializer is close to what I'd written and seems to handle all our requirements. I'll submit a pr with changes (mainly configuring individual serializers) and improved handling for data providers (for paging/collections).

Authentication / Authorization

Authentication should remain more or less the same. I like RBAC for authorization - is there a reason not to use it? Might need to improve performance, e.g. rbac\CacheManager that can use a Redis backend.

@Ragazzo

Dont think that we should support PUT as we are not "RESTiarians".

I think you're recommending that we not write actionReplace(), not that we ignore PUT? That sounds reasonable to me - replace is hard and specific to the dev's app and database. (E.g., handling a request like PUT /users/99999999999 {...} - how to allocate the pk? Handle future inserts? etc.)

Mapping both PUT and PATCH to actionUpdate() is a good default that many will recognize from Rails.

qiangxue commented 10 years ago

@danschmidt5189 Thank you for your comments. Do not submit PR for ModelSerializer because I'm considering dropping it and introducing an interface that may be implemented by models.

Authentication should remain more or less the same. I like RBAC for authorization - is there a reason not to use it? Might need to improve performance, e.g. rbac\CacheManager that can use a Redis backend.

RBAC will remain. It is for authorization. I was talking about authentication. Currently User component requires starting a session, while for REST api, people prefer it to be stateless without starting session. That's why I'm thinking to develop a separate User component.

dizews commented 10 years ago

it is an article about versioning which consider cons and pros of 3 ways.

Ragazzo commented 10 years ago

@danschmidt5189

(E.g., handling a request like PUT /users/99999999999 {...} - how to allocate the pk? Handle future inserts? etc.) Mapping both PUT and PATCH to actionUpdate() is a good default that many will recognize from Rails.

yeah, thats what i was trying to say, thank you.

danschmidt5189 commented 10 years ago

Two comments on "406 Not Acceptable" exceptions (as I write the NotAcceptableHttpException doc comment for #2106):

It would be pretty cool if throwing a NotAcceptableHttpException automatically added the acceptable content types to the response body. Just a thought.

Here's the spec.

teghnet commented 10 years ago

I was looking at the yii\web\User class and I've come to a connclusion that the loginByCookie() method could be "demoted" to a behavior, so as to make the User component more agnostic about the form of authentication used. Then, some other behaviors could be added to accomodate for the needs of RESTful APIs. For example: loginByCookie() => CookieAuthBehavior, and new OA2AuthBehavior could be created to login users by tokens they provide.

Of course I am open to any other approaches, as long they make the User component more universal.

tmsnvd commented 10 years ago

Is there any news about REST support in Yii2? (although not a argument, but this is only thing stopping develop new project with yii2..).

Ragazzo commented 10 years ago

(although not a argument, but this is only thing stopping develop new project with yii2..

you can take already developed extensions for Yii1 and use it, or rewrite them for Yii2. See in this article how you can use Yii1 components and extensions in Yii2.

danschmidt5189 commented 10 years ago

@qiangxue Any updates?

Model implementation via interface

I have a few ideas regarding an interface-based implementation. The basic concept is to define a ResourceInterface, the core of which defines:

/**
 * Returns the string representation of the object in a given MIME type
 * 
 * @param string $mimeType A MIME type, e.g. "application/json"
 * @param array $options Conversion options, e.g. "embed xyz resources"
 * 
 * @return string String representation of the object
 */
public function convertTo($mimeType, $options = array());

/**
 * Returns whether the object can be converted to the given MIME type
 * 
 * @param string $mimeType A MIME type, e.g. "application/json"
 * @param array $options Conversion options, e.g. "embed xyz resources"
 * 
 * @return bool Whether the object is convertible to the given type
 */
public function isConvertibleTo($mimeType, $options = array());

(It adds other features too, e.g. concepts of "freshness" and id.)

Controllers = Normal Flow + ResourceResponseFormatter

A ResourceResponseFormatter would handle content-type negotiation as well as setting headers and content based on the resource passed into Response::$data.

public function actionCreate(array $Customer)
{
    // Load the resource
    $customer = new Customer();

    // Something like this is possible because the resource knows what it can
    // be converted to. Probably better in a filter.
    if (!$this->isAcceptable($customer)) {
        throw new NotAcceptableHttpException();
    }

    // Do your thing
    $customer->setAttributes($Customer);
    $customer->save();

    // Response inspects resource state and the request to determine how to respond, eg:
    // 1. JSON representation with correct status code (201/422/500) if request wants JSON
    // 2. Redirect to "new" if there are errors
    // 3. Redirect to "view/id" if there are no errors
    Yii::$app->response->format = 'resource';
    Yii::$app->response->data   = $customer;
    return Yii::$app->response;
}

This is all strongly inspired by Responders / respond_with in Ruby on Rails 4, btw.

qiangxue commented 10 years ago

@loloroboros Thank you for your suggestions. I'm still working on this (about 60-70% done). It has taken quite a long time because I spent a lot of time evaluating existing solutions and many articles.

I thought about introducing ResourceInterface like you suggested but gave it up for various reasons. Currently, my design is using arrays as the common format, converting models to arrays, and converting arrays further to various formats (e.g. json, xml). Custom fields and embeds are both supported. HATEOAS is also supported.

Content type negotiation, result serialization, user authentication, allow check, etc. happen in action filters or before/afterAction() calls. As a result, you may find the following code very familiar:

public function actionCreate()
{
    $model = new Post;

    $model->load(Yii::$app->getRequest()->getBodyParams(), '');
    $model->save();

    return $model;
}
danschmidt5189 commented 10 years ago

Wow, that sounds fantastic. I'm really curious to see and comment on your implementation.

Do you plan to release a preview soon?

Also, is there the same/similar magic to Rails with partials/views and redirects when the request wants text/html? That's a great time-saving feature.

qiangxue commented 10 years ago

This is my top priority for now. Will certainly release it for preview before merging and when ready.

Ragazzo commented 10 years ago

@qiangxue well that is good, but please dont forget about filtering, pagination, custom fields in response (/api/v1/users?fields=name,email&limit=100&offset=10&filter[name]=%some_name%) and other things) Currently i dont like the approach where api is going because it will be more like some controller-helpers-useful-methods and not a separated API solution, but will see ;)

fragoulis commented 10 years ago

@qiangxue can you point us to these other solutions you are referring to? for academic reasons.

@Ragazzo pagination and custom fields is one thing but filtering .... nah ... filtering can be rather implementation specific. Anyway, my two cents here. Driven by the need for a more versatile filtering scheme, in one of my own implementations I supported multiple types of filters such as:

so your filter from the example above would become something like /api/v1/users?fields=name,email&limit=100&offset=10&filter[name:contains]=some_name

samdark commented 10 years ago

@jfragoulis the list will be huge. Check this one for the start: http://pages.apigee.com/web-api-design-ebook.html

Ragazzo commented 10 years ago

@jfragoulis does not matter, if web api does not provide filtering than it is bad api.

qiangxue commented 10 years ago

An initial implementation is done:

Your comments are welcome.

qiangxue commented 10 years ago

Remaining work:

acorncom commented 10 years ago

@qiangxue That's really elegant code. I look forward to adding APIs in easily in the future, I've been experimenting with Ember.js and this looks like it'd fit together quite nicely. Thanks for all the hard work fit in around your work and home life.

Ragazzo commented 10 years ago

so, we once again put everything in one model with huge number of scenarios (e.g. api-create)? not a good thing as for me)) will review other later.

nineinchnick commented 10 years ago

I don't like the idea of separating the html interface of an app from other formats. If I'd like to add an API to an existing application it would require to redefine all controllers and actions to return data passed to render instead of render result.

I think that API should be integral part of an application and the html view is just a degraded version to be used by a dumb client (human/browser) that can't use other http methods than GET/POST. If the html view contains navigation items like a menu and data items like a grid with item numbers and currently active filters then the same information should be available in other response formats like JSON or XML. The controller actions would remain the same, that is it would call render and return the result.

Format would be selected when rendering the output, not when sending it back, which the Response class is responsible for.

I recently did such implementation in Yii 1 and it looks really simple. I got a full html CRUD with an option to fetch JSON data utilizing all my filtering, pagination etc. It's also hyperlinked.

I only had to:

One more example, the API response for index action should return:

To sum this up, what I ask is is it possible to move format related login from Response to Controller::render()?

qiangxue commented 10 years ago

so, we once again put everything in one model with huge number of scenarios (e.g. api-create)? not a good thing as for me)) will review other later.

@Ragazzo: you are encouraged to subclass your base model classes. That is, you keep a set of model classes under common, and then subclass them in each tier. Creating specific resource classes is just another way of splitting the code, which you need to duplicate some logic available through AR.

I don't like the idea of separating the html interface of an app from other formats. If I'd like to add an API to an existing application it would require to redefine all controllers and actions to return data passed to render instead of render result.

@nineinchnick Thank you for your feedback. The idea of serving HTML together with JSON/XML using a common set of controllers sounds fascinating. However, in practice this will end up with very messy code in the end for the following reasons:

nineinchnick commented 10 years ago

Those are valid points but I don't think it's a response to my question. I don't say we have to make all apps have a web frontend bundled with an API. I just want it to be possible, probably for some more generic CRUD backends, to have a HTML format available in the API so a human could use it as a normal application instead of building a second one. I could provide my own formatter for HTML output to achieve this but that's not my point.

It seems like going away from the views concept and I don't see why. Even if preparing a specific format like JSON involves just a call to some serializer method it doesn't mean it can't be called rendering. I mean I'd like to keep the current flow of executing an action that prepares data, passes it to a renderer to format it and then returns it.

In Yii 1 an action sent the response contents, either through calling render or directly (echo). Now an action must return response contents. In the current API solution this flow is distrupted, the action returns raw data. A new concept of formatting is introduced that could be a bit confusing.

I ask my question again, why not move formatting from the Response class to a renderer and use it in API?

qiangxue commented 10 years ago

@nineinchnick You may treat yii\rest\Serializer as the view if you want. The point is that HTML views are mainly used to format complex outputs (call it free-form formatting if you like), while API data formatting is very regular and thus a single serializer is enough without the need to use different views to format different data. If you still insist in using a view, the current implementation doesn't prevent you from doing so.

subdee commented 10 years ago

Are you considering using OAuth2 as an alternative authentication mechanism or do we have to write/use 3rd party extensions for this?

qiangxue commented 10 years ago

Just improved the authentication by supporting different auth types, including HTTP Bearer which is supposed to be used by OAuth2. @subdee The implementation of oauth authorization server is beyond the scope of this rest api feature. We may or may not address it in a separate feature.

fernandezekiel commented 10 years ago

i think the simplest way to implement those hypermedia links is to have some kind of map of models to resources that can be used by the behavior attached to the model... also this could provide a mapping of resource attributes to model attributes i implement it like this:

$model->attachBehavior('hypermedia', new Hypermedia(function($model) use (&$controller){
    return $controller->createAbsoluteUrl($controller->resourceName. '/view', array('id'=>$model->id}));
});
danschmidt5189 commented 10 years ago

@qiangxue Initial thoughts (could be off base - I've spent 12+ hours working ;) -

And... that's enough for now. I'd like to hear what others think and take a deeper look myself.

Edit - Just saw Linkable/Link: a8c7d36c02e45fe490b16bf2f49de11262589c70. I like this - why not implement defaults for AR? Magically doing it would require checking for available API controllers.