MIME type - Githubissues

kyrias commented 9 years ago

It would be nice if there was a registered MIME type for msgpack, for use in eg. APIs.

methane commented 9 years ago

I agree. "application/x-msgpack" should be documented somewhere.

kyrias commented 9 years ago

Well, x- is explicitly unregistered, and it’s also deprecated now since they made registering a proper media type simpler, so it should be rather easy to get application/msgpack registered.

kuenishi commented 9 years ago

Fmmm. Do you know how easy is it to register? Looks like there's an application page and vendor/personal formats are rather easier to register. Standard formats requires being standardised, that's what library maintainers are not interested in.

edsiper commented 8 years ago

@frsyuki ^

arteymix commented 7 years ago

Hi, just doing a follow-up here as I need to use some media type for a data service.

Is it reasonable to assume that vnd.msgpack will eventually be registered and use it as a value?

Qix- commented 7 years ago

@kyrias Which RFC deprecates x-? To my knowledge it's still a thing (RFC 2045/2046, which have not been obsoleted yet).

Looks like there's an application page and vendor/personal formats are rather easier to register.

@kuenishi correct. You'll want to register it as application/vnd.messagepack or application/vnd.msgpack. MsgPack is considered a vendor media type as per RFC 6838.

Admittedly I've been reading way too much of the MIME type RFCs lately for work. While I haven't submitted a media type before, I hear the IANA is pretty good about it.

kyrias commented 7 years ago

@Qix-: RFC 6649

Qix- commented 7 years ago

@kyrias oh interesting, I had no idea. Thanks for the link!

cemremengu commented 7 years ago

Just wondering what's the current status of this issue ?

kyrias commented 7 years ago

The status essentially isn't.

tagomoris commented 7 years ago

@frsyuki ping? application/vnd.msgpack looks good to me.

frsyuki commented 7 years ago

"x-" was deprecated by RFC 6648 which was published June 2012, but msgpack exists since February 2009. From Github code search, application/x-msgpack is commonly used.

Therefore, registering application/x-msgpack officially is more beneficial than registering application/vnd.msgpack for the users, if possible.

RFC 6638 admits that there were wide use of "x-" names and Appendix A mentions that unfaceted "x-" names can also be registered. Although it also mentions some restrictions, and IANA or IESG or IETF or something unknown might advice me to take other ways, but I think it's worth to try.

frsyuki commented 7 years ago

I started crafting the application texts for https://www.iana.org/form/media-types based on RFC 6838. There're many examples I can refer to fill out the forms at Media Types list. Especially, applicatoin/json and vnd.apache.thrift.binary are similar to msgpack. application/x-www-form-urlencoded is an example of x- name, and application/sql is an example of format that has various implementatins clearly separated from the format specification.

Some fields are just obvious:

Type Name: "application"
Subtype Name: "Standards Tree (no prefix)", "x-msgpack"
Encoding Considerations: "binary"
Deprecated alias names for this type: N/A
Magic number: N/A
Macintosh File Type Code: N/A
Object Identifier: N/A

frsyuki commented 7 years ago

Forms that are easy to fill out:

Required Parameters
Optional Parameters

They are charset= of text/html; charset=utf-8, for example. Nothing available for MessagePack. They're N/A.

Fragment Identifier Considerations

This is +git of ssh+git, for example. N/A.

Restrictions on Usage

RFC 6848 section 4.11 says that this needs to describe something if the MIME type is not intended to be used widely. Maybe the RFC intends MIME types that wants to secure a name but doesn't want people to use it. N/A for MessagePack.

Published specification

Many existent MIME Types just mentions a website here. MessagePack can use http://msgpack.org. (O'Reilly's http://shop.oreilly.com/product/0636920032175.do is a good book that mentions about MessagePack, by the way)

Intended usage

Common.

Here could describe some details but that's mainly intended for other usage types (Limited use or Obsolete). So, nothing here for MessagePack.

frsyuki commented 7 years ago

The other forms are more complex.

Security Considerations

JSON says:

it is possible to use that language's "eval()" function to parse JSON texts. This generally constitutes an unacceptable security risk

But this doesn't apply to MessagePack.

Thrift says:

Apache Thrift does not enforce a security mechanism. Users define security features at application or transport level. The data carried by Apache Thrift is essentially arbitrary, and can include active or executable content. Apache Thrift does not provide or enforce a security mechanism, so appropriate mechanisms must be implemented at the application or transport level.

I think almost same text with Thrift works for MessagePack. But a difference is that Thrift includes RPC but MessagePack doesn't. Saying "transport level" is not always accurate. So, MessagePack would say...

MessagePack does not enforce a security mechanism. Users define security features at application, underlying transport level or data store. The objects serialized by MessagePack are essentially arbitrary objects that can contain executable or secret content. Appropriate mechanisms must be implemented at the user application, underlying transport level or data store. Deserializing too deeply nested objects, too long strings, or too long arrays can consume large amount of memory rapidly depending on the deserialization implementation. When the application deserializes objects serialized by untrusted sources, the application must reject them appropriately to be tolerant of DoS attack.

frsyuki commented 7 years ago

Application Usage

An existent mime type just says "Various" but it should be more specific to aim at standard tree, I think.

JSON says a good example:

JSON has been used to exchange data between applications written in all of these programming languages: ActionScript, C, C#, Clojure, ColdFusion, Common Lisp, E, Erlang, Go, Java, JavaScript, Lua, Objective CAML, Perl, PHP, Python, Rebol, Ruby, Scala, and Scheme.

MessagePack can take the same approach. So...

MessagePack has been used to exchange data between applications written in various programming languages and environments including: ActionScript, Arduino, C, C#, C++, Clojure, Crystal, D, Dart, Delphi, Elixir, Erlang, F#, Go, HHVM, Haskell, Haxe, J, Java, JavaScript, Julia, Lua, Node.js, Objective CAML, Objective-C, PHP, Pascal, Python, Qt, Ruby, Ruby on Rails, mruby, Rust, Standard ML, Scala, Scala.js, Scheme, Smalltalk, and Swift.

mnot commented 6 years ago

@frsyuki if you want to register application/x-msgpack, that counts as a registration in the standards tree, so you'll need to author an Internet-Draft and get it published as an RFC; you can't just submit the form. That can be fairly easy, and I'm happy to help if you need it.

Note that it'll need to have a convincing case for a "grandfathered'" media type if you want to keep the x-, as per Appendix A of RFC6838. I suspect that it may be felt that application/x-msgpack isn't widely used enough to justify grandfathering, but we can see what happens. Getting application/msgpack registered would be easier (although still require an RFC).

While you're at it, it might be worth considering whether people would want a +msgpack suffix, similar to +xml, so you could have things like application/vnd.me.myformat+msgpack. That's becoming more popular for meta-formats like this.

If you don't want to bother with an RFC, you can get application/vnd.msgpack without going through that process; the form will be fine for it.

If we can get a sense from the msgpack community as to what it wants, I can talk to folks at IETF next week and figure out the best path forward.

Qix- commented 6 years ago

Huge :+1: to application/msgpack if possible. Also :+1: to +msgpack as MsgPack can be used as an alternative representation of the same data (like you said, similar to +json and +xml). I think it'd be worth it if the process is minimal since it would only help to encourage adoption.

I think the security considerations clause could be reduced a bit. Something like the following should suffice IMO.

MessagePack is a generic data format specification that doesn't specify, or otherwise concern itself with, any security mechanisms, requirements or guarantees. Likewise, the MessagePack format freely allows secret or executable data, thus requiring any and all security measures be implemented at the underlying application or transport levels.

It's probably a safe assumption that any consumers of MessagePack libraries are aware of the usual considerations that come with software engineering; namely that consuming third-party libraries has the perpensity to cause overhead and that consumers should be aware of what that overhead looks like.

This includes the potential for high memory/CPU usage, etc. I don't see a need to include that, personally - I don't think it adds anything to a section about security considerations specific to MessagePack itself.

mnot commented 6 years ago

Had a quick chat with an Area Director about this; it should be pretty straightforward.

The decisions that this community needs to make are:

Which media type(s) do you want to register?
Do you want a suffix?
Do you want to have a draft just for registering the media type, or do you want to make MSGPACK's specification an RFC?

Happy to answer any process questions, and help with draft authoring / review.

frsyuki commented 6 years ago

@mnot Thank you for the comments & actions.

Certainly application/msgpack is the media type we would like to use: Reasons why we currently use application/x-msgpack is that application/msgpack is the straightforward MIME type MessagePack should use and x- was the common way to use such "unofficial" MIME types. I'm very sure that having application/msgpack officially registered is the ideal goal the community shares for this media type issue.
Having vnd + suffix is a compromise: I don't see technical benefits from having a suffix media type excepting a fact that it feels slightly more official compared to "application/x-msgpack". It seems that using application/x-msgpack is somewhat common way and I don't want to make confusion to those existent users. I thank that I should not decide to use it until we are certain that application/msgpack or application/x-msgpack is unrealistic.
If the standarization process of RFC makes the registered version of MessagePack incompatible to existent MessagePack spec, registering the media type only is the way to go. If we can make current specification of MessagePack an RFC without changes of the data format, I would like to make MessagePack an RFC.

There was a long story about RFC/IETF starting from this comment, 5 years ago. There was an unfortunate that the topic was mixed with a discussion whether MessagePack should have Binary type or not (now we have). But our conclusion was not to do it because making MessagePack RFC os too hard process we/I could take at the time, and even worse, it may break compatibility because IETF tends to start building specification from scratch, from our understanding. We can't accept incompatible changes because there are big amount data already serialized & stored in MessagePack format. It ended up having RFC 7049 Concise Binary Object Representation led by another person. It's the current status around standarization of us, although I really don't think it's a good situation.

I suspect that it may be felt that application/x-msgpack isn't widely used enough to justify grandfathering, but we can see what happens. Getting application/msgpack registered would be easier (although still require an RFC).

I understand about it and I didn't know have an idea that application/msgpack could be easier. What is the best way for us to continue this discussion?

chintan-mishra commented 6 years ago

Which MIME type should be used currently? I am using MsgPack for de-/serialization between Mobile app and Elixir servers. I cannot decide on which MIME type to select for HTTP request.

mnot commented 6 years ago

@frsyuki sorry for the delay.

It should be possible to make the current specification of MsgPack an RFC without changes; it would be published as an Informational RFC.

I mentioned a +msgpack suffix because it would allow other people to register media types like application/myformat+msgpack, to indicate that my format uses msgpack (just like application/myformat+xml does for XML). This doesn't rule out registering application/msgpack as well (see XML for an example here too).

Next step should be to generate an Internet-Draft that asks for the registration. I'm happy to help there; if you create a new repo (maybe media-type-registration?) and give me push access to it, I can sketch out the shape of what that would look like, if you want.

rustonaut commented 5 years ago

One argument against application/x-msgpack is that there is probably also application/x.msgpack out in the wild (I mean that is given RFC6838 the right think to use until msgpack is registered) , and just because application/msgpack is standarized it doesn't mean that your server stops accepting x-msgpack.

ziggythehamster commented 5 years ago

I'd like to give my +1 to application/msgpack and +msgpack, ignoring the current wild west situation of x-msgpack and vnd.msgpack. It's probably not too late to move to the correct MIME-type.