Noop specification deserves clarification(s)

[DbxDev]

Noop is defined as a valueless value that can be added to the elements of a container. That is not always true, in particular when the container is strongly typed and the type marker is omitted. In this case, there is no way to distinguish [N] from the integer 78. For this particular reason, using noop as a way to remove data should be used with care, and is in practice not efficient because you first need to find how the container is defined (with optimized format or not).

[rkalla]

Definitely a miss on my part, that's a great point.

On Thu, Oct 6, 2016 at 3:47 AM, Dorian notifications@github.com wrote:

Noop is defined as a valueless value that can be added to the elements of a container. That is not always true, in particular when the container is strongly typed and the type marker is omitted. In this case, there is no way to distinguish [N] from the integer 78. For this particular reason, using noop as a way to remove data should be used with care, and is in practice not efficient because you first need to find how the container is defined (with optimized format or not).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ubjson/universal-binary-json/issues/80, or mute the thread https://github.com/notifications/unsubscribe-auth/AARteHQ61EOAt9KluXhOdjzvPf-hLdVmks5qxNHBgaJpZM4KPzT2 .

[tsieprawski]

@DbxDev, what to you mean by when the container is strongly typed and the type marker is omitted. In this case, there is no way to distinguish [N] from the integer 78? According to [1] example, if type marker is ommited, then container is not optimized by type at all. And if type marker's argument is absent, then probably parser will fail. And if both type marker and argument N are present, then every value in container is noop already.

Do I think correctly?

[1] https://github.com/ubjson/universal-binary-json/blob/master/spec12/container-types.html#L146

[rkalla]

You are correct - the only way to omit the type marker for the container is if all the entries are typed... if you want to omit the type marker on the entries, then the container (value) must be typed.

On Tue, Nov 8, 2016 at 2:51 PM, tsieprawski notifications@github.com wrote:

@DbxDev https://github.com/DbxDev, what to you mean by when the container is strongly typed and the type marker is omitted. In this case, there is no way to distinguish [N] from the integer 78? According to [1] example, if type marker is ommited, then container is not optimized by type at all. And if type marker's argument is absent, then probably parser will fail. And if both type marker and argument N are present, then every value in container is noop already.

Do I think correctly?

[1] https://github.com/ubjson/universal-binary-json/blob/ master/spec12/container-types.html#L146

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ubjson/universal-binary-json/issues/80#issuecomment-259271021, or mute the thread https://github.com/notifications/unsubscribe-auth/AARteLBpmi_gG8kIMHjqyPu4LNBsWh9Rks5q8O75gaJpZM4KPzT2 .

[DbxDev]

You're right, the issue I raised actually cannot happen. This implies that noop usage is restricted to non-typed containers. A strongly typed array of NO-OP is valid but doesn't make sense does it ? Since the topic of this thread is noop clarification, here is an other suggestion. I think the website misses an example of the interaction between objects and noop. Noop is described as a valueless value to be added to the elements of a container. The only thing that makes sense is to treat noop as a key-value pair, but one can imagine cases like key : noop, so an example would certainly help. edit: Also a quite confusing case: a strongly type object of noop. If I follow the optimized format specifications it has to be a sequence of keys, but the result is an empty object. Not only it's quite disturbing to represent an empty element with so many data (all the keys), but also it's quite inconsistent with the example I used before where noop is an element (i.e a key value pair).

Am I missing something here ?

[tsieprawski]

@DbxDev Yeah, noop-typed containers are weird. Semantically array is empty (just the length of array-on-the-fly and array-after-ignoring-noops are different), but with objects there go keys+noops (which make no sense).

Maybe noops should not be allowed at all in containers? Or at least in non-optimized containers. Because, noop-typed make no sense (as above), and count-only optimized end up having less items than before parsing. For non-optimized ones, noops still serve as keepalive, and they can stay here.

[DbxDev]

Maybe noops should not be allowed at all in containers?

If so then when can you use them? Noop in general is not something I find well designed, because it is mixing a definition/action that belongs to a protocol (keep alive) inside a data format. I'm pretty sure that was already discussed in an other thread. That being said, the edge cases mentioned here deserve at least some examples, maybe a "caveats" section, because any parser has to carefully handle them.

[MikeFair]

My understanding of noop is for streaming channels and byte alignment issues.

When you have a keep alive requirement in your value stream, you have to send "something" or the connection gets closed on you; send a Noop. Also if you would like to pad the end of your data stream to align on a better byte boundary for retrieving the next data element, a Noop can be used to fill the space.

Nevermind about this, I found it. [ Also, as a side note related to Noop, there's a proposal floating around to change the Noop value to the "space" character (as it doesn't print anything to the screen and therefore looks like a Noop). The issue number for it seems missing; however and I think it's a brilliant suggestion and deserves attention. It's #75 ]

[ColinH]

IMHO no-op should be completely removed because, again IMHO, it is a violation of the separation-of-concerns principle.

Further, in the presence of no-op, applying the obvious optimisation of pre-allocating containers for arrays and objects when the size, but not the type, is given, can lead to unnecessarily large allocations since, at the extreme, the container might still be empty at the end.

[Frizlab]

See #88 and #89, No-op is now simply an invalid type for a container.