json_loadfd() for non-blocking file descriptors

[Zer0-One]

Currently, when attempting to call json_loadfd() for a non-blocking UNIX-domain socket with no input, I get the following, rather strange error:

']' expected near '-0' at Line: 1, Column: 7

with an error code of:

json_error_invalid_syntax

Should there be an error code specifically for the case when reading from a non-blocking socket would block?

[coreyfarrell]

json_loadfd does not support resuming so using it on non-blocking sockets has a high risk of data loss. I haven't looked into what it would take to support this but I suspect it would be difficult. It would require adding support for resuming loads, otherwise a special error code would have no purpose. This would require a large number of changes throughout src/load.c.

I'm inclined to say that we don't need a special code for non-blocking sockets as json_loadfd has no support for non-blocking sockets. Please don't close this ticket as I think this is a neat feature request for supporting resume of load operations. I'll try taking a more serious look at some point but it'll be at least a few months before I have the kind of time required to look into this.

[akheron]

At one point I had an idea that the parser could be changed to be incremental/callback based. I looked into it and came to conclusion that more or less the whole parser would need to be rewritten and I didn't (and don't) have time to do that.

The best we can do with little effort is to document the fact that non-blocking reads are not supported.

[lukelane124]

Up front: I have never looked at the parser code. Would it be possible to simply buffer the whole file and the loadb? Or is the loadfd an optimization for memory constrained systems?

[milasudril]

At least, I have some ideas about how an API that would support for non-blocking I/O.

• Add a user-exposed, but opaque type json_parser_ctxt_t
• Add a user-exposed type json_parse_result_t, like so

typedef struct json_parse_result {
  /** NULL until the parser as completed parsing of one object
   */
  json_t* object;  

  /** The offset in the latest buffer where the parser encountered the top level `}` or `]`
   */
  size_t buffer_offset;  
} json_parse_result_t;

/**
 * Creates a parser context that can be fed with data blocks
 *
json_parser_context* json_create_parser_context();

/**
 * Destroys context. Any json obejct returned by parse in the json_parse_result_t is still valid after a call to this function.
 */
void json_destroy_parser_context(json_parser_context* context)

/**
 * Parses the content of buffer, given the state in parser_context
 */
json_parse_result_t parse(json_parser_ctxt_t* parser_context, void const* buffer, size_t buffer_size);

I do not think that should be very hard to implement, since there must already be some kind of context managment for the parser. It is simply a more low-level API towards the library, so I guess some code will be moved to the new function parse, and all other parse functions will setup the context and pump the parser until EOF or the last ] or }, depending on the JSON_DISABLE_EOF_CHECK flag.