githubmanticore
commented
1 year ago ➤ Stan commented:
if persist is not available, may be let execute one(!) next command?
seems good to execute next command without error? maybe just warn about that
Update (plain old, by doc id selector) supports numbers update, now even strings update. But no support for 64-bits. And feature looks really easy to add, why not?
because all API needs that 64bit update functions and 64bit numbers is not standard there
There are several small issues or errors, see no need to make separate issue on each of them, let them be in one place.
[x] document not yet documented commands of API
[x] may be upgrade update command to support 64 bits?
[ ] for keywords make available sort-by docs/hits
[ ] if persist is not available, may be let execute one(!) next command?
[ ] for console status body is send as '1' and is marked as 'dummy body'. That is not true! That is global status request!
[ ] m.b. add predicted_time and dist_predicted time to output of Meta (globally) instead of just using it in 'status' API call.
[ ] for ISphOutputBuffer - consider to refactor SendLSBDword to platform format
[ ] check whether we collect and return status for ALL commands.
[x] SendAPIPercolateReply has something nasty in if (bQuery) branch.
[ ] Teach CallPQ to get list of indexes instead the only index.
[ ] Teach CallPQ to return warning usual way, not at the end!
[ ] packed factors are NOT byte-order agnostic.
[ ] m.b. refactor 'search' command; upgrade major version.
A bit of explanations
It is revealed that some API commands are available, but not even mentioned in API doc.
SetFilterStringList,SetQueryFlag,SetOuterSelect,SetTokenFilterUpdate (plain old, by doc id selector) supports numbers update, now even strings update. But no support for 64-bits. And feature looks really easy to add, why not?
Keywords command internally support sort by docs and sort by hits (see GetKeywordsSettings_t), but they're note exposed to API.
Logic of API implies running only one command per connection. If we issue 'persistent' command, and no persistence is possible (max_children exceeded), connection is dropped after the try. Let's for 'persistent' command especially give possibility to exec one actual command in the same connection.
For
searchd --statusin code body commented as 'dummy body', however actually it is for a long time is not dummy.Both API 'status' and sphinxql 'show meta' use one and same command BuildMeta. But API, in addition, returns 'predicted_time' and 'dist_predicted_time'. Let's just move both to commont BuildMeta: code will be simpler; sphinxql show meta will be enriched.
SendLSBDword is written with byte-by-byte processing. Since we're on Intel, let's make just direct write of values for LSB, as it is done in BSON manipulations.
Show status displays command statistics. Just check that ALL commands are mentioned there (AFAIK there are no tests whether some of them dismissed or not)
It is kind of copy-paste error; 3 times checked one and same condition.
Search/Update may take list of indexes. CallPQ - only if they're encapsulated in distributed index. Let's teach the command to process list of indexes also.
Usual way for our api is to return warnings with SEARCHD_WARNING status code, but CallPQ invokes totally another way, and it is not looking reasonable at all.
Everything is returned in predefined byte order (big-endian in general, little-endian for parts of bson and mysql). But packed factors is the only blob which is returned over network 'as is' without any sights of how it is built. The only way to guess - m.b. look into very first handshake of connection; no other hints available. So, it is quite possible to connect from ARM host to Intel agent and take a nonsense in 'packed factor' field because of no convention.
Search command itself look quite monstrous for now.
outer_orderby- which most likely first was added to master-agent extension, but later considered to be useful for clients also, and now exists in both places simultaneously. Same true forouter_limit, but it is even more confusion, since it always present in the message, and simultaneously MAY present in master-agent extension, depended from the version AND paramhas_outerin common section.So, for search it would be good to:
select * from * where match('test')the only thing is matching 'test'. And in this case even sphinxql would win, since it will not pass all non-necessary stuff. I considered different protocols and would give try to google's protobuf. Apart the fact that it is very compressed and versioned, it is also supported out of the box on may languages and platforms. At least for 'search' query it looks not as a big deal to play with .proto file and experiment with generated parser.