0.4 Queries

[danielsiders]

Tent 0.4 will include a dramatic restructuring of the query endpoints. Many features will be removed from the old query endpoint and a new endpoint with the ability to search the content of posts will be introduced.

Background

We always imagined a user's Tent server would also act as their personal search engine. During the initial development and testing period for Tent we lacked the resources to develop a server with full text search capabilities. As we move towards 1.0 that is beginning to change.

The 0.3 query endpoint has become significantly overloaded with URL query parameters over the past months. The addition of a new, richer endpoint provides us with an opportunity to slim down and speed up the old endpoint while adding more features to the new one.

Basics

The old endpoint will only sort by a single timestamp, version received at, as opposed to the current four. This means that new versions of posts will reappear in a feed as they are received. It will also lose the ability to use mentions and refs as neither exists in 0.4. This endpoint will likely be significantly faster than the complex query endpoint on most servers (since it can be generated in advance and cached).

The new endpoint will include complete full text search capabilities for the bodies of posts. Queries can also leverage the semantic tags included in 0.4 machine-readable post type schemas.

Update: Queries can also be submitted as posts. This should be used for complex and frequently used queries and gives the server the opportunity to optimize response time for frequent or complex queries (e.g. generating on insertion of new posts instead of on request).

Subscriptions

Subscriptions in 0.4 can be any complex query. update: The publishing server is not required to completely filter posts prior to delivery (delivered posts may be a superset of the query results).

[seanmonstar]

So, this would allow me to query for subscriptions with entites or profile names that start with a specific string?

My current work on an autocomplete feature requires downloading all subscriptions, and comparing locally. It's fine as long as a user a has small number of subscriptions, but will be less fine dealing with 1000s.

[titanous]

@seanmonstar Maybe. We haven't decided the specifics of how substring indexing will work.

[seanmonstar]

With subscriptions, that means I could subscribe to, say, only status posts by Daniel with content.text containing "#tentdev"? On Dec 23, 2013 11:49 AM, "Daniel Siders" notifications@github.com wrote:

Tent 0.4 will include a dramatic restructuring of the query endpoints. Many features will be removed from the old query endpoint and a new endpoint with the ability to search the content of posts will be introduced. Background

We always imagined a user's Tent server would also act as their personal search engine. During the initial development and testing period for Tent we lacked the resources to develop a server with full text search capabilities. As we move towards 1.0 that is beginning to change.

The 0.3 query endpoint has become significantly overloaded with URL query parameters over the past months. The addition of a new, richer endpoint provides us with an opportunity to slim down and speed up the old endpoint while adding more features to the new one. Basics

The old endpoint will only sort by a single timestamp, version received at, as opposed to the current four. This means that new versions of posts will reappear in a feed as they are received. It will also lose the ability to use mentions and refs as neither existshttps://github.com/tent/tent.io/issues/213in 0.4. This endpoint will likely be significantly faster than the complex query endpoint on most servers (since it can be generated in advance and cached).

The new endpoint will include complete full text search capabilities for the bodies of posts. Queries can also leverage the semantic tags included in 0.4 machine-readable post type schemashttps://github.com/tent/tent.io/issues/212 . Subscriptions

Subscriptions in 0.4 can be any complex query (more on this later).

— Reply to this email directly or view it on GitHubhttps://github.com/tent/tent.io/issues/215 .

[titanous]

@seanmonstar Yeah, that's the plan. It's possible that the filtering will end up happening on your server not Daniel's.

[seanmonstar]

Would there be any benefit? Filtering on Daniel 's server would reduce bandwidth, which he may be interested in if he's on a metered server. On Dec 28, 2013 9:25 AM, "Jonathan Rudenberg" notifications@github.com wrote:

@seanmonstar https://github.com/seanmonstar Yeah, that's the plan. It's possible that the filtering will end up happening on your server not Daniel's.

— Reply to this email directly or view it on GitHubhttps://github.com/tent/tent.io/issues/215#issuecomment-31300661 .

[titanous]

@seanmonstar Yeah, there are absolutely tradeoffs, which is why we are leaving it as a server implementation detail.

[danielsiders]

@seanmonstar different users/servers will have different needs, etc. We're trying to let each entity/provider determine what constitutes a DDoS for their implementation/infrastructure.

[seanmonstar]

To do that, my server would have to filter incoming posts, because there's no promise that the other server did so. But if it did, my server's filtering would just validate it and store it. Is that it?

[danielsiders]

@seanmonstar receiving servers are always expected to double check subscriptions