Add queued status, restore status array filter

[lidel]

This adds queued status for indicating scheduled pinning operations that did not start yet.

It is based on feedback from IPFS Cluster (cc @lanzafame) which suggested we should make it very clear when pinning is actively happening, and when it is pending due to service load etc. People coming to distributed systems should have better insight into what is happening, and where the delay comes from.

Because this introduces second "pinning-in-progress" status, we restore status parameter to accept multiple states.

cc @obo20 for visibility, as this partially reverts #28 and brings filtering over multiple states back

[obo20]

We have the ability to query on failed, pinning and queued all at once, but our system stores our pinned items in a completely separate table than active / failed pinning operations (this would include the failed, pinning and queued statuses).

As a result we wouldn't be able to accommodate either of these search situations:

• Return "all" statuses
• Return pinned items with any other status.

[lidel]

Are you sure this is impossible? DBs such as PostgreSQL have concept of VIEWS which can span across tables.

In case you are not able to work around this technical debt, you could return HTTP 401 Bad Request with message informing that one can't query for both pinned and pending/failed items.

[obo20]

@lidel this may work for us. I'll test things out and post a response here as soon as I learn more.

[obo20]

@lidel from further research, a VIEW doesn't seem to be a very good option here as postgres is unable to pass down dynamic where clauses (such as user id, etc) into the view query.

What this means is that we would effectively be creating a temporary table that includes every single pin in our DB, as well as all of our async pinning operations that are in progress. Only after that view is created would postgres apply the filters to the results.

There are materialized views, but these have the downside of requiring manual syncing which isn't feasible for what we're looking to do.

I'll keep exploring options for us, but this is looking like a fairly complex solution to pull off, especially when taking into account additional filtering like metadata, providers, and arrays of CIDs.

[lidel]

@obo20 thank you for looking into this on your end.

I believe in most cases users of this API will either care about pinned OR "in progress" states (queued || pinning || failed), which (iiuc) you should be able to ok. Returning "pinned + something else", while more expensive in your case due to legacy decisions, should be technically doable – returning error in the meantime should not impact users much.

Sadly we don't have any data to guide this decision other than basic devexp intuitions. This is a choice between making the most basic operation harder for all future users VS for some of preexisting pinning services.

I believe prioritizing the user experience in the long run is the right call here. Hope you understand.