Feature Request: Enable indexers to be able to keep one allocation open at all times to serve queries whilst manually closing

[Pete-LunaNova]

The default setup is to have 2 parallel allocations so that when queries are closed automatically the system will keep one allocation open at all times so queries can still be served. However, if an Indexer wishes to manually close their allocations they are not able to do so without disrupting their ability to serve queries.

There are two broad ways to tackle this problem and it may be sensible to implement both:

1) Provide fine-grained control of individual allocations.

Each allocation for a subgraph could be assigned an identifier, eg. A,B,C etc. Running the indexer status command could return the identifier for each allocation that is open, the amount and the Epoch it was opened. To close a specific allocation users could pass the identifier as an argument to a closing command eg. indexer allocation close --subid QmRhYzT8HEZ9LziQhP6JfNfd4co9A7muUYQhPMJsMUojSF --allocid A ........
It should also be possible for Indexers to specify the identifier they want whenever they open a new allocation (though it should default to the next unused idenitifier if none is provided).

2) Provide automation of some common Indexer procedures.

One of the most common procedures is for an Indexer to rebalance their allocations periodically. An optimal procedure with 2 parallel allocations is for a single allocation on each Subgraph to be closed, then a new single allocation opened on every desired Subgraph (the new Subgraphs do not need to be the same as the original ones). When this has been successfully completed, the original remaining allocations should then be closed and a second set of new allocations opened.

Indexers could provide a yaml or json file that lists all Subgraphs for which they want to close allocations and all Subgraphs they want to form part of their new allocation distribution along with updated allocation amounts. The agent could then automatically process these when it is initiated with the relevant command. It should perform some sort of sanity-checking, for example ensuring that the total allocations to be closed plus any unused allocation exceeds or matches the new allocations requested.

[cryptovestor21]

This is something indexers need and I think both approaches have merit.

Something I have been mulling over lately is the wide array of automation functionality that the agent could enable as the network matures - basic operations like @Pete-LunaNova is suggesting here, but also really complex stuff like fully automating the tuning of revenue share variables/staking intensity, indexer collaboration to improve profitability etc . The question in my mind is at what point does the agent's functionality end and the extension into automation begin? Which parts make sense as core functionality, and which parts form a separate product that simply uses the agent cli to achieve its goals?

I would argue that in this specific case the answer is core functionality, but I think that as indexers request new functionality, we need to come to a consensus on where the line is drawn between core functions and additional "nice to have" functions.

[fattox]

I didn't get a chance to read up on this until now, but i actually had a VERY similar line of thinking as #1 myself over the weekend. I'd like to be able to control allocations independantly this way, but my thoughts were that you needn't only have allocation IDs, but also 'sets'.

For example, you could make an allocation 'set' such as (group) A and then set up 2 parallels, which leads you to having A:1 and A:2, which could then be closed independently of each other. If you get more GRT delegated to you, you could then set up some additional allocations in set B (etc etc), each set with its own amount of parallels and GRT amounts per allocation. You could also control the sets as a whole by defining A rather than individually with A:1, etc. Or everything on a sub with all.

Gas costs might mount up this way, but as with anything else, it's better to have the option to choose. Currently, being deallocated (EG: stuck pending in a gas spike) loses you far more $ than additional gas would cost after a very short amount of time, so it's currently and often (imo) a very moot point either way.

Regarding what @cryptovestor21 says, i too see this kind of feature as core functionality, which i'm then able to use (or not) in my own automation.