Filecoin.StateLookupRobustAddress is not listed in Lotus Gateway API

[eshon]

Checklist

• [X] This is not a security-related bug/issue. If it is, please follow please follow the security policy.
• [X] I have searched on the issue tracker and the lotus forum, and there is no existing related issue or discussion.
• [X] I am running the Latest release, the most recent RC(release canadiate) for the upcoming release or the dev branch(master), or have an issue updating to any of these.
• [X] I did not make any code changes to lotus.

Lotus component

• [ ] lotus daemon - chain sync
• [ ] lotus fvm/fevm - Lotus FVM and FEVM interactions
• [ ] lotus miner/worker - sealing
• [ ] lotus miner - proving(WindowPoSt/WinningPoSt)
• [ ] lotus JSON-RPC API
• [ ] lotus message management (mpool)
• [ ] Other

Lotus Version

v1.27.1

Repro Steps

No response

Describe the Bug

Filecoin.StateLookupRobustAddress is not listed in Lotus Gateway API. This omission seems like an oversight.

It should be available in the Gateway API list here but is not listed.

Without it, it's difficult to convert from short ID to a non-account public-key address (multisig or miner, etc.) without a Lotus node. (BlockScout needs this endpoint to add native Filecoin addresses)

Logging Information

N/A

[rvagg]

Just to Chesterton's-fence this one: It could be because it's not a trivial operation, it has to iterate through actors in the init actor until it finds one with the right ID. So if you called this with an ID that doesn't exist, you'd cause the node to iterate through all of them before returning an error. Not super expensive, but maybe there's a reason for its omission.

@Stebalien is there a good reason not to expose this operation in the gateway? https://github.com/filecoin-project/lotus/blob/44569437a0538c77eca4caf581621761c35fb853/chain/stmgr/stmgr.go#L454-L461

[eshon]

Are ActorIDs randomly generated and not binary sortable?

[Stebalien]

So if you called this with an ID that doesn't exist, you'd cause the node to iterate through all of them before returning an error.

That's the reason it's not exposed. But maybe it's not too expensive compared to some of our other APIs like StateListMiners? We'd have to test it.

It would also be possible to cache a reverse table per-tipset. Not inexpensive, but it amortizes nicely.

Are ActorIDs randomly generated and not binary sortable?

Unfortunately, we don't keep a mapping of f0 to f2 (robust) address so ordering of ID addresses doesn't really help.

This omission seems like an oversight.

For some context, f2 addresses exist so that a user can send a sequence of messages without waiting for the chain to settle. I.e, they can send a message that creates a multisig immediately followed by a message to fund said multisig.

The idea was that everything else should just use f0 (ID) addresses.

[eshon]

BlockScout needs to index short ID to Robust Address mappings for many accounts (see here for an example.). They reported this issue:

"When attempting to call Filecoin.StateLookupRobustAddress on both our internal and public nodes, the request hangs and eventually times out.

What would be the most efficient way for them to make requests to their Lotus node to do this mapping?

[rjan90]

A couple of first things we should verify:

• Check/Benchmark the performance of Filecoin.StateLookupRobustAddress.
• TBD upcoming issue written by @aarshkshah1992 based on this internal Slack message
  • If we do this work, it negates the need to do the work listed in this ticket.

[rvagg]

A possible case for yet another sqlite db I suppose. We could build this mapping, but maintaining its consistency is going to be a problem for us, which is a problem we have with the other dbs; but perhaps if we can solve it in one place (events db) we can solve it in all of them, either by unifying them or having a consistent approach for them.

[aarshkshah1992]

Ongoing offline discussion with @eshon to understand the priority of this at https://filecoinproject.slack.com/archives/CP50PPW2X/p1722242663057079. Looks like Blockscout has found a way to do this using Beryx. Will wait on Eva to confirm.

[eshon]

@aarshkshah1992 - I can tell BlockScout to try using Beryx API for now but this will have wider impact in the future if we are telling EVM teams integrating Filecoin that they can also support f1/f2/f3 users using 0xFF00... addresses.

[rvagg]

Ooof, @aarshkshah1992 just asked me to time this on my mainnet node, which has very good nvme for its blockstore so should be fast; 5 seconds for the worse case of no-such-actor-id.

That's why it's not on the gateway.

[aarshkshah1992]

@rvagg Is the majority of the time spent on the blockstore reads for the Init Actor state ?

[rvagg]

(go tool pprof -raw -output=cpu.txt http://localhost:1234/debug/pprof/profile\?seconds=7 &) ; sleep 1; echo 'calling Filecoin.StateLookupRobustAddress'; time curl -s -X POST -H "Content-Type: application/js
on" --data '{"method":"Filecoin.StateLookupRobustAddress", "params":["f099999999991",[]],"id":1,"jsonrpc":"2.0"}' http://localhost:1234/rpc/v1

Gives me:

Which you can divide up into a few large categories:

1. GC
2. HAMT traversal a. Blockstore (mostly) b. Block decoding c. HAMT logic (minimal)

[rvagg]

Random thoughts on the complexity hidden behind the ways we could optimise this:

• Could build an sqlite db for this, but we have to figure out how to populate and keep it accurate
• Brute force option is to run it every epoch, but that's 5s of disk thrashing to go through the init actor (at its current size) each 30 seconds
• I don't know of good way to watch for init actor changes, perhaps inspecting messages and figuring out which ones would mutate it (I guess we can trace builtin-actors to look for anything that might mutate it)
• We could introduce a new builtin-actor event for this, that's a cheap option and we could just observe them; consistency might be a challenge but it's the same challenge as the events db and it would layer on top of the events db so maybe it ends up being easy
• Any solution we come up with could deal with the consistency challenges by wiping away and starting from scratch on start-up - do a full scan of the init actor on startup to populate a db and then figure out changes over time in a more lightweight way (tho 5s of blockstore thrash is quite a bit even for startup)
• We have HAMT diffing utilities, their nature makes them nice for diffing efficiently, although I've not used this implementation: https://github.com/filecoin-project/go-hamt-ipld/blob/299f45acca8ac7dacc4ad8af2004c7642819338f/diff_parallel.go#L16; you end up with a list of changes https://github.com/filecoin-project/go-hamt-ipld/blob/299f45acca8ac7dacc4ad8af2004c7642819338f/diff.go#L26C1-L31C2 of key/values, so it might be really efficient to run this between epochs to get the changeset and reconcile, we'd have to implement it to figure out the cost, perhaps super cheap if the init actor changes minimally over time

[rvagg]

Another option is to have a reverse mapping HAMT directly in state that builtin-actors is responsible for keeping up to date, but I imagine a FIP for duplicate data isn't going to get up

[aarshkshah1992]

@eshon @jennijuju Considering that this will need quiet a bit of engineering effort, are we okay punting on this unless there is a pressing user need for a trustless API to do this ? (otherwise can direct clients to Beryx for now who seem to have already indexed this).

[aarshkshah1992]

Removing the good first issue label as this is a non-trivial task if we want to make it performant.

[eshon]

Rather than on startup perhaps it could be a lotus-shed command (with warnings if there will be a node perf hit due to this op) that node operators can manually trigger at their discretion?

[rvagg]

Rather than on startup perhaps it could be a lotus-shed command (with warnings if there will be a node perf hit due to this op) that node operators can manually trigger at their discretion?

sure, but would we want to integrate a potentially out-of-date database into the node itself when responding to this API? seems more like something you could run separately and keep your own index and serve it in some other way, I don't think it'd be wise to have lotus depend on a db it has no idea about the consistency of

[rjan90]

@eshon Who, and what are the users looking for from this StateLookupRobustAddress method? Are they actually looking for the f2-address, or what kind of actor it is?

[eshon]

@rjan90 - https://filecoin.blockscout.com/accounts see all the 0xff000...{hex masked ID} addresses? That looks weird, would be nice for this to show native Filecoin addresses.

[eshon]

For now BlockScout will use Zondax's Beryx API to get this information.

[eshon]

Just to confirm, StateAccountKey is listed in the Lotus Gateway and is efficient? So StateLookupRobustAddress is only difficult for non-accounts?

[rvagg]

It's more efficient, it doesn't involve a loop through all actors to find a reverse mapping.