Built-in Simple Ledger Protocol (SLP) Support

[bitjson]

It would be great to add full support for Simple Ledger Protocol (SLP) queries. Chaingraph is actually very close to supporting SLP already, we likely just need a few well-designed Postgres functions. (I think our first goal should be a built-in, zero-cost implementation using only Postgres functions. Once it's clear which queries need better performance, we may also add some optional indexes or trigger-managed tables which help to precompute expensive queries.)

Building SLP Functionality in Chaingraph

Chaingraph's existing output_search_index was designed with OP_RETURN protocols like Simple Ledger Protocol (SLP) in mind. It indexes the first 26 bytes of every output (enough to cover P2PKH, the largest of typical output, and plenty to support searching for P2PK outputs) so OP_RETURN outputs can be searched by prefix (and often by contents) at no additional cost.

So it's already possible to query for SLP transactions, we just need to develop the Postgres functions (and maybe some additional indexes) to make common requests simple.

You can get a sense from this snippet for how complex features like SLP support can be added with just the PostgreSQL procedural language:

https://github.com/bitauth/chaingraph/blob/796ca96f92895af72aeff87f1c73ad5328c5b3d1/images/hasura/hasura-data/migrations/default/1616195337538_init/up.sql#L837-L899

Bitauth functionality is implemented in less than 100 lines – I expect basic SLP validation to be fairly similar. (And we can probably copy quite a bit from how collect_authchains traces the chain(s) of transactions descending from a parent transaction.)

I've already built out some of the utility functions we'll need – reverse_bytea, encode_uint32le, encode_uint64le, encode_bitcoin_var_int – and you can get a sense for how parsing might work by looking through parse_bytecode_pattern (parses VM bytecode to extract opcode "patterns", cutting out all the pushed data):

https://github.com/bitauth/chaingraph/blob/796ca96f92895af72aeff87f1c73ad5328c5b3d1/images/hasura/hasura-data/migrations/default/1616195337538_init/up.sql#L468-L519

These type of functions can be used to create views, which are much friendlier interfaces for querying:

https://github.com/bitauth/chaingraph/blob/796ca96f92895af72aeff87f1c73ad5328c5b3d1/images/hasura/hasura-data/migrations/default/1616195337538_init/up.sql#L922-L934

Finally, there are examples of e2e tests for the existing Postgres functions here:

https://github.com/bitauth/chaingraph/blob/796ca96f92895af72aeff87f1c73ad5328c5b3d1/src/e2e/e2e.spec.ts#L1052-L1441

So hopefully that helps to orient interested contributors to how we might implement SLP support in Chaingraph.

API

I haven't thought enough yet about how the SLP API should work, but maybe:

• output.slp – would be either NULL (if output is not a valid SLP SEND with an amount) or include amount, token_id, and token_type, and a token object relationship with the token's genesis details
• one or more fields on transaction (prefixed with slp) – would be NULL if the transaction was not an SLP transaction, and would otherwise provide information on GENESIS, MINT, and COMMIT transaction types (not sure what this should look like yet)

And I think the output.slp field would be sufficient to make search_output work for wallets looking up both satoshi values and token values.

I'm still familiarizing myself with the SLP spec, so any ideas or feedback is highly appreciated!

Progress

• [ ] Initial SLP support (via computed fields)
• [ ] SLP precomputation (depends on Accelerated columns: improve performance of computed fields #29)

[bitjson]

Cc: @scherrey and @blockparty-sh have both expressed interest in SLP support. I'd love to hear your thoughts on the API, and if you're interested in working on an implementation, I'd be very happy to work with you. (Either here or in the Chaingraph Devs Telegram group.)

[christroutner]

As I understand it, there are two parts to validating an SLP transaction:

• The OP_RETURN data needs to comply with the specification. This can be validated using the slp-parser library.
• The DAG back to the GENESIS transaction needs to be validated.

That second part is the more computationally intense operation.

The primary metric that I'm focused on with the psf-slp-indexer is memory usage. A secondary metric is output speed. Using LevelDB accomplishes both goals: it keeps the memory footprint to around 2GB or less, and the output is simply key-value lookup, so it's very fast.

I'll be interested in following the development of Chaingraph and its ability to track SLP transactions, but I don't think I'll be able to contribute very much. If there is room for synergy, or code that can be shared between projects, I'm happy to contribute on that front.

[bitjson]

Thanks for chiming in @christroutner, I'll look forward to seeing that develop.

Just to add to this issue: I think we can get SLP support working reasonably with just a few Postgres functions, but https://github.com/bitauth/chaingraph/issues/29 would allow us to precompute everything and make lookups instant.

So I'll probably consider there to be 2 stages for this issue:

• Initial, real-time SLP support
• SLP precomputation (depends on #29)

[bitjson]

With CashTokens now supported (https://github.com/bitauth/chaingraph/commit/3ebddcedeb288dfe9c873c872586051f60026753), I don't expect to ever add SLP support to Chaingraph.

I'd be happy to take a PR if anyone is interested, but going to close this issue for now.