search

shruggr / 1sat-indexer

18 stars 4 forks source link

issue from scrypt #13

Closed zhfnjust closed 1 year ago

zhfnjust commented 1 year ago

v2:

zhfnjust commented 1 year ago

v2:

  1. token deploy in a p2pkh, but api returns nothing deploy tx: https://whatsonchain.com/tx/8e42e29e7a39ec11eed7000834b6c062945be458368b8577173ec0a1d41d4804 api: https://v3.ordinals.gorillapool.io/api/bsv20/1sM8JmXi8WBwtDU28XsBJdR5i428E1dYA/id/8e42e29e7a39ec11eed7000834b6c062945be458368b8577173ec0a1d41d4804_0
shruggr commented 1 year ago

@zhfnjust the web-server was missing the deploy+mint utxo on these. This should be resolved now.

zhfnjust commented 1 year ago

@zhfnjust the web-server was missing the deploy+mint utxo on these. This should be resolved now.

ok, it fixed. but how about above issue?

zhfnjust commented 1 year ago

an other different results:

https://v3.ordinals.gorillapool.io/api/bsv20/1sM8JmXi8WBwtDU28XsBJdR5i428E1dYA/tick/VIVO return 2 token utxo

https://ordinals.gorillapool.io/api/utxos/address/1sM8JmXi8WBwtDU28XsBJdR5i428E1dYA/tick/VIVO?fromHeight=0&fromIdx=0&limit=100 returns 4 token utxo.

zhfnjust commented 1 year ago

Summary of API testing

NFT:

FT V1

deploy :

mint and transfer :

FT V2

some case old api returns nothing may returns correctly after manay blocks confirmed.

xhliu commented 1 year ago
freedomhero commented 1 year ago

Another concern is about the indexing speed. Even for the testnet blocks, indexing appears to be slow.

For example, the following log fragment focuses on the process of block #1147909, which has 29 transactions. It appears that about 8 seconds were spent on processing from the start of the job 'crawling,' which began around '2023/10/09 03:54:19,' to the end of the job 'Validating BSV20 mint,' which ended around '2023/10/09 03:54:27'.

2023/10/09 03:54:19 Done validating BSV20 1147907
txcount 4
Crawling Forward 1147909 0000000000001b49880157efbe5141a6d7da9a732c45624e4340483dd63e5191
Completed: 1147902 txns: 4
Validating bsv20 for height 1147908
2023/10/09 03:54:20 Validating BSV20 1147908
Processing inscription ids for height 1147902
2023/10/09 03:54:20 Validating BSV20 mint 1147908
Validating BSV20 mint 1147902
2023/10/09 03:54:20 Validating BSV20 transfers 1147908
Processing Block 1147909 0000000000001b49880157efbe5141a6d7da9a732c45624e4340483dd63e5191
txcount 29
2023/10/09 03:54:20 Done validating BSV20 1147908
Completed: 1147903 txns: 29
Validating bsv20 for height 1147909
2023/10/09 03:54:23 Validating BSV20 1147909
Processing inscription ids for height 1147903
Crawling Forward 1147910 00000000000023043dbcb4541dce2b9c7adade82a44757582dc2de5cb0b251f3
2023/10/09 03:54:23 Validating BSV20 mint 1147909
Validating BSV20 mint 1147903
2023/10/09 03:54:23 Validating BSV20 transfers 1147909
2023/10/09 03:54:23 Done validating BSV20 1147909
Processing Block 1147910 00000000000023043dbcb4541dce2b9c7adade82a44757582dc2de5cb0b251f3
txcount 12
Completed: 1147904 txns: 12
Validating bsv20 for height 1147910
Processing inscription ids for height 1147904
2023/10/09 03:54:24 Validating BSV20 1147910
Crawling Forward 1147911 000000000000093071c31bcf98faef2117764038a0ed9b14e6886c8d49318897
2023/10/09 03:54:24 Validating BSV20 mint 1147910
Validating BSV20 mint 1147904
2023/10/09 03:54:24 Validating BSV20 transfers 1147910
2023/10/09 03:54:24 Done validating BSV20 1147910
Processing Block 1147911 000000000000093071c31bcf98faef2117764038a0ed9b14e6886c8d49318897
txcount 10
Crawling Forward 1147912 0000000000001ddfb2211dcaf2f3d305b4a007e6c9356bab47bb52797317c39b
Completed: 1147905 txns: 10
Validating bsv20 for height 1147911
2023/10/09 03:54:24 Validating BSV20 1147911
Processing inscription ids for height 1147905
2023/10/09 03:54:25 Validating BSV20 mint 1147911
Validating BSV20 mint 1147905
2023/10/09 03:54:25 Validating BSV20 transfers 1147911
Processing Block 1147912 0000000000001ddfb2211dcaf2f3d305b4a007e6c9356bab47bb52797317c39b
2023/10/09 03:54:25 Done validating BSV20 1147911
txcount 1
Crawling Forward 1147913 0000000000002ec97f8387e47cfb6c3e5abbb6aecda3516039c3d682ddb975fe
Completed: 1147906 txns: 1
Validating bsv20 for height 1147912
2023/10/09 03:54:25 Validating BSV20 1147912
Processing inscription ids for height 1147906
2023/10/09 03:54:25 Validating BSV20 mint 1147912
Validating BSV20 mint 1147906
2023/10/09 03:54:25 Validating BSV20 transfers 1147912
2023/10/09 03:54:25 Done validating BSV20 1147912
Processing Block 1147913 0000000000002ec97f8387e47cfb6c3e5abbb6aecda3516039c3d682ddb975fe
txcount 13
Completed: 1147907 txns: 13
Validating bsv20 for height 1147913
2023/10/09 03:54:26 Validating BSV20 1147913
Processing inscription ids for height 1147907
Crawling Forward 1147914 0000000000001a0f7591fe9deed0ff52ef830242d28b4d53c2a5f09aeed62cf6
2023/10/09 03:54:26 Validating BSV20 mint 1147913
Validating BSV20 mint 1147907
2023/10/09 03:54:26 Validating BSV20 transfers 1147913
2023/10/09 03:54:26 Done validating BSV20 1147913
Processing Block 1147914 0000000000001a0f7591fe9deed0ff52ef830242d28b4d53c2a5f09aeed62cf6
txcount 13
Crawling Forward 1147915 000000000000350cfb16336383f3d6a364574b77ad0bc8e5bcb3e9610dd234de
Completed: 1147908 txns: 13
Processing inscription ids for height 1147908
Validating bsv20 for height 1147914
2023/10/09 03:54:27 Validating BSV20 1147914
2023/10/09 03:54:27 Validating BSV20 mint 1147914
Validating BSV20 mint 1147908
2023/10/09 03:54:27 Validating BSV20 transfers 1147914
2023/10/09 03:54:27 Done validating BSV20 1147914
Processing Block 1147915 000000000000350cfb16336383f3d6a364574b77ad0bc8e5bcb3e9610dd234de
txcount 1
Crawling Forward 1147916 000000000000254e089ec5669e9cac55134c2dbb83475976c0f66ee0da52e9fc
Completed: 1147909 txns: 1
Validating bsv20 for height 1147915
2023/10/09 03:54:27 Validating BSV20 1147915
Processing inscription ids for height 1147909
2023/10/09 03:54:27 Validating BSV20 mint 1147915
Validating BSV20 mint 1147909
2023/10/09 03:54:27 Validating BSV20 transfers 1147915

Here are some more samples:

txs secs
255 62
387 71
2275 900
2533 1110
2547 1455
2690 1260

Is there anything I can do to speed up the process?

By the way, the testnet node is running on the same host as the indexer, so the RPCs should have low network latency. Also, the host's CPU and memory usage are very low, so I do not think that is a bottleneck for indexing.

shruggr commented 1 year ago

How big of a box were you indexing on? How many cores/RAM?

freedomhero commented 1 year ago

2 cores / 4GB RAM. The CPU usage is less than 5%, and the memory usage is less than 40%.