search

ordinals / ord

👁‍🗨 Rare and exotic sats
https://ordinals.com
Creative Commons Zero v1.0 Universal
3.79k stars 1.33k forks source link

Ord indexing "loops" or "resets" back to a previous level and never completes #3804

Open so7ow opened 3 months ago

so7ow commented 3 months ago

On the Ordicord we've seen many reports of issues since the halving, where ord indexing loops indefinitely, returning to previous levels and never completing successfully. We've been recommending folks add a --commit-interval 100 (or even lower value sometimes) and that generally seems to fix it. Rather than just continue doing that forever, I wanted to bring it up here, and see if any of the devs are aware of the situation and/or working toward a fix. Or maybe our workaround is enough? Don't know, seems like there should be some way to adapt that commit interval when the machine it's running on is struggling.

Low RAM seems to be a factor, but, to be clear, this issue has not affected me directly so I don't have any specifics to report, just generalities!

cryptoni9n commented 2 months ago

On the Ordicord we've seen many reports of issues since the halving, where ord indexing loops indefinitely, returning to previous levels and never completing successfully. We've been recommending folks add a --commit-interval 100 (or even lower value sometimes) and that generally seems to fix it. Rather than just continue doing that forever, I wanted to bring it up here, and see if any of the devs are aware of the situation and/or working toward a fix. Or maybe our workaround is enough? Don't know, seems like there should be some way to adapt that commit interval when the machine it's running on is struggling.

Low RAM seems to be a factor, but, to be clear, this issue has not affected me directly so I don't have any specifics to report, just generalities!

Thanks for reporting this, so7ow. I've been experiencing it for sometime and can confirm that a large swath of Ordicord users experience it as well. I'm not sure the best way to handle it, but my initial thought is to just drastically lower the default commit interval (which is currently at 5000). My machine has 32GB of RAM and I've had to lower the interval to single digits at times.

Perhaps if this setting was changed to be based on GB of RAM rather than number of blocks?

huguojunsy commented 2 months ago

On the Ordicord we've seen many reports of issues since the halving, where ord indexing loops indefinitely, returning to previous levels and never completing successfully. We've been recommending folks add a --commit-interval 100 (or even lower value sometimes) and that generally seems to fix it. Rather than just continue doing that forever, I wanted to bring it up here, and see if any of the devs are aware of the situation and/or working toward a fix. Or maybe our workaround is enough? Don't know, seems like there should be some way to adapt that commit interval when the machine it's running on is struggling.

Low RAM seems to be a factor, but, to be clear, this issue has not affected me directly so I don't have any specifics to report, just generalities!

I have encountered the same problem as well.

HollywodMeta commented 2 months ago

Yup, I'm having the same issue myself. I've spent a few days trying to resolve it with no success. The reporting of any fix or workaround would certainly be appreciated.

cryptoni9n commented 2 months ago

Yup, I'm having the same issue myself. I've spent a few days trying to resolve it with no success. The reporting of any fix or workaround would certainly be appreciated.

The workaround that seems to resolve the issue for everyone is to set the --commit-interval to something much lower like 100, and see if that works. If not, keep lowering it until it does.

cryptoni9n commented 2 months ago

I was digging around and found that this is a dupe of #1630

I think a fix for this needs to be a priority as it is increasingly difficult for many users to index without using the --commit-interval flag. @raphjaph could we add this to the tracker?

alexagawddess commented 1 month ago

Been experiencing this issue for the past 3 weeks. I’ve attempted the —commit-interval flag with different amounts and it still loops. Only way I can successfully index right now is to let it run and then ctrl c to safely shutdown and it registers. It’s just very time consuming doing this and wish I could just let it run like normal 😅

Windows 64GB Ram plenty of storage.

so7ow commented 1 month ago

Been experiencing this issue for the past 3 weeks. I’ve attempted the —commit-interval flag with different amounts and it still loops. Only way I can successfully index right now is to let it run and then ctrl c to safely shutdown and it registers. It’s just very time consuming doing this and wish I could just let it run like normal 😅

Windows 64GB Ram plenty of storage.

How low a value for commit interval did you try?

alexagawddess commented 1 month ago

Been experiencing this issue for the past 3 weeks. I’ve attempted the —commit-interval flag with different amounts and it still loops. Only way I can successfully index right now is to let it run and then ctrl c to safely shutdown and it registers. It’s just very time consuming doing this and wish I could just let it run like normal 😅 Windows 64GB Ram plenty of storage.

How low a value for commit interval did you try?

Went as low as 25

so7ow commented 1 month ago

Went as low as 25

Go lower. 👍🏻

HollywodMeta commented 1 month ago

Yeah, it finally worked for me at 5.

cryptoni9n commented 1 month ago

@raphjaph I think we could use the expertise of mconst on the fix for this issue. It should be easy enough just to lower the default value for the --commit-interval flag. However, I worry that if it is lowered by too much, it could lengthen the index time for a beefier machine that doesn't have any issues with the current default value. Is this an issue that needs benchmarking to determine the optimal default value, or would setting it very low (single digits) not impact the overall indexing time by that much?

It seems like it could be a zero-sum change. For example, an index with a default commit interval might spend 5 minutes every 5000 blocks updating the db with those blocks, but lowering that value to 100 could take the same time if the server then only spent 6s per update. If that were the case, the lower value wouldn't impact overall indexing time. I'm just not sure that is the way it works.

alexagawddess commented 1 month ago

Yeah, it finally worked for me at 5.

I wanted to follow up and say 5 worked for me. Thank you!