search

backblaze-b2-samples / b2fs4chia

MIT License
27 stars 8 forks source link

The latency is sometimes >10s, so it's not usable for hpool #10

Open qazmichaelgw opened 3 years ago

qazmichaelgw commented 3 years ago

It is similar to rclone command. I don't see the speed up.

ppolewicz commented 3 years ago

I ran into this a few times. This is most probably due to improper installation of chiapos patch which parallelizes the full proof reads.

You can see in the logs if the full proof is parallelized or not, there is a concurrent thread count at the far side of the line

the-crypt-keeper commented 3 years ago

I seem to be struggling to keep even the initial 7 reads under 10sec. My latency to the api is 8ms, I am in the same region as my bucket but sometimes a 32kb read will still take 3 seconds for no particular reason. Is this something I need to take up with backblaze support?

ppolewicz commented 3 years ago

Ok so this doesn't really need to be bad - as far as I know, you have 28 seconds to submit a proof, so if the first 7 takes 10s and the second series fetching a tree of height 7 takes 10s, that's still 8s short of the target. It is not as fast as a local HDD would be, but it's fast enough to win Chia (and that's the only thing that matters for this project).

Contacting support may actually bring up some solutions. They are friendly. Maybe email them to say hi!

the-crypt-keeper commented 3 years ago

My ping to the API gateway is 8ms but still I see sometimes a single 16kb read taking over 2sec:

2021-06-18 18:59:45,261:INFO:downloading from b2: plot-k32-2021-06-18-11-55-b4da5ac7cb714b2c75ce3fc63db82887125c318219f0c03756b59fa51d8663da.plot; offset = 43239689224; length = 16384; time=2.546183, thr=1

Even one of these while fetching the tree and its basically over. I have reached out to support, but I do not beleive the p99 here is good enough to actually win a reward.

ppolewicz commented 3 years ago

I think you are using a 10s target, right? Why? To get it all done in 28s you need to fetch 7 data ranges for quality check and 7 layers of tree for proof, which makes it 14 rounds. If the average is below 2s, you should generally be ok, I think? Is your average >2s?

the-crypt-keeper commented 3 years ago

External constraint; the client I am using is not actually chiapos, and will reject any plot folder that cannot load quantities in 10sec. Cheers on the great hack here I learned a lot from your code, but I dont think B2 has the quite has the performance I am looking for. No issues at 2 other providers..

On Fri, Jun 18, 2021 at 10:00 PM Paweł Polewicz @.***> wrote:

I think you are using a 10s target, right? Why? To get it all done in 28s you need to fetch 7 data ranges for quality check and 7 layers of tree for proof, which makes it 14 rounds. If the average is below 2s, you should generally be ok, I think? Is your average >2s?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Backblaze-B2-Samples/b2fs4chia/issues/10#issuecomment-864340230, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUGCACF3NDHWL6PTRM3DEJTTTP233ANCNFSM463HNUQQ .

ppolewicz commented 3 years ago

That sounds fascinating @the-crypt-keeper - what client are you using?

BTW I still think you should speak with b2 support about this - there may be a way to reduce your latency.

qazmichaelgw commented 3 years ago

@ppolewicz Even I use the customed chiapos, the latency is still more than 10s.

the-crypt-keeper commented 3 years ago

@ppolewicz this is the client I run: https://github.com/hpool-dev/chia-miner

For anyone under 200TB a solo block win is very much chasing the dragon at this point and official pools are a day late and a buck short. Hoping the situation improves and I can go back to mainline.

I do have a b2 support ticket open about this, haven't quite given up yet.

ppolewicz commented 3 years ago

I'm sure you've been told - hpool is closed source and it has access to your private key. I would never use it due to security concerns. In fact, I don't,

The point of b2fs4chia is to have full proofs generating in time for solo farming or farming by the official chia pool protocol, where the timeout is at 28s or so. Hpool is... unofficial. If your target is 10s then B2 may still be viable, however you should talk to B2 support and see if they can help you.

the-crypt-keeper commented 3 years ago

Hpool compromises your plotting wallet, sure. This is not a big deal in practice, hpool allows withdrawals to any address you like.. the wallet with my proceeds in it is safe and nobody has keys but me. At worst, having multiple wallets is slightly inconvenient.

I plan to switch back to mainnet when a portable plotter gets released and the protocol modifications to be able to bind a pool on the blockchain will go live.. but replotting will take time and my feeling is Netspace is likely to double again before this all happens. It is almost triple now vs what it was in May when official pools were originally promised, the long tail has begun.

On Sat, Jun 19, 2021, 6:51 PM Paweł Polewicz, @.***> wrote:

I'm sure you've been told - hpool is closed source and it has access to your private key. I would never use it due to security concerns. In fact, I don't,

The point of b2fs4chia is to have full proofs generating in time for solo farming or farming by the official chia pool protocol, where the timeout is at 28s or so. Hpool is... unofficial. If your target is 10s then B2 may still be viable, however you should talk to B2 support and see if they can help you.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Backblaze-B2-Samples/b2fs4chia/issues/10#issuecomment-864473129, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUGCACEXFHLTSACLMEEUESDTTUNQXANCNFSM463HNUQQ .