bug: uploading a directory fails due to excessive payments being deducted which repeatedly drain the wallet

[happybeing]

This occurs with safe files upload -p <DIRECTORY> as well as with a client app I'm developing which uses the FilesApi.

Report here, including uploaded logs: https://forum.autonomi.community/t/assemble-at-the-start-line-the-beta-is-about-to-begin/39882/568?u=happybeing

[mickvandijke]

Hi @happybeing ,

Thank you for reporting this and providing all the context and files on the Autonomi forum!

I have just tried uploading the v1-autonomi-demo folder directly using the latest client (v0.93.9): safe files upload ~/Downloads/v1-autonomi-demo. And it seemed to have worked as expected:

mick@MBP-van-MacBook ~ % safe files upload ~/Downloads/v1-autonomi-demo
Logging to directory: "/Users/mick/Library/Application Support/safe/client/logs/log_2024-07-17_10-14-15"
safe client built with git version: b1e0cc9 / stable-2024-07-08 / b1e0cc9 / 2024-07-08
Instantiating a SAFE client...
Connecting to the network with 50 peers
🔗 Connected to the Network                                                     Splitting and uploading "/Users/mick/Downloads/v1-autonomi-demo" into 1 chunks
**************************************
*          Uploaded Files            *
*                                    *
*  These are not public by default.  *
*     Reupload with `-p` option      *
*      to publish the datamaps.      *
**************************************
"index.html" 0919a563d7e7a3d2908cda1fd526f75d8a54ea371599470676213b668c4aef91
"index-ants.png" b1ccf5989cc287532cae5a98adabc03bfc2cd95368c09e41dd066335a5e2a8f9
"more-ants.html" 59ec4ad76c51eb5e3a1a8fa02f7089de2395e54cbb769afd6d1b3d9c8ae8b546
"index.html" 0919a563d7e7a3d2908cda1fd526f75d8a54ea371599470676213b668c4aef91
"index-ants.png" b1ccf5989cc287532cae5a98adabc03bfc2cd95368c09e41dd066335a5e2a8f9
"another-ant.png" fe0d0b9bb2ac1bc85bc410a59ea7c0e462e420c1540e7a860239bba68c820f4f
"more-ants.html" 59ec4ad76c51eb5e3a1a8fa02f7089de2395e54cbb769afd6d1b3d9c8ae8b546
Among 1 chunks, found 0 already existed in network, uploaded the leftover 1 chunks in 31 seconds
**************************************
*          Payment Details           *
**************************************
Made payment of NanoTokens(10) for 1 chunks
Made payment of NanoTokens(1) for royalties fees
New wallet balance: 1.999999989
Completed with Ok(()) of execute "Files(Upload { file_path: \"/Users/mick/Downloads/v1-autonomi-demo\", batch_size: 16, make_data_public: false, retry_strategy: Balanced })"

Could you please confirm that your issue was fixed on the latest client?

[happybeing]

Thanks @mickvandijke

I'll need a solution to this before I can test any fix against my own client: https://forum.autonomi.community/t/how-to-build-a-client-app-against-the-correct-version-of-the-network/40089?u=happybeing

Can you help with that?

[happybeing]

EDIT: and I'll also need some nanos,

BTW: did your test use a wallet with just enough tokens according to the estimated cost?

The problem doesn't arise if you have a wallet with a whole token (eg from a faucet), because the excessive cost doesn't cause the upload to fail. So any test should happen with a wallet that has enough nanos but not a billion nanos.

With a few nanos (say a couple of thousand I think) in the wallet, each time it failed it still charged the wallet for chunks that were not uploaded.

Also be aware that if those chunks are already on the network the test won't be valid.

Can you confirm that this issue was found and fixed explicitly? Or are you just testing with the latest client and thinking, ah the problem isn't happening any more?

[mickvandijke]

Thanks @mickvandijke

I'll need a solution to this before I can test any fix against my own client: https://forum.autonomi.community/t/how-to-build-a-client-app-against-the-correct-version-of-the-network/40089?u=happybeing

Can you help with that?

Absolutely agree with you that the instructions with the keys and version number are a bit unclear. Could you perhaps open an issue with the same contents you posted here: https://forum.autonomi.community/t/how-to-build-a-client-app-against-the-correct-version-of-the-network/40089?u=happybeing ?

I'll bring it up with the team.

[mickvandijke]

EDIT: and I'll also need some nanos,

BTW: did your test use a wallet with just enough tokens according to the estimated cost?

The problem doesn't arise if you have a wallet with a whole token (eg from a faucet), because the excessive cost doesn't cause the upload to fail. So any test should happen with a wallet that has enough nanos but not a billion nanos.

With a few nanos (say a couple of thousand I think) in the wallet, each time it failed it still charged the wallet for chunks that were not uploaded.

Also be aware that if those chunks are already on the network the test won't be valid.

Can you confirm that this issue was found and fixed explicitly? Or are you just testing with the latest client and thinking, ah the problem isn't happening any more?

I did test it with the latest client and 2 full tokens initially. I have also tried it again with a new wallet that only had 850 nanos and new files in a folder in a similar structure folder/(4 items). Still I couldn't reproduce the issue on the latest client:

mick@MBP-van-MacBook ~ % safe files upload ~/Desktop/test_folder                                                                                            
Logging to directory: "/Users/mick/Library/Application Support/safe/client/logs/log_2024-07-17_12-04-52"
safe client built with git version: b1e0cc9 / stable-2024-07-08 / b1e0cc9 / 2024-07-08
Instantiating a SAFE client...
Connecting to the network with 50 peers
🔗 Connected to the Network                                                                                                                                                                                           Splitting and uploading "/Users/mick/Desktop/test_folder" into 6 chunks
**************************************
*          Uploaded Files            *
*                                    *
*  These are not public by default.  *
*     Reupload with `-p` option      *
*      to publish the datamaps.      *
**************************************
"logo.png" 5a428530d574a99abdb288d04f9eccebc738a058119db9a917056b4223d5070c
"logo.png" 5a428530d574a99abdb288d04f9eccebc738a058119db9a917056b4223d5070c
"meneerzorg-logo.png" 8491912c695a07049cf5ec9a37951436d2097ffb005cb1409d6217fa345778f9
"RCAX.pdf" b4fea043b866e10e328d9a73d6604d2d07dc115ba90fa0fb2c588bc4bf922d7a
Among 6 chunks, found 0 already existed in network, uploaded the leftover 6 chunks in 13 minutes 2 seconds
**************************************
*          Payment Details           *
**************************************
Made payment of NanoTokens(276) for 6 chunks
Made payment of NanoTokens(42) for royalties fees
New wallet balance: 0.000000532
Completed with Ok(()) of execute "Files(Upload { file_path: \"/Users/mick/Desktop/test_folder\", batch_size: 16, make_data_public: false, retry_strategy: Balanced })"

If you still think I'm testing it wrong, let me know!

Also, if you want to test again for yourself, but need some nanos, please send me your address here or on Discord (my username is @mick_maidsafe) and I'll send you some.

[happybeing]

Sounds like a good test :grin: phew! So fingers crossed. I will now need to sort the client build issue. One thing:

"logo.png" 5a428530d574a99abdb288d04f9eccebc738a058119db9a917056b4223d5070c "logo.png" 5a428530d574a99abdb288d04f9eccebc738a058119db9a917056b4223d5070c "meneerzorg-logo.png" 8491912c695a07049cf5ec9a37951436d2097ffb005cb1409d6217fa345778f9 "RCAX.pdf" b4fea043b866e10e328d9a73d6604d2d07dc115ba90fa0fb2c588bc4bf922d7a Among 6 chunks, found 0 already existed in network, uploaded the leftover 6 chunks in 13 minutes 2 seconds

I noticed the above seems a bit odd. Ignoring that logo.png appears twice (presumably in separate directories) that means three files were chunked for upload, which should be a minimum of 3 x 3 = 9 chunks unless self-encryption has changed significantly. Can you ask if that's the case, or if there's some discrepancy in the output there (or my understanding of self-encryption!).

I opened the issue as requested, which you've probably seen.

Once I am able to build my client to connect to the current beta I will need some nanos. I may be able to get enough from my own nodes which someone is kindly running on my behalf, but if I do need more I will be in touch. So thanks for that!

[happybeing]

Also, if you want to test again for yourself, but need some nanos, please send me your address here or on Discord (my username is @mick_maidsafe) and I'll send you some.

Actually @mickvandijke please can you send me say 3000 nanos so I'm ready to start testing as soon as I get the client connecting again:

b30d8cfd16eae0f2b7d92c5f86045051339a6d31b66d81c17a3e0137201334cf8faaab77d8af9347711125702d7c5805

I already have a donation of 311 and expect more from the community and my nodes, but having a few ready in the wallet would help.

Also, please can you send a friend request to me on Discord as happybeingagain. I'm reluctant to try friending anyone there now as it caused a lot of hassle when I tried friending Bux and my account was suspended. She has now friended me instead.

Thank you :pray:

[happybeing]

Thanks very much for your help. Things are improved and I've been able to upload chunks with plenty of nanos to spare, although so far only 20 out of the 76 chunks for my new test because of other problems.

The original bug appears to be fixed though, so I will close this. :muscle:

I've opened a new issue for what I'm seeing now which is that it is stopping partway through with a panic that appears to be in the crates used by autonomi. For more see: https://github.com/maidsafe/safe_network/issues/1979