issues with distributing spl tokens using tokens cli

[dzmitry-lahoda]

Problem

1. it is hard to find program.

several people I know and me searched, not found this cli. also its folder named tokens which is very unclear name. like tokens what? in crates it also has no info.

2. it failed and hard to find what account failed

Total in input_csv: 0.000019 tokens
Distributed: 0 tokens
Undistributed: 0.000019 tokens
Total: 0.000019 tokens
Recipient                                             Expected Balance
5cExDmw8fWbwhNQpEM4c2Kh4iWsFN9Bva89G79t5cQWs                  0.000001
HFxtKzGGLQA11FT1sjbVNdJuQU4fESTSH8wXg1bXJ6dW                  0.000001
6SNsU7iFaam7K5jJhoj7sbPsJ3tBFKunuVocfhjwYSa                   0.000001
HykEavTUgj3iqxYYQYK2DU3T82eG7BMiuZksYqqrg6u7                  0.000001
EybSv9Hhh1T5oXN5HxabA8TMZsFFTaHrjfxUGSFQ9yY9                  0.000001
8xe4YLrFHAB3kSc2cHskPtXMiEvCs5jgYT1591evWA1T                  0.000001
69kY83qgxwVcKLAn2So8PGEdSAwGNjr6RdisR9jFCgZ9                  0.000001
J6DV46TYWk359ym6Uks5FT7ehfnWhcjVrHYowCbYLfXR                  0.000001
8R9bBM3dUdFfDwfadbkPK9MHz3BHKvhLSLMrXRFjJ6Ej                  0.000001
3BFUHDDcj9Wc8VHaDmmFqVt4VqsSZUCCATYkhYWRPtUk                  0.000001
6UiBPmrnDWmhYYta6cLWiyewKfH65vnVddiWPqY33Xxq                  0.000001
HBx1znomocgCa2PYh5FctHskMAi9Hpt6a7VjaVj53Q18                  0.000001
6ayimYcc3XoWBALjiqa9bRUKrDQFgSDK4fxmGGDLQbL3                  0.000001
4JUxDZ3gwDAPyG13SUMzBJXZWdbfDp79PmQ29FvZievm                  0.000001
FGqDUUmscDat8xW3AmmLiXneZCjwY91FiWt4cs1AqjqJ                  0.000001
51upaDk3L8C9AcWme5J4v1VHG3BJGQSznfoLbbT4FFjM                  0.000001
FsBnTcSrcrjKwCutS8NoZrV8eUGeTp589rhBoN4ip48w                  0.000001
8oBqBmM5tozfEUJ3kEygj662UTGY3VGqUikJYjtsgzEB                  0.000001
 blockhash expired. Transaction either dropped or the validator purged the transaction status.
No work to do

So after some time get this. On rerun get this:

Total in input_csv: 0.000019 tokens
Signature not found 4TD51edoQUA2FXDghr9hVbDEvXkujw4Fj31sYrMnzYRfk8YzWJoAanHw8KFSkVscxqRrxUtX2y7YZ7HVD1xLZbMa and blockhash expired. Transaction either dropped or the validator purged the transaction status.
Signature not found 5xVmb5vpZ4uotLqgc6nRSoUDJNpMJYiEyCdojatjMsHUSgRRWqPT6Jq7huThBVsZovEWYKMo5woZPM39RPQWouP and blockhash expired. Transaction either dropped or the validator purged the transaction status.
No work to do

Really checked manually each account, and people did not get amount.

Failed accounts are stored in format I cannot read:

- 68\n  - 117\n  - 218\n  - 7\n  - 223\n  - 217\n  - 56\n  - 29\n  - 41\n  - 24\n  - 234\n  - 108\n  - 90\n  - 10\n  - 233\n  - 18\n  - 49\n  - 249\n  - 175\n  - 169\n  - 214\n  - 150\n  - 118\n  - 20\n  - 25\n  - 188\n  - 31\n

there is no command to dump of possible failed drops into new csv.

3. checking balance for spl mint does not work

   
   Error: Found argument '--mint' which wasn't expected, or isn't valid in this context

USAGE: solana-tokens.exe balances [OPTIONS] --input-csv

also it documented.

4. not clear model if diff of amount on account to be airdropped - not clear if it will about missed airdrop or double spend - clear that is not.

seems tool will not work reasonably if account already has an amount. it is not clear what transaction semantics. I am not sure if it will not double bill.  tx is send by only and data is saved. it must SAVE data into db, and then send.

#### Proposed Solution

some things to address above and generate helpers.

1. rename folder to be `tokens-distribute` or `tokens-airdrop`. Add airdrop tag to crates and into readme of program. make crates description relevant.

2. current approach does not scale to avoid double-spend or missed drops. if tx fails, or network fails after tx sent, there is no validation on the next run to avoid issues. the current approach does not work.

2.1. can work. we know sender and amount and first run. if we failed to bill, on next run check all TX on known token which are exactly  same within period from run to now. so if amount was given from to, do not bill again. 

2.2. alternative, create airdrop account on chain. for each airdropped account create void account derived from airdrop.. with rent enough for several days. on next run check if account exists, if not create and airdrop. so if account was created, tx will fail. this will ensure we do not double spend nor failed to airdrop. i think it is good approach to be robust.

3. run in batches. it seems 20 tx work.  will make tool faster.

4. allow to run creation of ata separate from airdrop.  ata costs much more. so can run ata creation before drop. so drop takes less time. so then during drop recepinets without ata just fail.

5. on tools run, show warning when Solana is degraded performance and suggest to run tool later.

6. if use db, use rocksdb. at least there is reader for it with ui. can copy paste keys. 

[steveluscher]

Thanks for filing this issue @dzmitry-lahoda! This GitHub issue is massive though, and can't possibly be addressed by a single PR.

Can you take the interventions that you think are most high value, create a separate GitHub issue for each one (with an explanation of its motivation), and optionally send PRs?