synapse-alpha / mirror-neuron

Experiments on bittensor reward models to find exploits
BSD 2-Clause "Simplified" License
1 stars 0 forks source link

Fix intermittent bug which occurs when subtensor tries to get delegates #42

Closed steffencruz closed 1 year ago

steffencruz commented 1 year ago
  0%|                                                                                                                                       | 0/10 [00:03<?, ?it/s]
Traceback (most recent call last):
  File "/Users/steffencruz/Desktop/py/bittensor/mirror_neuron/main.py", line 101, in <module>
    main()
  File "/Users/steffencruz/Desktop/py/bittensor/mirror_neuron/main.py", line 81, in main
    run_query(model=model, data=data)
  File "/Users/steffencruz/Desktop/py/bittensor/mirror_neuron/query.py", line 95, in run_query
    run_train(model)
  File "/Users/steffencruz/Desktop/py/bittensor/mirror_neuron/query.py", line 30, in run_train
    model.train(max_iter=1)
  File "/Users/steffencruz/Desktop/py/bittensor/mirror_neuron/sources/neuron.py", line 425, in train
    self.my_nominators = { nomin[0]: nomin[1] for nomin in self.subtensor.get_delegated( self.wallet.coldkeypub.ss58_address )[0][0].nominators }
IndexError: list index out of range
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/steffencruz/Desktop/py/bittensor/mirror_neuron/main.py:101 in <module>                    │
│                                                                                                  │
│    98                                                                                            │
│    99                                                                                            │
│   100 if __name__ == "__main__":                                                                 │
│ ❱ 101 │   main()                                                                                 │
│                                                                                                  │
│ /Users/steffencruz/Desktop/py/bittensor/mirror_neuron/main.py:81 in main                         │
│                                                                                                  │
│    78 │   # Run the queries                                                                      │
│    79 │   if config.get('query'):                                                                │
│    80 │   │   print(f'{"- "*40}\nRunning queries:')                                              │
│ ❱  81 │   │   run_query(model=model, data=data)                                                  │
│    82 │   │   print("\n>>> Queries executed successfully\n")                                     │
│    83 │                                                                                          │
│    84 │   # Run the analysis                                                                     │
│                                                                                                  │
│ /Users/steffencruz/Desktop/py/bittensor/mirror_neuron/query.py:95 in run_query                   │
│                                                                                                  │
│    92 │                                                                                          │
│    93 │   method_name = template.method.get('name')                                              │
│    94 │   if method_name == 'train':                                                             │
│ ❱  95 │   │   run_train(model)                                                                   │
│    96 │   elif method_name == 'forward':                                                         │
│    97 │   │   run_forward(model, data)                                                           │
│    98 │   elif method_name == 'inference':                                                       │
│                                                                                                  │
│ /Users/steffencruz/Desktop/py/bittensor/mirror_neuron/query.py:30 in run_train                   │
│                                                                                                  │
│    27 │   for i in tqdm.tqdm(range(max_iter)):                                                   │
│    28 │   │   t0 = time.time()                                                                   │
│    29 │   │   qsize = model.history.qsize()                                                      │
│ ❱  30 │   │   model.train(max_iter=1)                                                            │
│    31 │   │   # add step time and step number to the added queue items                           │
│    32 │   │   add_call_metrics(model.history.queue, t0, i, added_size=model.history.qsize()-qs   │
│    33                                                                                            │
│                                                                                                  │
│ /Users/steffencruz/Desktop/py/bittensor/mirror_neuron/sources/neuron.py:425 in train             │
│                                                                                                  │
│   422 │   │   │   # Resync metagraph before returning. (sync every 15 min or ~75 blocks)         │
│   423 │   │   │   if last_epoch_block % 10 == 0:                                                 │
│   424 │   │   │   │   self.metagraph.sync()                                                      │
│ ❱ 425 │   │   │   │   self.my_nominators = { nomin[0]: nomin[1] for nomin in self.subtensor.ge   │
│   426 │   │   │                                                                                  │
│   427 │   │   │   # Check if enough epoch blocks have elapsed since the last epoch.              │
│   428 │   │   │   epoch_length = self.subtensor.validator_epoch_length(self.config.netuid) if    │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
IndexError: list index out of range

specifically, the error occurs in this line:

self.my_nominators = { nomin[0]: nomin[1] for nomin in self.subtensor.get_delegated( self.wallet.coldkeypub.ss58_address )[0][0].nominators }
steffencruz commented 1 year ago

Using a mock subtensor implementation will fix this #9

steffencruz commented 1 year ago

Fixed in #48