opentensor / bittensor

Internet-scale Neural Networks
https://www.bittensor.com/
MIT License
867 stars 304 forks source link

Benchmark BTCLI #1735

Open garrett-opentensor opened 6 months ago

garrett-opentensor commented 6 months ago

Description: The BTCLI is currently operating too slow for metagraph pulls and other functionality. In order to identify the core issues, we need to benchmark the various features within the CLI.

Note: It is suspected that one issue may be related to importing too many large packages.

AC:

mjurbanski-reef commented 5 months ago
(.venv)  rooter@reef-vm ➤ ~/Downloads/testenv ➤ python -m timeit -r1 -n1 'import numpy'
1 loop, best of 1: 143 msec per loop
(.venv)  rooter@reef-vm ➤ ~/Downloads/testenv ➤ python -m timeit -r1 -n1 'import torch'
1 loop, best of 1: 1.28 sec per loop

So you can easily attribute at least 1s extra CLI delay to torch use. This is almost completely addressed by #1777 PR. I say almost completely, since I cannot guarnatee it doesn't load torch IF it is installed, as some CLI functions still use it then.