search

SakanaAI / AI-Scientist

The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery 🧑‍🔬
Apache License 2.0
7.46k stars 990 forks source link

How long for installation experimentation -- python experiment.py --out_dir run_0 && python plot.py #88

Open MrEdwards007 opened 2 weeks ago

MrEdwards007 commented 2 weeks ago

Good day, I was performing the installation process as shown in the model card instructions located at https://github.com/SakanaAI/AI-Scientist

I executed "cd templates/nanoGPT && python experiment.py --out_dir run_0 && python plot.py" After two hours of repeatedly cycling through building the model, I had no idea if this was normal, should it have finished after a few minutes or was it nearly finished and I just didn't know better.

OS : Ubuntu 22.04.4 LTS Driver Version : 535.183.01 CUDA Version : 12.2 Attached GPUs : 1 Product Name : NVIDIA RTX A4500 Laptop GPU Product Brand : NVIDIA RTX

I terminated the process at the keyboard after two hours of cycling. I've added a screenshot after restarting this same process, which has been running for about 40 minutes.

Questions

  1. Is there a progress bar (not just the number of iterations the process is on) or some indicator of what is normal?
  2. Have I damaged the process by terminating at the keyboard?

image image

MrEdwards007 commented 2 weeks ago

I let the process run and found that it stopped, though I'm not entirely sure about completion after (I believe) was 100,000 iterations six hours later.

Some indication that I wasn't in an endless loop and what percentage of completion is highly desirable, even if this is machine independent time to completion.

Yet, this was followed by the notification that I am in need of assistance in resolving.

Average tokens per second: 977.64 /home/homer/anaconda3/envs/ai_scientist/lib/python3.11/site-packages/numpy/core/fromnumeric.py:3504: RuntimeWarning: Mean of empty slice. return _methods._mean(a, axis=axis, dtype=dtype, /home/homer/anaconda3/envs/ai_scientist/lib/python3.11/site-packages/numpy/core/_methods.py:129: RuntimeWarning: invalid value encountered in scalar divide ret = ret.dtype.type(ret / rcount)

conglu1997 commented 2 weeks ago

This should be solved by: https://github.com/SakanaAI/AI-Scientist/issues/89#issuecomment-2323129188

xprabhudayal commented 2 weeks ago

the nanogpt part would iterate till 1,00,000 times for i guess 2-3 times as per me but the nanogpt_lite one would be faster

xprabhudayal commented 2 weeks ago

& it would be faster in the H100 gpu