How can I find out how long it would take to train my model if implemented in CMOS?

[rfernand2]

Hi, I'm new to analog world of NNs and SPICE simulator. Is there a way to find the expected time for training and inference on a model I have built in EBANA?

[mawatfa]

@rfernand2

Hello,

The time it takes to do an inference depends on the size of the circuit, the electrical components used, the simulator you choose, and how faster your computer is. To give you an idea, on my computer, it takes about 2 seconds to do inference on a single sample of the MNIST dataset. This time goes into creating the spice netlist, which can be almost 100,000 lines long, starting the spice simulator, running the simulation, and retrieving and processing the result from the spice simulator.

As for training, the time it takes depends on the factors given above in addition to the number of epochs and batch size. Since all the simulations within a batch are independent, it is possible to lunch as many simulations as cores available. So, the more cores you have, the larger the batch size, the faster the simulations run. Unlike inference, where only one simulation is run for each sample in the dataset, two simulations are run during training for each sample. This makes training at least twice as slow for each sample. To give you a number, it takes about a week to train MNIST (60,000 samples) for about 10 epochs on my computer.

Hope these answer your question.

[rfernand2]

Thanks for the detailed information, which will be helpful. I had no idea it would take a week to train MNIST! However, my primary question is how can I estimate how fast the analog circuit implemented in CMOS as a chip would take to train and run/inference. Does SPICE provide this kind of information?

---

From: mawatfa @.> Sent: Friday, March 15, 2024 1:23 AM To: mawatfa/ebana @.> Cc: Mention @.>; Author @.> Subject: Re: [mawatfa/ebana] How can I find out how long it would take to train my model if implemented in CMOS? (Issue #1)

@rfernand2https://github.com/rfernand2

Hello,

The time it takes to do an inference depends on the size of the circuit, the electrical components used, the simulator you choose, and how faster your computer is. To give you an idea, on my computer, it takes about 2 seconds to do inference on a single sample of the MNIST dataset. This time goes into creating the spice netlist, which can be almost 100,000 lines long, starting the spice simulator, running the simulation, and retrieving and processing the result from the spice simulator.

As for training, the time it takes depends on the factors given above in addition to the number of epochs and batch size. Since all the simulations within a batch are independent, it is possible to lunch as many simulations as cores available. So, the more cores you have, the larger the batch size, the faster the simulations run. Unlike inference, where only one simulation is run for each sample in the dataset, two simulations are run during training for each sample. This makes training at least twice as slow for each sample. To give you a number, it takes about a week to train MNIST (60,000 samples) for about 10 epochs on my computer.

Hope these answer your question.

— Reply to this email directly, view it on GitHubhttps://github.com/mawatfa/ebana/issues/1#issuecomment-1999150627 or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABAY3XV2FCHRN622KEZ6ZYLYYKVXRBFKMF2HI4TJMJ2XIZLTSOBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLLDTOVRGUZLDORPXI6LQMWWES43TOVSUG33NNVSW45FGORXXA2LDOOJIFJDUPFYGLKTSMVYG643JORXXE6NFOZQWY5LFVE2TCNRUGE3DONBWQKSHI6LQMWSWS43TOVS2K5TBNR2WLKRSGE4DONBZHAYDINNHORZGSZ3HMVZKMY3SMVQXIZI. You are receiving this email because you were mentioned.

Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[mawatfa]

@rfernand2, to give you a very short answer to your question, yes, SPICE can provide that information provided you model things accurately. That is, you use real components, model effects of the real circuit like RC delays in the wires, etc. Then, using transient simulation, SPICE will tell you the time it takes the circuit to achieve steady-state.