search

jcampbell05 / ml-training-benchmark

An easy to use CreateML project for benchmarking ML for evaluating ML performance on Macbooks
2 stars 1 forks source link

Questionable Results? #1

Open dclive opened 1 year ago

dclive commented 1 year ago

I've unclear results in my latest; please see the results I'm getting with 13.2 + i7-8700:

Screenshot 2023-02-05 at 8 35 43 PM

Looks to me like ~20 minutes, which can't be right given your results; can you comment?

And on the 8/256 M2:

image

...looks like about 8 minutes on the base M2.

jcampbell05 commented 1 year ago

So for some reason on your version of CreateML it's showing 25 iteration. Is it possible to set it to 100 iterations which is what the previous benchmarks have shown?

I'll update the instructions to mention this as it appears this might be something stored between sessions rather than something stored in the project file.

dclive commented 1 year ago

Hmm...I flipped things to 100 iterations with all bits at the bottom ticked, per the docs, and my i7-8700 ran overnight; I finally killed it today.

I then re-ran it; even the initial 'extracting features' takes hours now. Something's off...

jcampbell05 commented 1 year ago

Looks like it could be a RAM issue as that step involved copying the data in the CNN.

dclive commented 1 year ago

Very slow on M2 Pro too.

jcampbell05 commented 1 year ago

Can I ask which Xcode you are running ? I'm wondering if it's if they changed the neural network they are using to run this task and if so I may need to either re-run the results or find some way to make sure results are being trained on the same network.

dclive commented 1 year ago

14.2.