search

apple / turicreate

Turi Create simplifies the development of custom machine learning models.
BSD 3-Clause "New" or "Revised" License
11.19k stars 1.14k forks source link

Turi Create picking the wrong GPU #1063

Open srikris opened 6 years ago

srikris commented 6 years ago

How did you get it to work with the internal AMD Pro 560? In my case the Turicreate select the Intel UHD graphic when I train a image classification model with 30 000 pictures. I have a MacBook Pro 15" 2018, Mojave beta 10 and Xcode-beta 6. I use Jupyter to run my python code. I have tried to use a eGPU also. Same results as you but internal Intel GPU is running. If I use PlaidML as backend to Keras or Apples Create ML it will use the internal AMD card or eGPU without any problem.

srikris commented 6 years ago

@nickjong Any idea why this might be happening?

nickjong commented 6 years ago

@aicts What version of TuriCreate are you using? Note that GPU support for image classification was only added in turicreate 5.0. How are you determining what GPU is being used?

AIBCTS commented 6 years ago

I use the GPU history in Activity monitor to see which GPU is running Python 3.6.5 (anaconda) and Turicreate 5.0. I have made some more tests and its seems only affect my new MacBook Pro 2018. Same configuration in my old Powerbook Pro 2017, Turicreate use the AMD GPU. An interesting notice is that when you show the GPU history the iGPU (intel) is listed first in my old computers like Mac Pro 2017 but in this new one the dGPU(AMD) is listed first. Turicreate always chose the second one. I have tested some different python environments such as python 2.7 but the same issue. Finally I have tried to use my eGPU (Sonnet) with AMD Radeon RX 580, but independently if you set the numbers of GPU to -1, 1 or 2 I can not see any difference.

AIBCTS commented 6 years ago
screenshot 2018-09-05 at 21 09 06
AIBCTS commented 6 years ago

@nickjong Followup. After I have been restarted the computer with the eGPU connected Turi now pick the right GPU (AMD). What happens was that the eGPU was listed as GPU #1 and intel as GPU #2 and finally the AMD as #3. After I disconnected the eGPU. The Intel GPU (iGPU) become #1 and the dGPU (AMD) #2. In summary the turicreate seem to be dependent on which order the GPU driver is loaded when the computer is booting. My guess. Any plan to let turicreate use an eGPU?

nickjong commented 6 years ago

@aicts Thanks for the followup! We definitely would like to use eGPU here, but for image classification the relevant computation is actually performed by CoreML. (We just perform forward inference to obtain a fixed feature vector for each image, from which we train a logistic classifier, as described here.) So we're dependent on a code change there, which would have to happen in a new version of macOS....

It's interesting that you observed CreateML using the correct GPU. If you change the feature model for the TuriCreate image classifier to "VisionFeaturePrint_Screen" you should in theory obtain the same behavior as CreateML, e.g.

model = tc.image_classifer.create(dataset, target, model='VisionFeaturePrint_Screen')

Does that use the right GPU?

And to confirm, you're seeing Turi Create using the correct GPU as long as no eGPU is connected? (We have an internal bug report tracking better behavior in CoreML when an eGPU is present.)

noahgift commented 3 years ago

I can confirm I also have this issue I have several GPUs in my Mac Pro and it defaults to the eGPU when I would prefer it to use my dual AMD Radeon Pro W5700X