int03h
commented
7 years ago Mesa / Clover / amdgpu stack doesn't work. You need amdgpu-pro. It's not the Vega it's your driver's.
Open dkgeorge opened 7 years ago
int03h
commented
7 years ago Mesa / Clover / amdgpu stack doesn't work. You need amdgpu-pro. It's not the Vega it's your driver's.
int03h
commented
7 years ago Interesting though .. is that.just one card being enumerated in the machine or do you actually have a Furry and a RX480/580 in there too ??
dkgeorge
commented
7 years ago Hi int03h. Thanks for the reply. The rig has an RX 560, R9 Fury X and an RX Vega 64. I did try amdgpu-pro 17.30 + rocm 1.5 opencl but with this driver the Vega is the only one not recognized as an OpenCL device even though AMD lists the RX Vegas as fully supported in 17.30. Clover/Mesa/amdgpu was the closest I could get (so I thought) to get everything recognized. I followed these steps for both amdgpu-pro and mesa/clover/amdgpu attempts.
int03h
commented
7 years ago Oh ..ok. 3 cards cool. Vega is basically a mishmash of a Fury and Polaris so I was wondering .. nevermind.
Last time I looked, OpenCL on mesa was WIP and was months away. (Mostly because of AMD holding back blobs because of either Vulkan and/or ROCM and/or them just being there usual "helpful" selves). Which ROCM did you try ?? The KFD HSA kernel one ?
int03h
commented
7 years ago I did get this one working - NOT ON A VEGA - but it says it supports Vega Frontier - its ROCM 1.6, but it's more complicated than just using the amdgpu-pro drivers. https://github.com/RadeonOpenCompute/ROCm
On a sort of sidenote, I did find a very intersting discussion going on here which has direct impact on AMD's intent regarding OpenCL's overall direction (which hasn't been at all clear) : https://github.com/RadeonOpenCompute/ROCm/issues/132
To save you some effort .. this the compiler flags I used after installing ROCM (the one above) - not the one that comes with amdgpu-pro.
CFLAGS="-O2 -march=native -I/opt/rocm/opencl/include" LDFLAGS="-L/opt/rocm/opencl/lib/x86_64" ./configure --enable-adl --disable-git-version
ALSO: Opencl is also installed with the amdgpu-pro driver. ( You no longer need the AMD SDK installed ). Here are my flags for using those : FLAGS="-O2 -march=native -I/opt/amdgpu-pro/include/" LDFLAGS="-L/opt/amdgpu-pro/lib/x86_64-linux-gnu" ./configure --enable-adl --disable-git-version
dkgeorge
commented
7 years ago Yeah, it's ROCm 1.6 from https://github.com/RadeonOpenCompute/ROCm. It creates a KFD kernel when set up, but the Vega didn't get listed. I had to install 4.12.0-vegaphx kernel from amd-staging-4.12 to get Vega as an OpenCL device but as the guide explains:
This headless support on the mainline kernel is only practical if having an integrated GPU or secondary GPU installed where you could use that for driving a display and offloading the rendering to the Vega GPU using DRI_PRIME. It could also be used for OpenCL/compute systems, but the necessary ROCm kernel driver changes have yet to be mainlined, so ROCm is still relying upon its older kernel branch for now.
I do have the RX 560 as the primary GPU but looks like there are still pending changes to the open source ROCm that would make OpenCL possible to Vegas.
I'll try using your instructions to compile using both ROCms, thanks! If nothing works I'll try reinstalling Ubuntu 16.04.3 from scratch and make a clean install of amdgpu-pro + ROCm. If it still doesn't work I'll give a shot with RHEL or CentOS.
int03h
commented
7 years ago WRT to any kernel iteration you are trying to use, your fundamental problem is this: https://en.wikipedia.org/wiki/Mesa_(computer_graphics) - until the status changes from " in dev. by Gallium Compute (Clover): work in (very slow) progress (WIP) " that route is a total dead end.
Ubuntu 16.04 or 16.10 work fine. 16.04 with the HW enablement kernel is pretty good. I have gone as far as 16.10 with a 4.10 kernel and it all works fine.. It doesn't really get you anything, it's just a PITA to keep it maintained.
17.30 amdgpu-pro with these flags SHOULD work - it says Vega RX is supported : FLAGS="-O2 -march=native -I/opt/amdgpu-pro/include/" LDFLAGS="-L/opt/amdgpu-pro/lib/x86_64-linux-gnu" ./configure --enable-adl --disable-git-version
I would not install or try ROCM, and most definitely stay away from Clang and LLVM. ( I ended up wasting SOOOO much time trying to get AMD's compilation stack going .. I did not succeed, but it seems the dude from that thread above did, and got poor results for his efforts .. I don't feel so bad now for giving up eventually ).
int03h
commented
7 years ago I read that article on Phoronix. Ignore all of it. Install Ubuntu server LTS or 16.10, install the deps as per instructions ( you seem to be good there if you got a binary compiled ). Download amdgpu-pro 17.30 and install it with the --compute option. Compile with my amdgpu-pro flags, not ROCM.
The point here is to avoid X11 and it's friends (Unless you need the display ). It adds unnecessary complications and you can't twiddle any of the power tuning options anyway under Linux.
Windoze is the only way if you want to tweak, powertune etc..
My build script .. if you want the easy way ..
git clone https://github.com/genesismining/sgminer-gm/
cd sgminer-gm git submodule init git submodule update autoreconf -fi CFLAGS="-O2 -march=native -I/opt/amdgpu-pro/include/" LDFLAGS="-L/opt/amdgpu-pro/lib/x86_64-linux-gnu" ./configure --enable-adl --disable-git-version make -j4
Profit !!! ??? ;)
Some fiddling can be done with this https://github.com/matszpk/amdcovc and/or with this ..
https://raw.githubusercontent.com/RadeonOpenCompute/ROC-smi/roc-1.6.x/rocm-smi (ROC cruft not required to actually use it)
Check back in and let me know how it goes.. I've been debating grabbing a Vega RX but I haven't been able to do any math that would compel me to do it. XBOX Scorpio has Vega RX in it, maybe I will wait and try get RoCM to compile on that. ;) ESPECIALLY if that other thread above gets anywhere..
dkgeorge
commented
7 years ago Tried using ROCm from amdgpu-pro and these are the results :(
~# env LLVM_BIN=/opt/amdgpu-pro/bin /opt/amdgpu-pro/bin/clinfo | grep -i board
pci id for fd 10: 1002:687f, driver (null)
pci id for fd 10: 1002:687f, driver (null)
Board name: AMD Radeon (TM) R9 Fury Series
Board name: 67FF:CFIn windows the Vega is working quite well for CryptoNight. Using wolf's miner it's pushing 1550 H/s (pretty much same as 2 Fury X). I'm curious to see the performance in Linux with sgminer-gm.
Going to try now 16.10 server and install 17.30 with --compute. Thanks for the suggestions. Hope it works!
int03h
commented
7 years ago That actually looks like firmware. What does lspci | grep VGA say ?
Wow .. 2 Fury X's .. that's pretty good!
Actually ... try this .. (edited to add this:)
This will ID the VGA Cards only :
GET THE SLOT ID .. from above .. and change the 04 to the one that the Vega is in .. :
AND/OR:
and this will tell you the firmware loaded :
dkgeorge
commented
7 years ago Running update-pciids helped identify the Vega properly when running lshw -c video. Thanks! That's a step forward. However amdgpu-pro's ROCm fails to identify the Vega as an OpenCL device still. I've tried 16.10 without success so now I'm sticking to 16.04.3 with 4.10 kernel as it's (supposedly) recommended/supported by AMD for the RX Vegas.
# update-pciids
Downloaded daily snapshot dated 2017-08-29 03:15:01
lspci | grep -E "VGA|3D"
00:02.0 VGA compatible controller: Intel Corporation Core Processor Integrated Graphics Controller (rev 02)
06:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 XT [Radeon RX Vega 64] (rev c0)
08:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Fiji [Radeon R9 FURY / NANO Series] (rev c8)
09:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Baffin [Radeon RX 560] (rev cf)
root@dell:/lib/firmware/amdgpu# lspci -v -s 06:00.0
06:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 XT [Radeon RX Vega 64] (rev c0) (prog-if 00 [VGA controller])
Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Device 6b76
Flags: bus master, fast devsel, latency 0, IRQ 31
Memory at b0000000 (64-bit, prefetchable) [size=256M]
Memory at c0000000 (64-bit, prefetchable) [size=2M]
I/O ports at dc00 [size=256]
Memory at f7880000 (32-bit, non-prefetchable) [size=512K]
Expansion ROM at 000c0000 [disabled] [size=128K]
Capabilities: [48] Vendor Specific Information: Len=08 <?>
Capabilities: [50] Power Management version 3
Capabilities: [64] Express Legacy Endpoint, MSI 00
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [150] Advanced Error Reporting
Capabilities: [200] #15
Capabilities: [270] #19
Capabilities: [2a0] Access Control Services
Capabilities: [2b0] Address Translation Service (ATS)
Capabilities: [2c0] #13
Capabilities: [2d0] #1b
Capabilities: [320] Latency Tolerance Reporting
Kernel driver in use: amdgpu
Kernel modules: amdgpu
# lshw -c video
*-display
description: VGA compatible controller
product: Vega 10 XT [Radeon RX Vega 64]
vendor: Advanced Micro Devices, Inc. [AMD/ATI]
physical id: 0
bus info: pci@0000:06:00.0
version: c0
width: 64 bits
clock: 33MHz
capabilities: pm pciexpress msi vga_controller bus_master cap_list rom
configuration: driver=amdgpu latency=0
resources: irq:32 memory:b0000000-bfffffff memory:c0000000-c01fffff ioport:dc00(size=256) memory:f7880000-f78fffff memory:c0000-dffff
*-display
description: VGA compatible controller
product: Fiji [Radeon R9 FURY / NANO Series]
vendor: Advanced Micro Devices, Inc. [AMD/ATI]
physical id: 0
bus info: pci@0000:08:00.0
version: c8
width: 64 bits
clock: 33MHz
capabilities: pm pciexpress msi vga_controller bus_master cap_list rom
configuration: driver=amdgpu latency=0
resources: irq:33 memory:d0000000-dfffffff memory:c0200000-c03fffff ioport:cc00(size=256) memory:f76c0000-f76fffff memory:f7700000-f771ffff
*-display UNCLAIMED
description: VGA compatible controller
product: Core Processor Integrated Graphics Controller
vendor: Intel Corporation
physical id: 2
bus info: pci@0000:00:02.0
version: 02
width: 64 bits
clock: 33MHz
capabilities: msi pm vga_controller cap_list
configuration: latency=0
resources: memory:f7c00000-f7ffffff memory:a0000000-afffffff ioport:ec90(size=8)
*-display
description: VGA compatible controller
product: Baffin [Radeon RX 560]
vendor: Advanced Micro Devices, Inc. [AMD/ATI]
physical id: 0
bus info: pci@0000:09:00.0
version: cf
width: 64 bits
clock: 33MHz
capabilities: pm pciexpress msi vga_controller bus_master cap_list rom
configuration: driver=amdgpu latency=0
resources: irq:34 memory:e0000000-efffffff memory:f0000000-f01fffff ioport:bc00(size=256) memory:f74c0000-f74fffff memory:f7500000-f751ffff
# dmesg | grep -E 'drm|radeon' | grep -iE 'firmware|microcode'
[ 1.315000] [drm] Found UVD firmware Version: 1.57 Family ID: 17
[ 1.315015] [drm] PSP loading UVD firmware
[ 1.317896] [drm] Found VCE firmware Version: 53.32 Binary ID: 4
[ 1.317910] [drm] PSP loading VCE firmware
[ 2.505075] [drm] Found UVD firmware Version: 1.91 Family ID: 12
[ 2.521199] [drm] Found VCE firmware Version: 53.14 Binary ID: 3
[ 3.738210] [drm] Found UVD firmware Version: 1.79 Family ID: 16
[ 3.740529] [drm] Found VCE firmware Version: 52.4 Binary ID: 3
# env LLVM_BIN=/opt/amdgpu-pro/bin /opt/amdgpu-pro/bin/clinfo
Number of platforms: 1
Platform Profile: FULL_PROFILE
Platform Version: OpenCL 2.0 AMD-APP (2442.7)
Platform Name: AMD Accelerated Parallel Processing
Platform Vendor: Advanced Micro Devices, Inc.
Platform Extensions: cl_khr_icd cl_amd_event_callback cl_amd_offline_devices
Platform Name: AMD Accelerated Parallel Processing
Number of devices: 2
Device Type: CL_DEVICE_TYPE_GPU
Vendor ID: 1002h
Board name: AMD Radeon (TM) R9 Fury Series
Device Topology: PCI[ B#8, D#0, F#0 ]
Max compute units: 64
Max work items dimensions: 3
Max work items[0]: 256
Max work items[1]: 256
Max work items[2]: 256
Max work group size: 256
Preferred vector width char: 4
Preferred vector width short: 2
Preferred vector width int: 1
Preferred vector width long: 1
Preferred vector width float: 1
Preferred vector width double: 1
Native vector width char: 4
Native vector width short: 2
Native vector width int: 1
Native vector width long: 1
Native vector width float: 1
Native vector width double: 1
Max clock frequency: 1050Mhz
Address bits: 64
Max memory allocation: 3019898880
Image support: Yes
Max number of images read arguments: 128
Max number of images write arguments: 8
Max image 2D width: 16384
Max image 2D height: 16384
Max image 3D width: 2048
Max image 3D height: 2048
Max image 3D depth: 2048
Max samplers within kernel: 16
Max size of kernel argument: 1024
Alignment (bits) of base address: 2048
Minimum alignment (bytes) for any datatype: 128
Single precision floating point capability
Denorms: No
Quiet NaNs: Yes
Round to nearest even: Yes
Round to zero: Yes
Round to +ve and infinity: Yes
IEEE754-2008 fused multiply-add: Yes
Cache type: Read/Write
Cache line size: 64
Cache size: 16384
Global memory size: 4278120448
Constant buffer size: 3019898880
Max number of constant args: 8
Local memory type: Scratchpad
Local memory size: 32768
Max pipe arguments: 0
Max pipe active reservations: 0
Max pipe packet size: 0
Max global variable size: 0
Max global variable preferred total size: 0
Max read/write image args: 0
Max on device events: 0
Queue on device max size: 0
Max on device queues: 0
Queue on device preferred size: 0
SVM capabilities:
Coarse grain buffer: No
Fine grain buffer: No
Fine grain system: No
Atomics: No
Preferred platform atomic alignment: 0
Preferred global atomic alignment: 0
Preferred local atomic alignment: 0
Kernel Preferred work group size multiple: 64
Error correction support: 0
Unified memory for Host and Device: 0
Profiling timer resolution: 1
Device endianess: Little
Available: Yes
Compiler available: Yes
Execution capabilities:
Execute OpenCL kernels: Yes
Execute native function: No
Queue on Host properties:
Out-of-Order: No
Profiling : Yes
Queue on Device properties:
Out-of-Order: No
Profiling : No
Platform ID: 0x7faf3cdff478
Name: Fiji
Vendor: Advanced Micro Devices, Inc.
Device OpenCL C version: OpenCL C 1.2
Driver version: 2442.7
Profile: FULL_PROFILE
Version: OpenCL 1.2 AMD-APP (2442.7)
Extensions: cl_khr_fp64 cl_amd_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_fp16 cl_khr_gl_sharing cl_amd_device_attribute_query cl_amd_vec3 cl_amd_printf cl_amd_media_ops cl_amd_media_ops2 cl_amd_popcnt cl_khr_image2d_from_buffer cl_khr_spir cl_khr_gl_event
Device Type: CL_DEVICE_TYPE_GPU
Vendor ID: 1002h
Board name: 67FF:CF
Device Topology: PCI[ B#9, D#0, F#0 ]
Max compute units: 16
Max work items dimensions: 3
Max work items[0]: 256
Max work items[1]: 256
Max work items[2]: 256
Max work group size: 256
Preferred vector width char: 4
Preferred vector width short: 2
Preferred vector width int: 1
Preferred vector width long: 1
Preferred vector width float: 1
Preferred vector width double: 1
Native vector width char: 4
Native vector width short: 2
Native vector width int: 1
Native vector width long: 1
Native vector width float: 1
Native vector width double: 1
Max clock frequency: 1196Mhz
Address bits: 64
Max memory allocation: 3019898880
Image support: Yes
Max number of images read arguments: 128
Max number of images write arguments: 8
Max image 2D width: 16384
Max image 2D height: 16384
Max image 3D width: 2048
Max image 3D height: 2048
Max image 3D depth: 2048
Max samplers within kernel: 16
Max size of kernel argument: 1024
Alignment (bits) of base address: 2048
Minimum alignment (bytes) for any datatype: 128
Single precision floating point capability
Denorms: No
Quiet NaNs: Yes
Round to nearest even: Yes
Round to zero: Yes
Round to +ve and infinity: Yes
IEEE754-2008 fused multiply-add: Yes
Cache type: Read/Write
Cache line size: 64
Cache size: 16384
Global memory size: 4277456896
Constant buffer size: 3019898880
Max number of constant args: 8
Local memory type: Scratchpad
Local memory size: 32768
Max pipe arguments: 0
Max pipe active reservations: 0
Max pipe packet size: 0
Max global variable size: 0
Max global variable preferred total size: 0
Max read/write image args: 0
Max on device events: 0
Queue on device max size: 0
Max on device queues: 0
Queue on device preferred size: 0
SVM capabilities:
Coarse grain buffer: No
Fine grain buffer: No
Fine grain system: No
Atomics: No
Preferred platform atomic alignment: 0
Preferred global atomic alignment: 0
Preferred local atomic alignment: 0
Kernel Preferred work group size multiple: 64
Error correction support: 0
Unified memory for Host and Device: 0
Profiling timer resolution: 1
Device endianess: Little
Available: Yes
Compiler available: Yes
Execution capabilities:
Execute OpenCL kernels: Yes
Execute native function: No
Queue on Host properties:
Out-of-Order: No
Profiling : Yes
Queue on Device properties:
Out-of-Order: No
Profiling : No
Platform ID: 0x7faf3cdff478
Name: Baffin
Vendor: Advanced Micro Devices, Inc.
Device OpenCL C version: OpenCL C 1.2
Driver version: 2442.7
Profile: FULL_PROFILE
Version: OpenCL 1.2 AMD-APP (2442.7)
Extensions: cl_khr_fp64 cl_amd_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_fp16 cl_khr_gl_sharing cl_amd_device_attribute_query cl_amd_vec3 cl_amd_printf cl_amd_media_ops cl_amd_media_ops2 cl_amd_popcnt cl_khr_image2d_from_buffer cl_khr_spir cl_khr_gl_event Wondering if anybody else with an RX Vega 64 was able to get amdgpu-pro's 17.30 ROCm 1.5 working with it or if it's just with me. Perhaps I'll give a shot with the open source ROCm with the new updated pci ids. Thanks again for all the help.
int03h
commented
7 years ago So .. the kernel drivers are enumerating it fine.
Everything else sees it, but opencl doesn't !?
Platform Name: AMD Accelerated Parallel Processing Number of devices: 2
Very strange! I would go open a ticket with AMD. NP. Glad I could help.(sorta). :)
dkgeorge
commented
7 years ago Yes, it is strange, and indeed I opened a ticket with AMD this morning pasting all this information. Could be amdgpu-pro's ROCm only supports Vegas FE at the moment. Hope they reply. Until then, back to Windows...
ducho
commented
7 years ago How can I compile sgminer aginst Vega 56 with rocm/libOpenCL ? libOpenCL.so.1 => /opt/rocm/opencl/lib/x86_64/libOpenCL.so.1
I tried to use this options
CFLAGS="-Os -Wall -march=native -I/opt/AMDAPPSDK-3.0/include" LDFLAGS="-L/opt/rocm/opencl/lib/x86_64" ./configure --disable-git-version --disable-adl --prefix=/opt/sgminer-5.5.5
but not result ist
ldd sgminer | grep libOpenCL libOpenCL.so.1 => /opt/amdgpu-pro/lib/x86_64-linux-gnu/libOpenCL.so.1 (0x00007f5cbcf8c000)
OhGodAPet
commented
7 years ago Cheap and dirty hack is to LD_PRELOAD the amd ROCm OCL shared lib specifically.
informer2016
commented
6 years ago @dkgeorge Could you please remember and tell, which CPU you have been using while obtaining the logs in your comment here - https://github.com/genesismining/sgminer-gm/issues/66#issuecomment-325765687 I just noticed some lines like
[ 1.315015] [drm] PSP loading UVD firmware
[ 1.317896] [drm] Found VCE firmware Version: 53.32 Binary ID: 4
[ 1.317910] [drm] PSP loading VCE firmwareand would like to see, whether this PSP (Platform Secure Processor) was a part of your modern AMD CPU - if you have been using it - or a part of your AMD RX Vega GPU
dkgeorge
commented
6 years ago Hello @informer2016 It's an Intel i5 Quad-Core CPU. The PSP information should relate to the GPU's drm. Below is the complete CPU info (pasting just 1 core for reference).
# cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 37
model name : Intel(R) Core(TM) i5 CPU 650 @ 3.20GHz
stepping : 5
microcode : 0x3
cpu MHz : 3192.295
cache size : 4096 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 2
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt aes lahf_lm tpr_shadow vnmi flexpriority ept vpid dtherm ida arat
bugs :
bogomips : 6384.59
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
Hi, I'm trying to mine using the new RX Vega 64 but it seems it's incompatible with sgminer-gm? Any insights will be much appreciated. Thanks!