Mac Vulkan & GL support tracking issue

baldurk commented 5 years ago

Description

NOTE: Metal or iOS support is not planned at the moment and I can't say anything about when I might look at it. Please keep discussion here to GL & Vulkan issues as that's the scope for an initial shipping version.

As found in the survey I ran at the end of 2018 there is a noticeable segment of users who strongly want macOS support. I've looked into what would be involved and done some initial work, but I have no familiarity with macOS so input and assistance from domain experts would be very helpful here.

This issue tracks any known problems or missing features on macOS, with an aim to stabilising it enough to ship in an official build. As of c289290ffa5956fdcaacb874a76ad83089749c86 the build should be considered early alpha: It works well enough for the basics on Vulkan, GL, etc, but certainly requires a lot more work before it ships. I currently have no planned timeline for when mac support would ship and it remains medium priority, so these things will get fixed as/when I have time.. As of 2020 the build is no longer stable as far as I know, either having bit rotted or having been broken by an OS update. Any fixes and contributions are very welcome but currently I have no plans to continue work on mac support.

Any help with any of the below - particularly the very mac specific issues like sorting out building and packaging - would be greatly appreciated.

[ ] The build needs significantly more testing and bugfixing. At the moment I've really only tried it on a handful of demo programs, so more real-world testing is desperately needed. If you have any non-trivial projects please share them so I can use them for testing.

This also just means testing the build for any weird UI bugs or strange behaviour on mac. E.g. I seem to sometimes get 'sticky' context menus where right clicking will pop one up but then left clicking will continue to pop it up. This might be a holdover of mac not really having right clicking as a very native concept?
[ ] On GL we always create a 4.1 core context whenever we need one. In theory we should work OK on 3.3 as well - is it necessary to detect what versions are available and downscale, or are 3.3 devices/OSs not relevant anymore?
[ ] Key input in the injected application for triggering captures is not supported yet, so the only way to trigger captures is via the UI. I'm not sure what library/input method is best for minimally-intrusive keyboard input in windows created by the application that I don't control.
[ ] High-DPI screens aren't handled properly in at least the GL overlay, we need to detect the DPI scaling factor and multiply up the detected window bounds and the size of the rendering.
[ ] High-DPI screens may also have issues with sizing widgets in the UI, and with rendering displays on at least GL and maybe Vulkan.
[ ] When launching .app folders we need to parse the Info.plist to determine the executable to launch.
[ ] A couple of key functions in GL rely on compute shaders which thus aren't implemented on macOS. Namely: min/max texture range, histogram, and mesh picking. The simplest workaround is just to emulate on the CPU and take the performance hit on that, but it's possible the min/max/histogram shaders could still run on the GPU with pixel shaders doing a smaller downsample pyramid.
[ ] Performance in the UI seems very slow for some reason. I don't know if it's because of my hardware though.
[ ] renderdoccmd has some stubs that need to find a way to create a window on apple to display previews.
[ ] Is there a minimum macOS version that should be tested against? what about minimum hardware support? Currently I only have one apple device - a mid-2013 macbook air running 10.14.3. This is probably close to the minimum end on hardware I'd guess but it's the latest OS.
[ ] Some dialog boxes in the Qt UI have a large question mark - I don't know if that's normal for the OS or if that's a missing resource/setup on my side to provide RenderDoc's icon.
[ ] Currently the Xcode generator in cmake doesn't produce a working build. It seems to run into issues with the GL/vulkan driver projects and doesn't build them correctly.

If there is a simple/minimal impact fix to get this working then that would be great, but since it compiles fine on the command line this isn't critical.
[ ] Some modern mac programs are likely to require OpenCL interop for compute, which would need at least minimal hooking to serialise data at sync points (similar to the DX interop).
[x] There seems to be an issue with MVK not getting initial contents of PREINITIALIZED linear images - which repros with vkcube. The memory has the right contents but the image isn't handling properly.
[x] Related to the above, there are GL warnings that fire when opening a capture that indicate 'degraded hardware support'. We should hardcode these to never fire on mac as the support is software related not hardware, and will never appear later so there's no point annoying users about it.
[x] macOS GL contexts are assumed to be core contexts, but it's valid (and the default) to create a legacy 2.1 context. We should be able to detect this from the attributes in a pixel format on creation
[x] There's no package build process for macOS.

At the moment the cmake build produces a qrenderdoc.app, but it also needs to package the vulkan loader and a MoltenVK build as backup, and also include android apps. I don't know if anything else is expected to be packaged in a macOS binary that's packaged up, and I don't know how such a package is typically distributed or constructed. Is there an equivalent of an .msi installer on macOS?
[x] At the moment I don't have any nightly builds set up for macOS, and it seems like doing this one way or another would require apple hardware running constantly. Possibly the travis builds could be repurposed to store their output on S3 for macOS, which can then be downloaded each night and have the packaging script run from linux instead.

To build the mac build, consult the information in CONTRIBUTING.md, in particular the dependencies. To build it is the same cmake build steps as on linux, running mkdir build && cd build && cmake -DCMAKE_BUILD_TYPE=Debug .. && make -j4.

GL capture should mostly work as expected out of the box. Of course bugs are likely and expected.

Vulkan capture will require a bit more setup. If you have vulkan registered with the system properly it should work the same as on other platforms - this means libvulkan.1.dylib should be in the dylib search path somewhere - I put it in ~/lib. Also the driver should be registered in ~/.local/share/vulkan/icd.d. You can also use VK_ICD_FILENAMES but be sure it's set when running qrenderdoc. The vulkan layer will be registered as on other platforms when running a loose build, which writes its json into ~/.local/share/vulkan/implicit_layer.d.

On recent mac vulkan SDKs the default install location from the install_vulkan.py script should be detected by similarly recent RenderDoc builds.

Environment

RenderDoc build: Latest HEAD code.
Operating System: macOS 10.14.3 (minimum spec to be determined)
API: Vulkan and OpenGL only

mcclure commented 5 years ago

Awesome!

Vague responses: OS X has a default installer format, .pkg, and there is an Apple tool for making the PKGs. PKGs can do things of arbitrary complexity including running shell scripts. However most people try to distribute a single .app. On OS X is a "Package" format which is not so much a format as a standard way of laying out directories. You put your executable inside the Package at a particular point and you also pack in all the libraries, resources, command-line/subprocess helper tools etc you need etc. If a directory follows the Package conventions and has a recognized extension such as .app, OS X will present it as a single file. Because you can fit most things an installer would normally install in a .app, usually people distribute OS X apps by just compressing a .app in a .dmg file.

significant-bit commented 5 years ago

For Blender 2.8 (now in beta) we chose OpenGL 3.3 and Mac OS 10.9 as min spec. The OS choice was mostly for better C++11 runtime. As a bonus that let us use GL 3.3 instead of 3.2.

uliwitness commented 5 years ago

Apple generally only supports current OS - 1, and important security updates may get back-ported into current - 2. So by default I'd say aim for current OS or at most one earlier, unless you have a good reason to support older.

uliwitness commented 5 years ago

Right clicking is a native concept on macOS X (only classic macOS had issues with that). The only oddity is that Ctrl-click gets automatically presented/handled as a right click by the OS. If you're encountering issues, it's unlikely that it is a problem of macOS itself.

If you're using Qt, Qt fakes its own contextual menus and re-implements a number of standard controls with its own code, so that's where I would look for the bug.

uliwitness commented 5 years ago

Not sure what you mean about key input. If these are Cocoa windows you created, you can implement -keyDown: or -interpretKeyEvents: or any of the other NSResponder key press methods.

If these are Qt windows, then I would recommend not injecting those into other applications (Qt is far from least-invasive on macOS). Instead, maybe you want to make your app an XPC Service and talk to that from the app you're injected in, so the least amount of work happens in the injected app. IOSurface might come in handy there as well, for sharing textures between apps.

If you need to intercept key events globally, look at https://github.com/Kentzo/ShortcutRecorder (particularly PTHotKey). If it's only for that one application, event monitors may be your thing. There is one local to the app (the global one needs special permissions).

baldurk commented 5 years ago

Yeh I assume a certain familiarity with how RenderDoc works when reading the list above. For key input I mean specifically being able to tell if a key is pressed from within a given process without owning any of the windows. That way when RenderDoc is injected I can tell if the user presses a hotkey to trigger the capture of a frame. So generally listening for keypresses process-wide - on Windows this uses GetAsyncKeyState so I'm looking for the closest equivalent to that.

uliwitness commented 5 years ago

I think the only API that lets you poll the keyboard would be the ancient GetKeys() API, and I'm not sure that made it to 64 bit. I'd recommend the event monitor target in CoreGraphics, in that case.

redorav commented 5 years ago

How else could we help to get MoltenVK running on renderdoc? I've managed to get as far as to compile my own version of renderdoc, install the Vulkan SDK, configure the layers, etc and launch the application through qrenderdoc. Once there, I took a capture and it loaded correctly. I was getting really excited at that point.

However, inspecting resources and vertex buffers and the like doesn't work properly. Would sending you example applications help? I could configure simple ones and send them over plus do testing, if that is of any help.

baldurk commented 5 years ago

Sending repros would help definitely, though I only spare limited time for mac issues so debugging them to find a likely cause and/or opening a PR with a fix would be even better.

For Vulkan I've tested a couple of Sascha Willems' samples, so in principle it should largely work, though I can certainly believe there will be MVK-specific bugs. I did run into some MoltenVK bugs I believe that I haven't yet narrowed down enough to report.

redorav commented 5 years ago

Yeah, the repro in my case is basically resources of any kind aren't visible on any drawcall, they're all black, including textures, render targets, vertex buffers, etc. All the rest like inspecting shaders, pipeline state, etc. seems to work great. Overlays also don't work properly (crash), but my guess is it's because it cannot read the vertex info. I'll prepare a capture showing the issues and you be the judge of whether it's something I could help with. I don't really know where to start as well given that I cannot attach XCode to put breakpoints.

baldurk commented 5 years ago

I've never used XCode with RenderDoc. One of the items in the list above is that XCode projects from cmake don't work, I don't know why, and even in other projects I've never been able to figure it out. I rarely debug on mac but the little I've done I used lldb directly. There's always printf debugging too :smile:.

Sending me a capture might be difficult. I only have one very old macbook air which I think has an Intel GPU, and I'm not sure if MoltenVK captures will be compatible between different GPUs. I can certainly try though.

redorav commented 5 years ago

I've sent you an email :)

cmartel commented 4 years ago

On GL we always create a 4.1 core context whenever we need one. In theory we should work OK on 3.3 as well - is it necessary to detect what versions are available and downscale, or are 3.3 devices/OSs not relevant anymore?

I can quantify this a little as I've been working on a macOS GL application that's been deployed on a range of different macs.

A little unintuitively, if you request a 4.1 core context, the Mac driver will give you such a context even if the hardware doesn't support it. In other words, you get a software emulated context.

Conversely, if you request a 3.3 context and the hardware supports 4.1, it will silently upgrade you to a 4.1 context.

Lastly, if you use share contexts internally, macOS not only refuses to share between 3.3 and 4.1 contexts, but it'll also refuse to share between a request 4.1 context and a 4.1 context that was upgraded from 3.3.

So yeah, a 4.1 context will work in all cases, but might considerably slower than a 3.3 context if the hardware doesn't support it. Of course, for replaying captures, that's not as important as a realtime application so that might still be an acceptable fallback. 🙃

baldurk commented 4 years ago

That's good to know, thanks! It does suggest that perhaps creating a 4.1 context on the assumption that the large majority of users of RenderDoc will have the capability would be a solution. Do you know how to detect if a 4.1 context is being emulated? It does sound like it'd be harder to fallback to 3.3 (which we could do at the risk of losing some features) if we wanted to prioritise a hardware context.

cmartel commented 4 years ago

If I recall correctly, the GL_RENDERER string shows up as "Apple Software Renderer": https://feedback.wildfiregames.com/report/opengl/device/Apple%20Software%20Renderer

This is the renderer information from some tests I ran a while back: API OpenGL Renderer Apple Software Renderer Vendor Apple Inc. Version 4.1 APPLE-16.7.4 GLSL Version 4.10

Since we need context sharing with our Qt application's own contexts, the implementation I went for basically tries sharing using a 4.1 context, tries again with 3.3 if that doesn't work.

StarWitch commented 4 years ago

Hi all. I checked out the branch v1.x today and gave it a spin. Here are some of the issues I encountered:

compiling ../../qrenderdoc/Widgets/CustomPaintWidget.cpp
../../qrenderdoc/Widgets/CustomPaintWidget.cpp:122:16: error: member access into incomplete type 'QEvent'
  if(m_Ctx && e->type() == QEvent::UpdateRequest)

Missing include of #include <QEvent> for the APPLE section. Adding it fixes this.

compiling ../../qrenderdoc/Windows/Dialogs/PerformanceCounterSelection.cpp
../../qrenderdoc/Windows/Dialogs/PerformanceCounterSelection.cpp:441:44: error: 'toList' is deprecated: Use values()
      instead. [-Werror,-Wdeprecated-declarations]
      SetSelectedCounters(selectedCounters.toList());

Changing this to values() works as expected.

The application will successfully compile, load, and even capture an OpenGL 3.3 application that I'm working on to improve performance on macOS. However, I'm getting an error whenever I attempt to load an *.rdc file from the capture.

%  bin/qrenderdoc.app/Contents/MacOS/qrenderdoc ~/Downloads/test.rdc
[1]    31306 illegal hardware instruction  bin/qrenderdoc.app/Contents/MacOS/qrenderdoc ~/Downloads/test.rdc

This also happens whenever I try to load a file generated on another machine (like a Windows box). I can confirm the file captured on the Mac is valid because I loaded it on said Windows box and it parses just fine.

This applies to the renderdoccmd command as well, and attempting to replay over the network doesn't work either. This error seems to occur whenever a capture log file is loaded.

Maybe this is an issue with how macOS 10.15 doesn't support 32-bit anymore? Just a guess.

System: macOS 10.15.3 (Xcode 11.3) Qt 5.14.1 MacBook Pro 15" (mid-2018) AMD Radeon 560X

baldurk commented 4 years ago

The missing QEvent header is definitely a real problem, I'll fix that. It's strange that it doesn't show up on CI but it might be a Qt version difference - the CI is running on Qt 5.13.

The deprecation warning should not be appearing - I have deprecation notices turned off globally in Qt (QT_NO_DEPRECATED_WARNINGS) because many of them deprecate in favour of things added in Qt versions above our base (5.6). Meaning it's impossible to have code that compiles on old Qt versions as well as avoid things deprecated in newest versions. You'd need to investigate further to see why that is showing up at all.

As to the error you're encountering, I don't know what is going wrong there. I did previously have GL capturing and replaying working on simple examples, but last time I tested it that seemed to have been broken by an OS update. Since mac is not a supported platform I will not be investigating any bugs that only happen on mac. I'd welcome any contributions from yourself or anyone else to fix issues like this, but this issue is here to track my progress (currently stalled) and give a reference point for anyone who wants to help.

StarWitch commented 4 years ago

OK, thanks for letting me know. I'll give it a shot, although low-level stuff like this isn't my strong suit. (And if anyone else reading this has any better ideas, you should probably just go for it.)

albertofem commented 4 years ago

Hi there,

I've been trying renderdoc on Mac OS X using the 1.x branch. All seems to be working fine, but when I try to capture a frame, the following crash happens:

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000000008
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Segmentation fault: 11
Termination Reason:    Namespace SIGNAL, Code 0xb
Terminating Process:   exc handler [82421]

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   librenderdoc.dylib              0x0000000105c41b12 WrappedVulkan::SubmitAndFlushImageStateBarriers(BarrierSequence<VkImageMemoryBarrier>&) + 322
1   librenderdoc.dylib              0x0000000105d5f86c WrappedVulkan::Prepare_InitialState(WrappedVkRes*) + 29548
2   librenderdoc.dylib              0x0000000105c4810e ResourceManager<VulkanResourceManagerConfiguration>::PrepareInitialContents() + 542
3   librenderdoc.dylib              0x0000000105c4755e WrappedVulkan::StartFrameCapture(void*, void*) + 4174
4   librenderdoc.dylib              0x000000010640ac2c RenderDoc::StartFrameCapture(void*, void*) + 44
5   librenderdoc.dylib              0x0000000105c4efc1 WrappedVulkan::Present(void*, void*) + 193
6   librenderdoc.dylib              0x00000001060b86dd WrappedVulkan::vkQueuePresentKHR(VkQueue_T*, VkPresentInfoKHR const*) + 7901
7   libvulkan.1.dylib               0x0000000106bc2b05 vkQueuePresentKHR + 53
8   chippuhachi                     0x0000000105568e3f glfwvulkan::imgUiFramePresent() + 175 (glfwvulkan.cpp:813)
9   chippuhachi                     0x0000000105567939 glfwvulkan::run(system*) + 5961 (glfwvulkan.cpp:377)
10  chippuhachi                     0x0000000105574397 emulator::run() + 167 (emulator.cpp:10)
11  chippuhachi                     0x000000010555ca22 main + 82 (main.cpp:6)
12  libdyld.dylib                   0x00007fff5fff23d5 start + 1

Any idea as to why this could happen? The application otherwise runs fine. I'll try to debug a little bit tomorrow to provide more information as well. Thanks!

baldurk commented 4 years ago

I can't suggest anything off the top of my head, that callstack could mean a lot of things you'd need to debug it to be sure. It's worth double checking that your program doesn't have any vulkan validation layer warnings or errors as they can cause problems.

It's worth noting that as above the mac build is not currently supported so I won't be investigating bugs that only reproduce on mac. If you can reproduce your problem on windows or linux I can look into it.

cmartel commented 4 years ago

It looks like commit feafbde8 introduced a crash on macOS GL capture replay. Reverting that commit works around the crash.

While creating the NSOpenGLContext, the [context update] call ends up calling glGetIntegerv internally. But because glGetIntegerv is replaced with the interposed function, it calls the CheckImplicitThread before the CGL driver is fully initialized, and crashes on a null pointer execution.

Call stack for the crashing thread looks like:

0   librenderdoc.dylib              0x000000010cb91ad0 WrappedOpenGL::CheckImplicitThread() + 16 (gl_driver.cpp:998)
1   librenderdoc.dylib              0x000000010cf28328 glGetIntegerv_renderdoc_hooked(RDCGLenum, int*) + 56 (gl_hooks.cpp:151)
2   librenderdoc.dylib              0x000000010cf284eb interposed_glGetIntegerv + 27 (gl_hooks.cpp:151)
3   com.apple.AppKit                0x00007fff391415a1 -[NSOpenGLContext(NSOpenGLContextInternal) _updateOpenGLViewport] + 91
4   com.apple.AppKit                0x00007fff39141c1e -[NSOpenGLContext update] + 408
5   librenderdoc.dylib              0x000000010d0d625a NSGL_createContext + 634 (cgl_platform.mm:69)
6   librenderdoc.dylib              0x000000010d0d850e CGLPlatform::MakeOutputWindow(WindowingData, bool, GLWindowingData) + 814 (cgl_platform.cpp:196)
7   librenderdoc.dylib              0x000000010cb6ec5a GLReplay::MakeOutputWindow(WindowingData, bool) + 250 (gl_outputwindow.cpp:215)
8   librenderdoc.dylib              0x000000010e16e301 ReplayOutput::ReplayOutput(ReplayController*, WindowingData, ReplayOutputType) + 545 (replay_output.cpp:116)
9   librenderdoc.dylib              0x000000010e16e580 ReplayOutput::ReplayOutput(ReplayController*, WindowingData, ReplayOutputType) + 64 (replay_output.cpp:127)
10  librenderdoc.dylib              0x000000010e189fa6 ReplayController::CreateOutput(WindowingData, ReplayOutputType) + 854 (replay_controller.cpp:1798)
11  org.renderdoc.qrenderdoc        0x000000010b196898 BufferViewer::OnCaptureLoaded()::$_22::operator()(IReplayController*) const + 104 (BufferViewer.cpp:2303)
12  org.renderdoc.qrenderdoc        0x000000010b196820 void std::__1::__invoke_void_return_wrapper<void>::__call<BufferViewer::OnCaptureLoaded()::$_22&, IReplayController*>(BufferViewer::OnCaptureLoaded()::$_22&&&, IReplayController*&&) + 80 (__functional_base:350)
13  org.renderdoc.qrenderdoc        0x000000010b196719 std::__1::__function::__func<BufferViewer::OnCaptureLoaded()::$_22, std::__1::allocator<BufferViewer::OnCaptureLoaded()::$_22>, void (IReplayController*)>::operator()(IReplayController*&&) + 57 (functional:1562)
14  org.renderdoc.qrenderdoc        0x000000010acab384 std::__1::function<void (IReplayController*)>::operator()(IReplayController*) const + 132 (functional:1913)
15  org.renderdoc.qrenderdoc        0x000000010acaaf42 ReplayManager::run(int, QString const&, ReplayOptions const&, std::__1::function<void (float)>) + 1858 (ReplayManager.cpp:482)
16  org.renderdoc.qrenderdoc        0x000000010acaf283 ReplayManager::OpenCapture(QString const&, ReplayOptions const&, std::__1::function<void (float)>)::$_0::operator()() const + 115 (ReplayManager.cpp:55)
17  org.renderdoc.qrenderdoc        0x000000010acaf1fd void std::__1::__invoke_void_return_wrapper<void>::__call<ReplayManager::OpenCapture(QString const&, ReplayOptions const&, std::__1::function<void (float)>)::$_0&>(ReplayManager::OpenCapture(QString const&, ReplayOptions const&, std::__1::function<void (float)>)::$_0&&&) + 45 (__functional_base:350)
18  org.renderdoc.qrenderdoc        0x000000010acaeda9 std::__1::__function::__func<ReplayManager::OpenCapture(QString const&, ReplayOptions const&, std::__1::function<void (float)>)::$_0, std::__1::allocator<ReplayManager::OpenCapture(QString const&, ReplayOptions const&, std::__1::function<void (float)>)::$_0>, void ()>::operator()() + 41 (functional:1562)
19  org.renderdoc.qrenderdoc        0x000000010acac636 std::__1::function<void ()>::operator()() const + 102 (functional:1913)
20  org.renderdoc.qrenderdoc        0x000000010acabb94 LambdaThread::process() + 68 (QRDUtils.h:339)
21  org.renderdoc.qrenderdoc        0x000000010acac5ae QtPrivate::FunctorCall<QtPrivate::IndexesList<>, QtPrivate::List<>, void, void (LambdaThread::*)()>::call(void (LambdaThread::*)(), LambdaThread*, void**) + 126 (qobjectdefs_impl.h:152)
22  org.renderdoc.qrenderdoc        0x000000010acac503 void QtPrivate::FunctionPointer<void (LambdaThread::*)()>::call<QtPrivate::List<>, void>(void (LambdaThread::*)(), LambdaThread*, void**) + 99 (qobjectdefs_impl.h:186)
23  org.renderdoc.qrenderdoc        0x000000010acac416 QtPrivate::QSlotObject<void (LambdaThread::*)(), QtPrivate::List<>, void>::impl(int, QtPrivate::QSlotObjectBase*, QObject*, void**, bool*) + 166 (qobjectdefs_impl.h:415)
24  org.qt-project.QtCore           0x0000000110658d0b QMetaObject::activate(QObject*, int, int, void**) + 2219
25  org.qt-project.QtCore           0x0000000110460868 QThread::started(QThread::QPrivateSignal) + 56
26  org.qt-project.QtCore           0x0000000110461059 QThreadPrivate::start(void*) + 297
27  libsystem_pthread.dylib         0x00007fff63b29661 _pthread_body + 340
28  libsystem_pthread.dylib         0x00007fff63b2950d _pthread_start + 377
29  libsystem_pthread.dylib         0x00007fff63b28bf9 thread_start + 13

Null checking glhook.driver before calling CheckImplicitThread on it also works as a naive fix, but it might be a bit too much so. Without a full understanding of the thread switching issue that required the addition of that context tracking functionality, I can't be certain it won't break anything... But as far as I can tell, the only time when glhook.driver would be unexpectedly null is before/during context creation, and I should hope that the CGL implementation doesn't change the context in the middle of this.

Should it be a good enough fix to commit, I've setup pull request #1920 to save you the 30 seconds of typing. ;)

vladimir-nazarenko commented 3 years ago

I was able to run and explore vkcubeapp of Vulkan SDK, but renderdoc API seems to fail to initialize. I'm using the following code (from Renderdoc documentation) to initialize on Linux and MacOS (works just fine on Linux):

rdoc_api_ = NULL;
int ret = 0;

if (void *mod = dlopen("librenderdoc.so", RTLD_NOW | RTLD_NOLOAD))  {
    pRENDERDOC_GetAPI RENDERDOC_GetAPI = (pRENDERDOC_GetAPI) dlsym(mod, "RENDERDOC_GetAPI");
    ret = RENDERDOC_GetAPI(eRENDERDOC_API_Version_1_1_2, (void **) &rdoc_api_);
}

Is there a known issue on that? What the cause may be and can I do something about it?

baldurk commented 3 years ago

I'm not aware of any differences between the platforms but perhaps OS X has a different dynamic symbol API than linux and dlopen/dlsym doesn't work as needed for that code snippet. As mentioned above, mac is not a supported platform for RenderDoc so you'd have to investigate yourself and contribute any fixes that are necessary.

JannikGM commented 3 years ago

@vladimir-nazarenko

if (void *mod = dlopen("librenderdoc.so", RTLD_NOW | RTLD_NOLOAD))  {

This feels incorrect for MacOS because that'd likely be librenderdoc.dylib. It also assumes that the file is in the linker path (which is a different linker from GNU, so it also uses other environment variables / paths). There might also be some variation in the supported dlopen arguments, but I didn't check. Do you have some repository where people can actually test and debug your snippet?

cmartel commented 3 years ago

I've been able to use the RenderDoc API on macOS, under certain conditions. As @JannikGM mentions, the mac library extension is .dylib:

    void* mod = dlopen("librenderdoc.dylib", RTLD_NOW | RTLD_NOLOAD);

    if (mod != nullptr)
    {
        RENDERDOC_GetAPIFunc = reinterpret_cast<pRENDERDOC_GetAPI>(dlsym(mod, "RENDERDOC_GetAPI"));
        libraryLoaded = true;
    }

The other caveat is that I haven't been able to get it to pick up RenderDoc's dynamic library by merely adding it alongside my application's binaries. There must be some nuance to macOS library paths that I'm missing.

Instead, just starting my executable through RenderDoc's API will make the dynamic library available to dlopen and dlsym and behave correctly.

HiImJulien commented 3 years ago

The other caveat is that I haven't been able to get it to pick up RenderDoc's dynamic library by merely adding it alongside my application's binaries.

@cmartel This likely due the configured search path. Does it work, if you try dlopen("./librenderdoc.dylib", RTLD_NOW | RTLD_NOLOAD) instead? (Assuming, that the dylib is inside the current working directory of the application)

vladimir-nazarenko commented 3 years ago

@JannikGM thank you for your advice, as @cmartel points out, it was really sufficient to just use the correct dynamic library name. By the way, could someone please point me out to the mechanism/part of renderdoc code, that allows to, I guess, dynamically manipulate the rpath of an application for it to be able to load the renderdoc shared library? Really curious how it works under the hood.

hrydgard commented 3 years ago

I tried running RenderDoc on an M1 Macbook Air, but getting the following output, trying to debug OpenGL graphics of an ARM binary:

I assume that's expected at this point? Rosetta might not be able to handle cross-arch API hooking...

> file /Applications/RenderDoc.app/Contents/bin/renderdoccmd
/Applications/RenderDoc.app/Contents/bin/renderdoccmd: Mach-O 64-bit executable x86_64

baldurk commented 3 years ago

As mentioned above RenderDoc does not support mac in any way, so there is no expectation that it works regardless of architecture or API.

I'm closing this issue since it seems to falsely gives the impression that RenderDoc development is active on mac, and it doesn't seem currently that there's much chance of external contributions or fixes to support mac meaningfully.

hrydgard commented 3 years ago

Ah, fair enough! Having Mac builds available as Nightlies on the website kind of gave that impression too.

baldurk commented 3 years ago

Yes, while I never released official stable builds or listed mac as a supported platform the nightly builds being there is slightly misleading, so I've removed them now also. I kept them there after my initial work for the use of anyone who wanted to be able to contribute and have official builds for testing, but that never happened sadly.

Lord-Kamina commented 3 years ago

Yes, while I never released official stable builds or listed mac as a supported platform the nightly builds being there is slightly misleading, so I've removed them now also. I kept them there after my initial work for the use of anyone who wanted to be able to contribute and have official builds for testing, but that never happened sadly.

This kinda sucks, honestly... I had been using these builds and issues as they might have had, they did help me out quite a bit sometimes.

mcclure commented 3 years ago

For the record: I'm still using a Mac copy of RenderDoc I nabbed at some point in the last year and a half. I don't use it to test Mac. I use it to deploy the renderdoc Android debugger to Oculus Quest and view traces from the mac, this works great. I'd consider renderdoc for mac a useful tool even if it couldn't attach to mac executables at all.

This said I'm on an older MacOS version and so if someone's going to keep RenderDoc up to date with Apple's ever changing OS and hardware rules I can't help with that.

baldurk commented 3 years ago

Yeh I understand that some people were using some of the nightly builds, as I say it was somewhat alpha quality on my machine when I left it off but it seems to have bitrotted incredibly quickly judging by some of the reports from people. The problem is that I tend to agree with the earlier comment that it's quite misleading to have nightly builds listed without it being very obvious that they're completely unsupported and liable to be entirely broken. I didn't want to set up a completely different section to make clear that mac is unsupported, and also from experience sadly many people would ignore any simple warnings.

Similarly if someone gets a build they should expect all features to work and not for it to only work in certain circumstances. Even the android workflow still requires a certain amount of rendering code to run on the local machine for proxying textures, buffers etc, and of course all the UI is still running on mac.

The source hasn't changed and for the moment I'm happy to keep mac in the CI rotation to make sure it stays compiling, so you can always build yourself from source if you need a mac build and are aware of the level of support.

Unfortunately after doing the initial prototyping work on mac the platform proved just too unfriendly/unpleasant to develop for - especially on GL - so I dropped plans to work on it myself.

manwithsteelnerves commented 2 years ago

For the record: I'm still using a Mac copy of RenderDoc I nabbed at some point in the last year and a half. I don't use it to test Mac. I use it to deploy the renderdoc Android debugger to Oculus Quest and view traces from the mac, this works great. I'd consider renderdoc for mac a useful tool even if it couldn't attach to mac executables at all.

This said I'm on an older MacOS version and so if someone's going to keep RenderDoc up to date with Apple's ever changing OS and hardware rules I can't help with that.

Can you share the build? I don't find it anywhere and have the same use-case as yours (to test on android devices).

LostPeter commented 6 months ago

I built MacOS version successfully, but when launch app, there is showing: "API none" and can not capture! !

baldurk commented 6 months ago

Please do not comment on old issues that have been closed.

Mac is not supported in RenderDoc at all.

baldurk / renderdoc

Mac Vulkan & GL support tracking issue #1272

Description

Environment