search

KhronosGroup / MoltenVK

MoltenVK is a Vulkan Portability implementation. It layers a subset of the high-performance, industry-standard Vulkan graphics and compute API over Apple's Metal graphics framework, enabling Vulkan applications to run on macOS, iOS and tvOS.
Apache License 2.0
4.83k stars 428 forks source link

Official Homebrew tap? #509

Closed yurikoles closed 1 year ago

yurikoles commented 5 years ago

There is a tap rafaga/r2k by @rafaga, he is mostly done bring-up of MoltenVK formula with all dependencies. Is there any chance creating official tap, or at least mention of unofficial tap in readme?

billhollings commented 5 years ago

@yurikoles Thanks for pointing this out.

@rafaga Thanks for taking the initiative! I have some questions:

Pinging @karl-lunarg & @lenny-lunarg regarding SDK homebrew.

yurikoles commented 5 years ago

vulkan-sdk was removed from Caskroom, https://github.com/Homebrew/homebrew-cask/pull/52782

rafaga commented 5 years ago

Thanks @yurikoles, for metioning the repo in this thread I was trying to put in contact with KhronosGroup for a couple of questions related to the repos, and this maybe give me some answers.

@yurikoles Thanks for pointing this out.

@rafaga Thanks for taking the initiative! I have some questions:

* Are you intending this to be a published and maintained homebrew installation?

Yes that is the intention, my principal intention is to keep the formulas in this repo until i submit them for inclusion in homebrew-core, I know that some of them never will be approved to thier inclusion in homebrew-core, and these formulas would be keep here as-is, and yes maintaining the formulas in the possibilities of my time.

* Is the purpose of this to install the entire MoltenVK development repo into `/usr/local`...or just the MoltenVK compiled libraries (ie- the stuff in the `Package/Latest` directory? I should think that someone using `homebrew` to install MoltenVK would be after the compiled libraries.

I can't just make MoltenVK libraries available, it has a lot of dependencies that need to be installed and compiled separated as standalone formulas. trying to just compile MoltenVK it is same that just download the tarball and follow the build instruction including the execution of the fetchDependencies script, and that's not my goal. 

* In that case...have you considered the Vulkan SDK? It is the existing "official" mechanism for installing a Vulkan environment on macOS. Ideally...it would be the subject of a `homebrew` installation (although it doesn't cover iOS yet).

I didn't know that was the official mechanism to get Vulkan. So I'll check it later and try to use it instead of the current Vulkan-headers vulkan-tools formulas.

the official homebrew-core rules doesn't allow to submit formulas that compile or provides dependency for other plataforms including iOS, but if there is sufficent interest to bring iOS support I can take the task, maybe the iOS related stuff would be keep in 3rd party tap (like my repo).

* The formulas seem to recreate a lot of build logic that is already contained within MoltenVK and its existing `fetchDependencies` and `Makefile` scripts. That feels like it will make this fairly fragile. Is there a reason why the homebrew formula couldn't just leverage the existing tools and execute a Git clone, then `fetchDependencies` and `make`?

Ok I'm agree with that is not a very robust approach but there is a lot of reasons behind this:

As context: I would like to give the official instance about the use of fetchDependecies script

in this issue https://github.com/Homebrew/homebrew-core/issues/26951#issuecomment-383852055 one the maintainers of homebrew-core explains why homebrew formula can't rely on the fetchDependencies script (that is maybe the official posture about the problem), but the problem using fetchDependecies is more complicated.

system "./fetchDependencies" is not acceptable. We do not allow formulas to download things outside of resource blocks, to allow for Homebrew caching mechanism to operate.

system "lipo" is also not acceptable, we want to provide macOS software. Please drop the iOS part of the build.

Ok, one the first versions that i have made of the MoltenVK formula used resources to download each one of the repositories of the dependencies, but i noted that each one of these rosources could work as a formula as itself, I still needed to heavily edit the makefiles to make it works, and as separate package would be useful to more developers, so i tried to replicate the build logic to install the built libraries in the expected paths . This will help anyone using the headers and libraries independent if they want to use MoltenVk or not.

but doing this work i have noted some ambiguities in the way these files compiles. One example: if clone the SPIRV-Tool and follows the build instruction it makes an static library named libSPIRV-Tools.a, libSPIRV-Tools-comp.a, libSPIRV-Tools-link.a, libSPIRV-Tools-opt.a, libSPIRV-Tools-reduce.a and just one Shared library named libSPIRV-Tools-Shared.dylib , but if i run ./fetchDependencies just build libSPIRV-Tools.a, libSPIRV-Tools-shared.dylib. At this point I don't know if the fectchDependencies just build a subset of SPIRV-Tools or everything is compiled in one static library.

later I noted there is a inconsistency in the naming of the libraries. Why libSPIRV-Tools is uppercase and spirv-cross don't. there is any reasoning behind this? i know it is a detail but this would be a problem in filesystem with case sensibility enabled.

another problem is that xcodebuild it is a terrible tool to build something from CLI, and currently i having a problem linking MoltenVK library because the linker has a -lglslang flags that appears of the nowhere

I would like if there is oportunity to fix some of these issues with the help and expertise of the KhronosGroup team, to help to improve the experience of making homewbrew formulas or just making easier to other developers to adopt the libraries using homebrew or another package managers, in the long term fixing some of these problems would help to make more predictable the use of the libraries, keeping the consistency in the naming and build process across the whole set of Vulkan libraries.

I hope this answer would open a door to adress all these details and the to improve the overall experience for everyone invloved . And sorry for the wall of text.

Pinging @karl-lunarg & @lenny-lunarg regarding SDK homebrew.

rafaga commented 5 years ago

I have checked vulkansdk-macos and are the same libraries that I trying to implement , Comparing the sdk to rafaga/r2k tap this is the current status

Using the vulkan-sdk tarball will be more problematic because the archieve is stripped from build scripts, the tarball contains the sourcecode and the compiled libraries and nothing else. with that situation i will need to create the build scripts from zero and thats not a good idea, knowing that I not involved too much in the Vulkan development process. So to keep the things simple, I'll stick with the git repos and the recomended commits.

That should have the result as compiling from zero the sdk.

I would like to Know if there is any comments about what i have posted until now.

billhollings commented 5 years ago

@rafaga

Do you need the full MoltenVK distribution?

Have you thought about including only the compiled MoltenVK libraries in the homebrew formula? By that I mean just the built files that appear in the MoltenVK/Package/Release directory.

yurikoles commented 5 years ago

@billhollings Do you mean a binary package? What is purpose of such a formula? One possible solution is to have vulkan-sdk Caskroom tap, but it still will have problems that is mentioned in official repo.

yurikoles commented 5 years ago

Oh, there is cask tap already: apenngrace/vulkan. It can be installed by brew cask install apenngrace/vulkan/vulkan-sdk

billhollings commented 5 years ago

Do you mean a binary package? What is purpose of such a formula?

Yes...I mean a binary package of MoltenVK. It's purpose would be to install the MoltenVK libraries that a developer could use to develop their game.

For example...the Vulkan SDK consists only of binary libraries...so any Vulkan app/game dev using the SDK is already only working with binary libraries. This would be the same...but for the MoltenVK binary libraries only...without the rest of the full SDK binary libraries.

rafaga commented 5 years ago

Oh, there is cask tap already: apenngrace/vulkan. It can be installed by brew cask install apenngrace/vulkan/vulkan-sdk

good catch @yurikoles, well that would cover the propourse of the rafaga/homebrew-r2k.

The problem that i can see is that he is installing the libraries outside the system paths like /usr/lib /usr/local/lib etc. the .pc files are missing from sdk distribution.

Making use of the built-in build script will help solving some of these problems but maybe I'm overthinking this and need another perpective.

Don't know maybe making minor revision to this Cask would solve the problem.

apenngrace commented 5 years ago

Hi @rafaga, I wrote the apenngrace/vulkan tap.

It sounds like you have some questions about it. I am open to collaborating and improving the cask if something can be improved.

The problem that i can see is that he is installing the libraries outside the system paths like /usr/lib /usr/local/lib etc. the .pc files are missing from sdk distribution.

Most of the files are installed to /usr/local/lib, /usr/local/bin, /usr/local/include/vulkan, and /usr/local/share/vulkan. I am wondering why you consider those outside of the standard MacOS system paths. The reason why files are placed there (instead of /etc/ for example) is because they can be installed there without sudo privileges. Those are some of the directories that the Vulkan Loader looks for files. Last August I discussed this with Karl Schultz from LunarG in this thread on the LunarG forum.

The .pc files are for pkg-config, right? Which ones are missing?

Regarding the rafaga/r2k, are you working with LunarG? Or are you trying to reverse engineer the build system for vulkan? I am curious if LunarG changed their mind about open-sourcing their build script for vulkan.

In that LunarG thread I mentioned earlier, Karen Ghavam from LunarG told me:

The only piece that is not open source is the script to build the macOS SDK.

yurikoles commented 5 years ago

Hi @apenngrace,

this thread on the LunarG forum.

Seems to be a link to open issues list.

apenngrace commented 5 years ago

@yurikoles, sorry you're right. I updated my last comment to fix those links.

rafaga commented 5 years ago

It sounds like you have some questions about it. I am open to collaborating and improving the cask if something can be improved.

The problem that i can see is that he is installing the libraries outside the system paths like /usr/lib /usr/local/lib etc. the .pc files are missing from sdk distribution. The .pc files are for pkg-config, right? Which ones are missing?

It's not your fault, I'm not criticizing you, I'm just saying that SDK distribution comes without the pc files, but they are present in some of the source repos. As you are using the sdk release well is not possible to auto-generate such files, but i Don't know if generating the .pc files manually (is not that difficult) from the sdk, would bring some hidden side-effects.

Regarding the rafaga/r2k, are you working with LunarG? Or are you trying to reverse engineer the build system for vulkan? I am curious if LunarG changed their mind about open-sourcing their build script for vulkan.

In that LunarG thread I mentioned earlier, Karen Ghavam from LunarG told me:

Hi, I'm not related with LunarG in any way, I'm just an "freelance" developer that want to improve something. I don't have an account in LunarG site.

I just tried to edit the build files to make my best effort to follow the guidelines of homebrew with the objetive that the formulas could be approved in homebrew-core repo and have an automated way to build all the libraries and not depending solely on the binary files

The only piece that is not open source is the script to build the macOS SDK.

I didn't know that the build scripts are not open source (that explains a LOT of things). I hope that my edits not infringe something, if something does, please just notify me and i would delete it.

As addional comment i haven't touched my repo in a month because i have so much work to-do lately and because your formula does the job.

I have found too much problems in the build process, I would think because the I'm not involved in the Vulkan-sdk development process. If you need something I would glad to help, i made forks from some repos to create patches and improve the build scripts in part of the repos.

As i said before I just want to make my best effort to improve something and give back something to the open source community

yurikoles commented 5 years ago

@rafaga it seems like you had sent two similar comments.

rafaga commented 5 years ago

@yurikoles sorry, my connection is unstable .... fixed

rafaga commented 5 years ago

Just to clarify I'm still want to continue my effort to bring MoltenVK and all the Vulkan libraries to homebrew, I just don't want to duplicate efforts or working in a methond that goes to a dead-end.

With this I'm refering to the method that I'm following, It doesn't make sense to invest more time in something taht doesn't have any real benefit, And I prefer to help in your formula, if this helps to reach the goal. We can open an iusse on your/my repo or via Hangouts/Skype to discuss this.

KaoCC commented 5 years ago

Just want to share the information that there's already one formula here: molten-vk

One can utilize it by extending the following CMake template:

cmake_minimum_required(VERSION 3.10)
project(TestVK)
set(CMAKE_CXX_STANDARD 17)
add_executable(TestVK test.cpp)

set(SDK_PATH "/usr/local/Cellar/molten-vk/1.0.34")

set(Vulkan_INCLUDE_DIRS "${SDK_PATH}/include")
set(Vulkan_INCLUDE_DIRS_Extra "${SDK_PATH}/libexec/include")
set(Vulkan_LIBRARY_DIRS "${SDK_PATH}/lib")

target_include_directories(TestVK PUBLIC ${Vulkan_INCLUDE_DIRS} ${Vulkan_INCLUDE_DIRS_Extra})
target_link_directories(TestVK PUBLIC ${Vulkan_LIBRARY_DIRS})
target_link_libraries(TestVK MoltenVK)

C++ Code for testing:

#include <iostream>
#include <vulkan/vulkan.h>

int main(int argc, const char * argv[]) {
    VkInstance instance;
    VkResult result;
    VkInstanceCreateInfo info = {};

    result = vkCreateInstance(&info, NULL, &instance);
    std::cout << "vkCreateInstance result: " << result  << "\n";

    vkDestroyInstance(instance, nullptr);
    return 0;
}
KaoCC commented 2 years ago

Note that conan , the C++ package manager, also supports moltenvk

After installing conan, one can add the followings to the conanfile.txt 

[requires]
moltenvk/1.1.4

[generators]
cmake

And CMake:

cmake_minimum_required(VERSION 3.10)
project(TestVK)
set(CMAKE_CXX_STANDARD 17)

include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup()

add_executable(TestVK vk.cpp)

target_link_libraries(TestVK  ${CONAN_LIBS})

As an example for building, one can do the followings in the project folder:

This might be the easiest way to use MoltenVK

rcoreilly commented 2 years ago

It would be great if there was an official maintained homebrew cask for vulkan-sdk!

I followed the discussion here: https://github.com/Homebrew/homebrew-cask/issues/51330 and the README here: https://github.com/apenngrace/homebrew-vulkan -- it sounds like maintaining this install process is significantly more complex than the user experience of clicking on the installer program... My naive impression was that homebrew cask is just a way to run the standard install program from the CLI. The standard install seems to put the links into /usr/local/* as you'd expect. What am I missing?

apenngrace commented 2 years ago

Yeah, once they put that install script in their package, I stopped updating that homebrew cask.

Before then, you had to rely on environment variables and was really annoying when they had point updates. That was the motivation to create the cask back then.

As for homebrew, they would not allow vulkan to be part of their repo as a cask for whatever reason. They wanted to be able to build from source since vulkan has source code repos on github. Last time I looked, the vulkan build process was not entirely open source.

On Jun 2, 2022, at 3:39 AM, rcoreilly @.***> wrote:

 It would be great if there was an official maintained homebrew cask for vulkan-sdk!

I followed the discussion here: Homebrew/homebrew-cask#51330 and the README here: https://github.com/apenngrace/homebrew-vulkan -- it sounds like maintaining this install process is significantly more complex than the user experience of clicking on the installer program... My naive impression was that homebrew cask is just a way to run the standard install program from the CLI. The standard install seems to put the links into /usr/local/* as you'd expect. What am I missing?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

qcksilver commented 1 year ago

Has there been any updates about this?

I just tried running brew install molten-vk and it seemed to work. (https://formulae.brew.sh/formula/molten-vk and https://github.com/Homebrew/homebrew-core/blob/master/Formula/molten-vk.rb)

But after I installed it I found this thread and now I'm wondering if shouldn't have installed moltenvk through homebrew?

What is the best way of checking if it installed everything ok? There is no "Moltenvk" folder in my user folder, am I wrong in thinking that it should be there if it was installed correctly?

I'm on a M1 Mackbook Pro with MacOS 13.2 and xcode 14.2

yurikoles commented 1 year ago

The official tap is unneeded, since it's already available in a homebrew/core.

yurikoles commented 1 year ago

@qcksilver MoltenVK is intended for software developers. If you can't even find a copy that you installed from Homebrew, then you probably don't need it.