Open ThomasAlban opened 1 year ago
Possibly. I'm unfamiliar with the signing process and unsure if the brew libraries we link against require the system to have certain packages installed. If we can figure that out and add it to CI, I see no reason why not to.
That sounds great. I just tried to follow the steps to build Riistudio for M1 Mac (I have an M1 Pro Macbook Pro), but when I ran this command:
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=/opt/homebrew/bin/clang++
I got this error:
CMake Error at CMakeLists.txt:3 (project):
The CMAKE_CXX_COMPILER:
/opt/homebrew/bin/clang++
is not a full path to an existing compiler tool.
Tell CMake where to find the compiler by setting either the environment
variable "CXX" or the CMake cache entry CMAKE_CXX_COMPILER to the full path
to the compiler, or to the compiler name if it is in the PATH.
Not sure what to do to fix this, as I'm not very knowledgable about CMake and C++! I'd really love to be able to use Riistudio natively on my Mac without having to use a Windows VM.
I think it's not seeing clang++
from brew? Were you able to install the llvm
package?
yeah, when I run brew install llvm
it gives me a message saying it is already installed and up-to-date. I checked the /opt/homebrew/bin folder and there is no file called 'clang++'. I ran the command which installed all the brew packages and everything seemed to install fine.
Perhaps it's being installed to a different directory? E.g. llvm
's own bin
subdirectory?
Yes this was the problem, I found the clang++ file in the llvm bin directory and changed the file pathway to this.
If it helps anyone, on Apple Silicon, the pathway is:
/opt/homebrew/Cellar/llvm/18.1.8/bin/clang++
(or replace 18.1.8
with whichever version number it is)
I ran the rest of the commands, and they all seemed to work. Having run all the commands where is the final build located? I can't seem to find it to run Riistudio and see if everything worked.
It should be in the frontend
subdirectory in your build folder! Also keep in mind the latest Assimp version has some regressions, so the Windows build stays at a known good version. Otherwise the MacOS build should be basically feature-complete. :)
Here's what I have in my build folder, there is no frontend
subdirectory so something must have gone wrong.
This might be the problem:
/Users/thomasalban/Downloads/RiiStudio/source/librii/objflow/ObjFlow.cpp:100:40: error: no member named 'split' in namespace 'ranges::views'
for (auto word : std::ranges::views::split(param.AssetsListing, ";")) {
~~~~~~~~~~~~~~~~~~~~^
1 error generated.
make[2]: *** [source/librii/CMakeFiles/librii.dir/objflow/ObjFlow.cpp.o] Error 1
make[1]: *** [source/librii/CMakeFiles/librii.dir/all] Error 2
make: *** [all] Error 2
I just worked around that; pulling from upstream should do the trick.
Just ran git pull and ran the cmake commands again, now there is this error:
[ 85%] Linking CXX executable rszst
clang-15: error: no such file or directory: '/opt/homebrew/Cellar/freetype/2.13.0/lib/libfreetype.dylib'
make[2]: *** [source/cli/rszst] Error 1
make[1]: *** [source/cli/CMakeFiles/cli.dir/all] Error 2
make: *** [all] Error 2
Try pulling again--should be fixed. I bumped the project's freetype
version to match latest on brew
.
Unfortunately I'm still getting the same error, the only difference is the version number of freetype
:
[ 85%] Linking CXX executable rszst
clang-15: error: no such file or directory: '/opt/homebrew/Cellar/freetype/2.13.0_1/lib/libfreetype.dylib'
make[2]: *** [source/cli/rszst] Error 1
make[1]: *** [source/cli/CMakeFiles/cli.dir/all] Error 2
make: *** [all] Error 2
Just tried running brew install freetype
and I get this message but the version number is different, is this the issue?
Warning: freetype 2.12.1 is already installed and up-to-date.
To reinstall 2.12.1, run:
brew reinstall freetype
brew update
I believe. It seems your package definitions are out of date?
Turns out I had some weird issue with brew update
where it would just hang, but after some googling running brew update-reset
fixed it for me. After doing that it was a success and I got it working!! Thank you for this wonderful piece of software :)
is there a way to get it to run without the terminal opening up? And can I install it as though it were any other piece of software in the 'Applications' folder of my Mac?
Thanks! To be honest, I'm not sure. I've never really touched Mac packaging, and I don't know how that would interact with the current GitHub auto-updater. I'll leave this as an open issue to track that.
Hi! I just wanted to ask how do i launch riistudio on mac?, i've installed it following the instructions, i had a few issues but i was able to complete the installation without errors, still i dont know how to launch it 🙈
Hi, you would open the executable in build/source/frontend/RiiStudio
. Let me know if that works for you!
Hey, are you still planning on doing this? Unfortunately I know little about CMake, but I have built MacOS apps with Rust before so I do have some info that might be helpful that I'll just dump here:
A MacOS .app
file is just a directory, this stack overflow post was really helpful for understanding its structure.
So you should be able to automate the creation of that once you have an executable. I used this really helpful tool called cargo-bundle to do this for me.
In terms of the signing process, it seems like you need an Apple developer account to sign the app costing $100 a year (which is ridiculous ofc). Apple will add a quarantine flag to any app intalled which isn't signed meaning the app can't be opened, but the user can get around this by running xattr -d com.apple.quarantine path/to/binary
.
I added a very hacky Rust wrapper for the RiiStudio app, which if adapted for MacOS should allow deployment with cargo-bundle itself. I'll look into this a bit more later--though I am not enthused about the $100/yr developer subscription.
Oh cool. About the $100 a year thing, you could just not sign the apps then ask users to run the xattr
command when they download it, which isn't ideal but would be better than nothing.
I'm happy to help test anything!
I've been trying to build Riistudio again, but am running into more issues (probably me not understanding how to do this properly!). I pulled the latest changes and ran brew upgrade
and everything, before running:
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=/opt/homebrew/Cellar/llvm/16.0.6/bin/clang++
(changing the path to clang++ as before so that it works), this first command seems to work fine. However running cmake --build . --config Release --parallel
produces a huge amount of errors, the end of which being:
/Applications/RiiStudio/source/oishii/../core/common.h:73:11: error: redefinition of 'views' as different kind of symbol
namespace views {
^
/opt/homebrew/Cellar/llvm/16.0.6/bin/../include/c++/v1/__ranges/views.h:29:11: note: previous definition is here
namespace views = ranges::views;
^
1 error generated.
gmake[2]: *** [source/LibBadUIFramework/CMakeFiles/LibBadUIFramework.dir/build.make:146: source/LibBadUIFramework/CMakeFiles/LibBadUIFramework.dir/RichNameManager.cpp.o] Error 1
1 error generated.
1 error generated.
gmake[2]: *** [source/LibBadUIFramework/CMakeFiles/LibBadUIFramework.dir/build.make:104: source/LibBadUIFramework/CMakeFiles/LibBadUIFramework.dir/Plugins.cpp.o] Error 1
gmake[2]: *** [source/oishii/CMakeFiles/oishii.dir/build.make:118: source/oishii/CMakeFiles/oishii.dir/util/util.cxx.o] Error 1
gmake[2]: *** Waiting for unfinished jobs....
1 error generated.
gmake[2]: *** [source/oishii/CMakeFiles/oishii.dir/build.make:160: source/oishii/CMakeFiles/oishii.dir/writer/node.cxx.o] Error 1
1 error generated.
gmake[2]: *** [source/LibBadUIFramework/CMakeFiles/LibBadUIFramework.dir/build.make:76: source/LibBadUIFramework/CMakeFiles/LibBadUIFramework.dir/ActionMenu.cpp.o] Error 1
1 error generated.
1 error generated.
gmake[2]: *** [source/LibBadUIFramework/CMakeFiles/LibBadUIFramework.dir/build.make:118: source/LibBadUIFramework/CMakeFiles/LibBadUIFramework.dir/PropertyView.cpp.o] Error 1
gmake[2]: *** [source/oishii/CMakeFiles/oishii.dir/build.make:132: source/oishii/CMakeFiles/oishii.dir/writer/binary_writer.cxx.o] Error 1
1 error generated.
gmake[2]: *** [source/LibBadUIFramework/CMakeFiles/LibBadUIFramework.dir/build.make:132: source/LibBadUIFramework/CMakeFiles/LibBadUIFramework.dir/Reflection.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:1710: source/LibBadUIFramework/CMakeFiles/LibBadUIFramework.dir/all] Error 2
1 error generated.
gmake[2]: *** [source/oishii/CMakeFiles/oishii.dir/build.make:146: source/oishii/CMakeFiles/oishii.dir/writer/linker.cxx.o] Error 1
1 error generated.
gmake[2]: *** [source/oishii/CMakeFiles/oishii.dir/build.make:104: source/oishii/CMakeFiles/oishii.dir/reader/binary_reader.cxx.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:1618: source/oishii/CMakeFiles/oishii.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2
I'd really appreciate some help on fixing this, thanks!
Is this on a Sonoma preview build?
No I'm on the latest version of Ventura still
Thanks for spotting that. I've fixed the build on clang-15 on Ventura, though have not validated clang-16 yet.
unfortunately I'm still getting errors:
[ 98%] Linking CXX executable tests
default.blight: Success
Warning: Material data is compressed. (in MAT3)
ReverseGravity2DLiftPlanet.bdl: Success
posteffect.blmap: Success
default.blmap: Success
Error: ai_json.dae: Rebuild does not match!
--> Expected: a76072344956a6036e78757def00a998
--> Actual: 2917d80cefb7c992817eaf245e4e297e
Error: Failed to read MDL0 course: (in BRRES)
Error: old_town_ds.brres: Rebuild does not match!
--> Expected: 5908868b3d9c0f083ad3c23a1396d4ed
--> Actual: ef3b76c1c9534de7c9eb52ad1150b8ae
ReverseGravity2DRoofActionPlanet.bdl: Success
rszst(25165,0x16e347000) malloc: *** error for object 0x45e87540c58a62ad: pointer being freed was not allocated
rszst(25165,0x16e347000) malloc: *** set a breakpoint in malloc_error_break to debug
b'RiiStudio CLI Alpha 5.10.8 (Hotfix 1) (Built Jul 4 2023 at 22:16:20, Homebrew Clang 16.0.6)\nProcessing... [> ] 0 %\rExcluding tangents/bitangents\nExcluding boneweights\nExcluding animations\nExcluding lights\nExcluding cameras\nOptimizing meshes 0/0 [> ] 0 %\rOptimizing meshes (0 / 22) [> ] 0 %\r' None -6
Error: course.dae failed to rebuild
/Applications/RiiStudio/build/source/cli/rszst import-command /Applications/RiiStudio/source/tests/../../tests/samples/course.dae /Applications/RiiStudio/source/tests/../../tests/out/course.dae
Error: tests.py encountered a critical error
Traceback (most recent call last):
File "/Applications/RiiStudio/source/tests/../../tests/tests.py", line 189, in <module>
run_tests(sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4])
File "/Applications/RiiStudio/source/tests/../../tests/tests.py", line 180, in run_tests
run_test(test_exec, rszst, in_file, out_file)
File "/Applications/RiiStudio/source/tests/../../tests/tests.py", line 146, in run_test
rebuild(test_exec, rszst, path, rebuild_path, md5 == expected, bps)
File "/Applications/RiiStudio/source/tests/../../tests/tests.py", line 131, in rebuild
raise RuntimeError(err)
RuntimeError: None
gmake[2]: *** [source/tests/CMakeFiles/tests.dir/build.make:138: source/tests/tests] Error 1
gmake[2]: *** Deleting file 'source/tests/tests'
gmake[1]: *** [CMakeFiles/Makefile2:1843: source/tests/CMakeFiles/tests.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2
There seems to be an issue with the Apple std::expected
in this configuration. I'm falling back to tl::expected
to fix that until the feature stabilizes more.
Master should build now, although still with some tests diverging (for instance, the MacOS build is on a much newer version of Assimp than Windows for now (affecting the .dae output)).
thanks it worked! looks like a few of the tests failed though as you said
I've added a MacOS build to the latest release (Apple Silicon). Note that it's neither signed nor compatible with the auto-updater.
I'll see about integrating trust at some point to automatically publish MacOS builds when new git tags are created.
I just downloaded the build, unzipped, right clicked and opened it, but got this error: Looking into the log, this is the reason:
Termination Reason: Namespace DYLD, Code 1 Library missing
Library not loaded: /Users/*/Documents/*/libgctex.dylib
Referenced from: <A6D96C6C-7FF9-3AF6-8BC5-D57B77D37F2E> /Applications/RiiStudio.app/Contents/MacOS/rust_bundle
Reason: tried: '/Users/*/Documents/*/libgctex.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/*/Documents/*/libgctex.dylib' (no such file), '/Users/*/Documents/*/libgctex.dylib' (no such file), '/usr/local/lib/libgctex.dylib' (no such file), '/usr/lib/libgctex.dylib' (no such file, not in dyld cache)
(terminated at launch; ignore backtrace)
I am not sure why those are being linked as dylibs when my rustc flags explicitly declare them staticically.
println!("cargo:rustc-link-search=native={}", cargo_dir);
println!("cargo:rustc-link-lib=static=gctex");
println!("cargo:rustc-link-lib=dylib=llvm_sighandler");
println!("cargo:rustc-link-lib=static=avir_rs");
println!("cargo:rustc-link-lib=static=riistudio_rs");
println!("cargo:rustc-link-lib=static=szs");
println!("cargo:rustc-link-lib=static=c_wbz");
println!("cargo:rustc-link-lib=static=c_discord_rich_presence");
I'm also noticing that the brew deps are dylib references too.
I'll see what I can do. It would certainly be my preference to ship a single static binary.
I am not sure why those are being linked as dylibs when my rustc flags explicitly declare them staticically.
println!("cargo:rustc-link-search=native={}", cargo_dir); println!("cargo:rustc-link-lib=static=gctex"); println!("cargo:rustc-link-lib=dylib=llvm_sighandler"); println!("cargo:rustc-link-lib=static=avir_rs"); println!("cargo:rustc-link-lib=static=riistudio_rs"); println!("cargo:rustc-link-lib=static=szs"); println!("cargo:rustc-link-lib=static=c_wbz"); println!("cargo:rustc-link-lib=static=c_discord_rich_presence");
I'm also noticing that the brew deps are dylib references too.
I'll see what I can do. It would certainly be my preference to ship a single static binary.
That's weird... any progress on this? I noticed you removed the build from the latest release which makes sense haha. Until github releases Apple Silicon macos runners I'm assuming you'd have to add the build yourself which wouldn't be ideal
The current build on tip should have shed most of these dependencies, although a few remain:
[..]
0000000100C68A68 _glfwSetScrollCallback /opt/homebrew/opt/glfw/lib/libglfw.3.dylib
[..]
0000000100C68AF8 _FT_Done_Face /opt/homebrew/opt/freetype/lib/libfreetype.6.dylib
[..]
0000000100C68B58 Assimp::Intern::AllocateFromAssimpHeap::operator delete(void *) /opt/homebrew/opt/assimp/lib/libassimp.5.dylib
[..]
0000000100C68C00 std::__1::__fs::filesystem::recursive_directory_iterator::__dereference(void) /usr/lib/libc++.1.dylib
That is, for a published build to work at the moment, you'd need to have installed glfw
/freetype
/assimp
via brew
which is certainly not ideal.
I'll update this issue if I solve that.
Hiya, just tried to build Riistudio from source again and got a whole bunch of errors which all seem to centre around calls to the function lexicographical_compare_three_way
, e.g.:
In file included from /Users/thomasalban/Documents/Code/C++/RiiStudio/source/librii/rhst/RHST.cpp:1:
/Users/thomasalban/Documents/Code/C++/RiiStudio/source/librii/rhst/RHST.hpp:74:10: error: call to 'lexicographical_compare_three_way' is ambiguous
74 | return std::lexicographical_compare_three_way(
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@riidefi Is there any progress on including macos .app
files in the releases section? It looks like github actions now supports M1 as well as intel, so including both would be ideal!
In other news, trying to compile on Apple SIlicon gives a whole barrage of errors:
You should also probably update the README.md
to reflect the fact that on Apple Silicon, running this command:
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=/opt/homebrew/bin/clang++
doesn't work. You need to change it to
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=/opt/homebrew/Cellar/llvm/18.1.8/bin/clang++
and potentially change 18.1.8
to whatever the version is.
Is there any progress on including macos .app files in the releases section? It looks like github actions now supports M1 as well as intel, so including both would be ideal!
I can try to set that up. Hopefully the 10x minute multiplier doesn't burn through too many credits lol
The current build on tip should have shed most of these dependencies, although a few remain:
[..] 0000000100C68A68 _glfwSetScrollCallback /opt/homebrew/opt/glfw/lib/libglfw.3.dylib [..] 0000000100C68AF8 _FT_Done_Face /opt/homebrew/opt/freetype/lib/libfreetype.6.dylib [..] 0000000100C68B58 Assimp::Intern::AllocateFromAssimpHeap::operator delete(void *) /opt/homebrew/opt/assimp/lib/libassimp.5.dylib [..] 0000000100C68C00 std::1::fs::filesystem::recursive_directory_iterator::__dereference(void) /usr/lib/libc++.1.dylib That is, for a published build to work at the moment, you'd need to have installed glfw/freetype/assimp via brew which is certainly not ideal.
I'll update this issue if I solve that.
I saw this--hopefully should be pretty easy to make the .app standalone / ready for distribution.
In other news, trying to compile on Apple SIlicon gives a whole barrage of errors:
Should be fixed now.
ok just tried agian with your changes and it got so close but failed at the linking stage:
[ 97%] Linking CXX executable rszst
ld: warning: ignoring duplicate libraries: '-lSystem', '../core/libcore.a', '../librii/liblibrii.a', '../oishii/liboishii.a', '../plate/libplate.a', '../rsl/librsl.a', '../vendor/libvendor.a'
ld: Undefined symbols:
std::__1::__is_posix_terminal(__sFILE*), referenced from:
rsmeshopt::RingIterator<unsigned long>::RingIterator(unsigned long, std::__1::span<unsigned long const, 18446744073709551615ul>) in librsmeshopt.a[6](2e40c9e35e9506f4-RsMeshOpt.o)
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [source/cli/rszst] Error 1
make[1]: *** [source/cli/CMakeFiles/cli.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 98%] Linking CXX static library liblib_frontend.a
[ 98%] Built target lib_frontend
make: *** [all] Error 2
That bundle_dylibs
thing looks great, really exciting that this could finally work!
Does your auto updater work using github releases? Would be cool if we could auto update on mac as well. Also just curious, as for my own project I'm going to have to implement an auto updater and am wondering how you've done it!
Mhm that's because I edited rsmeshopt
without bumping the version number just yet (so a cached build is being linked). Should be fine on a clean build!
Does your auto updater work using github releases?
Yes! It's kind of bespoke and there are probably more standard libraries out there, though. Code here.
That bundle_dylibs thing looks great, really exciting that this could finally work!
I added it to CI and it's building an ostensibly standalone .app file, although I have yet to verify on a system without those libraries installed.
Let me know if that app file works in any case.
Thanks, just successfully built and bundled myself a Riistudio .app
and it seems to work!
However, downloading and running the RiiStudio app output from your CI gives me this:
(even after running the xattr
command on it to remove any apple related restrictions)
so something must be broken there...
Okay, thank you. That was pretty annoying to fix but I think the latest CI build should work, though needs xattr and chmod'ing--certainly not ideal.
💀
(latest CI build)
I tried doing differnet things recommended here, by default I do open the zip with the unarchiver (it says archive utility sometimes breaks things and I have experienced this before), but it could be something to do with how the file's being zipped as I know it's quite easy to break a macos app this way. It might be better to include the .app
file directly, or make a dmg
with something like this (I don't have experience using this tho).
I suppose once you can get it working, then the next step is for me to uninstall all your dependencies and check it still works.
In terms of xattr
and chmod
, I'm pretty sure an alternative to running the xattr
command is right clicking and opening the app, rather than double clicking. This then gives you a dialogue with a button 'open anyway' which does the same thing as xattr
(I only learnt this recently!). Looks like in the CI you're running a chmod
command? So hopefully you'll just be able to tell users to right click and open and this is all they'll need to do.
To be clear, I had to run xattr
/chmod
on the rust_bundle
executable in the MacOS
subfolder directly, not just on the .app itself. I can look into dmg
support though.
Can confirm that when installing from .dmg, just xattr -d com.apple.quarantine RiiStudio.app
is sufficient.
Yeah, this worked for me! It seems like the .app
file included in the zip file is still broken, but installing from a dmg then running the above command works!
seems like my 'right click and open' trick doesn't work for this annoyingly though.
I also just uninstalled all the brew dependencies and the app still runs!
Thanks for all your work on getting this working! Will you now be able to add this to the releases section and maybe get the auto update functionality working? Not sure how that would work with the dmg though...
Nice! Thanks for testing that. Yea I think you'd need to xattr
the rust_bundle
executable directly for the former to work. It wouldn't be much effort to include these builds in the release section, though I'd worry many would gloss over the xattr
step and assume they're nonfunctional.
Not sure about the auto-updater. On Windows I rely on being able to rename a running executable--don't know if I can do that with a .app. Presumably command-line installing a DMG wouldn't be too much effort, though.
I reckon it's still worth including it in the releases section, you could add a note at the bottom of each release with the CI
telling users to run xattr -d com.apple.quarantine /Applications/RiiStudio.app
in the teminal once installing, probably saying why it's necessary so people don't worry about typing in a scary looking command. This could alternatively be in the readme if you don't want to add it to releases. If users are techy enough to be wanting to use RiiStudio then I would hope they can run a command in the command line! There won't be large numbers of MacOS users anyway, but for those that use MacOS it should be on them to read the instructions haha.
I'm happy to look into MacOS auto updaters, in fact I'll have to do this soon anyway as I'm planning on adding this to my project. I'm happy to let you know when I find out how auto updaters should work on MacOS and whether you can rename running apps or if there's another way of doing it.
@riidefi Ok, I've looked into implementing auto updaters, the rust crate self_update seems to have a nice implementation. In terms of replacing the file in-place, it relies on the crate self_replace. There is an explanation of how the replacement works cross-platform here. It actually seems like self replacement is a lot easier to implement on unix systems than windows! Here is their unix implementation. The one thing I'm not sure of is whether this works with .app
files as well as executables.
Another place to look for inspiration could be the dolphin auto updater which is cross-platform. Here is an overview of how it works. Looks like it doesn't update in-place though, but uses a seperate Dolphin Updater process to perform the actual update.
Any chance of adding a Mac Intel and M1 Build to the releases section?