Open alexhaugland opened 7 months ago
im confused, im already using moonlight with sunshine on my windows 11 pc and it works great. what does this introduce for vision os?
im confused, im already using moonlight with sunshine on my windows 11 pc and it works great. what does this introduce for vision os?
the biggest thing for me is window management - if you're using the ipad app compatibility mode you're pretty restricted in how you can resize the window
there's some secondary issues around UX that can be improved as well - the swipe to go back gesture is difficult / impossible to do with your eyes, the settings menu target is hard to hit, the individual settings are a bit crowded, etc
Just played with this on my Vision Pro. Super excited to see where this goes. Thanks for your work. I was able to load up my Sunshine server just fine but am unable to access the settings menu in Moonlight to tweak anything.
Just played with this on my Vision Pro. Super excited to see where this goes. Thanks for your work. I was able to load up my Sunshine server just fine but am unable to access the settings menu in Moonlight to tweak anything.
The settings menu does not exist yet - I'll implement it eventually but PRs are welcome :D
Ah! Well, that would make sense as to why it isn't working for me then. I'm no developer or I'd jump on helping out. Is there any chance you know the variables I could manually edit in Xcode to change the resolution? Thanks again.
the bottom of MainViewModel
holds the hard-coded settings
ah got it. is this also what it takes to remove the black bars on the top and bottom when casting?
ah got it. is this also what it takes to remove the black bars on the top and bottom when casting?
yeah pretty much
have a testflight or anything? i can share it around and have beta testers
have a testflight or anything? i can share it around and have beta testers
i might set one up once the settings menu works but no promises
This is a fantastic work that will also make it easier to rebuild the UI of the other platforms with SwiftUI, something I was investigating doing a few months ago but gave up on quickly.
The only issue I'm seeing with your fork is that my gaming PC is being discovered 3 times (so 3 sidebar entries). This may be because it has 3 IPs (IPv4, IPv6 and Tailscale which both AVP and my gaming PC have enabled)?
This is a fantastic work that will also make it easier to rebuild the UI of the other platforms with SwiftUI, something I was investigating doing a few months ago but gave up on quickly.
The only issue I'm seeing with your fork is that my gaming PC is being discovered 3 times (so 3 sidebar entries). This may be because it has 3 IPs (IPv4, IPv6 and Tailscale which both AVP and my gaming PC have enabled)?
Yeah, that's a known bug. It's probably related to how it compares TemporaryHost
s for equality.
Just want to say that I tried this on my Vision Pro and it works extremely well. The only change that I made was that I hacked in an option for 90 Hz streaming. Thank you so much for doing this.
oh, nice! @alexhaugland is making some amazing progress. all i’m waiting for is custom resolutions, custom bit rates and 90hz and this would be 100% usable for me as is. can’t wait to be able game on a 100” ultra wide display.
have a testflight or anything? i can share it around and have beta testers
i might set one up once the settings menu works but no promises
Oh yes, please!!! Would love to test this out via TestFlight!!!!
Thanks for this work! Trying to build and deploy with xcode. Do I need to change the bundle identifier? I think it's having issues installing/upgrading because I already have iPad version from App Store. Do I need to delete that first before installing onto my device, or can I have both so I can compare performance?
my build is failing too in xcode
working fine here... make sure to follow all the steps in the instructions and ensure you used --recursive when cloning the visionOS repo and branch, or after switching to the visionOS branch and updating the submodules if you forgot --recursive while cloning.
nvm, got it to build by deleting the old xcode version, and using only xcode beta 3
Working for me now, thanks, probably didn't clone properly the first time. On visionos
branch on github, I clicked code > Open with xcode, closed xcode, opened Terminal, cd
'd to moonlight-ios
dir, then git submodule update --init --recursive
to finish those linked dirs, opened in xcode, Targets > Moonlight Vision, Signing & Capabilities, Team: My Name, then changed Bundle identifier, built and launched on my vision pro within seconds.
Will continue to update this as updates are pushed, thanks!
The settings screen kind of works now - I'm waiting on review for a testflight link.
@MetricExpansion I merged your changes (minus some log lines) into the branch. Thanks!
Will you post the link to the TestFlight here once it is approved? I’m dying to try this out!
From: alexhaugland @.> Sent: Sunday, February 11, 2024 2:06:59 PM To: alexhaugland/moonlight-ios @.> Cc: VeniceNerd @.>; Comment @.> Subject: Re: [alexhaugland/moonlight-ios] WIP visionOS support (PR #1)
@MetricExpansionhttps://github.com/MetricExpansion I merged your changes (minus some log lines) into the branch. Thanks!
— Reply to this email directly, view it on GitHubhttps://github.com/alexhaugland/moonlight-ios/pull/1#issuecomment-1937739421, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AF7W2M4TIKIBWHS5IR7KWBLYTC67HAVCNFSM6AAAAABCOFTDQGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZXG4ZTSNBSGE. You are receiving this because you commented.Message ID: @.***>
Thank you so much for all of the hard work on this. After the recent updates, things are looking really amazing! Definitely a big step up from using the iPad app.
iPad App (20ft away & max size):
VisionOS App (20ft away & max size):
Latest build is looking awesome! Got flawless 3ms stream at 4k/90 HDR, with a screen that filled my entire wall. Thanks @alexhaugland and @MetricExpansion!
Here's a testflight link for the latest build: https://testflight.apple.com/join/4eE59dyH
already working so good, great job on this. are you using a real avp to test btw?
bug report: the bitrate setting doesn't save, i set it low, connect, disconnect, and its back to 200
already working so good, great job on this. are you using a real avp to test btw?
bug report: the bitrate setting doesn't save, i set it low, connect, disconnect, and its back to 200
yeah, I've got a physical device
technically the bitrate setting would save but it would be blown up when you reopened the settings menu :D - I've uploaded a fixed build, lmk if that works for you
Looks great so far! When using the magic trackpad (or a mouse with proper assistive touch settings) it seems like right click does not work, with or without mouse support enabled.
bug report: the bitrate setting doesn't save, i set it low, connect, disconnect, and its back to 200
@Geczy Most likely, the setting is actually getting saved and applied. It's just that the Settings UI is doesn't reload the setting when you go to look at it. Try setting the bitrate, framerate, or resolution super low and it will be noticeable.
@alexhaugland the problem is the use of @State
to store the index of the resolution, framerate, and bitrate settings. You should probably either
tag
values to actually just be settings values themselves and bind directly to the settings
model (as is already being done for the other settings), orBinding
where the get
and set
closures convert the index to/from the setting value the settings
model.Either way, you can also then dispense with the @State
variables and the use of the onChanged
modifier. I've attached a patch with the first solution.
moonlight-ios-c796c95-Fix settings roundtrip for resolution, framerate, and bitrate.patch
bug report: the bitrate setting doesn't save, i set it low, connect, disconnect, and its back to 200
@Geczy Most likely, the setting is actually getting saved and applied. It's just that the Settings UI is doesn't reload the setting when you go to look at it. Try setting the bitrate, framerate, or resolution super low and it will be noticeable.
@alexhaugland the problem is the use of
@State
to store the index of the resolution, framerate, and bitrate settings. You should probably either
- change the
tag
values to actually just be settings values themselves and bind directly to thesettings
model (as is already being done for the other settings), or- change them to each to be a
Binding
where theget
andset
closures convert the index to/from the setting value thesettings
model.Either way, you can also then dispense with the
@State
variables and the use of theonChanged
modifier. I've attached a patch with the first solution.moonlight-ios-c796c95-Fix settings roundtrip for resolution, framerate, and bitrate.patch
That's a lot cleaner, thanks. I've applied the patch and am uploading another build.
Here's a patch that adds more resolution options, to cover many more aspect ratios. Might still be good to have a custom option at some point.
Here's a patch that adds more resolution options, to cover many more aspect ratios. Might still be good to have a custom option at some point.
I'm not sure how to best do it so I've been avoiding it. I don't love the way the current apps have a popup when you select it. The way the visionOS settings app works is pretty nice for a mixture of default and custom options on a subpage of each option, but I'm not sure if there's a quick and easy way to do it.
Here's one way to do it, if you like it. Just two text fields, but also a menu to select from pre-defined options.
I don't entirely love it - most people probably want to pick 1080p/4k so that should be as easy as possible - but I won't let that stop the people from having custom resolutions for now 😁
That's totally fair. Ideally this would be like a few common resolutions plus a "Custom" option, which then reveals the text fields. Not too hard to implement, but a little more fuss than I was feeling like going through.
Another concept, maybe more in line with what you're thinking? Put the resolution options into a submenu instead of a popup menu, and add an ornament to enter in the custom resolution. Also remove some options since they can now be entered manually. Patch is attached if you like it.
moonlight-ios-7359549-Custom resolution options in submenu.patch
Yeah, I think that's a lot closer!
Thank you for this. This is infinitely better than running the iPad app on the AVP. I do notice a slight amount of input lag/delay, which is undetectable when I stream my PC to my Steam Deck or other devices. I wonder if it’s a software or hardware limitation.
Thank you for this. This is infinitely better than running the iPad app on the AVP. I do notice a slight amount of input lag/delay, which is undetectable when I stream my PC to my Steam Deck or other devices. I wonder if it’s a software or hardware limitation.
I've noticed the same thing - it got a bit better when I switched to a dualsense controller but I would not be surprised if we're losing at least one frame of latency due to the compositor. What refresh rate are you using?
just curious, does vision os dev have hot module reloading like a js environment? or do you have to rebuild each time and relaunch
just curious, does vision os dev have hot module reloading like a js environment? or do you have to rebuild each time and relaunch
You've gotta rebuild/relaunch but it's relatively quick. SwiftUI has a preview functionality for iterating on just UI that has hot reloading in the editor.
Thank you for this. This is infinitely better than running the iPad app on the AVP. I do notice a slight amount of input lag/delay, which is undetectable when I stream my PC to my Steam Deck or other devices. I wonder if it’s a software or hardware limitation.
I've noticed the same thing - it got a bit better when I switched to a dualsense controller but I would not be surprised if we're losing at least one frame of latency due to the compositor. What refresh rate are you using?
I’m capping the frame rate at 90 frames per second. I’ve been testing with an Xbox controller and keyboard/mouse, both connected to my hardwired PC. Streaming over WiFi 6e. I would estimate an added 20ms of input lag, compared to my Steam Deck OLED and LG OLED C3.
@alexhaugland @MetricExpansion you both are amazing. A little over a week and we've got a full featured AVP app. Thank you so much!
@anthonyjutchenko noticed the same but in my testing, this is also present in the iPad compatible version on the AVP. are you seeing the same?
i cnnect to a desktop, and take the headset off. any way to keep the desktop connected? is it intentional that it disconnects and closes the window? i want to put the headset back on and resume my desktop mirroring unless i explicitly close it. this is the way most apps work from what i can tell
@anthonyjutchenko noticed the same but in my testing, this is also present in the iPad compatible version on the AVP. are you seeing the same?
Yes, the same amount of input lag is present with the iPad app.
The TestFlight version behaves differently for me than the one I build and install via xcode. Same version (latest). I deleted the xcode version on my Vision Pro to just use Testflight for OTA updates, and it would not recognize my local IP. Says no machine found, see if Sunshine is running correctly (it is). Then when I enter my public ip using the [::ffff:x.x.x.x] workaround (what you have to do on all ios versions), it does recognize the machine, Sunshine receives a pin request, I enter the pin code, Sunshine web gui says success, but on Moonlight, the pin instructions disappear, and it still says Start Pairing for yet another pin request, then same behavior endlessly. I cannot pair my machine whether lan or wan. I went back to installing the xcode version, and I can enter in my local ip, and pairing is successful on first try.
Anyone else seeing this with the Testflight version?
i cnnect to a desktop, and take the headset off. any way to keep the desktop connected? is it intentional that it disconnects and closes the window? i want to put the headset back on and resume my desktop mirroring unless i explicitly close it. this is the way most apps work from what i can tell
AIUI it must disconnect when you remove the headset since the app is being put to sleep but in theory it could reconnect when the app is resumed - PRs welcome :D
The TestFlight version behaves differently for me than the one I build and install via xcode. Same version (latest). I deleted the xcode version on my Vision Pro to just use Testflight for OTA updates, and it would not recognize my local IP. Says no machine found, see if Sunshine is running correctly (it is). Then when I enter my public ip using the [::ffff:x.x.x.x] workaround (what you have to do on all ios versions), it does recognize the machine, Sunshine receives a pin request, I enter the pin code, Sunshine web gui says success, but on Moonlight, the pin instructions disappear, and it still says Start Pairing for yet another pin request, then same behavior endlessly. I cannot pair my machine whether lan or wan. I went back to installing the xcode version, and I can enter in my local ip, and pairing is successful on first try.
Anyone else seeing this with the Testflight version?
does the app store ipad version work with the same public IP?
@anthonyjutchenko noticed the same but in my testing, this is also present in the iPad compatible version on the AVP. are you seeing the same?
Yes, the same amount of input lag is present with the iPad app.
I actually think my issue isn't so much related to input lag as it is this: https://github.com/moonlight-stream/moonlight-ios/issues/607. Looking at the statistics overlay, everything should be good but the stream is stuttering constantly. Happening on both this and the iPad version on the AVP. Also occurring when testing on an iPad Mini, though. So, seems to be an issue with the iPadOS version of Moonlight itself.
An extremely bare bones port to visionOS. It works enough to pair, view apps and connect to a stream in the simulator. Mouse input and video seems to work fine.
Notable changes:
@Observable
in the uiKnown things to do:
Any and all constructive advice / criticism / suggestions are welcome!