framework question - does the info.plist need to change to reflect xros

[dstolarz]

I noticed the plist file inside the framework folder still references iOS not xros.... does that affect linking? I have added the framework to a sample file, added it to framework search paths, but get

/Users/user/Library/Developer/Xcode/DerivedData/WebRTC-Demo-aqkevpztdoqexhanwteklkoocwnv/SourcePackages/artifacts/webrtc/WebRTC/WebRTC.xcframework:1:1 While building for visionOS Simulator, no library for this platform was found in '/Users/user/Library/Developer/Xcode/DerivedData/WebRTC-Demo-aqkevpztdoqexhanwteklkoocwnv/SourcePackages/artifacts/webrtc/WebRTC/WebRTC.xcframework'.

Here's the plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>BuildMachineOSBuild</key>
    <string>23D60</string>
    <key>CFBundleDevelopmentRegion</key>
    <string>en</string>
    <key>CFBundleExecutable</key>
    <string>WebRTC</string>
    <key>CFBundleIdentifier</key>
    <string>org.webrtc.WebRTC</string>
    <key>CFBundleInfoDictionaryVersion</key>
    <string>6.0</string>
    <key>CFBundleName</key>
    <string>WebRTC</string>
    <key>CFBundlePackageType</key>
    <string>FMWK</string>
    <key>CFBundleShortVersionString</key>
    <string>1.0</string>
    <key>CFBundleSignature</key>
    <string>????</string>
    <key>CFBundleSupportedPlatforms</key>
    <array>
        <string>iPhoneOS</string>
    </array>
    <key>CFBundleVersion</key>
    <string>1.0</string>
    <key>DTCompiler</key>
    <string>com.apple.compilers.llvm.clang.1_0</string>
    <key>DTPlatformBuild</key>
    <string>21E210</string>
    <key>DTPlatformName</key>
    <string>iphoneos</string>
    <key>DTPlatformVersion</key>
    <string>17.4</string>
    <key>DTSDKBuild</key>
    <string>21E210</string>
    <key>DTSDKName</key>
    <string>iphoneos17.4</string>
    <key>DTXcode</key>
    <string>1530</string>
    <key>DTXcodeBuild</key>
    <string>15E204a</string>
    <key>MinimumOSVersion</key>
    <string>12.0</string>
    <key>NSPrincipalClass</key>
    <string></string>
    <key>UIDeviceFamily</key>
    <array>
        <integer>1</integer>
        <integer>2</integer>
    </array>
</dict>
</plist>

[devluz]

I will try to build a standalone test app to check on this. My first test build hasn't consumed the framework directly so I might have missed this issue.

I use another toolchain that will output something like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>DTSDKName</key>
    <string>xrsimulator1.0</string>
    <key>DTXcode</key>
    <string>1520</string>
    <key>DTSDKBuild</key>
    <string>21N301</string>
    <key>CFBundleDevelopmentRegion</key>
    <string>English</string>
    <key>CFBundleVersion</key>
    <string>1.0.2</string>
    <key>BuildMachineOSBuild</key>
    <string>22G513</string>
    <key>DTPlatformName</key>
    <string>xrsimulator</string>
    <key>CFBundlePackageType</key>
    <string>FMWK</string>
    <key>CFBundleShortVersionString</key>
    <string>1.0.2</string>
    <key>CFBundleSupportedPlatforms</key>
    <array>
        <string>XRSimulator</string>
    </array>
    <key>CFBundleInfoDictionaryVersion</key>
    <string>6.0</string>
    <key>CFBundleExecutable</key>
    <string>WebRTC</string>
    <key>DTCompiler</key>
    <string>com.apple.compilers.llvm.clang.1_0</string>
    <key>MinimumOSVersion</key>
    <string>1.0</string>
    <key>CFBundleIdentifier</key>
    <string>org.webrtc.WebRTC</string>
    <key>UIDeviceFamily</key>
    <array>
        <integer>7</integer>
    </array>
    <key>CFBundleSignature</key>
    <string>????</string>
    <key>DTPlatformVersion</key>
    <string>1.0</string>
    <key>CSResourcesFileMapped</key>
    <true/>
    <key>DTXcodeBuild</key>
    <string>15C500b</string>
    <key>DTPlatformBuild</key>
    <string>21N301</string>
</dict>
</plist>

[dstolarz]

Thank you - will try

On Sat, Mar 23, 2024 at 6:19 PM, Christoph Kutza < @.*** > wrote:

I will try to build a standalone test app to check on this. My first test build hasn't consumed the framework directly so I might have missed this issue.

I use another toolchain that will output something like this:

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" " http:/ / www. apple. com/ DTDs/ PropertyList-1.

0. dtd ( http://www.apple.com/DTDs/PropertyList-1.0.dtd ) "> <plist version="1.0"> DTSDKName

xrsimulator1.0 DTXcode 1520 DTSDKBuild 21N301 CFBundleDevelopmentRegion English CFBundleVersion 1.0.2 BuildMachineOSBuild 22G513 DTPlatformName xrsimulator CFBundlePackageType FMWK CFBundleShortVersionString 1.0.2 CFBundleSupportedPlatforms XRSimulator CFBundleInfoDictionaryVersion 6.0 CFBundleExecutable WebRTC DTCompiler com. apple. compilers. llvm. clang. 1_0 ( http://com.apple.compilers.llvm.clang.1_0/ ) MinimumOSVersion 1.0 CFBundleIdentifier org.webrtc.WebRTC UIDeviceFamily 7 CFBundleSignature ???? DTPlatformVersion 1.0 CSResourcesFileMapped DTXcodeBuild 15C500b DTPlatformBuild 21N301

— Reply to this email directly, view it on GitHub ( https://github.com/because-why-not/webrtc_visionpro_workspace/issues/3#issuecomment-2016653932 ) , or unsubscribe ( https://github.com/notifications/unsubscribe-auth/ACETLRXACQ4C3HQH4P3SZ43YZYS27AVCNFSM6AAAAABFENA226VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJWGY2TGOJTGI ). You are receiving this because you authored the thread. Message ID: <because-why-not/webrtc_visionpro_workspace/issues/3/2016653932 @ github. com>

[devluz]

I added a merge.sh script now that builds an xcframework with mac,ios, xros and xros simulator for use via xcode. It also patches the broken plist files. You can test it via the MinTestApp. It should on on ios, xros simulator and hopefully xros (mac is broken atm). It just prints Hello World and writes some debug log if WebRTC manages to create a factory. There was also an issue with objc symbols not being properly exported so I hope the latest version fixes your issue

[dstolarz]

okay test app worked!

1. I had to manually copy XROS1.0.sdk into the Xcode app; i had XROS1.1.sdk (from Xcode Version 15.3 (15E204a)) clang++: error: no such sysroot directory: '/Applications/Xcode.app/Contents/Developer/Platforms/XROS.platform/Developer/SDKs/XROS1.0.sdk' [-Werror,-Wmissing-sysroot]

2. i have these 'duplicate' warnings:

3. /libwebrtc.a ../../../../../../../Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: warning duplicate member name 'aarch64_cpudetect.o' from 'obj/third_party/libaom/libaom/aarch64_cpudetect.o(aarch64_cpudetect.o)' and 'obj/third_party/libvpx/libvpx/aarch64_cpudetect.o(aarch64_cpudetect.o)'

so i may need to clean out the folder and try again, but i was able to run the app.

Now I'll go for running it on the real device.

[dstolarz]

This is promising.

[devluz]

I hard coded the "XROS1.0.sdk" folder name into the setup_xros.sh replacing this with your version might work as well.

[dstolarz]

(audio_device_module.mm:21): CreateAudioDeviceModule (audio_device_module_ios.mm:45): current platform is IOS (audio_device_module_ios.mm:46): iPhone Audio APIs will be utilized. (input_volume_stats_reporter.cc:98): Will not log any WebRTC.Audio.Apm.AppliedInputVolume.* histogram stats. (input_volume_stats_reporter.cc:98): Will not log any WebRTC.Audio.Apm.RecommendedInputVolume.* histogram stats. (audio_processing_impl.cc:695): Injected APM submodules: Echo control factory: 0 Echo detector: 0 Capture analyzer: 0 Capture post processor: 0 Render pre processor: 0 (audio_processing_impl.cc:708): AudioProcessing: AudioProcessing::Config{ pipeline: { maximum_internal_processing_rate: 48000, multi_channel_render: 0, multi_channel_capture: 0 }, pre_amplifier: { enabled: 0, fixed_gain_factor: 1 },capture_level_adjustment: { enabled: 0, pre_gain_factor: 1, post_gain_factor: 1, analog_mic_gain_emulation: { enabled: 0, initial_level: 255 }}, high_pass_filter: { enabled: 0 }, echo_canceller: { enabled: 0, mobile_mode: 0, enforce_high_pass_filtering: 1 }, noise_suppression: { enabled: 0, level: Moderate }, transient_suppression: { enabled: 0 }, gain_controller1: { enabled: 0, mode: AdaptiveAnalog, target_level_dbfs: 3, compression_gain_db: 9, enable_limiter: 1, analog_gain_controller { enabled: 1, startup_min_volume: 0, clipped_level_min: 70, enable_digital_adaptive: 1, clipped_level_step: 15, clipped_ratio_threshold: 0.1, clipped_wait_frames: 300, clipping_predictor: { enabled: 0, mode: 0, window_length: 5, reference_window_length: 5, reference_window_delay: 5, clipping_threshold: -1, crest_factor_margin: 3, use_predicted_step: 1 }}}, gain_controller2: { enabled: 0, fixed_digital: { gain_db: 0 }, adaptive_digital: { enabled: 0, headroom_db: 6, max_gain_db: 30, initial_gain_db: 8, max_gain_change_db_per_second: 3, max_output_noise_level_dbfs: -50 }, input_volume_control : { enabled 0}} (webrtc_voice_engine.cc:354): WebRtcVoiceEngine::WebRtcVoiceEngine (webrtc_video_engine.cc:771): WebRtcVideoEngine::WebRtcVideoEngine() (webrtc_voice_engine.cc:376): WebRtcVoiceEngine::Init (webrtc_voice_engine.cc:385): Supported send codecs in order of preference: (webrtc_voice_engine.cc:388): opus/48000/2 { minptime=10 useinbandfec=1 } (111) (webrtc_voice_engine.cc:388): red/48000/2 { =111/111 } (63) (webrtc_voice_engine.cc:388): G722/8000/1 (9) (webrtc_voice_engine.cc:388): ILBC/8000/1 (102) (webrtc_voice_engine.cc:388): PCMU/8000/1 (0) (webrtc_voice_engine.cc:388): PCMA/8000/1 (8) (webrtc_voice_engine.cc:388): CN/8000/1 (13) (webrtc_voice_engine.cc:388): telephone-event/48000/1 (110) (webrtc_voice_engine.cc:388): telephone-event/8000/1 (126) (webrtc_voice_engine.cc:391): Supported recv codecs in order of preference: (webrtc_voice_engine.cc:394): opus/48000/2 { minptime=10 useinbandfec=1 } (111) (webrtc_voice_engine.cc:394): red/48000/2 { =111/111 } (63) (webrtc_voice_engine.cc:394): G722/8000/1 (9) (webrtc_voice_engine.cc:394): ILBC/8000/1 (102) (webrtc_voice_engine.cc:394): PCMU/8000/1 (0) (webrtc_voice_engine.cc:394): PCMA/8000/1 (8) (webrtc_voice_engine.cc:394): CN/8000/1 (13) (webrtc_voice_engine.cc:394): telephone-event/48000/1 (110) (webrtc_voice_engine.cc:394): telephone-event/8000/1 (126) (audio_device_module_ios.mm:70): Init (audio_device_buffer.cc:66): AudioDeviceBuffer::ctor (audio_device_ios.mm:107): AudioDeviceIOS::ctor<NSThread: 0x600001790500>{number = 7, name = (null)},bypass_voice_processing=0 (audio_device_module_ios.mm:50): AttachAudioBuffer (audio_device_ios.mm:124): AudioDeviceIOS::AttachAudioBuffer (audio_device_ios.mm:131): AudioDeviceIOS::Init (audio_device_ios.mm:74): LogDeviceInfo (audio_device_ios.mm:76): system name: visionOS (audio_device_ios.mm:77): system version: 1.1 (audio_device_ios.mm:78): device type: Apple Vision Pro (audio_device_ios.mm:79): device name: arm64 (audio_device_ios.mm:80): process name: MinTestApp (audio_device_ios.mm:81): process ID: 75084 (audio_device_ios.mm:82): OS version: Version 1.1 (Build 21O209) (audio_device_ios.mm:83): processing cores: 12 (audio_device_ios.mm:84): low power mode: 0 (audio_device_ios.mm:86): TARGET_IPHONE_SIMULATOR is defined (audio_device_ios.mm:88): DeviceIsSimulator: 0 (audio_device_ios.mm:637): AudioDeviceIOS::UpdateAudioDevicebuffer (audio_device_buffer.cc:192): SetPlayoutSampleRate(48000) (audio_device_buffer.cc:212): SetPlayoutChannels(1) (audio_device_buffer.cc:186): SetRecordingSampleRate(48000) (audio_device_buffer.cc:206): SetRecordingChannels(1) (audio_device_module_ios.mm:406): SetPlayoutDevice(0) (audio_device_ios.mm:1005): virtual int32_t webrtc::ios_adm::AudioDeviceIOS::SetPlayoutDevice(uint16_t): Not implemented (audio_device_module_ios.mm:109): InitSpeaker (audio_device_module_ios.mm:311): StereoPlayoutIsAvailable (audio_device_module_ios.mm:318): output: 0 (audio_device_module_ios.mm:323): SetStereoPlayout(0) (audio_device_ios.mm:1058): virtual int32_t webrtc::ios_adm::AudioDeviceIOS::SetStereoPlayout(bool): Not implemented (audio_device_module_ios.mm:330): stereo playout is not supported (adm_helpers.cc:57): Failed to set stereo playout mode. (audio_device_module_ios.mm:468): SetRecordingDevice(0) (audio_device_ios.mm:1107): virtual int32_t webrtc::ios_adm::AudioDeviceIOS::SetRecordingDevice(uint16_t): Not implemented (audio_device_module_ios.mm:115): InitMicrophone (audio_device_module_ios.mm:278): StereoRecordingIsAvailable (audio_device_module_ios.mm:285): output: 0 (audio_device_module_ios.mm:290): SetStereoRecording(0) (adm_helpers.cc:77): Failed to set stereo recording mode. (audio_device_module_ios.mm:581): RegisterAudioCallback (audio_device_buffer.cc:86): RegisterAudioCallback (webrtc_voice_engine.cc:479): WebRtcVoiceEngine::ApplyOptions: AudioOptions {aec: 1, agc: 1, ns: 0, hf: 1, swap: 0, audio_jitter_buffer_max_packets: 200, audio_jitter_buffer_fast_accelerate: 0, audio_jitter_buffer_min_delay_ms: 0, } (webrtc_voice_engine.cc:498): Always disable AEC on iOS. Use built-in instead. (webrtc_voice_engine.cc:508): Always disable AGC on iOS. Use built-in instead. (audio_device_module_ios.mm:597): BuiltInAECIsAvailable (audio_device_generic.cc:18): virtual bool webrtc::AudioDeviceGeneric::BuiltInAECIsAvailable() const: Not supported on this platform (audio_device_module_ios.mm:600): output: 0 (audio_device_module_ios.mm:613): BuiltInAGCIsAvailable (audio_device_generic.cc:28): virtual bool webrtc::AudioDeviceGeneric::BuiltInAGCIsAvailable() const: Not supported on this platform (audio_device_module_ios.mm:616): output: 0 (audio_device_module_ios.mm:629): BuiltInNSIsAvailable (audio_device_generic.cc:38): virtual bool webrtc::AudioDeviceGeneric::BuiltInNSIsAvailable() const: Not supported on this platform (audio_device_module_ios.mm:632): output: 0 (audio_processing_impl.cc:911): AudioProcessing::ApplyConfig: AudioProcessing::Config{ pipeline: { maximum_internal_processing_rate: 48000, multi_channel_render: 0, multi_channel_capture: 0 }, pre_amplifier: { enabled: 0, fixed_gain_factor: 1 },capture_level_adjustment: { enabled: 0, pre_gain_factor: 1, post_gain_factor: 1, analog_mic_gain_emulation: { enabled: 0, initial_level: 255 }}, high_pass_filter: { enabled: 1 }, echo_canceller: { enabled: 0, mobile_mode: 0, enforce_high_pass_filtering: 1 }, noise_suppression: { enabled: 0, level: High }, transient_suppression: { enabled: 0 }, gain_controller1: { enabled: 0, mode: FixedDigital, target_level_dbfs: 3, compression_gain_db: 9, enable_limiter: 1, analog_gain_controller { enabled: 1, startup_min_volume: 0, clipped_level_min: 70, enable_digital_adaptive: 1, clipped_level_step: 15, clipped_ratio_threshold: 0.1, clipped_wait_frames: 300, clipping_predictor: { enabled: 0, mode: 0, window_length: 5, reference_window_length: 5, reference_window_delay: 5, clipping_threshold: -1, crest_factor_margin: 3, use_predicted_step: 1 }}}, gain_controller2: { enabled: 0, fixed_digital: { gain_db: 0 }, adaptive_digital: { enabled: 0, headroom_db: 6, max_gain_db: 30, initial_gain_db: 8, max_gain_change_db_per_second: 3, max_output_noise_level_dbfs: -50 }, input_volume_control : { enabled 0}} (webrtc_video_engine.cc:775): WebRtcVideoEngine::~WebRtcVideoEngine (webrtc_voice_engine.cc:362): WebRtcVoiceEngine::~WebRtcVoiceEngine (audio_device_module_ios.mm:532): StopPlayout (audio_device_ios.mm:243): AudioDeviceIOS::StopPlayout (audio_device_module_ios.mm:536): output: 0 (audio_device_module_ios.mm:563): StopRecording (audio_device_ios.mm:297): AudioDeviceIOS::StopRecording (audio_device_module_ios.mm:567): output: 0 (audio_device_module_ios.mm:581): RegisterAudioCallback (audio_device_buffer.cc:86): RegisterAudioCallback (audio_device_module_ios.mm:93): Terminate (audio_device_ios.mm:159): AudioDeviceIOS::Terminate (audio_device_ios.mm:243): AudioDeviceIOS::StopPlayout (audio_device_ios.mm:297): AudioDeviceIOS::StopRecording (audio_device_module_ios.mm:56): ~AudioDeviceModuleIOS (audio_device_buffer.cc:80): AudioDeviceBuffer::~dtor (audio_device_ios.mm:117): AudioDeviceIOS::~dtor<NSThread: 0x600001790500>{number = 7, name = (null)} (audio_device_ios.mm:159): AudioDeviceIOS::Terminate "done" nw_socket_copy_info [C1:2] getsockopt TCP_INFO failed [102: Operation not supported on socket] nw_socket_copy_info getsockopt TCP_INFO failed [102: Operation not supported on socket]

[dstolarz]

okay now i'm gonna try to get some video working

[dzeitman]

@dstolarz I'm curious if you made any further progress ?

[devluz]

@dstolarz I'm curious if you made any further progress ?

As far as I know everything works. The unity version has receiving video, receiving audio, sending audio, sending/receiving data all working. Only accessing local camera to send video is not possible due to visionOS restrictions.

I would expect this to work for the swift / objective C version as well by now. Only pitfall I am aware of is that users have to render incoming video frames themselves because the premade UI element doesn't work yet

[LFatZei]

May I ask if it works also directly when imported to Unity Plugins?

[devluz]

This version here has no C# API. For Unity there are two options:

a) com.unity.webrtc is a C# Unity created. I created a AVP port for it here: https://github.com/because-why-not/uwebrtc_visionpro . They use an older WebRTC version so I can not easily combine them into one. Note Unity does no longer appear to update this.

b) I am the developer of the unity asset WebRTC Video Chat. If you have purchased a license for this you can send me an e-mail and I share the latest AVP test builds. This asset focuses on high level video chats that you can customise yourself rather than building an app from scratch based on WebRTC like option a.