`deviceId` of `AudioDeviceManager` is not shareable with other package

[nguyenhy]

Describe the bug We're making a desktop app for Windows and macOS users. In this app, there's a feature that allows the user to preview their camera and microphone in case users have more than one video/audio input device.

We decide to use another package to make this, since we can't stream from the one cam/mic and preview the exact cam/mic to allow user previews their cam/mic. Because we can't find a solution to use the same cam/mic one for stream and one for preview, and these 2 have different configs. Even, when we create 2 different RtcEngine with the same/different appId

With record and camera_macos. We were able to make it but there's one problem. The deviceId return from these 2 packages is different than the deviceId from Agora SDK

To Reproduce Note that I plugin in 2 similar camera

final engine = createAgoraRtcEngine();
engine
    .initialize(
  const RtcEngineContext(
    appId: appId,
  ),
)
    .then((value) async {
  final videoManager = engine.getVideoDeviceManager();
  final audioManager = engine.getAudioDeviceManager();
  final videoDevices = await videoManager.enumerateVideoDevices();
  final audioDevices = await audioManager.enumerateRecordingDevices();

  for (var element in videoDevices) {
    print(['videoDevices', element.deviceId, element.deviceName]);
  }
  /// [Log] 
  /// flutter: [videoDevices, 0x2100000046d0825, USB Camera VID:1133 PID:2085]
  /// flutter: [videoDevices, 0x2200000046d0825, USB Camera VID:1133 PID:2085]

  for (var element in audioDevices) {
    print(['audioDevices', element.deviceId, element.deviceName]);
  }
  /// flutter: [audioDevices, 99, default (Unknown USB Audio Device)]
  /// flutter: [audioDevices, 95, Unknown USB Audio Device]
});

final record = Record();
record.listInputDevices().then((inputDevices) {
  for (var element in inputDevices) {
    print([
      'record',
      '|',
      element.id,
      '|',
      element.label,
      '|',
      element.channels,
    ]);
  }
});
/// Log
/// flutter: [record, |, AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:D5ACA760:3, |, Unknown USB Audio Device, |, null]
/// flutter: [record, |, AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:33829440:3, |, Unknown USB Audio Device, |, null]

CameraMacOS.instance
    .listDevices(deviceType: CameraMacOSDeviceType.video)
    .then((value) {
  for (var element in value) {
    print([
      'CameraMacOSDeviceType.video',
      '|',
      element.deviceId,
      '|',
      element.localizedName,
      '|',
      element.manufacturer,
    ]);
  }
});
/// Log
/// flutter: [CameraMacOSDeviceType.video, |, 0x2100000046d0825, |, USB Camera VID:1133 PID:2085, |, Unknown]
/// flutter: [CameraMacOSDeviceType.video, |, 0x2200000046d0825, |, USB Camera VID:1133 PID:2085, |, Unknown] 

CameraMacOS.instance
    .listDevices(deviceType: CameraMacOSDeviceType.audio)
    .then((value) {
  for (var element in value) {
    print([
      'CameraMacOSDeviceType.audio',
      '|',
      element.deviceId,
      '|',
      element.localizedName,
      '|',
      element.manufacturer,
    ]);
  }
});
/// Log
/// flutter: [CameraMacOSDeviceType.audio, |, AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:D5ACA760:3, |, Unknown USB Audio Device, |, Unknown Manufacturer]
/// flutter: [CameraMacOSDeviceType.audio, |, AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:33829440:3, |, Unknown USB Audio Device, |, Unknown Manufacturer]

As shown in the above sample code, record, and camera_macos are able to return the same id for audio input devices, whichs is AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:D5ACA760:3 and AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:33829440:3. But the id from enumerateRecordingDevices return is 95, 99.

$ system_profiler SPUSBDataType
USB:

    USB 3.0 Bus:

      Host Controller Driver: AppleEmbeddedUSBXHCIFL1100
      PCI Device ID: 0x1100
      PCI Revision ID: 0x0010
      PCI Vendor ID: 0x1b73

        Miscellaneous Device:

          Product ID: 0x0825
          Vendor ID: 0x046d  (Logitech Inc.)
          Version: 0.12
          Serial Number: 33829440
          Speed: Up to 480 Mb/s
          Location ID: 0x02200000 / 2
          Current Available (mA): 500
          Current Required (mA): 500
          Extra Operating Current (mA): 0

        Miscellaneous Device:

          Product ID: 0x0825
          Vendor ID: 0x046d  (Logitech Inc.)
          Version: 0.12
          Serial Number: D5ACA760
          Speed: Up to 480 Mb/s
          Location ID: 0x02100000 / 1
          Current Available (mA): 500
          Current Required (mA): 500
          Extra Operating Current (mA): 0

Expected behavior method enumerateRecordingDevices return value ofdeviceId as other package

Desktop (please complete the following information):

• macOS: 13.0.1
• SDK: 6.1.0

[littleGnAl]

We're making a desktop app for Windows and macOS users. In this app, there's a feature that allows the user to preview their camera and microphone in case users have more than one video/audio input device.

I'm so sorry that I might miss your point, is that mean you need to preview all the video/audio devices at the same time?

[nguyenhy]

We're making a desktop app for Windows and macOS users. In this app, there's a feature that allows the user to preview their camera and microphone in case users have more than one video/audio input device.

I'm so sorry that I might miss your point, is that mean you need to preview all the video/audio devices at the same time?

So we need to preview 1 video/audio input device at a time while using it to push to channel. Like, preview a cam/mic before use it to push stream.

[littleGnAl]

In our example, you can change and preview the device, and then you can control when to push with joinChannel. I'm not very sure if it meets your requirement. https://github.com/AgoraIO-Extensions/Agora-Flutter-SDK/blob/main/example/lib/examples/advanced/device_manager/device_manager.dart

[nguyenhy]

And the main reason for this issue is that we need to show the volume meter of current preview mic. I found that record package allow use to do this. But deviceId from record and agora_rtc_engine is not the same. So even if we finish the feature, we can't select device using AudioDeviceManager.setRecordingDevice(deviceId)

[nguyenhy]

In our example, you can change and preview the device, and then you can control when to push with joinChannel. I'm not very sure if it meets your requirement. https://github.com/AgoraIO-Extensions/Agora-Flutter-SDK/blob/main/example/lib/examples/advanced/device_manager/device_manager.dart

So our problem is that, saying we have 2 cameras: camA and camB. UserA is using camA to push the stream to other users in the channel and the user wants to swap the camera to camB. But he wants to preview it before making the change.

So our feature is that we will show a list of the available cameras inside a preview dialog. User can choose and preview it. Once he feels good he can click on the OK button to use this camera to push stream to other users in the channel.

Somehow it looks like this

So, at the time they preview, we have 2 AgoraVideoView. One in the preview dialog and one in the main view. If we use VideoDeviceManager.setDevice(deviceId). Then it also means the main player is also changed to current preview camera.

[littleGnAl]

Thanks for your detail, at this time our SDK only supports 2 cameras preview, you can try the VideoSourceType.videoSourceCameraSecondary, I still need more info on how to preview others' video devices internally.

[nguyenhy]

I still need more info on how to preview others' video devices internally.

I would love to know how to do it. Thanks for your support

[qiuyanli1990]

@nguyenhy The deviceID returned by our SDK is also the deviceID returned by macOS system. Can you please try directly put the device ID returned by our SDK to the deviceID of other SDKs to see if it will work? Maybe they have different numbers but can both be used? Please let us know your test result.

[nguyenhy]

Hi @qiuyanli1990.

Can you please try directly put the device ID returned by our SDK to the deviceID of other SDKs to see if it will work? Maybe they have different numbers but can both be used? Please let us know your test result.

I did try to use agora ID in record package but it not work.

@nguyenhy The deviceID returned by our SDK is also the deviceID returned by macOS system.

I'm really not sure about this. I try to create sample code from swift and objective-c but seem like it all return the same result. And every time I plug in and out my camera, Agora SDK returns a new different id.

    // macOS Objective-C
    NSArray *devices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeAudio];
    for (AVCaptureDevice *device in devices) {
        NSLog(@"=======");
        NSLog(@"localizedName: %@", device.localizedName);
        NSLog(@"uniqueID: %@", device.uniqueID);
        NSLog(@"modelID: %@", device.modelID);

    }

=======
localizedName: Unknown USB Audio Device
uniqueID: AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:5F6D2E80:3
modelID: Unknown USB Audio Device:046D:0825
=======
localizedName: Unknown USB Audio Device
uniqueID: AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:33829440:3
modelID: Unknown USB Audio Device:046D:0825

// macOS Swift
let discoverySession = AVCaptureDevice.DiscoverySession(
    deviceTypes: [.builtInMicrophone],
      mediaType: .audio, position: .unspecified
    )

for device in discoverySession.devices {
    print("===========")
    print("localizedName: ", device.localizedName);
    print("uniqueID: ", device.uniqueID);
    print("modelID: ", device.modelID);
}

===========
localizedName:  Unknown USB Audio Device
uniqueID:  AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:5F6D2E80:3
modelID:  Unknown USB Audio Device:046D:0825
===========
localizedName:  Unknown USB Audio Device
uniqueID:  AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:33829440:3
modelID:  Unknown USB Audio Device:046D:0825

Can you help me point out how to create this int id so we can somehow create ourself adapter to retrieve String id as in uniqueID

[littleGnAl]

For your feature, I think you can try using startSecondaryCameraCapture to preview the selected deviceId

[nguyenhy]

For your feature, I think you can try using startSecondaryCameraCapture to preview the selected deviceId

As I mention ealier, the main reason for this issue is that the deviceId from Agora SDK is not the device id of plugin devices.

You can read more about this in this comment

And every time I plug in and out my camera, Agora SDK returns a new different id.

[littleGnAl]

https://github.com/AgoraIO-Extensions/Agora-Flutter-SDK/issues/960#issuecomment-1473434723

So we need to preview 1 video/audio input device at a time while using it to push to channel. Like, preview a cam/mic before use it to push stream.

I think this can be achieved by using the startSecondaryCameraCapture and preview it without using the other packages, you can check our example for reference.

Capture by a deviceId: https://github.com/AgoraIO-Extensions/Agora-Flutter-SDK/blob/216eb04fddf6d2a81a75f9c37295e2d593bb9ba4/example/lib/examples/advanced/send_multi_camera_stream/send_multi_camera_stream.dart#L233

Then preview it: https://github.com/AgoraIO-Extensions/Agora-Flutter-SDK/blob/216eb04fddf6d2a81a75f9c37295e2d593bb9ba4/example/lib/examples/advanced/send_multi_camera_stream/send_multi_camera_stream.dart#L174

[nguyenhy]

Ok, I will check startSecondaryCameraCapture.

But how about deviceId of audioDevices and videoDevices not the same. And every time I plug in and out my camera, Agora SDK returns a new different id for audioDevices.

final engine = createAgoraRtcEngine();
engine
    .initialize(
  const RtcEngineContext(
    appId: appId,
  ),
)
    .then((value) async {
  final videoManager = engine.getVideoDeviceManager();
  final audioManager = engine.getAudioDeviceManager();
  final videoDevices = await videoManager.enumerateVideoDevices();
  final audioDevices = await audioManager.enumerateRecordingDevices();

  for (var element in videoDevices) {
    print(['videoDevices', element.deviceId, element.deviceName]);
  }
  /// [Log] 
  /// flutter: [videoDevices, 0x2100000046d0825, USB Camera VID:1133 PID:2085]
  /// flutter: [videoDevices, 0x2200000046d0825, USB Camera VID:1133 PID:2085]

  for (var element in audioDevices) {
    print(['audioDevices', element.deviceId, element.deviceName]);
  }
  /// flutter: [audioDevices, 99, default (Unknown USB Audio Device)]
  /// flutter: [audioDevices, 95, Unknown USB Audio Device]
});