Audio doesn't work properly on STM32U5

gamelaster commented 1 year ago

USBX version: 6.2.0 Target device: STM32U585 Toolchain and IDE: ARM GNU Toolchain for Windows, STM32CubeIDE

I have STM32CubeIDE project, where I use USBX with USB CDC and AUDIO devices. USB CDC worked fine, so I decided to integrate Audio Speaker, as that's only example I found (I use builder framework). I was able to integrate it, and it worked (host sent data to MCU), but I find out that callbacks ux_device_class_audio_stream_change and ux_device_class_audio_stream_frame_done were never called. I thought it is OK, so I moved on implementing microphone only descriptor, as that is my primary target (not speaker). I successfully integrated descriptor (at least, I hope so), but on Windows, it throws classical error "This device cannot start".

So this (probably?) means that descriptor is OK, but device don't talk. So I started to debug the stuff, and only thing, which is called is ux_slave_class_audio_instance_activate, after that, nothing. Mainly, ux_device_class_audio_stream_change is not called, so basically, nothing happens.

It seems that CubeMX is not supporting AUDIO applications that much, so maybe some configuration is missing? I tried everything, but couldn't figure out. I'm attaching snippets of codes:

Descriptor

```c static void USBD_FrameWork_AUDIO_Desc(USBD_DevClassHandleTypeDef *pdev, uint32_t pConf, uint32_t *Sze) { static USBD_IfDescTypedef *pIfDesc; static USBD_EpDescTypedef *pEpDesc; #if USBD_COMPOSITE_USE_IAD == 1 static USBD_IadDescTypedef *pIadDesc; #endif /* USBD_COMPOSITE_USE_IAD == 1 */ static USBD_AUDIOCCSIfDescTypeDef *pMicrophoneACCSIfDesc; static USBD_AUDIOSCSIfDescTypeDef *pMicrophoneASCSIfDesc; static USBD_AUDIOSFormatIfDescTypeDef *pMicrophoneASFormatDesc; static USBD_AUDIOSCSEpDescTypeDef *pMicrophoneASCSEpDesc; static USBD_AUDIOClockSourceDescTypeDef *pMicrophoneCSDesc; static USBD_AUDIOInputTerminalDescTypeDef *pMicrophoneITDesc; static USBD_AUDIOFeatureUnitPlayDescTypeDef *pMicrophoneFUDesc; static USBD_AUDIOOutputTerminalDescTypeDef *pMicrophoneOTDesc; #if USBD_COMPOSITE_USE_IAD == 1 pIadDesc = ((USBD_IadDescTypedef *)(pConf + *Sze)); pIadDesc->bLength = (uint8_t)sizeof(USBD_IadDescTypedef); pIadDesc->bDescriptorType = USB_DESC_TYPE_IAD; /* IAD descriptor */ pIadDesc->bFirstInterface = pdev->tclasslist[pdev->classId].Ifs[0]; pIadDesc->bInterfaceCount = 0x02U; pIadDesc->bFunctionClass = UX_DEVICE_CLASS_AUDIO_FUNCTION_CLASS; pIadDesc->bFunctionSubClass = UX_DEVICE_CLASS_AUDIO_FUNCTION_SUBCLASS_UNDEFINED; pIadDesc->bFunctionProtocol = UX_DEVICE_CLASS_AUDIO_FUNCTION_PROTOCOL_VERSION_02_00; pIadDesc->iFunction = 0U; /* String Index */ *Sze += (uint32_t)sizeof(USBD_IadDescTypedef); #endif /* USBD_COMPOSITE_USE_IAD == 1 */ /* Append AUDIO Interface descriptor to Configuration descriptor */ __USBD_FRAMEWORK_SET_IF(pdev->tclasslist[pdev->classId].Ifs[0], 0x00U, 0x00U, UX_DEVICE_CLASS_AUDIO_CLASS, \ UX_DEVICE_CLASS_AUDIO_SUBCLASS_CONTROL, UX_DEVICE_CLASS_AUTIO_PROTOCOL_VERSION_02_00, 0x00U); /* Append AUDIO USB Microphone Class-specific AC Interface descriptor to Configuration descriptor */ pMicrophoneACCSIfDesc = ((USBD_AUDIOCCSIfDescTypeDef *)(pConf + *Sze)); pMicrophoneACCSIfDesc->bLength = (uint8_t)sizeof(USBD_AUDIOCCSIfDescTypeDef); pMicrophoneACCSIfDesc->bDescriptorType = UX_DEVICE_CLASS_AUDIO_CS_INTERFACE; pMicrophoneACCSIfDesc->bDescriptorSubtype = UX_DEVICE_CLASS_AUDIO_AC_HEADER; pMicrophoneACCSIfDesc->bcdADC = 0x0200U; pMicrophoneACCSIfDesc->bCategory = UX_DEVICE_CLASS_AUDIO20_CATEGORY_MICROPHONE; pMicrophoneACCSIfDesc->wTotalLength = USBD_CONFIG_DESCRIPTOR_AC_TOTAL_SIZE; // TODO: neviem pMicrophoneACCSIfDesc->bmControls = 0x00; *Sze += (uint32_t)sizeof(USBD_AUDIOCCSIfDescTypeDef); /* Append USB Microphone Clock Source Descriptor to Configuration descriptor*/ pMicrophoneCSDesc = ((USBD_AUDIOClockSourceDescTypeDef *)(pConf + *Sze)); pMicrophoneCSDesc->bLength = (uint8_t)sizeof(USBD_AUDIOClockSourceDescTypeDef); pMicrophoneCSDesc->bDescriptorType = UX_DEVICE_CLASS_AUDIO_CS_INTERFACE; pMicrophoneCSDesc->bDescriptorSubtype = UX_DEVICE_CLASS_AUDIO20_AC_CLOCK_SOURCE; pMicrophoneCSDesc->bClockID = USB_AUDIO_CONFIG_PLAY_CLOCK_SOURCE_ID; // TODO: Should be 10 pMicrophoneCSDesc->bmAttributes = 0x05U; pMicrophoneCSDesc->bmControls = 0x01U; pMicrophoneCSDesc->bAssocTerminal = 0x00U; pMicrophoneCSDesc->iClockSource = 0x00U; *Sze += (uint32_t)sizeof(USBD_AUDIOClockSourceDescTypeDef); /* Append USB Microphone Input Terminal Descriptor to Configuration descriptor*/ pMicrophoneITDesc = ((USBD_AUDIOInputTerminalDescTypeDef *)(pConf + *Sze)); pMicrophoneITDesc->bLength = (uint8_t)sizeof(USBD_AUDIOInputTerminalDescTypeDef); pMicrophoneITDesc->bDescriptorType = UX_DEVICE_CLASS_AUDIO_CS_INTERFACE; pMicrophoneITDesc->bDescriptorSubtype = UX_DEVICE_CLASS_AUDIO_AC_INPUT_TERMINAL; pMicrophoneITDesc->bTerminalID = USB_AUDIO_CONFIG_PLAY_TERMINAL_INPUT_ID; // TODO: Should be 0x01 pMicrophoneITDesc->wTerminalType = UX_DEVICE_CLASS_AUDIO_MICROPHONE; // pMicrophoneITDesc->bAssocTerminal = 0x00U; pMicrophoneITDesc->bCSourceID = USB_AUDIO_CONFIG_PLAY_CLOCK_SOURCE_ID; // TODO: SHould be 0x10 pMicrophoneITDesc->bNrChannels = USB_AUDIO_CONFIG_PLAY_CHANNEL_COUNT; pMicrophoneITDesc->bmChannelConfig = 0x00000000U; pMicrophoneITDesc->iChannelNames = 0x00U; pMicrophoneITDesc->bmControls = 0x0000U; pMicrophoneITDesc->iTerminal = 0x00U; *Sze += (uint32_t)sizeof(USBD_AUDIOInputTerminalDescTypeDef); /*Append USB Microphone Audio Feature Unit Descriptor to Configuration descriptor */ pMicrophoneFUDesc = ((USBD_AUDIOFeatureUnitPlayDescTypeDef *)(pConf + *Sze)); pMicrophoneFUDesc->bLength = (uint8_t)sizeof(USBD_AUDIOFeatureUnitPlayDescTypeDef); pMicrophoneFUDesc->bDescriptorType = UX_DEVICE_CLASS_AUDIO_CS_INTERFACE; pMicrophoneFUDesc->bDescriptorSubtype = UX_DEVICE_CLASS_AUDIO_AC_FEATURE_UNIT; pMicrophoneFUDesc->bUnitID = USB_AUDIO_CONFIG_PLAY_UNIT_FEATURE_ID; // TODO: 0x02 pMicrophoneFUDesc->bSourceID = USB_AUDIO_CONFIG_PLAY_TERMINAL_INPUT_ID; // TODO: 0x01 pMicrophoneFUDesc->bmaControls[0] = 0x0F0F0F0FU; // TODO: USBD_AUDIO_FU_CONTROL_MUTE|USBD_AUDIO_FU_CONTROL_VOLUME; pMicrophoneFUDesc->bmaControls[1] = 0x00000000U; pMicrophoneFUDesc->bmaControls[2] = 0x00000000U; pMicrophoneFUDesc->iFeature = 0x00; *Sze += (uint32_t)sizeof(USBD_AUDIOFeatureUnitPlayDescTypeDef); /*Append USB Microphone Output Terminal Descriptor to Configuration descriptor*/ pMicrophoneOTDesc = ((USBD_AUDIOOutputTerminalDescTypeDef *)(pConf + *Sze)); pMicrophoneOTDesc->bLength = (uint8_t)sizeof(USBD_AUDIOOutputTerminalDescTypeDef); pMicrophoneOTDesc->bDescriptorType = UX_DEVICE_CLASS_AUDIO_CS_INTERFACE; pMicrophoneOTDesc->bDescriptorSubtype = UX_DEVICE_CLASS_AUDIO_AC_OUTPUT_TERMINAL; pMicrophoneOTDesc->bTerminalID = USB_AUDIO_CONFIG_PLAY_TERMINAL_OUTPUT_ID; // TODO: 0x03 pMicrophoneOTDesc->wTerminalType = UX_DEVICE_CLASS_AUDIO_USB_STREAMING; pMicrophoneOTDesc->bAssocTerminal = 0x00U; pMicrophoneOTDesc->bSourceID = USB_AUDIO_CONFIG_PLAY_UNIT_FEATURE_ID; // TODO: 0x02 pMicrophoneOTDesc->bCSourceID = USB_AUDIO_CONFIG_PLAY_CLOCK_SOURCE_ID; // TODO: 0x10 pMicrophoneOTDesc->bmaControls = 0x0000U; pMicrophoneOTDesc->iTerminal = 0x00U; *Sze += (uint32_t)sizeof(USBD_AUDIOOutputTerminalDescTypeDef); /* USB Microphone Standard AS Interface Descriptor - Audio Streaming Zero Bandwidth */ /* Interface 1, Alternate Setting 0*/ __USBD_FRAMEWORK_SET_IF(pdev->tclasslist[pdev->classId].Ifs[USBD_AUDIO_AS_PLAY_INTERFACE], 0x00U, 0x00U, UX_DEVICE_CLASS_AUDIO_CLASS, \ UX_DEVICE_CLASS_AUDIO_SUBCLASS_AUDIOSTREAMING, UX_DEVICE_CLASS_AUTIO_PROTOCOL_VERSION_02_00, 0x00U); /* USB Microphone Standard AS Interface Descriptor -Audio Streaming Operational */ /* Interface 1, Alternate Setting 1*/ __USBD_FRAMEWORK_SET_IF(pdev->tclasslist[pdev->classId].Ifs[USBD_AUDIO_AS_PLAY_INTERFACE], 0x01U, 0x01U, UX_DEVICE_CLASS_AUDIO_CLASS, \ UX_DEVICE_CLASS_AUDIO_SUBCLASS_AUDIOSTREAMING, UX_DEVICE_CLASS_AUTIO_PROTOCOL_VERSION_02_00, 0x00U); /* USB Microphone Audio Streaming Class-Specific Interface Descriptor */ pMicrophoneASCSIfDesc = ((USBD_AUDIOSCSIfDescTypeDef *)(pConf + *Sze)); pMicrophoneASCSIfDesc->bLength = (uint8_t)sizeof(USBD_AUDIOSCSIfDescTypeDef); pMicrophoneASCSIfDesc->bDescriptorType = UX_DEVICE_CLASS_AUDIO_CS_INTERFACE; pMicrophoneASCSIfDesc->bDescriptorSubtype = UX_DEVICE_CLASS_AUDIO_AS_GENERAL; pMicrophoneASCSIfDesc->bTerminalLink = USB_AUDIO_CONFIG_PLAY_TERMINAL_OUTPUT_ID; // TODO: 0x03 pMicrophoneASCSIfDesc->bmControls = 0x00U; pMicrophoneASCSIfDesc->bFormatType = UX_DEVICE_CLASS_AUDIO_FORMAT_TYPE_I; pMicrophoneASCSIfDesc->bmFormats = 0x01010101U; // 0x00000001U; pMicrophoneASCSIfDesc->bNrChannels = USB_AUDIO_CONFIG_PLAY_CHANNEL_COUNT; pMicrophoneASCSIfDesc->bmChannelConfig = USB_AUDIO_CONFIG_PLAY_CHANNEL_MAP; pMicrophoneASCSIfDesc->iChannelNames = 0x0000U; *Sze += (uint32_t)sizeof(USBD_AUDIOSCSIfDescTypeDef); /* USB Microphone Audio Format Interface Descriptor */ pMicrophoneASFormatDesc = ((USBD_AUDIOSFormatIfDescTypeDef *)(pConf + *Sze)); pMicrophoneASFormatDesc->bLength = (uint8_t)sizeof(USBD_AUDIOSFormatIfDescTypeDef); pMicrophoneASFormatDesc->bDescriptorType = UX_DEVICE_CLASS_AUDIO_CS_INTERFACE; pMicrophoneASFormatDesc->bDescriptorSubtype = UX_DEVICE_CLASS_AUDIO_AS_FORMAT_TYPE; pMicrophoneASFormatDesc->bFormatType = UX_DEVICE_CLASS_AUDIO_FORMAT_TYPE_I; pMicrophoneASFormatDesc->bSubslotSize = USB_AUDIO_CONFIG_PLAY_RES_BYTE; pMicrophoneASFormatDesc->bBitResolution = USB_AUDIO_CONFIG_PLAY_RES_BIT; *Sze += (uint32_t)sizeof(USBD_AUDIOSFormatIfDescTypeDef); /* Append Endpoint descriptor to Configuration descriptor */ __USBD_FRAMEWORK_SET_EP((pdev->tclasslist[pdev->classId].Eps[0].add), // TODO: 0x81 (USBD_EP_TYPE_ISOC|USBD_EP_ATTR_ISOC_ASYNC), (uint16_t)(pdev->tclasslist[pdev->classId].Eps[0].size), (0x01U), (0x01U)); /* Class-Specific AS Isochronous Audio Data Endpoint Descriptor*/ pMicrophoneASCSEpDesc = ((USBD_AUDIOSCSEpDescTypeDef *)(pConf + *Sze)); pMicrophoneASCSEpDesc->bLength = (uint8_t)sizeof(USBD_AUDIOSCSEpDescTypeDef); pMicrophoneASCSEpDesc->bDescriptorType = UX_DEVICE_CLASS_AUDIO_CS_ENDPOINT; pMicrophoneASCSEpDesc->bDescriptorSubtype = UX_DEVICE_CLASS_AUDIO_EP_GENERAL; pMicrophoneASCSEpDesc->bmAttributes = 0x00U; pMicrophoneASCSEpDesc->bmControls = 0x00U; pMicrophoneASCSEpDesc->bLockDelayUnits = 0x00U; pMicrophoneASCSEpDesc->wLockDelay = 0x0000U; *Sze += (uint32_t)sizeof(USBD_AUDIOSCSEpDescTypeDef); /* Update Config Descriptor and IAD descriptor */ ((USBD_ConfigDescTypedef *)pConf)->bNumInterfaces += 2U; ((USBD_ConfigDescTypedef *)pConf)->wDescriptorLength = *Sze; } ```

Device registration

```c audio_stream_parameter[0].ux_device_class_audio_stream_parameter_callbacks.ux_device_class_audio_stream_change = Audio_ReadChange; audio_stream_parameter[0].ux_device_class_audio_stream_parameter_callbacks.ux_device_class_audio_stream_frame_done = Audio_ReadDone; audio_stream_parameter[0].ux_device_class_audio_stream_parameter_max_frame_buffer_nb = 2; audio_stream_parameter[0].ux_device_class_audio_stream_parameter_max_frame_buffer_size = 196; // USBD_AUDIO_EPIN_FS_MPS; audio_stream_parameter[0].ux_device_class_audio_stream_parameter_thread_entry = ux_device_class_audio_write_thread_entry; /* Set the parameters for Audio device. */ audio_parameter.ux_device_class_audio_parameter_streams_nb = 1; audio_parameter.ux_device_class_audio_parameter_streams = audio_stream_parameter; audio_parameter.ux_device_class_audio_parameter_callbacks.ux_slave_class_audio_instance_activate = Audio_Init; audio_parameter.ux_device_class_audio_parameter_callbacks.ux_slave_class_audio_instance_deactivate = Audio_DeInit; audio_parameter.ux_device_class_audio_parameter_callbacks.ux_device_class_audio_control_process = Audio_Control; /* Get audio configuration number */ audio_configuration_number = USBD_Get_Configuration_Number(CLASS_TYPE_AUDIO, 0); /* Find audio interface number */ audio_interface_number = USBD_Get_Interface_Number(CLASS_TYPE_AUDIO, 0); /* Initialize the device Audio class. This class owns interfaces starting with 0. */ if( ux_device_stack_class_register(_ux_system_slave_class_audio_name, ux_device_class_audio_entry, audio_configuration_number, audio_interface_number, &audio_parameter) != UX_SUCCESS) { return UX_ERROR; } ```

It seems that this can be related to similar issue https://github.com/azure-rtos/usbx/issues/58 .

gamelaster commented 1 year ago

Update: I ported the code to use Audio 1.0, as it should be better for USB FS. Now the Audio is probed properly, and shows as Microphone in system. Sadly ux_device_class_audio_stream_change or ux_device_class_audio_stream_change is not called.

gamelaster commented 1 year ago

I was able to figure the issue after some painful debugging. This confirms that issue is not in USBX nor STM32's USBX controller drivers.

I will write here some of my notes, which could be useful for anyone else playing with USB on STM32 platform.

1) Always verify that your descriptor is correct. Test it both on Linux and Windows, most of times, Linux is more verbose about the problems. In my case of UAC 1.0 Microphone implementation, I forgot to change baInterfaceNr to correct interface number of Audio Stream. (Always check information from official USB specifications)

2) If things doesn't work , it is good to dig into the source code and look for calls which might be related, check out with Wireshark if when packet is sent from host to device, if USB OTG interrupt is triggered, and if it replies. If it replies, continue and check if the code will reach the function / callback you need.

3) My audio callbacks were not called, because I used endpoint ID, which was outside of supported endpoints range. I didn't knew about this, because I didn't knew how error handling and logging worked, because the documentation is really... weak, so I found the issue only by debugging in the USBX. Although, I found this useful function ux_utility_error_callback_register , which will call your callback when error occurs! Very handy!

4) I didn't received callbacks about successful "delivery" of IN packets, this was because I was missing USBD_HAL_ISOINCOMPLETE_CALLBACK definition in ux_stm32 header. (Of course, this is nowhere documented.)

5) Endpoint's wMaxPacketSize, HAL_PCD_SetTxFiFo and buffer size in audio parameters are somehow related. After proper configuration of those I was able to get smooth and nice working audio stream from device. I wasn't able to fully understood how it works (yet) 😅

xiaocq2001 commented 1 year ago

Good news. Thanks for the notes very much.

ra1nb0w commented 5 months ago

Hi, I am trying to implement the same composite device (CDC-ACM plus full-duplex UAC1.0). For the last, I can't find working examples and since STM32Cube doesn't generate the basic skeleton it is not easy to manage everything from scratch. @gamelaster, can you kindly share your code as example? thank you very much

gamelaster commented 5 months ago

hi @ra1nb0w . I am sorry, but I can't provide the example :( although, in this issue should be everything to make all of this functional.

ra1nb0w commented 5 months ago

Thanks for your answer. I started using https://github.com/STMicroelectronics/x-cube-azrtos-h7/tree/dev/usbx/Projects/STM32H743I-EVAL/Applications/USBX/Ux_Device_Audio as base. Have a nice weekend

gamelaster commented 5 months ago

@ra1nb0w yes, that's what I was inspiring from. If you will have any issue, let me know here.

ra1nb0w commented 5 months ago

Which changes have you done to move to UAC 1.0? Since it is more compatible with older OSs I would like to use that. It is an hobby project to interface an amateur radio to the computer with a single cable. Have you also tried to implement full-duplex? Thank you

gamelaster commented 5 months ago

Which changes have you done to move to UAC 1.0?

I am sorry, I do not fully understand what are you asking. About UAC1.0, I am not aware of any older OS which would have a problem with UAC 2.0. The reason why I used UAC 1.0 is because it's easier to implement.

Have you also tried to implement full-duplex?

Full-duplex, you mean having both microphone and speakers? Answer is no, I tried only microphone.

ra1nb0w commented 5 months ago

I am sorry, I do not fully understand what are you asking. About UAC1.0, I am not aware of any older OS which would have a problem with UAC 2.0. The reason why I used UAC 1.0 is because it's easier to implement.

UAC 2.0 was introduced natively in windows 10 r1703. Generally, *nix are way more supported. Moreover, clock domains are not always easy to manage. These are the main reasons why I prefer to use UAC 1.0 and not 2.0. The question was: except the descriptor, have you changed something else? you are able to share the descriptor part?

gamelaster commented 5 months ago

Oh, then my assumption was wrong, I did not look into UAC2.0. Thanks for correcting me :)

About descriptor, part of it is here in my first post, sadly, I can't share the final descriptor. Although, what problem do you have right now? When I was implementing this, I was trying to do 1:1 copy of sample descriptor in UAC1.0 manual. Do you have issues with Windows not accepting the descriptor?