search

thesofproject / sof

Sound Open Firmware
Other
560 stars 318 forks source link

[FEATURE]support for rpmsg #5068

Open lenghonglin opened 2 years ago

lenghonglin commented 2 years ago

image

I see the ppt say sof already support rpmsg, but in the sof project , i have not find this part code, how's it going?

dbaluta commented 2 years ago

This is a thing that i'm interested in seeing it done. I think there was no progress on this and I didnt see any patches.

My goal would be to have a look at this in 2022.

Anyhow, if you have some time I think the support is already in Zephyr so the only thing that might be needed would be a driver in the Linux kernel to implement remoteproc/rpmsg.

lgirdwood commented 2 years ago

@lenghonglin this slide is quite old and out of data as plans evolved. rpmsg patches for the kernel exist that @lyakh worked on in 2020. They went few a few rounds of patch submissions but were de-prioritised due to an HDA based virtualization interface being upstreamed. @lyakh is back next Tuesday so could point you to his code if you want to complete it.

The CMSIS work morphed into Zephyr, which is now supported upstream.

lenghonglin commented 2 years ago

This is a thing that i'm interested in seeing it done. I think there was no progress on this and I didnt see any patches.

My goal would be to have a look at this in 2022.

Anyhow, if you have some time I think the support is already in Zephyr so the only thing that might be needed would be a driver in the Linux kernel to implement remoteproc/rpmsg.

Zephyr already support with SOF and OpenAMP(implement with IPM) , but But SOF still uses IPC3/IPC4.

I Think if SOF run on Zephyr, SOF shound call zephyr drivers, such dma , ipm, openamp. or will maintain two sets of drivers.

lenghonglin commented 2 years ago

@lenghonglin this slide is quite old and out of data as plans evolved. rpmsg patches for the kernel exist that @lyakh worked on in 2020. They went few a few rounds of patch submissions but were de-prioritised due to an HDA based virtualization interface being upstreamed. @lyakh is back next Tuesday so could point you to his code if you want to complete it.

The CMSIS work morphed into Zephyr, which is now supported upstream.

yes, we already implement RPMSG with Zephyr , If SOF support RPMSG , we don't need rewrite ipc

hongshui3000 commented 2 years ago

I very much hope that zephyr and sof can unify the driver framework so that we don't need to maintain two sets of drivers. Of course it may take time to do it, but I think it’s worth it. Perhaps a hardware abstraction layer and OS porting interface can be provided first to isolate the current platform and hardware drivers and OS. Once this is done, RTOS and hardware drivers can be developed relatively independently

dbaluta commented 2 years ago

@hongshui3000 I think that's the intention in the future. We talked about this in this presentation: https://youtu.be/9kPSG5q6gmA?t=518

hongshui3000 commented 2 years ago

@dbaluta By the way, Zephyr currently has some issues with high-priority interrupts and interrupt nesting on the xtensa architecture. I think nxp may also encounter these issues . Maybe you can give me some suggestions. thanks https://github.com/zephyrproject-rtos/zephyr/issues/40946 https://github.com/zephyrproject-rtos/zephyr/issues/41039 https://github.com/zephyrproject-rtos/zephyr/issues/40942 https://github.com/zephyrproject-rtos/zephyr/issues/40974

lgirdwood commented 2 years ago

@hongshui3000 - yes, plan is to move all arch/, platform/ and drivers to Zephyr.

lenghonglin commented 2 years ago

@lgirdwood good news

lgirdwood commented 2 years ago

@dbaluta By the way, Zephyr currently has some issues with high-priority interrupts and interrupt nesting on the xtensa architecture. I think nxp may also encounter these issues . Maybe you can give me some suggestions. thanks zephyrproject-rtos/zephyr#40946 zephyrproject-rtos/zephyr#41039 zephyrproject-rtos/zephyr#40942 zephyrproject-rtos/zephyr#40974

@hongshui3000 thanks for logging the issues, fwiw both @dcpleung @andyross are working on them.

lenghonglin commented 2 years ago

@hongshui3000 - yes, plan is to move all arch/, platform/ and drivers to zephyr (西风) .

Is there a detailed plan？ We are now in the process of adapting the SOF driver or the Zephyr driver.(DAI, DMA, IPC) Zephyr alread support DMA and IPC driver. For example: If adapt SOF dma driver, will lack Zephyr dma driver, the other driver can not use dma. If adapt Zephyr dma driver,There is no good way to be called by SOF.

lgirdwood commented 2 years ago

@lenghonglin maybe this is something you want to work alongside @juimonen and discuss.

Fwiw, the general work going on now is @dcpleung is working on taking the DMA drivers into zephyr IIUC ? @andyross is working on the IPC and IDC drivers, this should be ready next week where @lyakh will adapt the SOF code to use the new Zephyr IPC/IDC drivers. @aiChaoSONG is working on changing the SOF log/trace calls to use native Zephyr LOG() API.

We have a plan to take the Zephyr i2s.h driver API and extend it as a generic DAI API (to support DMIC, HDA, SDW and other types). Once this is upstream in Zephyr we then plan to add the SOF audio drivers into Zephyr and the DMA drivers. @juimonen pls update any progress you have here.

lenghonglin commented 2 years ago

@lenghonglin maybe this is something you want to work alongside @juimonen and discuss.

Fwiw, the general work going on now is @dcpleung is working on taking the DMA drivers into zephyr IIUC ? @andyross is working on the IPC and IDC drivers, this should be ready next week where @lyakh will adapt the SOF code to use the new Zephyr IPC/IDC drivers. @aiChaoSONG is working on changing the SOF log/trace calls to use native Zephyr LOG() API.

We have a plan to take the Zephyr i2s.h driver API and extend it as a generic DAI API (to support DMIC, HDA, SDW and other types). Once this is upstream in Zephyr we then plan to add the SOF audio drivers into Zephyr and the DMA drivers. @juimonen pls update any progress you have here.

good, I wanted to be more involved because we had this problem with sof. The other things is that our soc is internally integrated with codec. How do we abstract this kind of interface?

lgirdwood commented 2 years ago

The other things is that our soc is internally integrated with codec. How do we abstract this kind of interface?

Probably a good to create a codec interface in Zephyr. It should include the DAI API and it should also create APIs for PM, clocks, jack insert, controls. Please take a look at Linux ASoC codec interface as most concepts can be reused. Feel free to send me an email to discuss.

lenghonglin commented 2 years ago

The other things is that our soc is internally integrated with codec. How do we abstract this kind of interface?

Probably a good to create a codec interface in Zephyr. It should include the DAI API and it should also create APIs for PM, clocks, jack insert, controls. Please take a look at Linux ASoC codec interface as most concepts can be reused. Feel free to send me an email to discuss.

I mean maybe we should re-design Zephyr Codec API. we could talk in this channel https://discord.com/channels/720317445772017664/930855494472589362

lgirdwood commented 2 years ago

@juimonen can you share your DAI work with @lenghonglin for discussion. Thanks