zephyrproject-rtos / zephyr

Primary Git Repository for the Zephyr Project. Zephyr is a new generation, scalable, optimized, secure RTOS for multiple hardware architectures.
https://docs.zephyrproject.org
Apache License 2.0
10.7k stars 6.54k forks source link

Bluetooth Controller: Coexistence GPIO feature #50463

Closed optical-o closed 1 year ago

optical-o commented 2 years ago

The SoftDevice Bluetooth controller(CONFIG_BT_LL_SW_SPLIT) got in recent release of NCS 2.1.0 coexistance feature using GPIO pins with other RF systems on board. We are looking into using Coexistence feature on NRF9160 and NRF52833 combo for Bluetooth coexistance with the LTE chip.

Is there a plan to integrate this feature into Zephyr Bluetooth controller ?

Our main use-case is Bluetooth advertisement. For the first implementation it would be sufficient to ignore radio transmissions when LTE COEX pin indicates that it is transmitting. Much more advance implementation, which would take into account connection with other Bluetooth device such as phones and etc. Would probably need to reschedule the transmissions.

Note: We are not really fans of using the closed sourced SoftDevice Controller.

kruithofa commented 2 years ago

Currently there is no plan for this

optical-o commented 2 years ago

Is there somebody who could guide me thought the process, who could tell me where is the best place to implement this functionality ? I'm willing to give it a go, if no one experienced has time to do it. I do not have any special equipment to verify it is working correctly in realtime. Only to test when i drive the coex pin that the Bluetooth advertisements will stop.

cvinayak commented 2 years ago

@optical-o Hi, I am maintaining the Controller, but adding Co-Ex is not in plan. As you have expressed interest in implementing, I can help in guiding you. But before that I would like to get the details of the protocol on the pin.

You can start by documenting here the protocol as a timing diagram. If you have a logic analyzer, you should be able to get real-time Bluetooth scheduling on GPIO by CONFIG_BT_CTLR_DEBUG_PINS=y of the current radio events. You can then prepare something that can be visualized as the requirement to achieve based on the timing diagram of Co-Ex pin. I should hopefully be able to guide you to the implementation where the needed changes are to be done.

cfriedt commented 2 years ago

Just my 2c, but latency will likely need to be a major consideration here. The Zephyr GPIO abstraction has some cost, even when used in kernel mode. It might need to be delegated to use hal-specific functions.

cvinayak commented 2 years ago

Just my 2c, but latency will likely need to be a major consideration here. The Zephyr GPIO abstraction has some cost, even when used in kernel mode. It might need to be delegated to use hal-specific functions.

Yes, the Controller to most extent is bare-metal, and will have to keep that way to meet soft real-time requirements.

optical-o commented 2 years ago

Firstly, i have raised an Devzone Ticket for further clarify the functionality of the COEX interface. Their documentation is quite lacking proper description of functionality.

Regarding the direction of from the Bluetooth controller to the LTE modem using COEX0, the external PA GPIO functionality, would be copied or utilized 'radio_gpio_pa_lna_enable' functionality. That one should be the easy one. In this case the Bluetooth would assert the pin during the tx transmision.

Regarding the other direction from LTE Modem to Bluetooth controller using COEX2 pin. There i have several question:

I will try to find time to create timing diagrams of the Radio events. I look at the Radio Events, which are available at the GPIO pins. The main events of interested i assume are the RADIO ones (PREPARE, START, STOP and CLOSE). Should i be also interested in the TICKER ones?

I assume that these event should correspond to the one inside nRF Radio documentation. Can i ask which of the transmit sequences is used inside the controller ? I assume that it will be the "Transmit sequence using shortcuts to avoid delays". Does the controller also uses transmission of multiple packets back to back ?

cvinayak commented 2 years ago

I would start with designing based on the requirement of the protocol.

This is what is happening in the timeline of a connectable advertiser, in the Zephyr Controller implementation (do not get confused with product specification radio states, they are for generic use): image

To this we need to add the CoEx protocol and decide what do we need the Controller to do on the CoEx transitions.

Firstly, i have raised an Devzone Ticket for further clarify the functionality of the COEX interface. Their documentation is quite lacking proper description of functionality.

ok. (as mentioned before, I may not be able to spend a lot of time to follow the discussions).

Regarding the direction of from the Bluetooth controller to the LTE modem using COEX0, the external PA GPIO functionality, would be copied or utilized 'radio_gpio_pa_lna_enable' functionality. That one should be the easy one. In this case the Bluetooth would assert the pin during the tx transmision.

Good, need to check if there are available PPI/DPPI and GPIOTE to achieve this side-by-side with PA/LNA.

What to do when we already ramped up the radio and we are in middle of transmission ? Should we assume that when we are using the COEX0 functionality this won't happen ?

Not sure. May be we can simply disable radio? Controller will gracefully handle such radio disable already.

When the controller detects the COEX0 should it ignore any event, which are planned should or can it even according Bluetooth specification somehow postpone the events ? Or should we just ignore these (in case of advertisements). ?

In the case of advertising events, it can be postponed say upto 10ms, or skipped to the next interval.

The main events of interested i assume are the RADIO ones (PREPARE, START, STOP and CLOSE). Should i be also interested in the TICKER ones?

The Zephyr Controller is architectured into these execution contexts: Thread, ULL_LOW, ULL_HIGH and LLL. CoEx interaction will be restricted to LLL execution context. In LLL, the execution points are prepare_cb, isr_tx, isr_rx, is_abort, abort_cb, isr_done to name some. Based on the CoEx protocol, each state or role will need to deal the CoEx requirements in its execution points.

Can i ask which of the transmit sequences is used inside the controller ? I assume that it will be the "Transmit sequence using shortcuts to avoid delays". Does the controller also uses transmission of multiple packets back to back ?

controller uses custom transmit and receive sequences using PPI/DPPI sw_switch implementation in the radio.c file.

optical-o commented 2 years ago

ok. (as mentioned before, I may not be able to spend a lot of time to follow the discussions).

It is fine that is just information, for us where i will be getting more information about timing of the CoEx. I really appreciate you guiding me in the implementation.

Not sure. May be we can simply disable radio? Controller will gracefully handle such radio disable already.

Maybe for the first implementation, we could support coexistance using only the COEX2 type of signal. Which indicates that the LTE modem will be active. Could we simply add simple skipping to next advertisement interval. This would resolve Bluetooth transmissions, which are not in progress.

Not sure. May be we can simply disable radio? Controller will gracefully handle such radio disable already.

For the ongoing could we somehow utilize another PPI channel to disable the radio, when it is in TX Enabled state. The PPI channel would be enabled when the radio enters TX State. Would this be possible to achieve with current controller implementation as you mentioned ?

Would this solution also cover other functionality of the controller such as Bluetooth connections and etc. ? Are there some time constrains, which apply the controller is connected to other bluetooth device ? I'm afraid of Bluetooth connection timeouts due to ignoring some transmissions. This is just to prepare for later parts of the implementation, first step is the advertisement.

Good, need to check if there are available PPI/DPPI and GPIOTE to achieve this side-by-side with PA/LNA.

This would be maybe first steps to support single wire CoEx. Later i would add support for the COEX0 (BT Controller -> LTE Modem).

This is what is happening in the timeline of a connectable advertiser, in the Zephyr Controller implementation (do not get confused with product specification radio states, they are for generic use): The Zephyr Controller is architectured into these execution contexts: Thread, ULL_LOW, ULL_HIGH and LLL. CoEx interaction will be restricted to LLL execution context. In LLL, the execution points are prepare_cb, isr_tx, isr_rx, is_abort, abort_cb, isr_done to name some. Based on the CoEx protocol, each state or role will need to deal the CoEx requirements in its execution points.

I will use sample advertisement on nrf9160dk, with enabled CONFIG_BT_CTLR_DEBUG_PINS=y together with the COEX pins to profile them using a logic analyzer and start to look at mentioned functions, where to place the CoEx implementation.

As for the code semantics i suggest using existing DTS binding already created(or some variation) for the SoftDevices Implementation instead of classic Kconfig options, if you agree.

cvinayak commented 2 years ago

Not sure. May be we can simply disable radio? Controller will gracefully handle such radio disable already.

Maybe for the first implementation, we could support coexistance using only the COEX2 type of signal. Which indicates that the LTE modem will be active. Could we simply add simple skipping to next advertisement interval. This would resolve Bluetooth transmissions, which are not in progress.

Not sure. May be we can simply disable radio? Controller will gracefully handle such radio disable already.

For the ongoing could we somehow utilize another PPI channel to disable the radio, when it is in TX Enabled state. The PPI channel would be enabled when the radio enters TX State. Would this be possible to achieve with current controller implementation as you mentioned ?

Would this solution also cover other functionality of the controller such as Bluetooth connections and etc. ? Are there some time constrains, which apply the controller is connected to other bluetooth device ? I'm afraid of Bluetooth connection timeouts due to ignoring some transmissions. This is just to prepare for later parts of the implementation, first step is the advertisement.

Please refer to: ll_radio_state_abort() ll_radio_state_is_idle() and ticker function usage in https://github.com/zephyrproject-rtos/zephyr/blob/f738f494eb23188aa4a30939a60e9963643992ea/drivers/flash/soc_flash_nrf_ticker.c

I am thinking aloud here, like how flash driver co-operatively halts the CPU during active Bluetooth states and roles, the COEX2 type signal should setup a ticker instance to expire immediately reserving a typical time duration of the LTE modem radio operation. This ticker expiry would use ll_radio_state_abort() and ll_radio_state_is_idle() to abort any Bluetooth radio events active beyond it's reserved time. Ticker implementation will ensure no new Bluetooth events are scheduled until the COEX2 reservation elapses. Ticker functions also provide means to yield any earlier completion of COEX use. Ticker functional interface is flexible being callable from ISRs and thread, and is designed to be context-safe.

@nvlsianpu has done an excellent work with the ticker usage in the nRF flash driver, and he could provide his guidance too.

Ideally, would be nice to tightly couple the co-ex inside the Bluetooth Controller, but general feeling in the community is that the Controller is very complex implementation and has very steep learning curve.

Taking the ticker use like in the flash driver seems a reasonable solution for input signal.

Good, need to check if there are available PPI/DPPI and GPIOTE to achieve this side-by-side with PA/LNA.

This would be maybe first steps to support single wire CoEx. Later i would add support for the COEX0 (BT Controller -> LTE Modem).

Sound reasonable.

As for the code semantics i suggest using existing DTS binding already created(or some variation) for the SoftDevices Implementation instead of classic Kconfig options, if you agree.

Agree.

optical-o commented 2 years ago

| Taking the ticker use like in the flash driver seems a reasonable solution for input signal.

I will take a look at the nrf_flash ticker. It looks very accessible to use as guide. Next week i will hopefully find time to create first implementation.

| Good, need to check if there are available PPI/DPPI and GPIOTE to achieve this side-by-side with PA/LNA.

Have you identified, which PPI/DPPI channels to use for the COEX0 (BT Controller -> LTE Modem). ? I saw that their are defined in bt_ctlr_used_resources.h i'm not quite sure how their are allocated. I would assume that they should be KConfig defined or somewhere documented for the users to not cause conflict, when the user use the PPI/DPPI channels. Is there some process ?

I can make KConfig options in Bluetooth Controller Advanced section for these new channels and maybe fill in even the PA/LNA channels, if it would be desired.

cvinayak commented 2 years ago

Have you identified, which PPI/DPPI channels to use for the COEX0 (BT Controller -> LTE Modem). ? I saw that their are defined in bt_ctlr_used_resources.h i'm not quite sure how their are allocated. I would assume that they should be KConfig defined or somewhere documented for the users to not cause conflict, when the user use the PPI/DPPI channels. Is there some process ?

No. The define in bt_ctlr_used_resources.h is used by zephyr drivers based on those that use PPI/DPPI and dynamically allocate the free ones. The ones used by Controller is reserved using that header file.

The Controller so far has very little documentation, but most of the ones are privately accessible in the form of presentations we hold in our regular F2F meetings with contributing member companies. There is nothing private about the documentation, just that we have not gotten to send in PRs.

I have started some work in #50842, also I will try to fix #43562 as part of which I will include the PPI/DPPI indices currently used.

I can make KConfig options in Bluetooth Controller Advanced section for these new channels and maybe fill in even the PA/LNA channels, if it would be desired.

Use devicetree, PA/LNA already have devicetree definitions.

optical-o commented 2 years ago

Hello, so i looked at the implementation of the soc_flash_nrf_ticker.c. I draw an flow diagram of its functionality for me to understand it correctly.

ticker_soc_flash

The implementation as i understands is using a low priority ticker to schedule the flash operation. When the low priority ticker timeouts it will start the high priority chain of aborting any ongoing radio operations and after wards executing the desired flash operation initiated by nrf_flash_sync_exe.

I have several questions to be able to adopt the same flow for the COEX functionality.

Firstly is whether the high priority ticker can in its callback be busy waiting for the COEX pin to be cleared. This would be pretty easy solution, which i suspect is not possible. Flow diagram for this implementation bellow:

ticker_coex

The other solution is for the high priority ticker to start itself again immediately to check the COEX pin again. Diagram bellow. Ideal solution would be to schedule the ticker with some period(not immediately). For this it would need some how block any tickers to be executed. In the soc flash implementation i can see the abort action ll_radio_state_abort. But i do not see any resuming of the radio operation. This i presume only aborts any ongoing radio operations. Is there a way to ticker to disable other tickers to be executed ?

ticker_coex

Example of implementation using blocking method (not tested): coex_ticker.zip

optical-o commented 2 years ago

@cvinayak I have obtained a nrf52 dk with logic analyzer. This week i will do some tests with actual implementation. Could you please answer the questions above to guide me with the actual ticker handler code ?

optical-o commented 2 years ago

So i implemented the ticker, which will abort any radio event, which are scheduled during assertion of the COEX Pins.

static void time_slot_callback_work(uint32_t ticks_at_expire,
                    uint32_t ticks_drift,
                    uint32_t remainder,
                    uint16_t lazy, uint8_t force,
                    void *context)
{
    uint8_t instance_index;
    uint8_t ticker_id;
    const struct device *dev = context;
    const struct coex_ticker_config *config = dev->config;

    __ASSERT(ll_radio_state_is_idle(),
         "Radio is on during flash operation.\n");

    /* Read inhibit pin */
    if(gpio_pin_get_dt(&config->inhibit_spec)) {
        if(!ll_radio_state_is_idle()) {
            ll_radio_state_abort();
        }
        /* Schedule another check for inhibit pin and abort any events scheduled */
        time_slot_delay(ticker_ticks_now_get(),
                0,
                time_slot_callback_work,
                context);
    } else {
        gpio_pin_interrupt_configure_dt(&config->inhibit_spec, GPIO_INT_EDGE_TO_ACTIVE);

        //If coex pin is low stop the ticker
        ll_timeslice_ticker_id_get(&instance_index, &ticker_id);

        /* Stop the time slot ticker */
        _ticker_stop(instance_index, 0, ticker_id);
    }
}

The ticker task is executed as expected during assertion of the COEX pin and schedule it self repeatedly until the COEX pin is deasserted. Note that this is dummy coexistance pin simulation by the processor it self. image

Advetisement event during coex deasserted: image

Advetisement event during coex asserted: image

@cvinayak Is this behavior acceptable for the BT controller ? Could you suggest the period of the radio abort task ?

cvinayak commented 2 years ago

@optical-o I will review your comments here on Friday, and reply back then.

optical-o commented 2 years ago

I have created pull request containing all of the changes made.

I have connected the COEX pin to actual nrf9160 Hardware with COEX2=4 configuration, which uses assertion as soon as possible only for TX RF operations. Minimal coexistence pin assertion, which i have observed was around ~1ms.

image

Currently i'm facing issue with the ticker scheduling issue of the ticker tasks. The COEX Inhibit in the picture should indicate the whole ticker duration when it is aborting radio events. However it is not called in some cases. The COEX GPIO IRQ handler is working fine.

optical-o commented 2 years ago

I have resolved the issue. The code should be functional. I have forgot to configure the pin as Input, however the IRQ logic still worked.

image

optical-o commented 1 year ago

@cvinayak I have prepared PR #51419, which should be somewhat functional and comply with the zephyr PR policies. Will you find some time to review it this week ?

cvinayak commented 1 year ago

@cvinayak I have prepared PR #51419, which should be somewhat functional and comply with the zephyr PR policies. Will you find some time to review it this week ?

Its been holidays last week, will try the PR this week.

optical-o commented 1 year ago

@cvinayak It has been a while since the post of the PR is there any chance for you to review it ? Do you need any additional information than the README provides ?