h2zero / NimBLE-Arduino

A fork of the NimBLE library structured for compilation with Arduino, for use with ESP32, nRF5x.
https://h2zero.github.io/NimBLE-Arduino/
Apache License 2.0
667 stars 138 forks source link

Support for new ESP32 Arduino core (3.x) and IDF (5.1) #641

Closed shlomozippel closed 4 months ago

shlomozippel commented 4 months ago

I started working with the new arduino core and espidf drivers and the NimBLE-Arduino library isn't compatible anymore. There are breaking changes on the arduino side (https://docs.espressif.com/projects/arduino-esp32/en/latest/migration_guides/2.x_to_3.0.html) and on the espidf side (https://docs.espressif.com/projects/esp-idf/en/stable/esp32/migration-guides/release-5.x/5.0/bluetooth-low-energy.html)

I didn't dig too deep yet but I noticed NimBLE-Arduino has it's own copy of nimble, presumably a version that has been ported to idf 4.x

h2zero commented 4 months ago

What is it that is no longer compatible? This library doesn't really use any Arduino core functions so there shouldn't be any changes required from there. This does contain a slightly older NimBLE version but that should not matter.

shlomozippel commented 4 months ago

When I first build it I get:


                 from .pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/porting/nimble/include/os/os.h:42,
                 from .pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/nimble/include/nimble/ble.h:26,
                 from .pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEAddress.h:23,
                 from .pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEAddress.cpp:19:
.pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/porting/npl/freertos/include/nimble/nimble_npl_os.h:457:4: error: 'esp_timer_handle_t' does not name a type; did you mean 'TimerHandle_t'?
  457 |    esp_timer_handle_t handle;
      |    ^~~~~~~~~~~~~~~~~~```

So I tried adding `CONFIG_BT_NIMBLE_USE_ESP_TIMER=n` to sdkconfig, which then resulted in a whole bunch of other errors:

```.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:90:62: error: 'CONFIG_BTDM_SCAN_DUPL_TYPE' was not declared in this scope; did you mean 'CONFIG_BTDM_SCAN_DUPL_TYPE_DATA'?
   90 | uint8_t                     NimBLEDevice::m_scanFilterMode = CONFIG_BTDM_SCAN_DUPL_TYPE;
      |                                                              ^~~~~~~~~~~~~~~~~~~~~~~~~~
      |                                                              CONFIG_BTDM_SCAN_DUPL_TYPE_DATA
In file included from .pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/porting/nimble/include/syscfg/syscfg.h:9,
                 from .pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/nimble/include/nimble/ble.h:25,
                 from .pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEAddress.h:23,
                 from .pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEAdvertisedDevice.h:20,
                 from .pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEScan.h:20,
                 from .pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.h:22,
                 from .pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:18:
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp: In static member function 'static void NimBLEDevice::init(const std::string&)':
.pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/esp_port/port/include/esp_nimble_cfg.h:13:49: error: 'MYNEWT_VAL_BLE_TRANSPORT_EVT_COUNT' was not declared in this scope; did you mean 'MYNEWT_VAL_BLE_HCI_ACL_OUT_COUNT'?
   13 | #define MYNEWT_VAL(x)                           MYNEWT_VAL_ ## x
      |                                                 ^~~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/common/api/include/api/../../../../controller/esp32c2/esp_bt_cfg.h:41:48: note: in expansion of macro 'MYNEWT_VAL'
   41 |     #define DEFAULT_BT_LE_HCI_EVT_HI_BUF_COUNT MYNEWT_VAL(BLE_TRANSPORT_EVT_COUNT)
      |                                                ^~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/include/esp32c2/include/esp_bt.h:235:33: note: in expansion of macro 'DEFAULT_BT_LE_HCI_EVT_HI_BUF_COUNT'
  235 |     .ble_hci_evt_hi_buf_count = DEFAULT_BT_LE_HCI_EVT_HI_BUF_COUNT,                     \
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:869:45: note: in expansion of macro 'BT_CONTROLLER_INIT_CONFIG_DEFAULT'
  869 |         esp_bt_controller_config_t bt_cfg = BT_CONTROLLER_INIT_CONFIG_DEFAULT();
      |                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/esp_port/port/include/esp_nimble_cfg.h:13:49: error: 'MYNEWT_VAL_BLE_TRANSPORT_EVT_DISCARDABLE_COUNT' was not declared in this scope; did you mean 'CONFIG_BT_NIMBLE_TRANSPORT_EVT_DISCARD_COUNT'?
   13 | #define MYNEWT_VAL(x)                           MYNEWT_VAL_ ## x
      |                                                 ^~~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/common/api/include/api/../../../../controller/esp32c2/esp_bt_cfg.h:42:48: note: in expansion of macro 'MYNEWT_VAL'
   42 |     #define DEFAULT_BT_LE_HCI_EVT_LO_BUF_COUNT MYNEWT_VAL(BLE_TRANSPORT_EVT_DISCARDABLE_COUNT)
      |                                                ^~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/include/esp32c2/include/esp_bt.h:236:33: note: in expansion of macro 'DEFAULT_BT_LE_HCI_EVT_LO_BUF_COUNT'
  236 |     .ble_hci_evt_lo_buf_count = DEFAULT_BT_LE_HCI_EVT_LO_BUF_COUNT,                     \
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:869:45: note: in expansion of macro 'BT_CONTROLLER_INIT_CONFIG_DEFAULT'
  869 |         esp_bt_controller_config_t bt_cfg = BT_CONTROLLER_INIT_CONFIG_DEFAULT();
      |                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/esp_port/port/include/esp_nimble_cfg.h:13:49: error: 'MYNEWT_VAL_BLE_MAX_PERIODIC_ADVERTISER_LIST' was not declared in this scope; did you mean 'DEFAULT_BT_LE_MAX_PERIODIC_ADVERTISER_LIST'?
   13 | #define MYNEWT_VAL(x)                           MYNEWT_VAL_ ## x
      |                                                 ^~~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/common/api/include/api/../../../../controller/esp32c2/esp_bt_cfg.h:32:56: note: in expansion of macro 'MYNEWT_VAL'
   32 |     #define DEFAULT_BT_LE_MAX_PERIODIC_ADVERTISER_LIST MYNEWT_VAL(BLE_MAX_PERIODIC_ADVERTISER_LIST)
      |                                                        ^~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/include/esp32c2/include/esp_bt.h:237:29: note: in expansion of macro 'DEFAULT_BT_LE_MAX_PERIODIC_ADVERTISER_LIST'
  237 |     .ble_ll_sync_list_cnt = DEFAULT_BT_LE_MAX_PERIODIC_ADVERTISER_LIST,                 \
      |                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:869:45: note: in expansion of macro 'BT_CONTROLLER_INIT_CONFIG_DEFAULT'
  869 |         esp_bt_controller_config_t bt_cfg = BT_CONTROLLER_INIT_CONFIG_DEFAULT();
      |                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/esp_port/port/include/esp_nimble_cfg.h:13:49: error: 'MYNEWT_VAL_BLE_TRANSPORT_ACL_SIZE' was not declared in this scope; did you mean 'CONFIG_BT_NIMBLE_TRANSPORT_ACL_SIZE'?
   13 | #define MYNEWT_VAL(x)                           MYNEWT_VAL_ ## x
      |                                                 ^~~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/common/api/include/api/../../../../controller/esp32c2/esp_bt_cfg.h:35:40: note: in expansion of macro 'MYNEWT_VAL'
   35 |     #define DEFAULT_BT_LE_ACL_BUF_SIZE MYNEWT_VAL(BLE_TRANSPORT_ACL_SIZE)
      |                                        ^~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/include/esp32c2/include/esp_bt.h:255:25: note: in expansion of macro 'DEFAULT_BT_LE_ACL_BUF_SIZE'
  255 |     .ble_acl_buf_size = DEFAULT_BT_LE_ACL_BUF_SIZE,                                     \
      |                         ^~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:869:45: note: in expansion of macro 'BT_CONTROLLER_INIT_CONFIG_DEFAULT'
  869 |         esp_bt_controller_config_t bt_cfg = BT_CONTROLLER_INIT_CONFIG_DEFAULT();
      |                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/esp_port/port/include/esp_nimble_cfg.h:13:49: error: 'MYNEWT_VAL_BLE_TRANSPORT_ACL_FROM_LL_COUNT' was not declared in this scope; did you mean 'CONFIG_BT_NIMBLE_TRANSPORT_ACL_FROM_LL_COUNT'?
   13 | #define MYNEWT_VAL(x)                           MYNEWT_VAL_ ## x
      |                                                 ^~~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/common/api/include/api/../../../../controller/esp32c2/esp_bt_cfg.h:36:41: note: in expansion of macro 'MYNEWT_VAL'
   36 |     #define DEFAULT_BT_LE_ACL_BUF_COUNT MYNEWT_VAL(BLE_TRANSPORT_ACL_FROM_LL_COUNT)
      |                                         ^~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/include/esp32c2/include/esp_bt.h:256:26: note: in expansion of macro 'DEFAULT_BT_LE_ACL_BUF_COUNT'
  256 |     .ble_acl_buf_count = DEFAULT_BT_LE_ACL_BUF_COUNT,                                   \
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:869:45: note: in expansion of macro 'BT_CONTROLLER_INIT_CONFIG_DEFAULT'
  869 |         esp_bt_controller_config_t bt_cfg = BT_CONTROLLER_INIT_CONFIG_DEFAULT();
      |                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/nimble/esp_port/port/include/esp_nimble_cfg.h:13:49: error: 'MYNEWT_VAL_BLE_TRANSPORT_EVT_SIZE' was not declared in this scope; did you mean 'MYNEWT_VAL_BLE_HCI_TRANSPORT_EMSPI'?
   13 | #define MYNEWT_VAL(x)                           MYNEWT_VAL_ ## x
      |                                                 ^~~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/common/api/include/api/../../../../controller/esp32c2/esp_bt_cfg.h:37:44: note: in expansion of macro 'MYNEWT_VAL'
   37 |     #define DEFAULT_BT_LE_HCI_EVT_BUF_SIZE MYNEWT_VAL(BLE_TRANSPORT_EVT_SIZE)
      |                                            ^~~~~~~~~~
/Users/shlomo/.platformio/packages/framework-espidf/components/bt/include/esp32c2/include/esp_bt.h:257:29: note: in expansion of macro 'DEFAULT_BT_LE_HCI_EVT_BUF_SIZE'
  257 |     .ble_hci_evt_buf_size = DEFAULT_BT_LE_HCI_EVT_BUF_SIZE,                             \
      |                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:869:45: note: in expansion of macro 'BT_CONTROLLER_INIT_CONFIG_DEFAULT'
  869 |         esp_bt_controller_config_t bt_cfg = BT_CONTROLLER_INIT_CONFIG_DEFAULT();
      |                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:873:16: error: 'struct esp_bt_controller_config_t' has no member named 'mode'
  873 |         bt_cfg.mode = ESP_BT_MODE_BLE;
      |                ^~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:874:16: error: 'struct esp_bt_controller_config_t' has no member named 'ble_max_conn'
  874 |         bt_cfg.ble_max_conn = CONFIG_BT_NIMBLE_MAX_CONNECTIONS;
      |                ^~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:876:16: error: 'struct esp_bt_controller_config_t' has no member named 'normal_adv_size'
  876 |         bt_cfg.normal_adv_size = m_scanDuplicateSize;
      |                ^~~~~~~~~~~~~~~
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEDevice.cpp:877:16: error: 'struct esp_bt_controller_config_t' has no member named 'scan_duplicate_type'
  877 |         bt_cfg.scan_duplicate_type = m_scanFilterMode;
      |                ^~~~~~~~~~~~~~~~~~~
*** [.pio/build/usba-c2/lib41b/NimBLE-Arduino/NimBLEDevice.cpp.o] Error 1
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEScan.cpp: In member function 'void NimBLEScan::clearDuplicateCache()':
.pio/libdeps/usba-c2/NimBLE-Arduino/src/NimBLEScan.cpp:456:5: error: 'esp_ble_scan_dupilcate_list_flush' was not declared in this scope
  456 |     esp_ble_scan_dupilcate_list_flush();
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
*** [.pio/build/usba-c2/lib41b/NimBLE-Arduino/NimBLEScan.cpp.o] Error 1```

Also, important to note that these errors only happen when my board is ESP32-C2. I get different errors when I build for ESP32-S3:

```In file included from .pio/libdeps/s3/NimBLE-Arduino/src/nimble/porting/nimble/include/modlog/modlog.h:30,
                 from .pio/libdeps/s3/NimBLE-Arduino/src/nimble/nimble/host/include/host/ble_hs_log.h:23,
                 from .pio/libdeps/s3/NimBLE-Arduino/src/nimble/nimble/host/include/host/ble_hs.h:38,
                 from .pio/libdeps/s3/NimBLE-Arduino/src/nimble/nimble/host/include/host/ble_gap.h:31,
                 from .pio/libdeps/s3/NimBLE-Arduino/src/nimble/nimble/host/src/ble_gap_priv.h:26,
                 from .pio/libdeps/s3/NimBLE-Arduino/src/nimble/nimble/host/src/ble_hs_priv.h:27,
                 from .pio/libdeps/s3/NimBLE-Arduino/src/nimble/nimble/host/src/ble_gap.c:26:
.pio/libdeps/s3/NimBLE-Arduino/src/nimble/nimble/host/src/ble_gap.c: In function 'ble_gap_log_duration':
/Users/shlomo/.platformio/packages/framework-espidf/components/log/include/esp_log.h:265:27: error: format '%d' expects argument of type 'int', but argument 6 has type 'int32_t' {aka 'long int'} [-Werror=format=]```

and more similar warnings that are treated as errors.

Feels like a combination of upgrading from idf 4.x to 5.x where some of the ble / nimble headers changed, and probably some build flags changed too
shlomozippel commented 4 months ago

Platformio.ini to repro:

[env]
framework = arduino, espidf
platform = https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5
lib_deps = 
  h2zero/NimBLE-Arduino @ ^1.4.1

[env:usba-c2]
board = esp32-c2-devkitm-1
h2zero commented 4 months ago

Oh, using this with IDF is not recommended since you have the option there to enable nimble. You will want to switch from this repo to esp-nimble-cpp.

shlomozippel commented 4 months ago

Thanks, confirming esp-nimble-cpp worked 🎉

I also tested NimBLE-Arduino with arduino framework (without espidf) and still ran into the same errors with the new esp32 core v3.x, so leaving this issue open for now

h2zero commented 4 months ago

I also tested NimBLE-Arduino with arduino framework (without espidf) and still ran into the same errors with the new esp32 core v3.x, so leaving this issue open for now

This is due to using platform = https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5; in that fork he has enabled NimBLE in the config, the official release will not.

h2zero commented 4 months ago

Closing this as it is compiling and working fine with the latest Arduino core master branch. The issue here is using a third party core which provides the NimBLE core sources via IDF config.