Bt bringup

[tao-j]

with the help of #30 now additional library can be put into the flash.

Trying to enable BT using NimbleBLE. But so far, I am unable to pair with the device. in freemote PR I made. It is obvious and tested that the manual focus can be controlled in a fine grained way. Their nrf52 has a 'soft device' which is essentially a closed source library/binary version of what nimble is trying to achieve (and it does that well).

I found the BT remote in your pic. And given the ptp manual focus is far from perfect, so it might worth a try to figure out how to do it, but right now I could not figure out how. Stuck at waiting a BT event from the camera after initiating the ->SecureConnection() call. And it timed out.

[frank26080115]

I feel like it would be good to encapsulate a bluetooth connection to a camera as a class on its own, and have it be a library that can be put into other projects

have you tried just not using security? unless I missed something, the freemote code doesn't have security

[tao-j]

Definitely yes for library, after fixing the bt pair (bond/security, whatever we call it), it will be put into a library.

in freemote they have conn->requestPairing(); which jumps to

bool BLEConnection::requestPairing(void)
{
  // skip if already paired
  if ( secured() ) return true;

  return Bluefruit.Security._authenticate(_conn_hdl);
}

which jumps to

bool BLESecurity::_authenticate(uint16_t conn_hdl)
{
  VERIFY_STATUS(sd_ble_gap_authenticate(conn_hdl, &_sec_param ), false);
  return true;
}

The sd_ble is the nrf52's function calling soft device 'to pair'. This is to setup a Bonded connection, where two party exchange key and cipher and use it later. My guess is nimble will save it in ESP32's nvm partition, nrf52's sd may have similar thing as well.

Since you have the actual remote, would you confirm/test that it requires first using camera's "bluetooth cfg -> pairing" and turn on the remote then the camera will show "pairing request from xxxx Yes/No". After select yes on camera, next time this remote can directly connect to camera and control it without pairing again.

To simulate this behavior, For Linux, I have to pair it in the system menu first, and then use the script in the freemote PR I opened to simulate the remote. If it is not paired(bonded), it can connect to the camera, but as long as a write request to the characteristic(0xff01), the camera will drop the connection (for safety, authenticated control only).

This is also the same for nrf connect app on Android. It has to be paired in system settings, then in the app it will show 'Bonded', after this I can send cmd and not being dropped. If not paired, but directly connect to the camera (without security), it just disconnects after any write to characteristics.

[frank26080115]

Since you have the actual remote, would you confirm/test that it requires first using camera's "bluetooth cfg -> pairing" and turn on the remote then the camera will show "pairing request from xxxx Yes/No". After select yes on camera, next time this remote can directly connect to camera and control it without pairing again.

confirmed

maybe this becomes a mashup of https://github.com/maxmacstn/ESP32-Canon-BLE-Remote/tree/master and freemote? Act

can you call loop2() from inside setup2()? I wouldn't want to mix in all my other init crap while experimenting with BLE

omg especially settings, I'm using EEPROM emulation and that probably screws with NimBLE's usage of NVS

[tao-j]

Thanks for the hint, actually I created an empty project with only ble stuff. However, judging by the debug output of nimble (which can be enabled by modifying header file nimconfig.h), it stuck at the same place like the previous commit in this PR.

I also erased the whole flash before tried this bt only crap to ensure nvm is empty.

Here is the log, in case someone with enough bt knowledge can spot some way out.

Starting NimBLE Client
I NimBLEDevice: BLE Host Task Started
I NimBLEDevice: NimBle host synced.
D NimBLEDevice: Setting bonding: 1, mitm: 0, sc: 1
D NimBLEDevice: >> setPower: 7 (type: 11)
D NimBLEDevice: << setPower
D NimBLEDevice: >> setLocalMTU: 517
D NimBLEDevice: << setLocalMTU
D NimBLEScan: >> start: duration=0
D NimBLEScan: << start()
I NimBLEScan: New advertiser: d0:40:ef:44:44:44
I NimBLEScan: Updated advertiser: d0:40:ef:44:44:44
Found Sony Camera
D NimBLEScan: >> stop()
Scan Ended
D NimBLEScan: << stop()
New client created
D NimBLEClient: >> connect(d0:40:ef:44:44:44)
D NimBLEClient: Got Client event BLE_GAP_EVENT_CONNECT 
I NimBLEClient: Connected event
D NimBLEClient: Got Client event BLE_GAP_EVENT_MTU
I NimBLEClient: mtu update event; conn_handle=0 mtu=517
I NimBLEClient: Connection established
D NimBLEClient: >> deleteServices
D NimBLEClient: << deleteServices
Connected -------------   on connection
D NimBLEClient: << connect()
D NimBLEClient: >> secureConnection()

Stuck at previous line for about 15-30s, it timed out(?). Then error output is like below: rt=13 sounds like timeout.

D NimBLEClient: Got Client event BLE_GAP_EVENT_ENC_CHANGE
E NimBLEClient: secureConnection: failed rc=13
===========-==-=========-=-=-=-=------------------================
Connected to: d0:40:ef:44:44:44
RSSI: D NimBLEClient: >> getRssi()
-32
D NimBLEClient: >> getService: uuid: 8000ff00-ff00-ffff-ffff-ffffffffffff
D NimBLEClient: >> retrieveServices
D NimBLEClient: Service Discovered >> status: 7 handle: -1
E NimBLEClient: serviceDiscoveredCB() rc=7 No open connection with the specified handle.
D NimBLEClient: << Service Discovered
E NimBLEClient: Could not retrieve services
D NimBLEClient: Got Client event BLE_GAP_EVENT_DISCONNECT
D NimBLEClient: << getService: not found
remoteI NimBLEClient: disconnect; reason=531, Remote User Terminated Connection
 service not found.
Done with this device!
d0:40:ef:44:44:44 Success! we should now be getting notifications, scanning for more!
 Disconnected - not Starting scan

Strangely BLE_GAP_EVENT_ENC_CHANGE is received after timeout.

I was able to trace into secureConnection() call in NimbleBLEClient.cpp. And it called ble_gap_security_initiate in ble_gap.c. (nrf52 equivalent of sd_ble_gap_authenticate, maybe). This call did not block. But there is ulTaskNotifyTake(pdTRUE, portMAX_DELAY); which blocked the execution. It seems that only NimBLEClient::handleGapEvent has xTaskNotifyGive to unblock that (this is my first time working free rtos, so I might be wrong)

All that being said, it seems that the ble_gap_security_initiate is not sending good info to the camera so that camera recognize this pairing request. Maybe some security flags are not set properly (but I tried about 2^4=16 combinations), or it could be nimble stack is not implemented generic enough to work with sony cam (the nrf softdevice works fine tho).

[tao-j]

Let me try the canon remote, actually it is not using the nimble stack with a 2x larger code footprint bt library, but it may work.

There is an improved version: https://github.com/combatistor/BR-M5-custom

[tao-j]

unfortunately, the canon one won't help since it's using unbonded access. Their pairing is just write the esp32's device name into a custom uuid (by canon). So technically not pairing at all?

if (pclient->connect(camera_address))
    {
        // Acquire reference to main service
        pRemoteService = pclient->getService(SERVICE_UUID);
        if (pRemoteService != nullptr)
        {
            // Acquire reference to BLE characteristics
            pRemoteCharacteristic_Pairing = pRemoteService->getCharacteristic(PAIRING_SERVICE);
            if ((pRemoteCharacteristic_Pairing != nullptr))
            {
                // Send request on pairing service from external device
                String device_name_ = " " + device_name + " ";          //Pairing message to send
                byte cmdPress[device_name_.length()];                   // Stocking list of Bytes char of the message
                device_name_.getBytes(cmdPress, device_name_.length()); // message Parser
                cmdPress[0] = {0x03};
                pRemoteCharacteristic_Pairing->writeValue(cmdPress, sizeof(cmdPress), false); // Writing to Canon_pairing_service
                log_e("Camera paring success");
                delay(200);
                disconnect();
                BLEDevice::setEncryptionLevel(ESP_BLE_SEC_ENCRYPT_NO_MITM);
                delay(200);
                connect();
                nvs.setString("cameraaddr", String(camera_address.toString().c_str()))

Also, the nvs is used to by the user to store an address of the camera. , not the bt stack to store crypto info

[tao-j]

I have implemented a working version just for taking pictures https://github.com/tao-j/alpha-ble Build with "bluedroid" works, build with "nimble" sometimes works, I believed the problem is connection settings like MTU and conn parameters like timeout interval that caused instability of the nimble stack.

I also made an integration into your code https://github.com/frank26080115/alpha-fairy/commit/6431c9ded87310438cc8fffc2592b0c9e605e1d6, however, it seems that either nvs is disturbed or the semaphones of the freertos is disturbed. See two locations of the bt_init/bt_loop one can trigger shutter, one is stuck.

[tao-j]

The reason for the locality of bt_init/bt_loop behaves differently is that when it is called early, it claims ~60k heap space and it worked. When it is called after png image is drawn, the bt stack fails, since the original impl w/o bt stack only has ~10k heap left. So, out of memory, time for PSRAM or painfully optimize the whole program.
Converted to data to bmp files totaled about 4MB, so the flash cannot accommodate. Converted to Jpg and it seems that the drawJpg function from m5display always crashes, tried to make jpg files 8 bit per component and baseline with quality 100, no avail.

The NimBLE stack problem seems to be related to ESP32 only. https://github.com/espressif/esp-idf/issues/8303. And ESP32 is pretty old now, I guess they are not of high priority for a fix.

In summary, when ESP32-S3-PICO-(R8F8) is available with PSRAM and FLASH in the same small package as ESP32-PICO and served in an upgraded m5stick (and I wish they replace side button with the same small up/down/button encoder as in m5ink), it may worth a second shot.

[tao-j]

https://electric.garden/m5stack-2an3w/photos/STICKCPLUSV11-Internal-Photos-M5Stack-2an3wstickcplusv11-ex-1-6.pdf

It seems that M5Stack is going to release a new version of the device, dubbed "m5Stick c plus2". This internal image from FCC (July 11 2023) shows that ESP32-PICO-V3-02 is used, which has 8M flash and 2M PSRAM. So we can use the bluedroid BLE stack instead of the NimBLE. Just let everyone know here all hope is not lost.