Can you switch the codes from ESP-IDF to ESP32 libraries in Arduino IDE?

[Paryavi]

Hi There,

Thank you so much for your codes for ESP32 MCU in ESP-IDF, I have lots of ESP camera boards and a Hologram Nova that has Sara-R410 and I want to connect ESP32 to it. But the problem is all of my previous application codings are in Arduino IDE as most ESP32 codes are written in Arduino IDE! So would you please modify your code so that we be able to use ubxlib library in Arduino IDE platform instead of IDF, like what Sparkfun people did?

Thank you so much!

[RobMeades]

Hi there, and thanks for you interest. We've never looked at the ESP32 Arduino IDE, it might not be an easy thing to do. I'll have a look at it tomorrow and comment again here.

[Paryavi]

Thanks, Rob for the quick response, if you look at something like Sparkfun LTE libs that would be great too; https://github.com/sparkfun/SparkFun_LTE_Shield_Arduino_Library https://learn.sparkfun.com/tutorials/lte-cat-m1nb-iot-shield-hookup-guide/all

Details: My project has some image classification with ESP32, take pictures, and send it with LTE (I am familiar with Python not much C++, but working on it!), I was able to send images with Pycom Gpy recently, using micropython!

Gpy is ESP32 plus Sequans Monarch, but they do not have good library support as you do, so that is why we switched to SaraR410M-2B and we wanna integrate it with our ESP custom-designed board. So I recently bought some Sparkfun level-shifters to connect ESP32 3.3 volts to Sara modem 1.8 V Uart pins, similar to Sparkfun Arduino shield Eagle design, but I need codes to send a message(eventually image) from ESP to our website using Sara cellular modem, unfortunately, ESP-IDF is not that optimal, however, I am looking at some easier ways like using Visual studio IDE for IDF integration now, but if you could migrate to Arduino Libs, that would be gold!

Thanks, MoZen

[RobMeades]

Hi again. I started writing a comment about how I didn't understand the relationship between Arduino and ESP-IDF but then I found this article which suggests that ESP-IDF is already there, inside the pre-built libraries included with the Arduino IDE. In other words, you should be able to take the ubxlib code plus the existing esp-idf platform port for ubxlib and simply persuade the Arduino IDE to build and link them against the pre-built ESP-IDF libraries it already has.

Then you would have the ubxlib APIs as well as the Arduino APIs, though of course you should probably NOT use the Arduino serial or GPIO APIs as ubxlib will be using that HW via the ESP-IDF APIs.

Have you tried doing anything like this?

[Paryavi]

Hi Rob,

Thank you, I looked at my Arduino lib folder, I found IDF-related files, so the integration sounds cool! Yes, I assume you mean ESP is using UART0 with Arduino IDE and ubxlib uses UART0 with Sara modem too which might cause problems.

I used Sparkfun Sara-r4 shield eagle files and added your LTE modem schematic to our ESP integration board in Eagle but I am worried on software side; It is a little vague how to use ESP-IDF ubxlib APIs for me (I worked with ESP-IDF previously, I need to install it again), For example, which ESP32 pins to connect to Sara-R4? I assume Uart0 to UART pins of Sara-r4 using a level shifter... I have components, but honestly, I could not understand how your cellular example work, what should we set when uploading the code, where to add APN? what your example will print in the terminal of ESP-IDF?

My goal (and I assume every developer's first goal) is to send a "Hello World" message using Sara LTE with ESP-IDF, how can I do that with ubxlib Github examples with ESP32? Then I will modify that to send my images...

Some LTE messaging examples; GPY messaging example: https://docs.pycom.io/tutorials/networks/lte/ Sparkfun: https://learn.sparkfun.com/tutorials/lte-cat-m1nb-iot-shield-hookup-guide/all#example-2-send-a-hologram-message Hologram: https://www.hologram.io/references/python-sdk

I know you guys are busy but if one of your engineers could add an LTE messaging Github example or record a video on how to use ubxlib/GPIO APIs with ESP-IDF on your ublox1 YouTube channel that would be great.

Thanks, MoZen

[RobMeades]

Having checked other priorities we will be able to make time to have a poke at this in the next few weeks. I can't promise that it will work, there might be impossibilities or too-hard-for-us-to-solve things hiding within it but I'm hoping this will be just another set of build metadata, no code required. If that is true it will indeed be a valuable addition.

I will report back.

[Paryavi]

Sounds good, Rob, really appreciate it! Also, adding more info on how to use the current cellular API examples will help too, I will dig more on cellular documentation too!

[RobMeades]

On the issue of documenting how to use the cellular API examples, can you provide some hints as to what needs to be clarified? Since we know how to use it and have made an attempt to explain we need your help to figure out where we can improve.

[Paryavi]

Sure, I think first I need to install IDF and run it asap to get a better user experience, However, today I checked the runner folder, it gave more clues on how it might work; https://github.com/u-blox/ubxlib/tree/master/port/platform/esp-idf/mcu/esp32/runner

It looks like you made some "general" examples here; https://github.com/u-blox/ubxlib/tree/master/example Where users can access them for all boards, previously I was looking for a cellular example in your ESP-IDF folder!

Also, the hardware part is not clear, if there was a schematic picture (fritzing, or eagle) that would help a lot; something like Sparkfun schematics; https://www.sparkfun.com/products/14997 My question is which esp32 and sarar410-2B UART ports are connected, or how do they do Uart handshake.

Thanks

[RobMeades]

Hi again. We have implemented Arduino support and have pushed a preview branch here, see the port/platform/arduino directory for more information. This is for you to have a look at, maybe try out, see if it is what you are after. Note that it is (a) still subject to change (it has not been reviewed internally yet) and (b) this preview branch will be DELETED when the feature is merged into master.

On the documentation, what would usually happen concerning hardware is that you would consult the u-blox integration manual for your chosen module and go from there. I will see if we can provide some form of "HW hints and tips" README.md in ubxlib which might point you in the right direction.

[Paryavi]

Hi Rob!

Thank you so much for the great explanation. I was able to copy the files using your Python code in the Arduino Library folder and build the ubxlib src folder, then include and compile a sketch in Arduino IDE successfully, more details after my question. Now I want to write the sketch code there...

My question:

1. I am not sure where to find these API functions: xxxInit() ,xxxAdd(), xxxConnect().

2. Assuming my first goal is to send an SMS like this code from Sparkfun (my end goal is to send the camera board picture); https://github.com/sparkfun/SparkFun_LTE_Shield_Arduino_Library/blob/master/examples/01_SMS_Send/01_SMS_Send.ino In the u_blox repo, which example should I look at?

There are two cell resources in ubxlib; Cell API:https://github.com/u-blox/ubxlib/tree/preview_feature_arduino/cell And https://github.com/u-blox/ubxlib/tree/preview_feature_arduino/example/cell/lte_cfg

My Updates; I am connecting 4 UART wires; from ESPCAM RX, TX, VCC, and Ground, to Sparkfun Arduino Shield I received yesterday; https://learn.sparkfun.com/tutorials/lte-cat-m1nb-iot-shield-hookup-guide/all

Things I need to do;

• configuring the pins for ESPCAM in u_cfg_app_platform_specific.h

• Need to install ESP-IDF and look at how to integrate it with Arduino from the medium article you shared again

• Then, if it works, we can integrate u-blox SaraR410M-02B-01 to our boards using Eagle.

Thanks, MoZen

[RobMeades]

That's great progress, thanks for looking at the preview branch. If you were able to include and compile/link the ubxlib Arduino library for ESP32 then you must already have the ESP-IDF stuff inside your Arduino environment - maybe ESP-IDF comes with Arduino by default anyway? I only followed the instructions on the Espressif site because that seemed the obvious route, since I don't really know very much about Arduino and I do know about Espressif.

On the examples, the cell examples are just a few small cellular examples, we try to direct you away from those towards the general examples which use the common APIs. The common APIs will work across our cellular and Wifi/BLE modules and are intended to be simpler to use so it is much better to start there, only go down to the cellular API if you need something slightly special.

So, if you want to open a socket to a server, look at the sockets examples, if you want to use MQTT look at the mqtt_client examples, if you want to do location stuff look at the location examples, etc. These examples each include all of the xxxInit() and xxxAdd() functions you need so you should be able to understand what's going on from there.

You mention SMS above: is that a requirement? I imagine you aren't going to send a picture over SMS? If it is a requirement then we will need to add SMS support on cellular (not something anyone has asked us for yet). Not a huge issue, it is on the TODO list, just needs to be moved up.

[Paryavi]

Aloha and Thanks, Rob, I will check the TCP example, that's what I want.

Oh, I thought you mean installing esp-idf independently, I did that previously not sure if I deleted it... But yes, I have esp stuff installed inside Arduino following the espressif link long ago (esp support inside Arduino you shared).

My goal is to send ESPCAM pictures using sockets, so I was thinking at SMS as a starting point that I know hardware works!, but thinking about it again... more relevant thing I see in Gpy or OpenMV codes as the first step is getting a website IP address using TCP (like Google!). Check GPu example at bottom of this page; https://docs.pycom.io/firmwareapi/micropython/usocket/ To send a picture I used this code first to know cellular module, antenna, hologram Simcard,... works(Sequans module inside Gpy); socket.getaddrinfo() So putting get address info from TCP work in Arduino(which is essentially C or C++) is a great TODO thing I think for many users, including me! Thanks!

So if you write a code that gets an IP address using TCP in Arduino using the ubxlib lib we have, that is great! Arduino coding is mostly the same. I will check the TCP example you shared too. I have direct access to WiFi from ESP, we do not use GPS and BLuthotth now, first priority is making sarar4 work using my Sparkfun sarar4 breakout board using ESPCAM(wanted to share a picture of my espcam and sparkfun on my table now but probably sending picture is not supported in Github!

Thanks, Mohsen

[RobMeades]

Sounds like you're having fun! For TCP sockets (assuming ignoring TLS security for now) this is the example you want. The entry point, I suppose the start of setup() in your terms, is this line, then you have to fit the following calls however you would like into the Arduino setup()/loop() world. I'd guess that up to this line you'd probably put in setup(), then you have to decide whether you want bringing the network up (i.e. registering with the cellular network and gaining an IP address) to be part of setup() or you might prefer to put that whole section (i.e. all of lines 212 to 281) in loop(), it entirely depends on how you would like your application to behave; does it connect to the network once, in setup() and stay on forever or does it connect only while it is in loop() and disconnect when it leaves loop()? Only you can decide :-).

I've never used socket.getaddrinfo() in Python, and you need to remember that, despite C being there, hiding, underneath Python, Python is almost the exact opposite of C in almost every way, but it looks like all it effectively does is return the IP address of the device, which you could get, in the context of that example, by calling uCellNetGetIpAddressStr() with the network handle after you've done uNetworkUp(), so something like:

uCellNetGetIpAddressStr(networkHandle, buffer);

...just re-using the buffer that is already there as it is big enough (it just happens to be U_CELL_NET_IP_ADDRESS_SIZE (i.e. 64) characters). The IP address is just a string in the buffer, so if you like you could print it with:

uPortLog("The IP address of this device is %s\n", buffer);

The thing to remember with the ubxlib C code is that everything is done using handles and all of the sockets stuff is pure BSD sockets, there should be no surprises to a BSD sockets user (which is pretty much all sockets users I think).

Looking forward to seeing a picture of your ESPCAM :-).

[Paryavi]

Good morning Rob!

Thanks for your patience, I know it's hard dealing with non-computer engineers, it took me a while as an electronics engineer to learn Python from Coursera (I knew C to some extent). Anyways, I tried copying the parts you mentioned in Arduino and call the functions, I got this error; exit status 1, 'networkHandle' was not declared in this scope I was not sure what to include and what to ignore at the beginning of the sketch based on the codes... I think if you write a sample .ino file in your arduino sketch example folder, that is great.

Here are two examples I was talking about, tested in OpenMV which I found useful for wireless projects (fetching IP address of a website and photo upload); https://github.com/openmv/openmv/blob/master/scripts/examples/OpenMV/14-WiFi-Shield/http_client.py https://github.com/openmv/openmv/blob/master/scripts/examples/OpenMV/14-WiFi-Shield/http_post.py

Ok, since you said you know IDF more, I will start with it to make sure the sarar4 hardware is working, then I will go back to Arduino for the photo upload project. I have the ESPCAM photo upload code in Arduino already.

Ok I installed ESP-IDF again, and I went to configure these files (pins for ESPCAM); https://github.com/u-blox/ubxlib/tree/master/port/platform/esp-idf/mcu/esp32/cfg I think you mentioned, u_cfg_app_platform_specific.h should be modified more based on ESP format which in my case is ESPCAM(ESP32S), there were many RX and TX to be set! so I panicked a bit, btw what is the short-range module?

It looks like I should change most of them to -1 as they are not connected to sara r4, I am trying to mimic Sparkfun shield wiring, they just connect rx and tx, and I will just connect rx,tx,vcc,gnd from espcam to sparkfun. 4 pins in ESPCAM UART are shown here; https://randomnerdtutorials.com/program-upload-code-esp32-cam/

In Sparkfun SaraR4 shield, they mention they connect all uart handshake signals to ground as recommended for a UART "with NO FLOW CONTROL" mentioned at the lower part of page two of their schematic; https://cdn.sparkfun.com/assets/3/9/d/5/f/lte_cat_m1_shield_schematic.pdf I assume I should do the same.

Hopefully, when I am able to run Sarar4 with ESPCAM, I will definitely send its success picture, haha!

Thanks, MoZen

[RobMeades]

Understood, in that case I will see if I can write an Arduino example, making some assumptions about how the application should behave, which does the same send/receive to a TCP echo server as the sockets example.

On the setting of pins etc., yes, you either edit the u_cfg_app_platform_specific.h under the ESP-IDF folder ('cos that's the chipset you're building for) or you override the values, or just set them, in the application itself, it is entirely up to you. All of the pins that might be required are present in the file; you might be using one of our Wi-Fi/BLE modules or GNSS modules etc., we can't know. But if you're only using a cellular module you only need to set the things with "CELL" in the name and if you're not using flow control you should set the CTS/RTS ones to -1 since they are not connected.

I won't close this issue until I've seen the picture :-).

[RobMeades]

Hi again: I have pushed an updated version of the Arduino preview branch, called preview_feature_arduino_updated (the previous preview branch is still there, unchanged, in case you need to refer to it for some reason).

Please delete any stuff that was created last time by running the Python script, read the revised README.md (some of the script names and certainly the script behaviours have changed). The most important thing about this update is that it includes an example sketch, an Arduino version of the sockets example.

This branch has also passed testing on our test system and so is likely quite close to the version I will merge. When I have merged Arduino support into master I will delete both of these preview branches.

[Paryavi]

Hi Rob, it is awesome to test the socket example in Arduino, I deleted my previous libraries to try the preview version.

However, I get the following error when trying to run u_arduino.py; No such file or directory: 'source_lib.txt'

I guess it is because in the code you have added; UBXLIB_LIB_POSTFIX = "_lib"

Source file listing file

SOURCE_FILES = "source" + UBXLIB_LIB_POSTFIX + ".txt"

But there is no "source_lib.txt" file in your Arduino folder...

[Paryavi]

Comparing to the previous Arduino preview folder; https://github.com/u-blox/ubxlib/tree/preview_feature_arduino/port/platform/arduino

It looks like you probably forgot to add these files in the new Arduino preview folder; source_lib.txt and include_lib.txt, so I will just copy those from the previous preview and hopefully it will work!

[RobMeades]

Sorry, my fault, I was testing with the automated test system which always specifies the names of those text files on the command-line to u_arduino.py rather than using the default names; it's just the name in u_arduino.py that is wrong, UBXLIB_LIB_POSTFIX should be "" not _"lib". Don't use the old files, the contents are different and it probably won't compile.

I have update the branch so that UBXLIB_LIB_POSTFIX is now correct.

[Paryavi]

No problem my friend, Yeah, it was not compiling, but it successfully copied the ublox in the Arduino Library using old files... Anyways, I will delete them and use your updated version now, thanks for the quick reply. Btw, I am curious how you test the code, what hardware do you use? Probably Nina...

[RobMeades]

We test ubxlib in general on the things listed here, where the Arduino ones are instances 11.1 and 22, so an ESP32-DevkitC and a NINA-W10.

[Paryavi]

Thanks.

I get the following error compiling empty sketch example in Arduino IDE;

Library ubxlib has been declared precompiled: Precompiled library in "C:\Users\myuser\Documents\Arduino\libraries\ubxlib\src\esp32" not found C:\Users\myuser\Documents\Arduino\libraries\ubxlib\src\u_port_uart.c: In function 'uPortUartOpen': C:\Users\myuser\Documents\Arduino\libraries\ubxlib\src\u_port_uart.c:275:19: error: 'uart_config_t {aka struct }' has no member named 'source_clk' config.source_clk = UART_SCLK_REF_TICK; ^ C:\Users\myuser\Documents\Arduino\libraries\ubxlib\src\u_port_uart.c:275:33: error: 'UART_SCLK_REF_TICK' undeclared (first use in this function) config.source_clk = UART_SCLK_REF_TICK; ^ C:\Users\myuser\Documents\Arduino\libraries\ubxlib\src\u_port_uart.c:275:33: note: each undeclared identifier is reported only once for each function it appears in exit status 1 Error compiling for board Adafruit ESP32 Feather.

[RobMeades]

That's strange: it looks like the version of the underlying ESP-IDF code is different in your case, their definition of uart_config_t has changed. Do you know what ESP-IDF version you have?

[Paryavi]

idf4.2_py3.8_env but I can install a version that worked for you...

[RobMeades]

All I did was follow Espressif's instructions; it compiled for you yesterday and none of that code has changed. Don't understand what's different.

[RobMeades]

Assuming you are Windows, can you look at the file:

C:\Users\<your_username>\AppData\Local\Arduino15\packages\esp32\hardware\esp32\2.0.0\tools\sdk\esp32\include\hal\include\hal\uart_types.h

...and tell me what the uart_config_t structure in that file looks like? I have:

typedef struct {
    int baud_rate;                      /*!< UART baud rate*/
    uart_word_length_t data_bits;       /*!< UART byte size*/
    uart_parity_t parity;               /*!< UART parity mode*/
    uart_stop_bits_t stop_bits;         /*!< UART stop bits*/
    uart_hw_flowcontrol_t flow_ctrl;    /*!< UART HW flow control mode (cts/rts)*/
    uint8_t rx_flow_ctrl_thresh;        /*!< UART HW RTS threshold*/
    union {
        uart_sclk_t source_clk;         /*!< UART source clock selection */
        bool use_ref_tick  __attribute__((deprecated)); /*!< Deprecated method to select ref tick clock source, set source_clk field instead */
    };
} uart_config_t;

...i.e. there is a source_clk field in the version I have under Arduino.

EDIT: I know that a long time ago the field was use_ref_tick, you can see above that they deprecated use_ref_tick and replaced it with source_clk but this was back in 2019 and all the versions of uart_types.h I can find under my Arduino packages directory (there are three versions of the file, one for ESP32, one for ESP32C3 and one for ESP32S3) have the source_clk field.

[Paryavi]

Ok, it looks like my esp32 is 1.0.6! instead of 2.0.0; C:\Users\my_user\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.6\tools\sdk From there I do not see hal folder... I guess I need to update it...

[RobMeades]

Oh! That's probably a good idea :-).

[Paryavi]

Ya thanks, it's updating now in Arduino IDE following the Espressif link, it was 1.0.6!

[Paryavi]

Cool, it compiled! Thanks! I have questions about gpio pins, will ask in the other thread now. God Bless you!

[Paryavi]

I get this warning too; Library ubxlib has been declared precompiled: Precompiled library in "C:\Users\myuser\Documents\Arduino\libraries\ubxlib\src\esp32" not found

[RobMeades]

That is intentional: the library.properties file is set up so that, if you wish, you may copy the .a file for the library once you have compiled it into the libraries\ubxlib\src\esp32 and it will use that instead of re-compiling the source code of the library each time. We use this internally so that we don't re-compile the library for each test. However, note that if you do this and make a change in the library source code you MUST delete the .a file from libraries\ubxlib\src\esp32 as your changed code is not going to be compiled/used otherwise (since it's using the compiled code from the .a file).

[Paryavi]

The board setup: From left part of feather board;

pin 2 is 3V, Pin 4 is GND GPIO 16 is ESP RX GPIO 17 is ESP TX GPIO 14 is used for Power on connected to pin D5 on Sparkfun shield to toggle power-on pin on the modem! I did not know I can just drag and drop an image to the thread, haha! Thanks, Rob for all your help, I think I am pretty close to making it work in Arduino!

[RobMeades]

The stuff that is present on the preview_arduino and preview_arduino_updated branches has now been pushed to master in commit c1dfbf01ba45eaa10550b676ca9dbdb6d3ebb1e8, so I will delete those branches.

How are you doing with your development?

[Paryavi]

Aloha and Thanks Rob!,

Sorry, I was busy with class finals, but recently we soldered our Sara-R4 module, I was not able to turn the module on yet!, 2 updates;

1. I get this message in Arduino IDE ; Done compiling, Library ubxlib has been declared precompiled: Precompiled library in "C:\Users\myusername\Documents\Arduino\libraries\ubxlib\src\esp32" not found.

2. To turn the Sara module ON, we use a P-channel MOSFET at pin VCC of SARA module, so in the other thread you said and I quote; "U_CFG_APP_PIN_CELL_ENABLE_POWER: if you have a relay or some such gating power to the cellular module then you need to set this to the MCU pin which controls that relay. If you don't you should set this to -1."

So using Adafruit ESP32 feather board, I have set it to

define U_CFG_APP_PIN_CELL_ENABLE_POWER 32

endif

This means I have connected pin 32 of it to the gate of P-MOS,

and 3.3 volts voltage supply of feather board is connected to its sources,

so the idea is the Drain of this P-MOS that is connected to VCC pin of Sara-R4, will have 3.3 volts only when the P-MOS Gate is pulled down(opposite of N-MOS as you know that works when Gate is Pulled Up), so our approach is similar to having a Relay you mentioned to provide power to the Sara module but NOT of it, so my question is how can I NOT the U_CFG_APP_PIN_CELL_ENABLE_POWER in the program, i.e. when it is zero volts? then the Sara module will have power at pin VCC and vice versa.

To test it(without having NOT, and running socket example), I put an LED at pin 32 of feather board, but it does not turn ON currently when I run the socket.ino code, so I will try ESP IDF today too...

Happy Holidays! MoZen

[RobMeades]

Hi there, hope finals went OK.

Precompiled library in "C:\Users\myusername\Documents\Arduino\libraries\ubxlib\src\esp32" not found.

That's fine; it's not there yet, you're compiling it for the first time. Setting it as pre-compiled just means it won't keep recompiling the library.

On inverting the state of the U_CFG_APP_PIN_CELL_ENABLE_POWER pin you'll have to modify the code to do that for you. Search for pinEnablePower in u_cell.c and u_cell_pwr.c and you should find the bits you'll need to change.

[Paryavi]

Yeah, One of the finals was promising, not sure about the other yet, haha. Cool, I was not sure where to modify the code, thanks for the big help!

By the way, we developed this module for ESPCAM and OPENMV H7, I guess with ESP32 it should be fine, what do you think about making ublox library work for OPENMV boards too?

[RobMeades]

OpenMV is quite a beast: looks as though they intend you to run MicroPython not C.

[Paryavi]

Aloha Rob,

I just connected the P-MOS input to the ground of the ESP feather(act like a relay), good news is I have 1.8 volts from Sara r410m-02B at 1.8V pin so ESP feather is supplying it properly.

Next as I am using hologram simcard, so in Arduino code I set it as 'hologram' in Honolulu!; static const uNetworkConfigurationCell_t configCell = {U_NETWORK_TYPE_CELL, U_CFG_TEST_CELL_MODULE_TYPE, NULL, / SIM pin, must be NULL / "hologram", / APN: NULL to accept default. If using a Thingstream SIM enter "tsiot" here / 240, / Connection timeout in seconds / U_CFG_APP_CELL_UART, ...

I receive this message in the Arduino terminal;

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:1 load:0x3fff0030,len:1284 load:0x40078000,len:12836 load:0x40080400,len:3032 entry 0x400805e4 Added network with handle -5. Bringing up the network... Network is not available! 0 Tests 0 Failures 0 Ignored

So in u-cfg-platform specific I only have defined and connected 3 pins;

define U_CFG_APP_PIN_CELL_PWR_ON 14

define U_CFG_APP_PIN_CELL_TXD 17

define U_CFG_APP_PIN_CELL_RXD 16

Looks like the power-on pin is working as it turns on the module as I read 1.8V on the multimeter and I double-checked TX and RX pins, are you suspicious of anything else?

Thanks, MoZen

[philwareublox]

Hi, Is there any way you can check to see if the Arduino is able to send AT commands through the UART and you get "OK" in response? Until you confirm the AT interface is working, I'd ignore the "Network is not available!" message. This could simply mean there is no communication to the SARA module. Phil.

[RobMeades]

Hi @Paryavi: for this project you will really need to understand embedded C in some considerable depth. I would seriously suggest taking a course in the subject, maybe alongside doing the project, as you will need it when you come to write your application. Making use of ubxlib requires good knowledge of embedded C.

But to your point, your general approach when debugging must be:

1) look at each piece of debug information you have, 2) trace that information back to the code to determine what has happened.

The debug information has a lot of clues in it, you just need to consider each one with care and then follow it through to the code. It is easy to gloss over debug messages, not read them carefully, not look at the sequence, I make that mistake myself all the time, I think it is why "rubber ducking" is so effective, as it makes you think about what your, ummm, thinking. But quite a lot of the time you'll find that the information is there in front of you, it just requires logical following-through.

The first debug print after the ESP32 boot prints above is:

Added network with handle -5.

You can search the code for the text Added network with handle but there's no need as it is front-and-centre in the top-level sockets.ino file:

https://github.com/u-blox/ubxlib/blob/master/example/sockets/sockets.ino#L144

So clearly uNetworkAdd() is returning -5. A function name starting with uNetwork is going to be in the common network API, so you can find its definition, with all the documentation about it, in common/network/api; it is here. Even if you didn't know that, a text editor recursively searching for uNetworkAdd in *.h from the root of the ubxlib directory will find it very quickly.

However it is simpler than that: as you will see from the definition of pretty much all ubxlib functions, a negative number indicates an error. The generic error codes are in common/error/api/u_error_common.h and you can see that -5 is U_ERROR_COMMON_INVALID_PARAMETER.

There are only two parameters to uNetworkAdd(): U_NETWORK_TYPE_CELL and the address of configCell. You are obviously opening a cellular network type and so the most likely candidate for a problem is configCell, since it is the large complicated thing.

The structure type for configCell is named uNetworkConfigurationCell_t, so again it is part of the common network API in common/network/api and its definition, with all the documentation about it, can be found there (or you can simply search the .h files for uNetworkConfigurationCell_t and you'll find its header file soon enough).

Does your structure contain valid values according to those definitions for each field? For instance, can you confirm that you have edited the .ino file to provided a value for U_CFG_TEST_CELL_MODULE_TYPE? The instructions just above the structure in the .ino file tell you how to do this:

https://github.com/u-blox/ubxlib/blob/master/example/sockets/sockets.ino#L31

I think you are using SARA-R410M-02B, so the line would need to be:

#define U_CFG_TEST_CELL_MODULE_TYPE U_CELL_MODULE_TYPE_SARA_R410M_02B

P.S. you're in Honolulu? I got married there (in 1996). Loved the place, and the islands.

[Paryavi]

Thanks, Rob, I knew it is about -5 error but thanks for helping me understand how to look at the API and its different components, I really liked the rubber ducking concept and it looks like Coursera has a nice embedded programming course that I should check out during the hollydays as it is my blindspot currently! I should learn about CMake, compiling details, ...; https://www.coursera.org/learn/introduction-embedded-systems#syllabus

You were right, previously, long ago I did set sarar4-2b in the code, but this time I forgot about it, so setting the Sara module config properly, it starts the AT commands but ends with error -7;

https://github.com/u-blox/ubxlib/blob/master/common/error/api/u_error_common.h#L55 U_ERROR_COMMON_NOT_RESPONDING -7

Not sure where to find more about this error,I got suspicious of my RX, TX pins, and level shifting transistors voltages which look fine, also I looked at hologram cloud documentation, and I am not sure if I need to set the SIM pin in arduino code; NULL, / SIM pin, must be NULL /

In Hologram API, the have device Key which I did set with hologram Nova and Raspb pi ; https://support.hologram.io/hc/en-us/articles/360035212714

But I guess it is not relevant here, I can text them to get my sim Pin if needed. Hologram Help Centr for Sarar410 modules; https://support.hologram.io/hc/en-us/articles/360038810214?input_string=not+sure+if+pin+is+required+for+turning+sara+module+on

Btw, thank you, we've had good progress so far and here is the serial response, not sure how to confront the common not responding error(-7)....

U_CELL_PWR: powering on. AT AT AT AT AT AT AT AT AT AT Added network with handle -7. Bringing up the network... The network is not available!

Thank you so much, P.S. Yes I am a Ph.D. student at UHM, Honolulu, Hawaii, there is an invasive insect, Coconut Rhino Beetle, attacking its Palms and I am hired as a graduate student to design Software and Hardware to delineate their infestation, we previously used WiFi to send our integrated camera board images to a server with ESP32 and OpenMV, so I did some Tensorflow CRB insect detection on OpenMV (using TFLite) and recently doing inference on Flask/Heroku using TensorFlow.

CRB Traps do not have WiFi accessibility, as some located on the road! So currently we soldered 2 Sara-R4 boards to connect to our camera boards eventually, this board turns ON, and I am working on it, hopefully, this will be part of my Ph.D. thesis and if we detect CRB in traps quickly, the team can respond quickly! Also, My advisor found another Sara-R4 with GPS capabilities and we will work with that design too but it is not available currently due to chip shortage!

Yes, I loved Hawaii, and happy you had a great time here. Feel free to visit again, I'd be here at least in the next two years, we can hang out!

Mahalo, MoZen

[RobMeades]

Good stuff, so the issue is serial comms; you appear to be sending AT commands to the module but either they are not getting to the module or it is not responding. A few things to check:

1. Remember that the module uses the DCE convention, i.e. TXD is the input pin at the module, it is where data to be transmitted goes, and RXD is the output pin at the module. People often get this mixed up and think the TXD pin of the module is an output.
2. If you are not using flow control. is the RTS pin of the module grounded? The module will come up with flow control on and so if RTS is not grounded it will never send anything back to you.

Ignore the SIM, that's irrelevant until we're talking to the module and it is very unlikely to be an issue anyway, just make sure no PIN is set.

Sounds like a very interesting project indeed, let's get it working! Seems as though your supervisor has found SARA-R422M8S: that will work out of the box with ubxlib and give you various location establishment options.

[Paryavi]

Cool, Yes in eagle design I had grounded RTS and DTR, pin 9 and 10 of Sara module are physically grounded under the ublox shield and I can't access those pins to test them, also I have set RTS and CTS to -1 in u-cfg-app-platform-specific, not sure if -1 is proper as it is grounded internally...

Today I checked the DCE convention and our logic shifting transistors are working, our setup looks like Figure 45 in "the Sara-r4 integration manual"; Figure 45-UART interface application circuit with a partial V.24 link instead of voltage translator we use level shifters, other difference is on the application processor side(ESP32) I did not connect RTS to CTS or DTR to DSR.... I think we are not using Flow control Btw, the labels RX-Cell and TX-Cell are misleading in this image, they are connected to proper level shifters, so RX of ESP goes to RXD, and TX of it goes to TXD of Sara!

[RobMeades]

I think you need to monitor the dynamic behaviour of the PWR_ON, TXD and RXD lines, as close to the module as possible, during the initial power-on sequence, using something like a storage scope or a Salaea logic probe. Need to check that PWR_ON is actually toggling, that it is the right way up, whether TXD is getting to the module and whether RXD is coming back.

[Paryavi]

Sure, I will check them out, can it be the level shifters I used instead of Texas Instrument unidirectional voltage translators suggested by the Sara datasheet(Fig 45)? I mean I am not sure our N-MOS transistors switching is fast enough (115200 bps?) Here Adafruit people say; While we designed it for use with I2C, this works as well for TTL Serial, slow <2MHz SPI, and any other digital interface both uni-directional and bidirectional. https://www.adafruit.com/product/757 So probably 115200 baudrate probably might be fine! but as you said we should test it with a probe!

Another difference is our level shifter is bidirectional but Fig 45 required unidirectional logic shifting...

[RobMeades]

I use the Sparkfun bidirectional level-shifters boards routinely, they are fine at 115,200, just don't try to use bi-directional level-shifters in flow control lines, that doesn't work.

Your O6/BSS138 (the thing between the MCU's U_CFG_APP_PIN_CELL_PWR_ON pin and the module's PWR_ON pin); is that inverting or not? If it is inverting then you will need to pass the conditional compilation flag U_CELL_PWR_ON_PIN_INVERTED into the build to get everything the right way up again.

If it is NOT inverting then there still may be an issue: the default drive for the U_CFG_APP_PIN_CELL_PWR_ON pin is open drain, intended to pull the module's PWR_ON pin low; will open drain drive work with your O6/BSS138? If not then it may be necessary to set the conditional compilation flag U_CELL_PWR_ON_PIN_DRIVE_MODE=U_PORT_GPIO_DRIVE_MODE_NORMAL for the build; this is set anyway if U_CELL_PWR_ON_PIN_INVERTED is set since there is plainly a driver transistor in the way.

[RobMeades]

FYI, assuming you are using arduino-cli, you would pass conditional compilation flags in with an additional command-line switch to arduino-cli compileas follows:

--build-property "compiler.cpp.extra_flags=STUFF_HERE"

...so maybe:

--build-property "compiler.cpp.extra_flags=\"-DU_CELL_PWR_ON_PIN_INVERTED\""

...or

--build-property "compiler.cpp.extra_flags=\"-DU_CELL_PWR_ON_PIN_DRIVE_MODE=U_PORT_GPIO_DRIVE_MODE_NORMAL\""

Make sure you do a clean build after introducing such a change of course.

[Paryavi]

Q6 transistor, The N-MOS transistor connected to Sara PWR_ON pin is inverting, i.e., when applying a positive voltage directly from ESP Feather pin 14 to this N-MOS gate, then the transistor channel will conduct and its DRAIN will be grounded similar to its source(so when applying positive gate voltage then drain is Zero hence inverting), so I will edit the code as you suggested.

Actually, I used the Sparkfun level shifter Eagle files to implement RX and TX in my eagle files with Sara module, as seen in shared pictures, so as you see MAIN RX and TX comes from ESP Feather and go to Sara directly through RX-Cell and TX-Cell labels in the two pictures above. I think RX and TX is non-inverting, similar to Sparkfun level-shifter.

I think the Power-ON is working because when I connect its pin and press boot on ESP feather I will have 1.8 volts, so looks like the Power_On pin is turning the module on, but as it is inverting using the Q6 N-MOS I will follow your inverting comment.

[RobMeades]

FYI, in commit bb05f55583e90105758c90febcebc293fa21005f I've added the facility for you to have an inverted enable-power pin. You just need to pass the conditional compilation flag U_CELL_ENABLE_POWER_PIN_INVERTED into the build.