Closed diplfranzhoepfinger closed 7 months ago
@diplfranzhoepfinger Does it mean that it works with the inflatable buffer? Looking at the logs, your'e hitting only the exit CMUX problem, which is related to the known issue of SIMCOM A7672
I think you're actually seeing this problem:
I was thinking about adding a debug/warning log here, but this could also mean a standard timeout, so decided against it.
I know that I told you to set ESP_MODEM_CMUX_DEFRAGMENT_PAYLOAD
to false
in the past, because your device uses a longer CMUX payload and esp_modem wasn't ready to handle that.
It is also documented:
that this mode is perfectly suited for DATA mode, but in the COMMAND mode, you might see get some timeout issues and this is probably the case. And it also explains why the logging verbosity makes such a difference, clearly sending long hexdump payloads to UART causes delays, so the data come in chunks and if the OK
delimiter is not present in the first chunk we have no other option (in this configuration) than to give up and report failure.
Now the esp_modem
was fixed to properly work in the default mode, as well. You can give it a try. (the data mode might be a bit slower, since we keep and copy buffers between CMUX and DTE layers). This config also handles corner cases, especially with your device A7672
, so It would give a warning if the defragmenting bufffer size is not big enough:
That said, I would recommend these configuration of esp_modem
to work with A7672:
ESP_MODEM_CMUX_DEFRAGMENT_PAYLOAD=y
, ESP_MODEM_USE_INFLATABLE_BUFFER_IF_NEEDED=n
-- default, optimized for "embedded" usecase, static buffer on CMUX layer, may need bigger buffer size, slower networkingESP_MODEM_CMUX_DEFRAGMENT_PAYLOAD=n
, ESP_MODEM_USE_INFLATABLE_BUFFER_IF_NEEDED=y
-- optimized for DATA throughput, buffer size could be small, but an allocation would take place if you get a long (or fragmented) command reply (which you do if you're using GNSS commands)ESP_MODEM_CMUX_DEFRAGMENT_PAYLOAD=n, ESP_MODEM_USE_INFLATABLE_BUFFER_IF_NEEDED=y
Set like you recommend
It does now at least get a IP Address; which was a big issue before. But how can we go further? We want not only get a IP but also get a Connection which seems not to be that case now.
Where does this "invalid header" warning come from?
But how can we go further?
This example terminates the connection in the end, so you either remove the exit or reconnect after it. Looking at your logs I can see that the connection and publishing to the MQTT broker succeeded -- in both verbose/info version.
Where does this "invalid header" warning come from?
I think we've discussed that already. Your modem doesn't comply to some control sequences of the CMUX protocol, I've even added a note about it in the docs:
Previous version of esp_modem
simply ignored it (the last command at the end didn't work), but I've recently added 8edbac697478bd7abef1b2f69e7e5c24b48e0369 to strictly check all violations, not only to emit warnings, but mainly to be able to recover the protocol. This proved very useful indeed, especially on UART if we miss an interrupt or buffer overflows. Very much applicable to devices like A7672, which define two byte payloads, so if such a glitch happen and is interpreted as a very long payload (like 32kB) we'd keep reading on and on, looking for the end of the frame (and CRC) which never comes.
so i will integrate your patch and make new Tests.
i do not think that anything runs in the current Version. The Device get an IP, but furthere communication does not happen.
i will do more tests later today, and post them here.
Thanks,
Franz
first Example
CONFIG_LOG_DEFAULT_LEVEL_INFO=y | log |
CONFIG_LOG_DEFAULT_LEVEL_VERBOSE=y | log |
that part looks good now ... wow.
but my Second Example lacks: see: repo with fix
CONFIG_LOG_DEFAULT_LEVEL_INFO=y | log |
CONFIG_LOG_DEFAULT_LEVEL_VERBOSE=y | log |
seems also working, only Problem is the Fragmentation of the AT AT+CPSI? Answer. can i increase the Default Buffer, so that is works in most cases without a Fragmentation ?
seems also working, only Problem is the Fragmentation of the AT AT+CPSI? Answer.
It comes fragmented, but should be passed to the command library multiple times, but the new fragments should be appended to the previous so in the end, it should arrive as a whole. This is also happening for other commands, that's why we parse them like this:
(returning command_result::TIMEOUT
to indicate that we need to read more, but after the next fragment arrives the buffer contains all accumulated data so far)
can i increase the Default Buffer, so that is works in most cases without a Fragmentation ?
Yes, as indicated above: https://github.com/espressif/esp-protocols/issues/375#issuecomment-1760953943 you can use the most default configuration (ESP_MODEM_CMUX_DEFRAGMENT_PAYLOAD=y
, ESP_MODEM_USE_INFLATABLE_BUFFER_IF_NEEDED=n
) to have the CMUX layer defragment the data. You can start with a smaller buffer and the library would tell you if you have to increase it.
(returning
command_result::TIMEOUT
to indicate that we need to read more, but after the next fragment arrives the buffer contains all accumulated data so far)
so, how to get the Data then ? can i grab it somehow ?
can i increase the Default Buffer, so that is works in most cases without a Fragmentation ?
Yes, as indicated above: #375 (comment) you can use the most default configuration (
ESP_MODEM_CMUX_DEFRAGMENT_PAYLOAD=y
,ESP_MODEM_USE_INFLATABLE_BUFFER_IF_NEEDED=n
) to have the CMUX layer defragment the data. You can start with a smaller buffer and the library would tell you if you have to increase it.
so you say maybe this config is the better one in some cases ? Data Througput is less then you say ? Even with a increased Buffer ?
Now I understand. We must change the Parser.
Now I understand. We must change the Parser.
Yes that's one option. The other one is to switch to the default config.
Data Througput is less then you say ? Even with a increased Buffer ?
I meant theoretical throughput, since the defragmentation takes place before we post the data to the network stack. This is of course negligible on UART speeds of few hundred kbauds.
thanks a lot, i will do a test.
@diplfranzhoepfinger You can also try to cherry-pick https://github.com/espressif/esp-protocols/pull/391/commits/1db1e1508d63c9d03161e3b7785f6cc7a2ac81b8 to see if your parser works with ESP_MODEM_CMUX_DEFRAGMENT_PAYLOAD=n
, ESP_MODEM_USE_INFLATABLE_BUFFER_IF_NEEDED=n
.
Context: There's a regression in v1.0.3
with handling USB DTE data and this fix actually mimics the previous behavior for that config.
but we use A7672 on 4-wire serial.
do you think USB has some siginficant advantages ? OK, it saves 2 Lines.
but i still need CMUX right ?
but we use A7672 on 4-wire serial.
This also affects UART in the configuration of ESP_MODEM_CMUX_DEFRAGMENT_PAYLOAD
=n, ESP_MODEM_USE_INFLATABLE_BUFFER_IF_NEEDED=n
do you think USB has some siginficant advantages ? OK, it saves 2 Lines.
Wires and speed
but i still need CMUX right ?
Not necessarily, A7670 supports two logical channels (endpoints) in USB, which could represent two physical channels (i.e. terminals) that you connect to the DTE. So you can have one channel for commands and another for data without multiplexing.
but i still need CMUX right ?
Not necessarily, A7670 supports two logical channels (endpoints) in USB, which could represent two physical channels (i.e. terminals) that you connect to the DTE. So you can have one channel for commands and another for data without multiplexing.
then i start ESP-Modem 2 times, once in DATA Mode, once in AT Mode ?
Just check the examples. This would create the corresponding DTE with these two terminals already included:
The rest is the exactly same (one DTE, one DCE, one network interface).
PS: This constructs a DTE which simply takes two terminals, and sets the mode to DUAL_MODE
so no further switching is necessary.
that is cool
i want to test it !
It looks like the tests pass. Please reopen if not.
YES !
we use it, and beside some initial Problems with the VBUS Pin ( USB still work when VBUS not on 5V, but Fail after this when activating GNSS)
it is much more cool than using CMUX.
so if somebody read this:
i would rate USB much better than CMUX.
I'm planning on enabling Discussions
in this repo, so people can share what they think about different components and modes.
It's probably a good place for you to post a feedback on CMUX vs USB
yep. might be a good thing !
Answers checklist.
General issue report
testing new Modem modem-v1.0.3 on a SIMCOM A7672E
Test 1:
Test is with Firmware
After adding some Delay and Sync, here it gets an IP Address. BUT only with INFO Logging, not with Verbose Logging.
It is interesting to see that at "verbose" logging the behaviour is different than in "info" logging.
@david-cermak
Test 2:
testing the 4-Wire Mode: repo
Not working.
Test 3:
Inflatable Buffer:
Test 4:
cross-Testing with Old Modem:
repo
log
[log]()
test with merged new Modem 1.0.3:
repo
log