search

meshtastic / python

The Python CLI and API for talking to Meshtastic devices
https://meshtastic.org
335 stars 150 forks source link

preference changes not saving in 1.1.8 #41

Closed geeksville closed 3 years ago

geeksville commented 3 years ago

reported by @dinthenorth in the alpha tester thread. alas, I haven't yet been able to reproduce this. My run seems successful:

~/development/meshtastic/Meshtastic-python$ meshtastic --set position_broadcast_secs 60 --set wait_bluetooth_secs 28800
rm: cannot remove 'log_*': No such file or directory
Node changed: {'num': 2885173400, 'user': {'id': '!2462abf84098', 'longName': 'Unknown 4098', 'shortName': '?98', 'macaddr': 'JGKr+ECY'}, 'position': {'batteryLevel': 100, 'time': 1605398970}}
Connected to radio
Setting position_broadcast_secs to 60
Setting wait_bluetooth_secs to 28800
Writing modified preferences to device
~/development/meshtastic/Meshtastic-python$ meshtastic --info
Node changed: {'num': 2885173400, 'user': {'id': '!2462abf84098', 'longName': 'Unknown 4098', 'shortName': '?98', 'macaddr': 'JGKr+ECY'}, 'position': {'batteryLevel': 100, 'time': 1605398970}}
Connected to radio
my_node_num: 2885173400
has_gps: true
num_channels: 10
region: "1.0"
hw_model: "tbeam"
firmware_version: "1.1.8"
packet_id_bits: 32
current_packet_id: 1401156767
node_num_bits: 32
message_timeout_msec: 300000
min_app_version: 172

preferences {
  position_broadcast_secs: 60
  wait_bluetooth_secs: 28800
  ls_secs: 300
  region: TW
}
channel_settings {
  modem_config: Bw125Cr48Sf4096
  psk: "\324\361\273: )\007Y\360\274\377\253\317Ni\277"
  name: "Default"
}

Channel URL https://www.meshtastic.org/c/#GAMiENTxuzogKQdZ8Lz_q89Oab8qB0RlZmF1bHQ=
Nodes in mesh:
{'num': 2885173400, 'user': {'id': '!2462abf84098', 'longName': 'Unknown 4098', 'shortName': '?98', 'macaddr': 'JGKr+ECY'}, 'position': {'batteryLevel': 100, 'time': 1605398970}}

To folks who see this problem a couple of questions:

  1. can you try "pip3 install --upgrade meshtastic" to make sure you have the latest python tool?
  2. What OS is your desktop computer?
geeksville commented 3 years ago

This issue has been mentioned on Meshtastic. There might be relevant details there:

https://meshtastic.discourse.group/t/alpha-tester-thread-1-1-8-device-code-ready-for-alpha-testing/298/551

DDVanMS2018 commented 3 years ago

PS C:> pip3 install --upgrade meshtastic Requirement already up-to-date: meshtastic in c:\users\dallas.platformio\penv\lib\site-packages (1.1.7) Requirement already satisfied, skipping upgrade: pyserial>=3.4 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (3.4) Requirement already satisfied, skipping upgrade: dotmap>=1.3.14 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (1.3.23) Requirement already satisfied, skipping upgrade: protobuf>=3.13.0 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (3.13.0) Requirement already satisfied, skipping upgrade: pygatt>=4.0.5 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (4.0.5) Requirement already satisfied, skipping upgrade: pexpect>=4.6.0 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (4.8.0) Requirement already satisfied, skipping upgrade: pypubsub>=4.0.3 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (4.0.3) Requirement already satisfied, skipping upgrade: pyqrcode>=1.2.1 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (1.2.1) Requirement already satisfied, skipping upgrade: setuptools in c:\users\dallas.platformio\penv\lib\site-packages (from protobuf>=3.13.0->meshtastic) (41.2.0) Requirement already satisfied, skipping upgrade: six>=1.9 in c:\users\dallas.platformio\penv\lib\site-packages (from protobuf>=3.13.0->meshtastic) (1.15.0) Requirement already satisfied, skipping upgrade: enum-compat in c:\users\dallas.platformio\penv\lib\site-packages (from pygatt>=4.0.5->meshtastic) (0.0.3) Requirement already satisfied, skipping upgrade: ptyprocess>=0.5 in c:\users\dallas.platformio\penv\lib\site-packages (from pexpect>=4.6.0->meshtastic) (0.6.0)

====================================

I'm using Windows 10 Home 64bit.

====================================

PS C:> meshtastic --set position_broadcast_secs 60 --set wait_bluetooth_secs 28800 --info Node changed: {'num': 2988158628, 'user': {'id': '!ac67b21baea4', 'longName': 'AE', 'shortName': 'AE', 'macaddr': 'rGeyG66k'}, 'position': {'batteryLevel': 100}} Node changed: {'num': 2988159592, 'user': {'id': '!ac67b21bb268', 'longName': 'Unknown b268', 'shortName': '?68', 'macaddr': 'rGeyG7Jo'}, 'position': {}, 'snr': 11.75} Connected to radio Setting position_broadcast_secs to 60 Setting wait_bluetooth_secs to 28800 Writing modified preferences to device my_node_num: 2988158628 has_gps: true num_channels: 13 region: "1.0" hw_model: "tbeam" firmware_version: "1.1.8" packet_id_bits: 32 current_packet_id: 1113913922 node_num_bits: 32 message_timeout_msec: 300000 min_app_version: 172

preferences { position_broadcast_secs: 60 wait_bluetooth_secs: 28800 ls_secs: 300 region: US } channel_settings { modem_config: Bw125Cr48Sf4096 psk: "\324\361\273: )\007Y\360\274\377\253\317Ni\277" name: "Default" }

Channel URL https://www.meshtastic.org/c/#GAMiENTxuzogKQdZ8Lz_q89Oab8qB0RlZmF1bHQ= Nodes in mesh: {'num': 2988158628, 'user': {'id': '!ac67b21baea4', 'longName': 'AE', 'shortName': 'AE', 'macaddr': 'rGeyG66k'}, 'position': {'batteryLevel': 100}} {'num': 2988159592, 'user': {'id': '!ac67b21bb268', 'longName': 'Unknown b268', 'shortName': '?68', 'macaddr': 'rGeyG7Jo'}, 'position': {}, 'snr': 11.75}

=====================================

PS C:> meshtastic --info Node changed: {'num': 2988158628, 'user': {'id': '!ac67b21baea4', 'longName': 'Unknown aea4', 'shortName': '?A4', 'macaddr': 'rGeyG66k'}, 'position': {'batteryLevel': 100}} Connected to radio my_node_num: 2988158628 has_gps: true num_channels: 13 region: "1.0" hw_model: "tbeam" firmware_version: "1.1.8" packet_id_bits: 32 current_packet_id: 1127327210 node_num_bits: 32 message_timeout_msec: 300000 min_app_version: 172

preferences { ls_secs: 300 } channel_settings { modem_config: Bw125Cr48Sf4096 psk: "\324\361\273: )\007Y\360\274\377\253\317Ni\277" name: "Default" }

Channel URL https://www.meshtastic.org/c/#GAMiENTxuzogKQdZ8Lz_q89Oab8qB0RlZmF1bHQ= Nodes in mesh: {'num': 2988158628, 'user': {'id': '!ac67b21baea4', 'longName': 'Unknown aea4', 'shortName': '?A4', 'macaddr': 'rGeyG66k'}, 'position': {'batteryLevel': 100}}

DDVanMS2018 commented 3 years ago

I just used the ESPHome-Flasher and downgraded to f/w 1.1.7alpha and the preferences appear to be saving.

DDVanMS2018 commented 3 years ago

I just re-paired the tbeam to my phone and tried applying the preferences again and they are gone and it no longer saves. I'm still on 1.1.7. I don't know what's wrong.

DDVanMS2018 commented 3 years ago

Strange, the preferences are now saved.

DDVanMS2018 commented 3 years ago

I flashed to f/w 1.1.8alpha and the preferences are saving. I have one more tbeam that had the same problem. I'll try to fix that one.

DDVanMS2018 commented 3 years ago

I tried flashing to 1.1.7 on my other tbeam and made the preference changes, but the settings didn't save. I flashed it to 1.1.8 and the same problem. I wasn't able to add the preferences on this unit.

DDVanMS2018 commented 3 years ago

I don't think the following --set command actually writes to the tbeam. From the debug panel in the app, the RadioConfig section doesn't show the new preferences.

PS C:> meshtastic --set position_broadcast_secs 60 --set wait_bluetooth_secs 28800 --info Node changed: {'num': 2988159592, 'user': {'id': '!ac67b21bb268', 'longName': 'B2', 'shortName': 'B2', 'macaddr': 'rGeyG7Jo'}, 'position': {'batteryLevel': 69}} Node changed: {'num': 2988158628, 'user': {'id': '!ac67b21baea4', 'longName': 'AE', 'shortName': 'AE', 'macaddr': 'rGeyG66k'}, 'position': {'batteryLevel': 100}, 'snr': 11.0} Connected to radio Setting position_broadcast_secs to 60 Setting wait_bluetooth_secs to 28800 Writing modified preferences to device

DDVanMS2018 commented 3 years ago

I flashed back to version 1.1.0 beta and it saves.

crossan007 commented 3 years ago

I noticed this using the dev-https branch that I built and flashed locally.

I installed the meshtastic python app for the first time last week.

I'm on mobile right now, but I'll try the troubleshooting steps listed above later tonight.

I'm on windows 10 with a ttgo tbeam

geeksville commented 3 years ago

hmm - mysterious - I'm still having trouble reproducing this. But multiple user reports.

sachaw commented 3 years ago

I have observed this a few times, most recently when I passed a fairly long psk, I'll see if I can replicate it again.

sachaw commented 3 years ago

Haven't looked into how the firmware uses the psk, but here are some that work, and some that don't works:

49, 80, 71, 55, 79, 105, 65, 112, 66, 49, 110, 119, 118, 80, 43, 114, 122, 48, 53, 112, 118, 119, 61, 61
112, 221, 197, 38, 32, 41, 55, 82, 120, 188, 215, 11, 37, 148, 125, 11

Doesn't work:

112, 221, 197, 38, 32, 41, 55, 82, 120, 188, 215, 11, 37, 148, 125, 11, 10
112, 221, 197, 38, 32, 41, 55, 82, 120, 188, 215, 11, 37, 148, 125

If I pass one that doesn't work one of two things seem to happen, either it fails silently and no config is saved, or the device disconnects.

crossan007 commented 3 years ago

Without digging in too much, it seems that hard-coding WiFi credentials via the two strcpy lines in MeshWifi.cpp and then flashing was sufficient to get these values into the preferences:

image

Additional compiling / flashing of the code with these two lines re-commented seems to allow the WiFi credentials to persist in memory.

This might suggest that there's a breakdown when transferring the settings to the radio from the python app.

geeksville commented 3 years ago

I found the bug in the python app (I think 😆). Would you mind updating the python tool and see if it is now happy?

DDVanMS2018 commented 3 years ago

Hi, I think I have the same problem. The settings still don't save.

PS C:> pip3 install --upgrade meshtastic Requirement already satisfied: meshtastic in c:\users\dallas.platformio\penv\lib\site-packages (1.1.20) Requirement already satisfied: pygatt>=4.0.5 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (4.0.5) Requirement already satisfied: pyqrcode>=1.2.1 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (1.2.1) Requirement already satisfied: pyserial>=3.4 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (3.4) Requirement already satisfied: dotmap>=1.3.14 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (1.3.23) Requirement already satisfied: pexpect>=4.6.0 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (4.8.0) Requirement already satisfied: protobuf>=3.13.0 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (3.13.0) Requirement already satisfied: pypubsub>=4.0.3 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (4.0.3) Requirement already satisfied: ptyprocess>=0.5 in c:\users\dallas.platformio\penv\lib\site-packages (from pexpect>=4.6.0->meshtastic) (0.6.0) Requirement already satisfied: six>=1.9 in c:\users\dallas.platformio\penv\lib\site-packages (from protobuf>=3.13.0->meshtastic) (1.15.0) Requirement already satisfied: setuptools in c:\users\dallas.platformio\penv\lib\site-packages (from protobuf>=3.13.0->meshtastic) (41.2.0) Requirement already satisfied: pyserial>=3.4 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (3.4) Requirement already satisfied: enum-compat in c:\users\dallas.platformio\penv\lib\site-packages (from pygatt>=4.0.5->meshtastic) (0.0.3) PS C:> python.exe -m pip install --upgrade pip Requirement already satisfied: pip in c:\users\dallas.platformio\penv\lib\site-packages (20.3.1)

PS C:> meshtastic --info Connected to radio my_node_num: 2988158628 has_gps: true num_channels: 13 region: "1.0-US" hw_model: "tbeam" firmware_version: "1.1.0" packet_id_bits: 32 current_packet_id: 1676219822 node_num_bits: 32 message_timeout_msec: 300000 min_app_version: 172

preferences { position_broadcast_secs: 900 send_owner_interval: 4 wait_bluetooth_secs: 120 screen_on_secs: 300 phone_timeout_secs: 900 phone_sds_timeout_sec: 7200 mesh_sds_timeout_secs: 7200 sds_secs: 31536000 ls_secs: 3600 }

PS C:> meshtastic --set position_broadcast_secs 60 --set wait_bluetooth_secs 28800 --info Connected to radio Setting position_broadcast_secs to 60 Setting wait_bluetooth_secs to 28800 Writing modified preferences to device my_node_num: 2988158628 has_gps: true num_channels: 13 region: "1.0-US" hw_model: "tbeam" firmware_version: "1.1.0" packet_id_bits: 32 current_packet_id: 1194212792 node_num_bits: 32 message_timeout_msec: 300000 min_app_version: 172

preferences { position_broadcast_secs: 60 send_owner_interval: 4 wait_bluetooth_secs: 28800 screen_on_secs: 300 phone_timeout_secs: 900 phone_sds_timeout_sec: 7200 mesh_sds_timeout_secs: 7200 sds_secs: 31536000 ls_secs: 3600 }

PS C:> meshtastic --info Connected to radio my_node_num: 2988158628 has_gps: true num_channels: 13 region: "1.0-US" hw_model: "tbeam" firmware_version: "1.1.0" packet_id_bits: 32 current_packet_id: 593152499 node_num_bits: 32 message_timeout_msec: 300000 min_app_version: 172

preferences { position_broadcast_secs: 900 send_owner_interval: 4 wait_bluetooth_secs: 120 screen_on_secs: 300 phone_timeout_secs: 900 phone_sds_timeout_sec: 7200 mesh_sds_timeout_secs: 7200 sds_secs: 31536000 ls_secs: 3600 }

crossan007 commented 3 years ago

I'm still seeing this with python version 1.1.21.

I also opened https://github.com/meshtastic/Meshtastic-python/pull/38 to make more visible the currently installed CLI version

crossan007 commented 3 years ago

Had a little discussion on the Slack channel about this: https://meshtasticdev.slack.com/archives/C0163V5NC79/p1607743902429100

DDVanMS2018 commented 3 years ago

Looks like the settings saved after a few tries. The settings I wanted to make are finally saved.

PS C:> pip3 install --upgrade meshtastic
Requirement already satisfied: meshtastic in c:\users\dallas.platformio\penv\lib\site-packages (1.1.22) Requirement already satisfied: pexpect>=4.6.0 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (4.8.0) Requirement already satisfied: pyqrcode>=1.2.1 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (1.2.1) Requirement already satisfied: pygatt>=4.0.5 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (4.0.5) Requirement already satisfied: dotmap>=1.3.14 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (1.3.23) Requirement already satisfied: pypubsub>=4.0.3 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (4.0.3) Requirement already satisfied: protobuf>=3.13.0 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (3.13.0) Requirement already satisfied: pyserial>=3.4 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (3.4) Requirement already satisfied: ptyprocess>=0.5 in c:\users\dallas.platformio\penv\lib\site-packages (from pexpect>=4.6.0->meshtastic) (0.6.0) Requirement already satisfied: setuptools in c:\users\dallas.platformio\penv\lib\site-packages (from protobuf>=3.13.0->meshtastic) (41.2.0) Requirement already satisfied: six>=1.9 in c:\users\dallas.platformio\penv\lib\site-packages (from protobuf>=3.13.0->meshtastic) (1.15.0) Requirement already satisfied: enum-compat in c:\users\dallas.platformio\penv\lib\site-packages (from pygatt>=4.0.5->meshtastic) (0.0.3) Requirement already satisfied: pyserial>=3.4 in c:\users\dallas.platformio\penv\lib\site-packages (from meshtastic) (3.4)

PS C:> meshtastic --info Connected to radio my_node_num: 2988158628 has_gps: true num_channels: 13 region: "1.0-US" hw_model: "tbeam" firmware_version: "1.1.0" packet_id_bits: 32 current_packet_id: 1871768910 node_num_bits: 32 message_timeout_msec: 300000 min_app_version: 172

preferences { position_broadcast_secs: 60 send_owner_interval: 4 wait_bluetooth_secs: 28800 screen_on_secs: 300 phone_timeout_secs: 900 phone_sds_timeout_sec: 7200 mesh_sds_timeout_secs: 7200 sds_secs: 31536000 ls_secs: 3600 }

geeksville commented 3 years ago

hmm - i'll investigate this today. thanks for the good reports! Has anyone seen this problem on a !Windows OS? I think it might have to do with how we are tearing down the command link.,\

geeksville commented 3 years ago

@timgunter if my commit above doesn't fix the bug you found, would you mind try calling time.sleep(0.2) in StreamInterface.__disconnected() before calling close() on the port. I bet that will fix this problem (by waiting for any last sent characters to be delivered from the PC to the device before the port is closed)

(My theory is that pyserial flush() is not working correctly on Windows - but I don't have a windows machine to test with)

timgunter commented 3 years ago

Just tried the latest commit, and I'm still having the same issue. A .2 second sleep in StreamInterface._disconnected() doesn't solve the issue. When I sleep for about 10 seconds, the message does get through.

I noticed that on raspbian, when the python app disconnects it doesn't reboot the device. However, on my Windows box, the device is rebooted both when the python api connects and when it disconnects. Perhaps this is the issue?

geeksville commented 3 years ago

I noticed that on raspbian, when the python app disconnects it doesn't reboot the device. However, on my Windows box, the device is rebooted both when the python api connects and when it disconnects. Perhaps this is the issue?

OOOH! Thats a good clue. Yeah - that's not supposed to happen and doesn't happen on my linux desktop. What model of lora device are you using?

geeksville commented 3 years ago

@timgunter Could you try removing this conditional and applying the same fix that worked for OS-X?

in __init__.py
        # OS-X seems to have a bug in its serial driver.  It ignores that we asked for no RTSCTS
        # control and will always drive RTS either high or low (rather than letting the CP102 leave
        # it as an open-collector floating pin).  Since it is going to drive it anyways we want to make
        # sure it is driven low, so that the TBEAM won't reset
        if platform.system() == 'Darwin':
            self.stream.rts = False
geeksville commented 3 years ago

This issue has been mentioned on Meshtastic. There might be relevant details there:

https://meshtastic.discourse.group/t/firmware-complete-with-wifi-enabled/1858/20

timgunter commented 3 years ago

I have experienced the device reboot when connecting from Windows with all the devices I have: TTGO T-Beam v1.1 w/ SX1276 TTGO T-Beam v1.1 w/ SX1262 Heltec WiFi Lora32 V2

The RTS fix also works on Windows. Now the device doesn't reset, and the messages get through.

geeksville commented 3 years ago

yay thanks for that info! I'll change the polarity of that fix to apply on all platforms that are not linux

timgunter commented 3 years ago

Just to make things more complicated, on my Windows system, setting rts to True in SerialInterface._disconnected() results in a single device reset(The old code reset two times; once upon entering the API, and once upon exit). The value of rts is "False" prior to setting it to True here. If I set rts to False in init(), and do nothing in_disconnected(), the message gets through, the device does not reset, and the reset button on the device still works. If I re-invoke meshtastic.py to send another message just after the first one, the value of rts upon entering init() is True.

geeksville commented 3 years ago

@timgunter ooh that's super helpful. Would it be possible for you to convert that fix into a PR? I don't have a windows machine to test on currently and it sounds like you've found the perfect solution (and a lot of people are having this problem)

geeksville commented 3 years ago

This issue has been mentioned on Meshtastic. There might be relevant details there:

https://meshtastic.discourse.group/t/python-api-release-notes/150/27

geeksville commented 3 years ago

1.1.25 released with this fix. Anyone who is in this bug thread, would you mind downloading and confirm it works for you? (ideally someone with Windows and someone else with OS-X - I just tested linux)

timgunter commented 3 years ago

Works on my Windows box. Thanks!

geeksville commented 4 months ago

This issue has been mentioned on Meshtastic. There might be relevant details there:

https://meshtastic.discourse.group/t/preferences-settings-sometimes-not-sticking-saving-cli-2-2-22-firmware-2-2-24-app-2-2-24/10560/1