Payload values incredibly inaccurate when using TTGO T-BEAM and Packed Data Format

[proffalken]

Hi folks,

I've followed the tutorial in the README file and I've got data flowing in to my TTN app, however the values are all over the place:

{
  "altitude": 60435,
  "hdop": 2.04,
  "latitude": -2011.9552,
  "longitude": -8.787961,
  "pax": 25877,
  "sats": 149,
  "wifi": 25877
}

This is from a device currently running in my house, with (at the very most!) 15 WiFi and 4 bluetooth devices in range.

My paxcounter.conf has the following content:

// ----- Paxcounter user config file ------
//
// --> adapt to your needs and use case <--
//
// Note: After editing, before "build", use "clean" button in PlatformIO!

#define PRODUCTNAME                     "PAXCNT"

// Verbose enables serial output
#define VERBOSE                         1       // comment out to silence the device, for mute use build option

// Payload send cycle and encoding
#define SEND_SECS                       30      // payload send cycle [seconds/2] -> 60 sec.
#define PAYLOAD_ENCODER                 2       // payload encoder: 1=Plain, 2=Packed, 3=CayenneLPP dynamic, 4=CayenneLPP packed

// Set this to include BLE counting and vendor filter functions
#define VENDORFILTER                    1       // comment out if you want to count things, not people
#define BLECOUNTER                      1       // activate if you want to use BLE count, at expense of memory

// BLE scan parameters
#define BLESCANTIME                     0       // [seconds] scan duration, 0 means infinite [default], see note below
#define BLESCANWINDOW                   80      // [milliseconds] scan window, see below, 3 .. 10240, default 80ms
#define BLESCANINTERVAL                 80      // [illiseconds] scan interval, see below, 3 .. 10240, default 80ms = 100% duty cycle

/* Note: guide for setting bluetooth parameters
*
* |< Scan Window >       |< Scan Window >       | ... |< Scan Window >       |
* |<    Scan Interval   >|<    Scan Interval   >| ... |<    Scan Interval   >|
* |<                   Scan duration                                        >|
* 
* Scan duration sets how long scanning should be going on, before starting a new scan cycle. 0 means infinite (default).
* Scan window sets how much of the interval should be occupied by scanning. Should be >= BLESCANINTERVAL.
* Scan interval is how long scanning should be done on each channel. BLE uses 3 channels for advertising.
* -> Adjust these values with power consumption in mind if power is limited.
* -> Scan interval can be changed during runtime by remote comammand.
*/

// WiFi scan parameters
#define WIFI_CHANNEL_MIN                1       // start channel number where scan begings
#define WIFI_CHANNEL_MAX                13      // total channel number to scan
#define WIFI_MY_COUNTRY                 "EU"    // select locale for Wifi RF settings
#define WIFI_CHANNEL_SWITCH_INTERVAL    50      // [seconds/100] -> 0,5 sec.

// LoRa payload default parameters
#define MEM_LOW                         2048    // [Bytes] low memory threshold triggering a send cycle
#define RETRANSMIT_RCMD                 5       // [seconds] wait time before retransmitting rcommand results
#define PAYLOAD_BUFFER_SIZE             51      // maximum size of payload block per transmit
#define LORASFDEFAULT                   9       // 7 ... 12 SF, according to LoRaWAN specs
#define MAXLORARETRY                    500     // maximum count of TX retries if LoRa busy
#define SEND_QUEUE_SIZE                 10       // maximum number of messages in payload send queue [1 = no queue]

// Ports on which the device sends and listenes on LoRaWAN and SPI
#define COUNTERPORT                     1       // Port on which device sends counts
#define RCMDPORT                        2       // Port on which device listenes for remote commands
#define STATUSPORT                      2       // Port on which device sends remote command results
#define CONFIGPORT                      3       // Port on which device sends config query results
#define GPSPORT                         4       // Port on which device sends gps data
#define BUTTONPORT                      5       // Port on which device sends button pressed signal
#define LPP1PORT                        1       // Port for Cayenne LPP 1.0 dynamic sensor encoding
#define LPP2PORT                        2       // Port for Cayenne LPP 2.0 packed sensor encoding
#define BEACONPORT                      6       // Port on which device sends beacon alarms
#define BMEPORT                         7       // Port on which device sends BME680 sensor data
#define BATTPORT                        8       // Port on which device sends battery voltage data
#define SENSOR1PORT                     10      // Port on which device sends User sensor #1 data
#define SENSOR2PORT                     11      // Port on which device sends User sensor #2 data
#define SENSOR3PORT                     12      // Port on which device sends User sensor #3 data

// Some hardware settings
#define RGBLUMINOSITY                   30      // RGB LED luminosity [default = 30%]
#define DISPLAYREFRESH_MS               40      // OLED refresh cycle in ms [default = 40] -> 1000/40 = 25 frames per second
#define HOMECYCLE                       30      // house keeping cycle in seconds [default = 30 secs]

// Settings for BME680 environmental sensor (if present)
#define BME_TEMP_OFFSET                 5.0f    // Offset sensor on chip temp <-> ambient temp [default = 5°C]
#define STATE_SAVE_PERIOD               UINT32_C(360 * 60 * 1000) // update every 360 minutes = 4 times a day

// OTA settings
#define USE_OTA                         1       // Comment out to disable OTA update
#define WIFI_MAX_TRY                    5       // maximum number of wifi connect attempts for OTA update [default = 20]
#define OTA_MAX_TRY                     5       // maximum number of attempts for OTA download and write to flash [default = 3]
#define OTA_MIN_BATT                    3600    // minimum battery level for OTA [millivolt]
#define RESPONSE_TIMEOUT_MS             60000   // firmware binary server connection timeout [milliseconds]

// settings for syncing time of node and external time sources
#define TIME_SYNC_INTERVAL_GPS          5       // sync time each .. minutes from GPS [default = 5], comment out means off          
#define TIME_SYNC_INTERVAL_RTC          60      // sync time each .. minutes from RTC [default = 60], comment out means off          
//#define TIME_SYNC_INTERVAL_LORA       60      // sync time each .. minutes from LORA network [default = 60], comment out means off

// time zone, see https://github.com/JChristensen/Timezone/blob/master/examples/WorldClock/WorldClock.ino
#define DAYLIGHT_TIME                   {"CEST", Last, Sun, Mar, 2, 120}     // Central European Summer Time
#define STANDARD_TIME                   {"CET ", Last, Sun, Oct, 3, 60}      // Central European Standard Time

// LMIC settings
// moved to src/lmic_config.h

And I'm uploading via the command line using platformio run -t upload

I've uploaded the "packed" decoder and encoder from https://github.com/cyberman54/ESP32-Paxcounter/blob/master/src/TTN/packed_decoder.js and https://github.com/cyberman54/ESP32-Paxcounter/blob/master/src/TTN/packed_converter.js to TTN console, but the data is wildly inaccurate.

I'm sure it's just me, but what am I doing wrong?!

[cyberman54]

Since data is arriving this looks like a conversion problem on TTN console side. Did you put anything in the encoder tab? If so, delete it. You need the decoder and the converter.

If the problem persists, delete all entries from all payload format tabs in TTN console, then copy/paste the raw data seen in console to here.

[proffalken]

Thanks.

I've removed all of the decoders/encoders/converters and the raw data is as follows:

{
  "time": "2019-02-25T15:29:35.574791447Z",
  "frequency": 868.5,
  "modulation": "LORA",
  "data_rate": "SF7BW125",
  "coding_rate": "4/5",
  "gateways": [
    {
      "gtw_id": "ttnp-25-gw1",
      "gtw_trusted": true,
      "timestamp": 381172555,
      "time": "",
      "channel": 2,
      "rssi": -55,
      "snr": 7,
      "rf_chain": 1,
      "latitude": <CORRECT LAT>,
      "longitude": -<CORRECT LNG>
    }
  ]
}

What's really weird is that everything seems to be coming in via port 1 with multiple payload lengths, whereas I thought things would be coming in on different ports - is that just for the plain decoder?

A sample payload is 15650000148807E87AFF95CB001A90

[proffalken]

Yup, this is latest master, and this is a brand new setup in TTN - new app, devices, everything.

I'll dive into the code and shotgun debug with a few println's and see what I can get out of it

[cyberman54]

@proffalken Allright, here's the solution: You're using Payload Encoder Type "3", this Cayenne LPP 2.0 format. The above sample payload contains a Wifi count (value 0) and a GPS location value.

Probably the code was compiled with a paxcounter.conf file containing the line #define PAYLOAD_ENCODER 3

The default setting, needed for packed decoder/converter is #define PAYLOAD_ENCODER 2

Please make sure you have the correct setting in paxcounter.conf and rebuild & reflash code.

Important: Press "Clean" before "build" in platformio, since for some reason modifications in the .conf file sometimes are not recompiled by platformio.

[cyberman54]

@proffalken You can validate on your device which payload encoder was compiled. See console output during startup. Payload encoding is listed on the line starting with Features:. If you see LPPDYN here, not PACKED, the wrong encoder was compiled.

[proffalken]

OK, no idea what caused it, but after following the instructions for the vim plugin and modifying the make file, then running make clean and make upload, it now works...

I'm more than happy to put this down to User Error and close this off.

Thanks anyway :)