After build on RPI3 failed to start concentrator

[hallard]

Hi kersing I've build from scratch on my PI3 with IMST IC880a board on raspbian etcher.

When I launch old packet forwarder all is working fine.

root@ttn-gw02(ro):/opt/ttn-gateway/bin# ./poly_pkt_fwd
*** Poly Packet Forwarder for Lora Gateway ***
Version: 2.1.0
*** Lora concentrator HAL library version info ***
Version: 3.1.0; Options: native;
***
INFO: Little endian host
INFO: found global configuration file global_conf.json, parsing it
INFO: global_conf.json does contain a JSON object named SX1301_conf, parsing SX1301 parameters
INFO: lorawan_public 1, clksrc 1
INFO: Configuring TX LUT with 16 indexes
...
INFO: Contact email configured to "contact@ch2i.eu"
INFO: Description configured to "ch2i-gw02-ic880a"
INFO: Successfully contacted server router.eu.thethings.network
INFO: [main] Starting the concentrator
INFO: [main] concentrator started, radio packets can now be received.
INFO: [up] Thread activated for all servers.
INFO: [down] Thread activated for all server router.eu.thethings.network
INFO: [down] for server router.eu.thethings.network PULL_ACK received in 368 ms
^C

Now I reset the concentrator with reset script and I launch new mp packet forwarder failed to start concentrator

root@ttn-gw02(rw):/opt/ttn-gateway/mp_pkt_fwd# ./mp_pkt_fwd
*** Multi Protocol Packet Forwarder for Lora Gateway ***
Version: 3.0.0
*** Lora concentrator HAL library version info ***
Version: 5.0.1; Options: native;
***
INFO: Little endian host
INFO: found global configuration file global_conf.json, parsing it
INFO: global_conf.json does contain a JSON object named SX1301_conf, parsing SX1301 parameters
...
ERROR: [main] failed to start the concentrator

Ok now try to launch old packet forwarder that was working just before

root@ttn-gw02(rw):/opt/ttn-gateway/bin# ./poly_pkt_fwd
*** Poly Packet Forwarder for Lora Gateway ***
Version: 2.1.0
*** Lora concentrator HAL library version info ***
Version: 3.1.0; Options: native;
***
...
NFO: [main] Starting the concentrator
ERROR: Version of calibration firmware not expected, actual:4 expected:2
ERROR: [main] failed to start the concentrator

humm, seems concentrator config has been changed, let's restart with a reset before

root@ttn-gw02(rw):/opt/ttn-gateway/bin# ./reset_lgw.sh start
Accessing concentrator reset pin through GPIO17...
root@ttn-gw02(rw):/opt/ttn-gateway/bin# ./poly_pkt_fwd
*** Poly Packet Forwarder for Lora Gateway ***
Version: 2.1.0
*** Lora concentrator HAL library version info ***
Version: 3.1.0; Options: native;
***
INFO: Little endian host
INFO: found global configuration file global_conf.json, parsing it
INFO: global_conf.json does contain a JSON object named SX1301_conf, parsing SX1301 parameters
...INFO: [main] Starting the concentrator
INFO: [main] concentrator started, radio packets can now be received.
INFO: [up] Thread activated for all servers.
INFO: [down] Thread activated for all server router.eu.thethings.network
INFO: [down] for server router.eu.thethings.network PULL_ACK received in 375 ms

works fine, so I suspect there is a problem with reset pin and I don't know were to setup it in the new packet forwarder when building it on target ?

Any idea of this problem, maybe it's not reset relatated?

Thanks for your help

For information official TTN binary packet forwarder starts fine also after a reset of the concentrator

root@ttn-gw02(rw):/opt/ttn-gateway# ./packet-forwarder --config .pktfwd.yml start
  INFO Packet Forwarder for LoRa Gateway        HALVersionInfo=Version: 4.0.0; Options: native;
  INFO Reset pin specified, resetting concentrator... ResetPin=17
  INFO Found gateway parameters, getting frequency plans AuthServer=https://account.thethingsnetwork.org GatewayID=ch2i-                                                   ttn-gw02 URL=https://account.thethingsnetwork.org/api/v2/frequency-plans/EU_863_870
  INFO Refreshed account server information     TokenExpiry=2017-12-19 16:31:26.443866781 +0100 CET
  INFO Connecting to TTN discovery server       Address=discover.thethingsnetwork.org:1900
  INFO Connected to discovery server - getting router address
  INFO Connecting to router...                  RouterID=ttn-router-eu
  INFO Downlinks queuing routine started
  INFO SX1301 board configured                  ClockSource=1 LorawanPublic=true
  INFO Configured TX Lut                        Indexes=16
  INFO Radio configured                         EnabledTX=true Frequency=867500000 RSSIOffset=-166 Radio=0 Type=SX1257
  INFO Radio configured                         EnabledTX=false Frequency=868500000 RSSIOffset=-166 Radio=1 Type=SX1257
  INFO LoRa multi-SF channel configured         Freq=-400000 Lora multi-SF channel=0 RFChain=1
  INFO LoRa multi-SF channel configured         Freq=-200000 Lora multi-SF channel=1 RFChain=1
  INFO LoRa multi-SF channel configured         Freq=0 Lora multi-SF channel=2 RFChain=1
  INFO LoRa multi-SF channel configured         Freq=-400000 Lora multi-SF channel=3 RFChain=0
  INFO LoRa multi-SF channel configured         Freq=-200000 Lora multi-SF channel=4 RFChain=0
  INFO LoRa multi-SF channel configured         Freq=0 Lora multi-SF channel=5 RFChain=0
  INFO LoRa multi-SF channel configured         Freq=200000 Lora multi-SF channel=6 RFChain=0
  INFO LoRa multi-SF channel configured         Freq=400000 Lora multi-SF channel=7 RFChain=0
  INFO LoRa standard channel configured
  INFO FSK channel configured
  WARN No GPS chip configured, ignoring
  INFO Starting concentrator...                 DateTime=2017-09-17 17:31:27.49858626 +0200 CEST
  INFO Concentrator started, packets can now be received and sent DateTime=2017-09-17 17:31:30.414484019 +0200 CEST
  INFO Waiting for uplink packets
  INFO Waiting for downlink messages
  INFO Received uplink packets                  NbPackets=1
  INFO Determined concentrator boot time        BootTime=2017-09-17 17:31:28.020336475 +0200 CEST
  INFO Sending valid uplink packets             NbValidPackets=1
  INFO Uplink message transmission successful.  CodingRate=4/5 DataRate=SF7BW125 Frequency=867500000 GatewayID=ch2i-ttn-                                                   gw02 Modulation=LORA PayloadSize=21 RSSI=-21 SNR=8
  INFO Received uplink packets                  NbPackets=1
  INFO Sending valid uplink packets             NbValidPackets=1
  INFO Uplink message transmission successful.  CodingRate=4/5 DataRate=SF10BW125 Frequency=868100000 GatewayID=ch2i-ttn                                                   -gw02 Modulation=LORA PayloadSize=21 RSSI=-75 SNR=8.25
  INFO Received uplink packets                  NbPackets=2
  INFO Sending valid uplink packets             NbValidPackets=2
  INFO Uplink message transmission successful.  CodingRate=4/5 DataRate=SF7BW125 Frequency=868300000 GatewayID=ch2i-ttn-gw02 Modulation=LORA PayloadSize=28 RSSI=-31 SNR=9
  INFO Uplink message transmission successful.  CodingRate=4/5 DataRate=SF7BW125 Frequency=868300000 GatewayID=ch2i-ttn-gw02 Modulation=LORA PayloadSize=28 RSSI=-31 SNR=9
  INFO Sending status to the network server     Altitude=110 CpuPercentage=1.578475 FrequencyPlan=EU_863_870 Latitude=46.62585 Load1=0.09 Load15=0 Load5=0.02 Longitude=0.434929 MemoryPercentage=11.510501 RTT=34 RXPacketsReceived=4 RXPacketsValid=4 TXPacketsReceived=0 TXPacketsValid=0 Uptime=17.430026s

[kersing]

1) The MP forwarder does not reset the concentrator. Use the same command sequence used for the poly forwarder to reset. (Should be in the reset_lgw.sh script)

2) There are known issues with all packet forwarders on the RPi3 due to SPI timing not being correct. In my experience if the startup fails it usually works after a few attempts (with reset in between). If it continuously fails you can try the spi_speed branch (requires spi_speed branch of lora_gateway as well!) and specify the SPI speed on the command-line when starting the packet forwarder. To fix the SPI timings check this thread on the TTN forum

[hallard]

Jac Thanks for this helpfull information, I was not aware about SPI because to be honest I never had this problem and I'm using RPI3 (at least on 4 gateways) with my fork of your old packed forwarder. Good to know that reset is not done by your packet forwarder. What is strange is that I've got 3 packet forwarder on this RPI3, 2 works fine (TTN new official, and old poly) only this one has issue with Concentrator. Anyway funny that SPI is wrong, I had same problem with console uart (my plate board has FTDI) and I already set core_freq=250 and set_turbo=1 and problem still exist. May be I missed something in build process, I'll get a chance with spi speed branch one day ;-) Anyway, thanks for helping me.

[kersing]

This forwarder is the only one of the three using the newest lora libraries from Semtech. May-be that one is more particular?

[hallard]

good catch, do you think I can test it with old lora lib ?

[hallard]

so strange got compiled spi_speed branch of libloragw and mp_packet tried all (with reset in between of course)

./mp_pkt_fwd -s 8000000
./mp_pkt_fwd -s 5000000
./mp_pkt_fwd -s 4000000
./mp_pkt_fwd -s 1000000

none worked ;-( Anyway I wanted to know CPU usage of your new packet forwarder ? on my RPI3 old is 1% and TTN offcial is 10% (too much for my taste), hoping your in between ;-)

[kersing]

Old lora lib has different API. Will not work without some work. Could you try resin with https://github.com/jpmeijers/ttn-resin-gateway-rpi to see if that works?

Top inside resin container shows ~ 1% for forwarder.

[hallard]

Yeah, I would like to, not sure to understand what resin is in fact, something like docker ?

[kersing]

Check resin.io : Resin.io brings the benefits of Linux containers to the IoT. Develop iteratively, deploy safely, and manage at scale.

[hallard]

Need to try this ;-) Ok I done something interesting, gone to IMST LoraLite gateway (not a RPI3) rebuild your packet forwarder same thing unable to start concentrator, I think I missed something in build process ;-)

[kersing]

Have you configured it for use with 'native' spi and the correct device in lora_gateway/libloragw/library.cfg? Or are you accidentally trying to use the USB version? (SPI does not require libmpsse so if you need that you know you're not building for the right hardware)

[kersing]

For building on an RPi, you could use https://github.com/jpmeijers/ttn-resin-gateway-rpi/blob/master/dev/build.sh That version is for iC880a and works with RAK831 as well.

[hallard]

Thanks for the information

I think I'm in no luck with MP packet forwarder I just tried from another GW, an official IMST Gateway Lite (ic880a + Rpi B+) so not a PI3 I launched build script build.shfrom dev folder of resin.io, build fine and just changed install folder to /opt/ttn-gateway/resin.io to avoid any conflict with original poly packet forwarder Since resin.io script install wiringPi I've done reset script and changed pin to GPIO5 to be sure

#!/bin/sh
RESET_PIN=5
echo "Accessing concentrator reset pin through GPIO$RESET_PIN..."
WAIT_GPIO() {
    sleep 0.1
}
gpio -g mode $RESET_PIN out ; WAIT_GPIO
gpio -g write  $RESET_PIN 0 ; WAIT_GPIO
gpio -g write  $RESET_PIN 1 ; WAIT_GPIO
gpio -g write  $RESET_PIN 0 ; WAIT_GPIO

launched

root@ttn-gw07(rw):/opt/ttn-gateway/resin.io# ../reset.sh
Accessing concentrator reset pin through GPIO5...
root@ttn-gw07(rw):/opt/ttn-gateway/resin.io# ./mp_pkt_fwd 
*** Multi Protocol Packet Forwarder for Lora Gateway ***
Version: 3.0.0
*** Lora concentrator HAL library version info ***
Version: 5.0.1; Options: native;
***
INFO: Little endian host
INFO: found global configuration file global_conf.json, parsing it
...
INFO: [main] Starting the concentrator
ERROR: Failed to load fw 1
ERROR: Version of calibration firmware not expected, actual:0 expected:2
ERROR: [main] failed to start the concentrator
root@ttn-gw07(rw):/opt/ttn-gateway/resin.io#

and of course launching original poly packet forwarder starts the concentrator fine

root@ttn-gw07(rw):/opt/ttn-gateway/bin# ../reset.sh 
Accessing concentrator reset pin through GPIO5...
root@ttn-gw07(rw):/opt/ttn-gateway/bin# ./poly_pkt_fwd 
*** Poly Packet Forwarder for Lora Gateway ***
Version: 2.1.0
*** Lora concentrator HAL library version info ***
Version: 3.1.0; Options: native;
***
INFO: Little endian host
INFO: found global configuration file global_conf.json, parsing it
...
INFO: [main] concentrator started, radio packets can now be received.
INFO: [down] Thread activated for all server router.eu.thethings.network
INFO: [up] Thread activated for all servers.
INFO: [down] for server router.eu.thethings.network PULL_ACK received in 48 ms

I'm totally lost, building on my own or with resin.io on 2 different targets (RPI3 and RPI2 B+) with each is own ic880a board never succeeded to have it working.

[kersing]

Please check the global_config.json you are using, what is the clock source set to?

[hallard]

Jac you got it !!! I changed clock source, it was default setup for multitech not for other. I changed to setup clock source to 1 and now it works. Worth mentioning this on resin.io build script :-)

        /*  "clksrc": 1, radio_1 provides clock to concentrator for most devices except MultiTech */
    "clksrc": 0, /* radio_0 provides clock to concentrator for MultiTech */

to

        "clksrc": 1, /* radio_1 provides clock to concentrator for most devices except MultiTech */
    /* "clksrc": 0, radio_0 provides clock to concentrator for MultiTech */

Works on IMST Lite GW, I'll try on PI3 now

Thank you for your help

[kersing]

The full resin.io setup uses a python script to download the latest configuration at startup. By just using the build script you are not 'enjoying' the full experience :wink: and hence encountered this issue.