search

AllStarLink / ASL3

AllStarLink Version 3
https://www.allstarlink.org
GNU Affero General Public License v3.0
7 stars 1 forks source link

Tx gets stuck on during bootup... #76

Open w8cpt opened 1 week ago

w8cpt commented 1 week ago

Pi 3 A+ and a AURSINC Allstar RF Node UHF SR-FRS Radio Programming Module for Allstar hangs up and leaves TX on when I shut down the node properly after the install and reboot it. Here's a video and my buddies Shari Pi-hat did the same thing. He put a 30 second delay in Asterisk an entry in system d service file and it seemed to correct it...I don't know how to do that! https://youtu.be/mnH9BxGZWqQ?si=lEwRawGfE0p0Ch5a

Have you run a software update and rebooted?

**Pi 3 A+

jxmx commented 6 days ago

I'm a bit confused by this report. You're talking about a PiHat but linking to a video of a USB-attached appliance board. To confirm, this is what you're trying to use, correct? https://www.amazon.com/AURSINC-Programming-Echolink-Compatible-Raspberry/dp/B0BNKS4GGH

JayCue702 commented 6 days ago

heres quick solution for delay. in terminal

sudo nano /lib/systemd/system/asterisk.service

then add following line above ExecStart: ExecStartPre=/bin/sleep 30

then restart asterisk and see if that worked

jxmx commented 6 days ago

@JayCue702 - You do not want to do that. /lib/systemd/system/asterisk.service will get blown away on the next upgrade. If you really want to change the systemd unit, then the correct procedure is to copy the /lib/systemd/system unit to /etc/systemd/system, edit the file there, and then do a systemctl daemon-reload.

jxmx commented 6 days ago

Philosophically though, adding a sleep to ExecStartPre is really a bandge over a different problem. If this is only happening at boot, then most likely this unit is very slow to initialize and/or requires some other system process to be available before asterisk tries to use it.

tsawyer commented 6 days ago

One of my testers is having this issue. I think we need to find the root cause and a solution. Any debugging suggestions?

jxmx commented 6 days ago

We either need someone on the development team who has the hardware that’s a problem or get SSH and root access to someone’s note that is having the problem.

KB4MDD commented 6 days ago

As @jxmx noted, we need hardware to test or dedicated access to a computer.

jxmx commented 6 days ago

Tim what hardware is your tester seeing the issue on?

W6HBR commented 6 days ago

One of my testers is having this issue. I think we need to find the root cause and a solution. Any debugging suggestions?

Tim, I have a HotSpotRadio that exhibits this issue. First boot keys up and stays. If I go into the tune menu and update settings, then restart Asterisk, it stops. Unfortunately, I do not have time to debug right now, but you're welcome to come by and grab the unit if you wish.

jxmx commented 6 days ago

@W6HBR what model?

W6HBR commented 6 days ago

Pi 3 w/ hat https://hotspotradios.com/hotspotradio

tsawyer commented 6 days ago

Tim what hardware is your tester seeing the issue on?

@jxmx Rpi3b with DMK URIxb.

JayCue702 commented 6 days ago

I was having issues getting simpleusb.conf settings to save on reboot. Could this be possibility for this issue? Does it need usbinvert?

JayCue702 commented 6 days ago

Post your tail

tail /etc/asterisk/simpleusb.conf

w8cpt commented 6 days ago

I don't know what that means...can you explain more?

On Tue, Jul 2, 2024, 3:33 PM JayCue702 @.***> wrote:

Post your tail

tail /etc/asterisk/simpleusb.conf

— Reply to this email directly, view it on GitHub https://github.com/AllStarLink/ASL3/issues/76#issuecomment-2204189280, or unsubscribe https://github.com/notifications/unsubscribe-auth/BBX5CKMZWXEDEIK2665ZSATZKL57BAVCNFSM6AAAAABKGX35F6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBUGE4DSMRYGA . You are receiving this because you authored the thread.Message ID: @.***>

JayCue702 commented 6 days ago

Go to terminal and type that line in above then copy and paste what it says here

w8cpt commented 6 days ago

asl@node588840:~ $ tail /etc/asterisk/simpleusb.conf

588840

ctcssfrom = no

;;;;; ASL3 Tune settings ;;;;; devstr = 1-1:1.0 rxmixerset = 650 txmixaset = 650 txmixbset = 650

w8cpt commented 6 days ago

anything else while im at the terminal?

w8cpt commented 6 days ago

I'm a bit confused by this report. You're talking about a PiHat but linking to a video of a USB-attached appliance board. To confirm, this is what you're trying to use, correct? https://www.amazon.com/AURSINC-Programming-Echolink-Compatible-Raspberry/dp/B0BNKS4GGH

yes thats what im using Amazon called it that I copied it from Amazon...

JayCue702 commented 6 days ago

Not sure if all these hats are equal but can you try adding line to end of simpleusb.conf

carrierfrom = usb

w8cpt commented 6 days ago

Not sure if all these hats are equal but can you try adding line to end of simpleusb.conf

carrierfrom = usb

ive tried that in the tune menu on the asl-menu and it still does it....

JayCue702 commented 6 days ago

I had issues with parameters not saving using tune menu. Go to expert settings and edit simpleusb.conf. I'm just checking if you had same issue I had

w8cpt commented 6 days ago

anything else before I shut down this node? log files needed etc? you have to tell me what to type to get them but cause im not a pro by far but I'll try...

w8cpt commented 6 days ago

I had issues with parameters not saving using tune menu. Go to expert settings and edit simpleusb.conf. I'm just checking if you had same issue I had

that happend to me once but the way its currently configured it saves the settings. not sure what I changed other than when it didn't save the USB settings the "Restart Asterisk" popped up. when it actually saved the settings i went and restarted asterisk myself in the menu...

w8cpt commented 6 days ago

here's my settings. i have tried "I" and "J" no, USB and USBINVERT and I still had the restart issue...

Active Simple USB Radio device is [588840]. 1) Select active USB device 2) Set Rx Voice Level (using display) 3) Set Transmit A Level (currently '650') 4) Set Transmit B Level (currently '650') B) Toggle RX Boost (currently 'disabled') C) Toggle Pre-emphasis (currently 'disabled') D) Toggle De-emphasis (currently 'disabled') E) Toggle Echo Mode (currently 'disabled') F) Flash (Toggle PTT and Tone output several times) G) Toggle PL Filter (currently 'enabled') H) Toggle PTT mode (currently 'ground') I) Change Carrier From (currently 'usbinvert') J) Change CTCSS From (currently 'no') K) Change RX On Delay (currently '0') L) Change TX Off Delay (currently '0') P) Print Current Parameter Values S) Swap Current USB device with another USB device T) Toggle Transmit Test Tone/Keying (currently 'disabled') V) View COS, CTCSS and PTT Status W) Write (Save) Current Parameter Values 0) Exit Menu

Please enter your selection now:

JayCue702 commented 6 days ago

thats because its not saving like i said using that menu and when you did that tail command it shows its not saving. so i need you to go sudo asl-menu > expert configuration menu > edit simpleusb.conf file

scroll all the way to bottom and add line carrierfrom = usb

press ctrl x, press y press enter

now restart your node

JayCue702 commented 6 days ago

not sure if it works for your device but this worked for someone elses setup. seems to be few different hats on the market

jxmx commented 6 days ago

So some of these adapters are COR=high? Is that what we're saying?

w8cpt commented 6 days ago

thats because its not saving like i said using that menu and when you did that tail command it shows its not saving. so i need you to go sudo asl-menu > expert configuration menu > edit simpleusb.conf file

scroll all the way to bottom and add line carrierfrom = usb

press ctrl x, press y press enter

now restart your node

But it says 650 and 650 so it saved them...

tsawyer commented 6 days ago

From what I can gather, the transmitter gets stuck only on first boot from a new microSB. This might be related to a kernel upgrade during the boot process.

Here are separate servers where USB dies on boot up. I believe these are both Rpi 3b. See snips from /var/log/asterisk/messages.log from each server:

This is HopSpotRadios node...

[2024-03-15 08:10:32.501] NOTICE[1100] app_rpt.c: Normal Repeater Init  1999
[2024-03-15 08:10:32.503] NOTICE[1105] chan_simpleusb.c: Channel 1999: Automatically assigned USB device 1-1.4:1.0 to SimpleUSB channel
[2024-03-15 08:10:32.507] NOTICE[1104] chan_simpleusb.c: Channel 1999: Set option TONE VERIFY, mode: OFF(0).
[2024-03-15 08:10:32.562] WARNING[1104] chan_simpleusb.c: Possibly stuck USB read channel. [1999]
[2024-03-15 08:10:32.587] WARNING[1104] chan_simpleusb.c: USB read channel [1999] was not stuck.
[2024-07-02 19:58:06.210] ERROR[1104] chan_simpleusb.c: Channel 1999: HID process has died or is not responding.
[2024-07-02 19:58:10.000] WARNING[1100] app_rpt.c: rpt_thread restarted on node 1999
[2024-07-02 19:58:10.011] NOTICE[1726] chan_simpleusb.c: Channel 1999: Set option TONE VERIFY, mode: OFF(0).
[2024-07-02 19:58:10.013] ERROR[1727] chan_simpleusb.c: Channel 1999: Unable to open DSP device 1: Device or resource busy.
[2024-07-02 19:58:14.003] ERROR[1726] chan_simpleusb.c: Channel 1999: HID process has died or is not responding.
[2024-07-02 20:03:02.328] NOTICE[1081] dnsmgr.c: dnssrv: host 'register.allstarlink.org' changed from 162.248.92.131:443 to 34.105.111.212:443

And this is the other server. It has a URI...

[2024-06-30 14:56:02.912] NOTICE[1024] app_rpt.c: Normal Repeater Init  47050
[2024-06-30 14:56:02.915] NOTICE[1028] chan_simpleusb.c: Channel 47050: Set option TONE VERIFY, mode: OFF(0).
[2024-06-30 14:56:36.695] ERROR[1028] chan_simpleusb.c: Channel 47050: HID process has died or is not responding.
[2024-06-30 14:56:37.127] WARNING[1024] app_rpt.c: rpt_thread restarted on node 47050
[2024-06-30 14:56:37.130] NOTICE[1281] chan_simpleusb.c: Channel 47050: Set option TONE VERIFY, mode: OFF(0).
[2024-06-30 14:56:37.160] ERROR[1282] chan_simpleusb.c: Channel 47050: Unable to open DSP device 1: Device or resource busy.
[2024-06-30 14:56:37.182] WARNING[1281] chan_simpleusb.c: Possibly stuck USB read channel. [47050]
[2024-06-30 14:56:37.207] WARNING[1281] chan_simpleusb.c: USB read channel [47050] was not stuck.
[2024-06-30 14:56:41.005] ERROR[1281] chan_simpleusb.c: Channel 47050: HID process has died or is not responding.
[2024-06-30 15:01:18.059] NOTICE[997] dnsmgr.c: dnssrv: host 'register.allstarlink.org' changed from 162.248.92.131:443 to 34.105.111.212:443

After a second reboot, all is well.

[2024-06-30 15:05:21.535] NOTICE[1882] app_rpt.c: Normal Repeater Init  47050
[2024-06-30 15:05:21.538] NOTICE[1885] chan_simpleusb.c: Channel 47050: Set option TONE VERIFY, mode: OFF(0).
[2024-06-30 15:05:21.614] WARNING[1885] chan_simpleusb.c: Possibly stuck USB read channel. [47050]
[2024-06-30 15:05:21.638] WARNING[1885] chan_simpleusb.c: USB read channel [47050] was not stuck.
[2024-06-30 15:06:08.804] DTMF[1885] channel.c: DTMF begin '*' received on SimpleUSB/47050
[2024-06-30 15:06:08.804] DTMF[1885] channel.c: DTMF begin passthrough '*' on SimpleUSB/47050
w8cpt commented 6 days ago

It's not just the 1st boot on a new SD card. It happens every boot for me....

tsawyer commented 6 days ago

Does your log show the usb is stuck?

w8cpt commented 6 days ago

I don't know what log to look at. If you show me I will be able to copy it Saturday if someone doesn't figure it out before then. Mine does it every boot up now it keys up and get stuck on tx and if I unplug the Aursinc and plug it back in it's not keyed up anymore but I can't TX or RX...my buddies Shari does the same on every boot up.

I'll attach a photo of something someone on Facebook said fixes it but I can't mess with it til Saturday so I don't know if it works...

He said " Don McJr because there is no stanza to drop the ground for ptt. That's why adding this line forces it to pull the ground hence stopping the PTT. All of this software is new and still in BETA! Take everything with open arms. But what do I know.

I can also tell you how to turn off cwid but I'm probably wrong too." IMG_20240703_003307_618

jxmx commented 5 days ago

For anyone having problems with the "TX on boot", I'd like people to try something. As root/via sudo, please do the following:

rm -f /etc/systemd/system/asterisk.service 
wget -O /lib/systemd/system/asterisk.service https://repo.allstarlink.org/hotfix/asl3_76_asterisk_systemd.service
systemctl daemon-reload
systemctl restart asterisk

After this, please test your device WITHOUT rebooting and confirm everything is okay. Then reboot the unit and see if the hung TX happens.

tsawyer commented 5 days ago

The new service did not work for my friend with the URI.

Tx locks up on every reboot (he wasn't clear that it was every reboot previously). Can be cleared with an Asterisk restart each time.

KB4MDD commented 5 days ago

It might help to get the simpleusb.conf and messages.log to review.

jxmx commented 5 days ago

It has been verified that RTC/wallclock/hwclock interactions are causing this at boot. Linked the issue in app_rpt above. Short term fix is to apply:

rm -f /etc/systemd/system/asterisk.service 
wget -O /lib/systemd/system/asterisk.service https://repo.allstarlink.org/hotfix/asl3_76_asterisk_systemd.service
systemctl daemon-reload

Note: this is not a permanent solution and may cause long delays in startup

Having the asterisk start depending on time-sync.target can delay asterisk starting for anywhere from 30 seconds to minutes depending on how quickly timesyncd is able to set the clock accurately. This will also fail to ever launch with a node is not a network with a viable Internet connection to set the clock. So this is a very temporary workaround.

mpannen1979 commented 4 days ago

A pi4 with SimpleUSB device locked on TX for me once, and both that unit and a pi3 with SimpleUSB devices had error messages in asterisk log file regarding 'USB HID' audio device lock ups. If I restarted asterisk things were OK.

This was the general message I was seeing:

chan_simpleusb.c: HID process has died or is not responding.

I am trying the 'delay startup of asterisk for time sync' in a service override file as mentioned above.

Both the pi3 and pi4, after the startup delay for time sync, worked on the first boot, no USB HID error messages listed in asterisk log file.

saffrey commented 3 days ago

For anyone having problems with the "TX on boot", I'd like people to try something. As root/via sudo, please do the following:

rm -f /etc/systemd/system/asterisk.service 
wget -O /lib/systemd/system/asterisk.service https://repo.allstarlink.org/hotfix/asl3_76_asterisk_systemd.service
systemctl daemon-reload
systemctl restart asterisk

After this, please test your device WITHOUT rebooting and confirm everything is okay. Then reboot the unit and see if the hung TX happens.

Can confirm that this worked for me. Pi3 with SA818 via USB. Thanks!

jxmx commented 2 days ago

The proposed fix will be included in 3.0.1

mpannen1979 commented 2 days ago

the 3.0.1 packages from devel seem to do the trick for me

tsawyer commented 1 day ago

The root issue was on Raspberry Pi nodes without a realtime clock. A large time shift while Asterisk was running would cause Asterisk to become unresponsive and stop the usb channel drivers among other problems. Depending on the timing the PTT would lock asserted. Some nodes would lock in Tx on every boot, others only the during first boot after an install, or never lock in PTT and only appear unresponsive. Restarting Asterisk would always "fix" the problem until the next reboot.

jxmx commented 1 day ago

This fix will now likely be included in the 3.0.2 production release. There is an issue that needs to be resolved with 3.0.1 before production release. New devel packages coming later today.

N0NB commented 1 day ago

On a Pi 3B+ with ASL3 using a Techno By George FOB with the Simple USB driver, the FOB appeared non-responsive to COS. An Asterisk restart brought to life. So far I've not seen the PTT lockup but I've not rebooted much in my testing this past week. At least now that I'm aware I can anticipate it until the update hits the repository.