aprsc crash on long filter string

snip commented 8 years ago

Hi,

i just got 2 occurrences of my both aprsc servers still running but were not able to respond anymore to any TCP requests (http ou aprs). Existing TCP connections were also dropped. Logs were no more updating.

Both servers which got the issue at same time was running aprsc 2.0.19-g05a4d35

Unfortunately, I don't have so much information, but maybe it can comes from one client having bad behavior ...

Any idea is welcome.

Thanks

snip commented 8 years ago

It is failing again and again :(

hessu commented 8 years ago

Can you confirm that:

The aprsc process is still up, running, visible in the "ps ax" process list?
Does it consume CPU (see 'top', is it in the top of the list, in a busy loop or something)?
What did you do to fix it?
When running "sudo service aprsc restart" or equivalent, does it successfully shut down the server, does the log file display the shutdown sequence before starting up again?
When it is stuck, can you connect to it locally from the server (telnet your-ip-address 14580)?

You can also attach the log file here so that I can inspect it. If it doesn't show anything interesting, and it's still failing, you can bump the log level to 'debug' (adjust the command line in /etc/default/aprsc or equivalent, replace '-e info' with '-e debug').

snip commented 8 years ago

Hi hessu,

in fact as i'm running 2 instances of aprsc, one was still running. But you are true: one is stopping without saying anything in normal logs.

So to fix that i was restarting process.

With debug i was able to identify one client which is using a very large aprs filter at login time. Error message in logs are the following:

Dropped packet (-18: long_packet) len 1473

When i filter client IP by ACL, i don't have anymore the issue. This client is also doing lot of different connection to the aprsc server.

Logs sent by mail.

Thanks

hessu commented 8 years ago

Ok, thanks. It seems like the log probably does not reveal the actual trigger of the crash, it dies before it has a chance of logging the input which causes the crash.

Does the 'dmesg' command print out a segmentation fault message for aprsc? Do you get a core dump of aprsc in /var/crash?

If yes, please provide the core dump (along with precise operating system and aprsc version numbers).

snip commented 8 years ago

Unfortunately not. It is possible my linux has coredump disabled.

hessu commented 8 years ago

I think I can now reproduce one crash which is probably the one seen here. Made a test case for it last night.

snip commented 8 years ago

Good news. Thanks.

hessu commented 8 years ago

Please try out v2.0.20 which is released now. It includes a fix for a bug that I could reproduce, which matches the description well.

snip commented 8 years ago

Hi, i upgraded 3 days ago. No issue so far. But i don't know if the aprs client is still sending the faulty message.

I will update if any. Thanks

hessu / aprsc

aprsc crash on long filter string #29