Open rudinyu opened 5 years ago
What accel-ppp version do you use?
I have 3 machines to run different version, but they have the same issues.
This is my accelpppd
net@250-50-accelpppd:~$ accel-cmd --version accel-cmd 7bcac049264393cf06c8be7743d96c66a7ad701c
net@250-51-accelpppd:~$ accel-cmd --version accel-cmd 1.11.2
net@250-55-accelpppd:~$ accel-cmd --version accel-cmd 2910add2238d419ad82e43f3ebf1fb8d5708e13b
Do you see in accel-cmd show stat
acct lost or interim lost?
How can I check this on my test server?
acct lost(total/5m/1m): 3713/0/0 acct avg query time(5m/1m): 0/0 ms interim sent: 0 interim lost(total/5m/1m): 0/0/0 interim avg query time(5m/1m): 0/0 ms
We use netstat -u netstat -u Active Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State udp 0 0 250-55-accelpppd:33883 10.1.25.2:radius-acct ESTABLISHED udp 0 0 250-55-accelpppd:64981 10.1.25.2:radius-acct ESTABLISHED udp 0 0 250-55-accelpppd:40809 10.1.25.2:radius-acct ESTABLISHED
When we stop dial-up testing program, the UDP sessions still be here.
Our stress testing is
I think it's possible if sessions have state "finish", do you may check this? After stress test run command accel-cmd show sessions match state finish
then wait closing sessions and see netstat -u
If radius server not reply onAcct-Status-Type Start
accel-ppp trying send again several times ([radius]max-try=3 by default and [radius]timeout=3 by defaults) and then close sessions. I think that's normal for this situation.
When we stop stress testing session, we use netstat -u to check. We get a more ESTABLISHED session. Even we wait for one day, these sessions still are on ESTABLISHED. We need to stop accel-ppp.
It looks like sock handle leak.
Thanks -Rudin
Do you may show me part of log with one of sessions which has state "start" or "finish" and has session long time. And part of accel-ppp.conf [ppp] and [pppoe]. Before log capture, you must set [log]level=5 [ppp]verbose=1 [ipoe]verbose=1 and run accel-cmd reload
. I think this problem not radius
hinet@250-55-accelpppd:~$ netstat -tun | more
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 127.0.0.1:54336 127.0.0.1:6379 TIME_WAIT
tcp 0 52 10.1.250.55:22 10.1.25.204:60493 ESTABLISHED
udp 0 0 10.1.250.55:33084 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:37180 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:28988 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:49468 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:20796 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:45372 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:16700 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:53564 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:12604 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:41276 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:24892 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:61756 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:57660 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:33085 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:37181 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:45373 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:41277 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:28989 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:61757 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:20797 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:53565 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:49469 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:57661 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:24893 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:12605 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:16701 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:37182 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:61758 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:24894 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:20798 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:33086 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:12606 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:16702 10.1.25.2:1813 ESTABLISHED
udp 0 0 10.1.250.55:28990 10.1.25.2:1813 ESTABLISHED
hinet@250-55-accelpppd:~$
hinet@250-55-accelpppd:~$
hinet@250-55-accelpppd:~$
hinet@250-55-accelpppd:~$ netstat -u | more
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
udp 0 0 250-55-accelpppd:33084 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:37180 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:28988 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:49468 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:20796 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:45372 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:16700 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:53564 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:12604 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:41276 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:24892 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:61756 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:57660 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:33085 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:37181 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:45373 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:41277 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:28989 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:61757 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:20797 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:53565 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:49469 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:57661 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:24893 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:12605 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:16701 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:37182 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:61758 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:24894 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:20798 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:33086 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:12606 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:16702 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:28990 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:45374 10.1.25.2:radius-acct ESTABLISHED
udp 0 0 250-55-accelpppd:49470 10.1.25.2:radius-acct ESTABLISHED
hinet@250-55-accelpppd:~$ netstat -u | wc -l
50899
hinet@250-55-accelpppd:~$ accel-cmd --version
accel-cmd 2910add2238d419ad82e43f3ebf1fb8d5708e13b
hinet@250-55-accelpppd:~$
hinet@250-55-accelpppd:~$ accel-cmd show sessions
ifname | username | calling-sid | ip | type | comp | state | uptime
--------+----------+-------------+----+------+------+-------+--------
hinet@250-55-accelpppd:~$ accel-cmd show stat
uptime: 10.23:59:18
cpu: 0%
mem(rss/virt): 398520/670772 kB
core:
mempool_allocated: 365603652
mempool_available: 856492
thread_count: 6
thread_active: 1
context_count: 684
context_sleeping: 0
context_pending: 0
md_handler_count: 50988
md_handler_pending: 0
timer_count: 51175
timer_pending: 0
sessions:
starting: 0
active: 0
finishing: 0
pppoe:
starting: 0
active: 0
delayed PADO: 0
recv PADI: 16030278
drop PADI: 17
sent PADO: 16030261
recv PADR(dup): 3997399(35476)
sent PADS: 3950769
filtered: 0
radius(1, 10.1.25.2):
state: active
fail count: 180956
request count: 0
queue length: 0
auth sent: 3679793
auth lost(total/5m/1m): 240/0/0
auth avg query time(5m/1m): 4/6 ms
acct sent: 7365211
acct lost(total/5m/1m): 186169/6/3
acct avg query time(5m/1m): 0/0 ms
interim sent: 0
interim lost(total/5m/1m): 0/0/0
interim avg query time(5m/1m): 0/0 ms
hinet@250-55-accelpppd:~$ accel-cmd show stat
uptime: 10.23:59:21
cpu: 0%
mem(rss/virt): 398520/670772 kB
core:
mempool_allocated: 365603652
mempool_available: 856620
thread_count: 6
thread_active: 1
context_count: 684
context_sleeping: 0
context_pending: 0
md_handler_count: 50988
md_handler_pending: 0
timer_count: 51175
timer_pending: 0
sessions:
starting: 0
active: 0
finishing: 0
pppoe:
starting: 0
active: 0
delayed PADO: 0
recv PADI: 16030278
drop PADI: 17
sent PADO: 16030261
recv PADR(dup): 3997399(35476)
sent PADS: 3950769
filtered: 0
radius(1, 10.1.25.2):
state: active
fail count: 180956
request count: 0
queue length: 0
auth sent: 3679793
auth lost(total/5m/1m): 240/0/0
auth avg query time(5m/1m): 4/6 ms
acct sent: 7365211
acct lost(total/5m/1m): 186169/6/3
acct avg query time(5m/1m): 0/0 ms
interim sent: 0
interim lost(total/5m/1m): 0/0/0
interim avg query time(5m/1m): 0/0 ms
hinet@250-55-accelpppd:~$
accel-ppp log
[2019-05-14 09:52:49]: info: ens9.3601: recv [PPPoE PADI 00:00:77:0b:01:02 => ff:ff:ff:ff:ff:ff sid=0000
debug log
[2019-05-14 09:52:49.219] ens9.3601: recv [PPPoE PADI 00:00:77:0b:01:02 => ff:ff:ff:ff:ff:ff sid=0000
Sorry for long answer time. Did you use module pppd_compat? Can you provide whole accel-ppp.conf
We don't enable pppd_compat.
It is our config #file
[modules] log_file
pppoe
auth_pap radius chap-secrets ippool
connlimit
[auth] any-login=0 noauth=0
timeout=9
[core] log-error=/var/log/accel-ppp/core.log thread-count=6
[common]
max-sessions=32100
[ppp] verbose=1 min-mtu=1280 mtu=1440 mru=1440 # accomp=deny
pcomp=deny ccp=0 check-ip=0
mppe=deny ipv4=require ipv6=deny
lcp-echo-interval=120 lcp-echo-failure=1 lcp-echo-timeout=0 unit-cache=2000
[pppoe] verbose=1 ac-name=250.50
called-sid=ifname:mac
ip-pool=pool1
mac-filter=/etc/mac-blacklist,deny
interface=re:ens9.
[dns] dns1=168.95.1.1 dns2=168.95.192.1
[radius] dictionary=/usr/local/share/accel-ppp/radius/dictionary
gw-ip-address=10.1.250.199 server=10.1.25.2,jalee123,auth-port=1812,acct-port=1813,max-lease-time=540,renew-time=180,lease-time=360,req-limit=400,fail-timeout=3,max-fail=2,weight=1
verbose=1 timeout=3 max-try=2
acct-timeout=3 acct-delay-time=1
[ip-pool] gw-ip-address=10.1.250.199
attr=Framed-Pool
10.167.0.0/18,name=pool1 10.167.64.0/20,name=pool1 10.167.80.0/21,name=pool1 10.167.88.0/22,name=pool1 10.167.92.0/24,name=pool1 10.167.93.0-192,name=pool1
[log] log-file=/var/log/accel-ppp/accel-ppp.log log-emerg=/var/log/accel-ppp/emerg.log log-fail-file=/var/log/accel-ppp/auth-fail.log log-debug=/var/log/accel-ppp/debug.log
copy=1
level=3
[log-pgsql] conninfo=user=log log-table=log
[pap-secrets] pap-secrets=/etc/ppp/pap-secrets
[cli] verbose=1 telnet=127.0.0.1:2000 tcp=127.0.0.1:2001
[snmp] master=0 agent-name=accel-ppp
[connlimit]
limit=150/s burst=3 timeout=30
[accel-dp] socket=/var/run/accel-dp.sock
Cannot reproduce with your config. Why Accounting-Request
without Acct-Status-Type
? You are modified code or log output?
Thanks
We found if radius request more this setting (req-limit=400), it will happen session leak.
In rad_server_req_exit function, I have a question if serv->req_cnt big than serv->req_limit, why we don't do anything.
acl-pppd create a lot of udp session to radius server,but we check radius server, these sessions should be finished.
We use netstat to check it show "ESTABLISHED"
Like
udp 0 0 10.1.250.55:29053 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:24957 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:45437 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:24958 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:41342 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:16766 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:53630 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:45438 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:37246 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:20862 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:57726 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:33150 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:49534 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:12670 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:29054 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:61822 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:37247 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:33151 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:16767 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:24959 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:12671 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd udp 0 0 10.1.250.55:49535 10.1.25.2:1813 ESTABLISHED 14206/accel-pppd
Do you have any idea about it?