search

FreeRADIUS / freeradius-server

FreeRADIUS - A multi-protocol policy server.
http://freeradius.org
GNU General Public License v2.0
2.09k stars 1.07k forks source link

DB query getting hang during database switchover in AWS #2610

Closed sraghunathan closed 5 years ago

sraghunathan commented 5 years ago

DB query getting hang during database switchover in AWS

Defect

Issue we are facing is during the database switchover freeRADIUS insert query is getting hanged , even after successful database switchover freeRADIUS is in hung state and not processing any other messages, query is not getting timeout.

We tried to perform the config change of query_timeout = 5 in /etc/raddb/mods-enabled/sql but its not reflecting during the radiusd -X

To solve this issue we have manually restart the service.

Observation

If there is no RADIUS traffic during this database switchover, there is no issue observed. After successful switchover , if we start the traffic its working fine from the new database. Kindly note, database hostname is same for both the DB. database switchiover is triggered by a DNS change for the hostname.

How to reproduce the issue

Create 2 database (both in sync) in two different instance. Create the DNS A record with 1st database which will act as primary. Perform the switch over based on changing the DNS A record. Observe how freeRADIUS behave during the DB change

Output of [radiusd|freeradius] -X showing issue occurring

[root@ip-10-12-239-140 centos]# radiusd -X
FreeRADIUS Version 3.0.13
Copyright (C) 1999-2017 The FreeRADIUS server project and contributors
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE
You may redistribute copies of FreeRADIUS under the terms of the
GNU General Public License
For more information about these matters, see the file named COPYRIGHT
Starting - reading configuration files ...
including dictionary file /usr/share/freeradius/dictionary
including dictionary file /usr/share/freeradius/dictionary.dhcp
including dictionary file /usr/share/freeradius/dictionary.vqp
including dictionary file /etc/raddb/dictionary
including configuration file /etc/raddb/radiusd.conf
including configuration file /etc/raddb/proxy.conf
including configuration file /etc/raddb/clients.conf
including files in directory /etc/raddb/mods-enabled/
including configuration file /etc/raddb/mods-enabled/always
including configuration file /etc/raddb/mods-enabled/attr_filter
including configuration file /etc/raddb/mods-enabled/cache_eap
including configuration file /etc/raddb/mods-enabled/chap
including configuration file /etc/raddb/mods-enabled/date
including configuration file /etc/raddb/mods-enabled/detail
including configuration file /etc/raddb/mods-enabled/detail.log
including configuration file /etc/raddb/mods-enabled/dhcp
including configuration file /etc/raddb/mods-enabled/digest
including configuration file /etc/raddb/mods-enabled/dynamic_clients
including configuration file /etc/raddb/mods-enabled/eap
including configuration file /etc/raddb/mods-enabled/echo
including configuration file /etc/raddb/mods-enabled/exec
including configuration file /etc/raddb/mods-enabled/expiration
including configuration file /etc/raddb/mods-enabled/expr
including configuration file /etc/raddb/mods-enabled/files
including configuration file /etc/raddb/mods-enabled/linelog
including configuration file /etc/raddb/mods-enabled/logintime
including configuration file /etc/raddb/mods-enabled/mschap
including configuration file /etc/raddb/mods-enabled/ntlm_auth
including configuration file /etc/raddb/mods-enabled/pap
including configuration file /etc/raddb/mods-enabled/passwd
including configuration file /etc/raddb/mods-enabled/preprocess
including configuration file /etc/raddb/mods-enabled/radutmp
including configuration file /etc/raddb/mods-enabled/realm
including configuration file /etc/raddb/mods-enabled/replicate
including configuration file /etc/raddb/mods-enabled/soh
including configuration file /etc/raddb/mods-enabled/sradutmp
including configuration file /etc/raddb/mods-enabled/unix
including configuration file /etc/raddb/mods-enabled/unpack
including configuration file /etc/raddb/mods-enabled/utf8
including configuration file /etc/raddb/mods-enabled/sql
including configuration file /etc/raddb/mods-config/sql/main/postgresql/queries.conf
including files in directory /etc/raddb/policy.d/
including configuration file /etc/raddb/policy.d/accounting
including configuration file /etc/raddb/policy.d/canonicalization
including configuration file /etc/raddb/policy.d/control
including configuration file /etc/raddb/policy.d/cui
including configuration file /etc/raddb/policy.d/debug
including configuration file /etc/raddb/policy.d/dhcp
including configuration file /etc/raddb/policy.d/eap
including configuration file /etc/raddb/policy.d/filter
including configuration file /etc/raddb/policy.d/operator-name
including files in directory /etc/raddb/sites-enabled/
including configuration file /etc/raddb/sites-enabled/rt-radius
including configuration file /etc/raddb/sites-enabled/testbrad.pcap
main {
 security {
        user = "radiusd"
        group = "radiusd"
        allow_core_dumps = no
 }
        name = "radiusd"
        prefix = "/usr"
        localstatedir = "/var"
        logdir = "/var/log/radius"
        run_dir = "/var/run/radiusd"
}
main {
        name = "radiusd"
        prefix = "/usr"
        localstatedir = "/var"
        sbindir = "/usr/sbin"
        logdir = "/var/log/radius"
        run_dir = "/var/run/radiusd"
        libdir = "/usr/lib64/freeradius"
        radacctdir = "/var/log/radius/radacct"
        hostname_lookups = no
        max_request_time = 30
        cleanup_delay = 5
        max_requests = 16384
        pidfile = "/var/run/radiusd/radiusd.pid"
        checkrad = "/usr/sbin/checkrad"
        debug_level = 0
        proxy_requests = yes
 log {
        stripped_names = no
        auth = no
        auth_badpass = no
        auth_goodpass = no
        colourise = yes
        msg_denied = "You are already logged in - access denied"
 }
 resources {
 }
 security {
        max_attributes = 200
        reject_delay = 1.000000
        status_server = yes
 }
}
radiusd: #### Loading Realms and Home Servers ####
 proxy server {
        retry_delay = 5
        retry_count = 3
        default_fallback = no
        dead_time = 120
        wake_all_if_all_dead = no
 }
 home_server localhost {
        ipaddr = 127.0.0.1
        port = 1812
        type = "auth"
        secret = <<< secret >>>
        response_window = 20.000000
        response_timeouts = 1
        max_outstanding = 65536
        zombie_period = 40
        status_check = "status-server"
        ping_interval = 30
        check_interval = 30
        check_timeout = 4
        num_answers_to_alive = 3
        revive_interval = 120
  limit {
        max_connections = 16
        max_requests = 0
        lifetime = 0
        idle_timeout = 0
  }
  coa {
        irt = 2
        mrt = 16
        mrc = 5
        mrd = 30
  }
 }
 home_server_pool my_auth_failover {
        type = fail-over
        home_server = localhost
 }
 realm example.com {
        auth_pool = my_auth_failover
 }
 realm LOCAL {
 }
radiusd: #### Loading Clients ####
 client localhost {
        ipaddr = 127.0.0.1
        require_message_authenticator = no
        secret = <<< secret >>>
        nas_type = "other"
        proto = "*"
  limit {
        max_connections = 16
        lifetime = 0
        idle_timeout = 30
  }
 }
 client localhost_ipv6 {
        ipv6addr = ::1
        require_message_authenticator = no
        secret = <<< secret >>>
  limit {
        max_connections = 16
        lifetime = 0
        idle_timeout = 30
  }
 }
 client 10.128.133.49 {
        ipaddr = 10.128.133.49
        require_message_authenticator = no
        secret = <<< secret >>>
  limit {
        max_connections = 16
        lifetime = 0
        idle_timeout = 30
  }
 }
 client 10.12.239.140 {
        ipaddr = 10.12.239.140
        require_message_authenticator = no
        secret = <<< secret >>>
  limit {
        max_connections = 16
        lifetime = 0
        idle_timeout = 30
  }
 }
Debugger not attached
 # Creating Auth-Type = mschap
 # Creating Auth-Type = digest
 # Creating Auth-Type = eap
 # Creating Auth-Type = PAP
 # Creating Auth-Type = CHAP
 # Creating Auth-Type = MS-CHAP
radiusd: #### Instantiating modules ####
 modules {
  # Loaded module rlm_always
  # Loading module "reject" from file /etc/raddb/mods-enabled/always
  always reject {
        rcode = "reject"
        simulcount = 0
        mpp = no
  }
  # Loading module "fail" from file /etc/raddb/mods-enabled/always
  always fail {
        rcode = "fail"
        simulcount = 0
        mpp = no
  }
  # Loading module "ok" from file /etc/raddb/mods-enabled/always
  always ok {
        rcode = "ok"
        simulcount = 0
        mpp = no
  }
  # Loading module "handled" from file /etc/raddb/mods-enabled/always
  always handled {
        rcode = "handled"
        simulcount = 0
        mpp = no
  }
  # Loading module "invalid" from file /etc/raddb/mods-enabled/always
  always invalid {
        rcode = "invalid"
        simulcount = 0
        mpp = no
  }
  # Loading module "userlock" from file /etc/raddb/mods-enabled/always
  always userlock {
        rcode = "userlock"
        simulcount = 0
        mpp = no
  }
  # Loading module "notfound" from file /etc/raddb/mods-enabled/always
  always notfound {
        rcode = "notfound"
        simulcount = 0
        mpp = no
  }
  # Loading module "noop" from file /etc/raddb/mods-enabled/always
  always noop {
        rcode = "noop"
        simulcount = 0
        mpp = no
  }
  # Loading module "updated" from file /etc/raddb/mods-enabled/always
  always updated {
        rcode = "updated"
        simulcount = 0
        mpp = no
  }
  # Loaded module rlm_attr_filter
  # Loading module "attr_filter.post-proxy" from file /etc/raddb/mods-enabled/attr_filter
  attr_filter attr_filter.post-proxy {
        filename = "/etc/raddb/mods-config/attr_filter/post-proxy"
        key = "%{Realm}"
        relaxed = no
  }
  # Loading module "attr_filter.pre-proxy" from file /etc/raddb/mods-enabled/attr_filter
  attr_filter attr_filter.pre-proxy {
        filename = "/etc/raddb/mods-config/attr_filter/pre-proxy"
        key = "%{Realm}"
        relaxed = no
  }
  # Loading module "attr_filter.access_reject" from file /etc/raddb/mods-enabled/attr_filter
  attr_filter attr_filter.access_reject {
        filename = "/etc/raddb/mods-config/attr_filter/access_reject"
        key = "%{User-Name}"
        relaxed = no
  }
  # Loading module "attr_filter.access_challenge" from file /etc/raddb/mods-enabled/attr_filter
  attr_filter attr_filter.access_challenge {
        filename = "/etc/raddb/mods-config/attr_filter/access_challenge"
        key = "%{User-Name}"
        relaxed = no
  }
  # Loading module "attr_filter.accounting_response" from file /etc/raddb/mods-enabled/attr_filter
  attr_filter attr_filter.accounting_response {
        filename = "/etc/raddb/mods-config/attr_filter/accounting_response"
        key = "%{User-Name}"
        relaxed = no
  }
  # Loaded module rlm_cache
  # Loading module "cache_eap" from file /etc/raddb/mods-enabled/cache_eap
  cache cache_eap {
        driver = "rlm_cache_rbtree"
        key = "%{%{control:State}:-%{%{reply:State}:-%{State}}}"
        ttl = 15
        max_entries = 0
        epoch = 0
        add_stats = no
  }
  # Loaded module rlm_chap
  # Loading module "chap" from file /etc/raddb/mods-enabled/chap
  # Loaded module rlm_date
  # Loading module "date" from file /etc/raddb/mods-enabled/date
  date {
        format = "%b %e %Y %H:%M:%S %Z"
  }
  # Loaded module rlm_detail
  # Loading module "detail" from file /etc/raddb/mods-enabled/detail
  detail {
        filename = "/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/detail-%Y%m%d"
        header = "%t"
        permissions = 384
        locking = no
        escape_filenames = no
        log_packet_header = no
  }
  # Loading module "auth_log" from file /etc/raddb/mods-enabled/detail.log
  detail auth_log {
        filename = "/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/auth-detail-%Y%m%d"
        header = "%t"
        permissions = 384
        locking = no
        escape_filenames = no
        log_packet_header = no
  }
  # Loading module "reply_log" from file /etc/raddb/mods-enabled/detail.log
  detail reply_log {
        filename = "/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/reply-detail-%Y%m%d"
        header = "%t"
        permissions = 384
        locking = no
        escape_filenames = no
        log_packet_header = no
  }
  # Loading module "pre_proxy_log" from file /etc/raddb/mods-enabled/detail.log
  detail pre_proxy_log {
        filename = "/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/pre-proxy-detail-%Y%m%d"
        header = "%t"
        permissions = 384
        locking = no
        escape_filenames = no
        log_packet_header = no
  }
  # Loading module "post_proxy_log" from file /etc/raddb/mods-enabled/detail.log
  detail post_proxy_log {
        filename = "/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/post-proxy-detail-%Y%m%d"
        header = "%t"
        permissions = 384
        locking = no
        escape_filenames = no
        log_packet_header = no
  }
  # Loaded module rlm_dhcp
  # Loading module "dhcp" from file /etc/raddb/mods-enabled/dhcp
  # Loaded module rlm_digest
  # Loading module "digest" from file /etc/raddb/mods-enabled/digest
  # Loaded module rlm_dynamic_clients
  # Loading module "dynamic_clients" from file /etc/raddb/mods-enabled/dynamic_clients
  # Loaded module rlm_eap
  # Loading module "eap" from file /etc/raddb/mods-enabled/eap
  eap {
        default_eap_type = "md5"
        timer_expire = 60
        ignore_unknown_eap_types = no
        cisco_accounting_username_bug = no
        max_sessions = 16384
  }
  # Loaded module rlm_exec
  # Loading module "echo" from file /etc/raddb/mods-enabled/echo
  exec echo {
        wait = yes
        program = "/bin/echo %{User-Name}"
        input_pairs = "request"
        output_pairs = "reply"
        shell_escape = yes
  }
  # Loading module "exec" from file /etc/raddb/mods-enabled/exec
  exec {
        wait = no
        input_pairs = "request"
        shell_escape = yes
        timeout = 10
  }
  # Loaded module rlm_expiration
  # Loading module "expiration" from file /etc/raddb/mods-enabled/expiration
  # Loaded module rlm_expr
  # Loading module "expr" from file /etc/raddb/mods-enabled/expr
  expr {
        safe_characters = "@abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789.-_: /äéöüàâæçèéêëîïôœùûüaÿÄÉÖÜßÀÂÆÇÈÉÊËÎÏÔŒÙÛÜŸ"
  }
  # Loaded module rlm_files
  # Loading module "files" from file /etc/raddb/mods-enabled/files
  files {
        filename = "/etc/raddb/mods-config/files/authorize"
        acctusersfile = "/etc/raddb/mods-config/files/accounting"
        preproxy_usersfile = "/etc/raddb/mods-config/files/pre-proxy"
  }
  # Loaded module rlm_linelog
  # Loading module "linelog" from file /etc/raddb/mods-enabled/linelog
  linelog {
        filename = "/var/log/radius/linelog"
        escape_filenames = no
        syslog_severity = "info"
        permissions = 384
        format = "This is a log message for %{User-Name}"
        reference = "messages.%{%{reply:Packet-Type}:-default}"
  }
  # Loading module "log_accounting" from file /etc/raddb/mods-enabled/linelog
  linelog log_accounting {
        filename = "/var/log/radius/linelog-accounting"
        escape_filenames = no
        syslog_severity = "info"
        permissions = 384
        format = ""
        reference = "Accounting-Request.%{%{Acct-Status-Type}:-unknown}"
  }
  # Loaded module rlm_logintime
  # Loading module "logintime" from file /etc/raddb/mods-enabled/logintime
  logintime {
        minimum_timeout = 60
  }
  # Loaded module rlm_mschap
  # Loading module "mschap" from file /etc/raddb/mods-enabled/mschap
  mschap {
        use_mppe = yes
        require_encryption = no
        require_strong = no
        with_ntdomain_hack = yes
   passchange {
   }
        allow_retry = yes
        winbind_retry_with_normalised_username = no
  }
  # Loading module "ntlm_auth" from file /etc/raddb/mods-enabled/ntlm_auth
  exec ntlm_auth {
        wait = yes
        program = "/path/to/ntlm_auth --request-nt-key --domain=MYDOMAIN --username=%{mschap:User-Name} --password=%{User-Password}"
        shell_escape = yes
  }
  # Loaded module rlm_pap
  # Loading module "pap" from file /etc/raddb/mods-enabled/pap
  pap {
        normalise = yes
  }
  # Loaded module rlm_passwd
  # Loading module "etc_passwd" from file /etc/raddb/mods-enabled/passwd
  passwd etc_passwd {
        filename = "/etc/passwd"
        format = "*User-Name:Crypt-Password:"
        delimiter = ":"
        ignore_nislike = no
        ignore_empty = yes
        allow_multiple_keys = no
        hash_size = 100
  }
  # Loaded module rlm_preprocess
  # Loading module "preprocess" from file /etc/raddb/mods-enabled/preprocess
  preprocess {
        huntgroups = "/etc/raddb/mods-config/preprocess/huntgroups"
        hints = "/etc/raddb/mods-config/preprocess/hints"
        with_ascend_hack = no
        ascend_channels_per_line = 23
        with_ntdomain_hack = no
        with_specialix_jetstream_hack = no
        with_cisco_vsa_hack = no
        with_alvarion_vsa_hack = no
  }
  # Loaded module rlm_radutmp
  # Loading module "radutmp" from file /etc/raddb/mods-enabled/radutmp
  radutmp {
        filename = "/var/log/radius/radutmp"
        username = "%{User-Name}"
        case_sensitive = yes
        check_with_nas = yes
        permissions = 384
        caller_id = yes
  }
  # Loaded module rlm_realm
  # Loading module "IPASS" from file /etc/raddb/mods-enabled/realm
  realm IPASS {
        format = "prefix"
        delimiter = "/"
        ignore_default = no
        ignore_null = no
  }
  # Loading module "suffix" from file /etc/raddb/mods-enabled/realm
  realm suffix {
        format = "suffix"
        delimiter = "@"
        ignore_default = no
        ignore_null = no
  }
  # Loading module "realmpercent" from file /etc/raddb/mods-enabled/realm
  realm realmpercent {
        format = "suffix"
        delimiter = "%"
        ignore_default = no
        ignore_null = no
  }
  # Loading module "ntdomain" from file /etc/raddb/mods-enabled/realm
  realm ntdomain {
        format = "prefix"
        delimiter = "\\"
        ignore_default = no
        ignore_null = no
  }
  # Loaded module rlm_replicate
  # Loading module "replicate" from file /etc/raddb/mods-enabled/replicate
  # Loaded module rlm_soh
  # Loading module "soh" from file /etc/raddb/mods-enabled/soh
  soh {
        dhcp = yes
  }
  # Loading module "sradutmp" from file /etc/raddb/mods-enabled/sradutmp
  radutmp sradutmp {
        filename = "/var/log/radius/sradutmp"
        username = "%{User-Name}"
        case_sensitive = yes
        check_with_nas = yes
        permissions = 420
        caller_id = no
  }
  # Loaded module rlm_unix
  # Loading module "unix" from file /etc/raddb/mods-enabled/unix
  unix {
        radwtmp = "/var/log/radius/radwtmp"
  }
Creating attribute Unix-Group
  # Loaded module rlm_unpack
  # Loading module "unpack" from file /etc/raddb/mods-enabled/unpack
  # Loaded module rlm_utf8
  # Loading module "utf8" from file /etc/raddb/mods-enabled/utf8
  # Loaded module rlm_sql
  # Loading module "sql" from file /etc/raddb/mods-enabled/sql
  sql {
        driver = "rlm_sql_postgresql"
        server = "sessioninfo.c7igpa4atzlo.eu-central-1.rds.amazonaws.com"
        port = 5432
        login = "radius"
        password = <<< secret >>>
        radius_db = "radius"
        read_groups = yes
        read_profiles = yes
        read_clients = no
        delete_stale_sessions = yes
        sql_user_name = "%{User-Name}"
        logfile = "/var/log/radius/sqllog.sql"
        default_user_profile = ""
        client_query = "SELECT id, nasname, shortname, type, secret, server FROM nas"
        authorize_check_query = "SELECT id, UserName, Attribute, Value, Op FROM radcheck WHERE Username = '%{SQL-User-Name}' ORDER BY id"
        authorize_reply_query = "SELECT id, UserName, Attribute, Value, Op FROM radreply WHERE Username = '%{SQL-User-Name}' ORDER BY id"
        authorize_group_check_query = "SELECT id, GroupName, Attribute, Value, op FROM radgroupcheck WHERE GroupName = '%{SQL-Group}' ORDER BY id"
        authorize_group_reply_query = "SELECT id, GroupName, Attribute, Value, op FROM radgroupreply WHERE GroupName = '%{SQL-Group}' ORDER BY id"
        group_membership_query = "SELECT GroupName FROM radusergroup WHERE UserName='%{SQL-User-Name}' ORDER BY priority"
        simul_count_query = "SELECT COUNT(*) FROM radacct WHERE UserName='%{SQL-User-Name}' AND AcctStopTime IS NULL"
        simul_verify_query = "SELECT RadAcctId, AcctSessionId, UserName, NASIPAddress, NASPortId, FramedIPAddress, CallingStationId, FramedProtocol FROM radacct WHERE UserName='%{SQL-User-Name}' AND AcctStopTime IS NULL"
        safe_characters = "@abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789.-_: /"
        query_timeout = 5
   accounting {
        reference = "%{tolower:type.%{%{Acct-Status-Type}:-none}.query}"
        logfile = "/var/log/radius/accounting.sql"
    type {
     accounting-on {
        query = "UPDATE radacct SET AcctStopTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctUpdateTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctSessionTime = (%{integer:Event-Timestamp} - EXTRACT(EPOCH FROM(AcctStartTime))), AcctTerminateCause = '%{%{Acct-Terminate-Cause}:-NAS-Reboot}' WHERE AcctStopTime IS NULL AND NASIPAddress= '%{%{NAS-IPv6-Address}:-%{NAS-IP-Address}}' AND AcctStartTime <= '%S'::timestamp"
     }
     accounting-off {
        query = "UPDATE radacct SET AcctStopTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctUpdateTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctSessionTime = (%{integer:Event-Timestamp} - EXTRACT(EPOCH FROM(AcctStartTime))), AcctTerminateCause = '%{%{Acct-Terminate-Cause}:-NAS-Reboot}' WHERE AcctStopTime IS NULL AND NASIPAddress= '%{%{NAS-IPv6-Address}:-%{NAS-IP-Address}}' AND AcctStartTime <= '%S'::timestamp"
     }
     start {
        query = "INSERT INTO radacct (AcctSessionId,AcctUniqueId,AcctMultiSessionId,SessionState,StartEvent,StopEvent,UserName,GroupName,Realm,NASIPAddress,NASPortId,NASPortType,NASIdentifier,AcctAuthentic,AcctDelayTime,AcctStartTime,AcctUpdateTime,AcctStopTime,AcctInterval,AcctSessionTime,AcctInputOctets,AcctOutputOctets,AcctInputPackets,AcctOutputPackets,AcctTerminateCause,ConnectInfo_start,ConnectInfo_stop,Class,CalledStationId,CallingStationId,DelegatedIPv6Prefix,FramedInterfaceId,FramedIPaddress,FramedIPv6address,FramedProtocol,ServiceType,m3GPP_Camel_Charging_Info,m3GPP_Charging_Characteristics,m3GPP_Charging_Gateway_Address,m3GPP_Charging_Id,m3GPP_GGSN_Address,m3GPP_GGSN_MCC_MNC,m3GPP_GPRS_Negotiated_QoS_profile,m3GPP_IMEISV,m3GPP_IMSI,m3GPP_IMSI_MCC_MNC,m3GPP_IPv6_DNS_Servers,m3GPP_MS_Timezone,m3GPP_Negotiated_DSCP,m3GPP_NSAPI,m3GPP_Packet_Filter,m3GPP_PDP_Type,m3GPP_RAT_Type,m3GPP_Selection_Mode,m3GPP_Session_Stop_Indicator,m3GPP_SGSN_Address,m3GPP_SGSN_MCC_MNC,m3GPP_Teardown_Indicator,m3GPP_User_Location_Info) VALUES('%{Acct-Session-Id}','%{Acct-Unique-Session-Id}','%{Acct-Multi-Session-Id}','Active','START-A','','%{SQL-User-Name}',NULL,NULLIF('%{Realm}', ''),'%{%{NAS-IPv6-Address}:-%{NAS-IP-Address}}',NULLIF('%{%{NAS-Port-ID}:-%{NAS-Port}}',''),'%{NAS-Port-Type}','%{NAS-Identifier}','%{Acct-Authentic}','%{Acct-Delay-Time}',TO_TIMESTAMP(%{integer:Event-Timestamp}),TO_TIMESTAMP(%{integer:Event-Timestamp}),NULL,0,0,0,0,0,0,NULL,'%{Connect-Info}',NULL,'%{Class}','%{Called-Station-Id}','%{Calling-Station-Id}',NULLIF('%{Delegated-IPv6-Prefix}','')::inet,'%{Framed-Interface-Id}',NULLIF('%{Framed-IP-Address}','')::inet,NULLIF('%{Framed-IPv6-Prefix}','')::inet,'%{Framed-Protocol}','%{Service-Type}','%{3GPP-CAMEL-Charging-Info}','%{3GPP-Charging-Characteristics}','%{%{3GPP-Charging-Gateway-IPv6-Address}:-%{3GPP-Charging-Gateway-Address}}','%{3GPP-Charging-ID}','%{%{3GPP-GGSN-IPv6-Address}:-%{3GPP-GGSN-Address}}','%{3GPP-GGSN-MCC-MNC}','%{3GPP-GPRS-Negotiated-QoS-profile}','%{3GPP-IMEISV}','%{3GPP-IMSI}','%{3GPP-IMSI-MCC-MNC}',NULL,'%{3GPP-MS-Time-Zone}','%{3GPP-Negotiated-DSCP}','%{3GPP-NSAPI}','%{3GPP-Packet-Filter}','%{3GPP-PDP-Type}','%{3GPP-RAT-Type}','%{3GPP-Selection-Mode}',NULL,'%{%{3GPP-SGSN-IPv6-Address}:-%{3GPP-SGSN-Address}}','%{3GPP-SGSN-MCC-MNC}',NULL,'%{3GPP-Location-Info}')"
     }
     interim-update {
        query = "UPDATE radacct SET FramedIPAddress = NULLIF('%{Framed-IP-Address}', '')::inet, FramedIPv6Address = NULLIF('%{Framed-IPv6-Prefix}', '')::inet, AcctSessionTime = %{%{Acct-Session-Time}:-NULL}, AcctInterval = (%{integer:Event-Timestamp} - EXTRACT(EPOCH FROM (COALESCE(AcctUpdateTime, AcctStartTime)))), AcctUpdateTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctInputOctets = (('%{%{Acct-Input-Gigawords}:-0}'::bigint << 32) + '%{%{Acct-Input-Octets}:-0}'::bigint), AcctOutputOctets = (('%{%{Acct-Output-Gigawords}:-0}'::bigint << 32) + '%{%{Acct-Output-Octets}:-0}'::bigint) WHERE AcctUniqueId = '%{Acct-Unique-Session-Id}' AND AcctStopTime IS NULL"
     }
     stop {
        query = "UPDATE radacct SET AcctStopTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctUpdateTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctSessionTime = COALESCE(%{%{Acct-Session-Time}:-NULL}, (%{integer:Event-Timestamp} - EXTRACT(EPOCH FROM(AcctStartTime)))), AcctInputOctets = (('%{%{Acct-Input-Gigawords}:-0}'::bigint << 32) + '%{%{Acct-Input-Octets}:-0}'::bigint), AcctOutputOctets = (('%{%{Acct-Output-Gigawords}:-0}'::bigint << 32) + '%{%{Acct-Output-Octets}:-0}'::bigint), AcctInputPackets = '%{Acct-Input-Packets}',AcctOutputPackets = '%{Acct-Output-Packets}',AcctTerminateCause = '%{Acct-Terminate-Cause}', FramedIPAddress = NULLIF('%{Framed-IP-Address}', '')::inet, FramedIPv6Address = NULLIF('%{Framed-IPv6-Prefix}', '')::inet, SessionState = 'Closed',StopEvent = 'STOP-A',m3GPP_Session_Stop_Indicator = '%{3GPP-Session-Stop-Indicator}',m3GPP_Teardown_Indicator ='%{3GPP-Teardown-Indicator}',ConnectInfo_stop = '%{Connect-Info}' WHERE AcctUniqueId = '%{Acct-Unique-Session-Id}' AND AcctStopTime IS NULL"
     }
    }
   }
   post-auth {
        reference = ".query"
        query = "INSERT INTO radpostauth (username, pass, reply, authdate) VALUES('%{User-Name}', '%{%{User-Password}:-Chap-Password}', '%{reply:Packet-Type}', NOW())"
   }
  }
rlm_sql (sql): Driver rlm_sql_postgresql (module rlm_sql_postgresql) loaded and linked
Creating attribute SQL-Group
  instantiate {
  }
  # Instantiating module "reject" from file /etc/raddb/mods-enabled/always
  # Instantiating module "fail" from file /etc/raddb/mods-enabled/always
  # Instantiating module "ok" from file /etc/raddb/mods-enabled/always
  # Instantiating module "handled" from file /etc/raddb/mods-enabled/always
  # Instantiating module "invalid" from file /etc/raddb/mods-enabled/always
  # Instantiating module "userlock" from file /etc/raddb/mods-enabled/always
  # Instantiating module "notfound" from file /etc/raddb/mods-enabled/always
  # Instantiating module "noop" from file /etc/raddb/mods-enabled/always
  # Instantiating module "updated" from file /etc/raddb/mods-enabled/always
  # Instantiating module "attr_filter.post-proxy" from file /etc/raddb/mods-enabled/attr_filter
reading pairlist file /etc/raddb/mods-config/attr_filter/post-proxy
  # Instantiating module "attr_filter.pre-proxy" from file /etc/raddb/mods-enabled/attr_filter
reading pairlist file /etc/raddb/mods-config/attr_filter/pre-proxy
  # Instantiating module "attr_filter.access_reject" from file /etc/raddb/mods-enabled/attr_filter
reading pairlist file /etc/raddb/mods-config/attr_filter/access_reject
[/etc/raddb/mods-config/attr_filter/access_reject]:11 Check item "FreeRADIUS-Response-Delay"    found in filter list for realm "DEFAULT". 
[/etc/raddb/mods-config/attr_filter/access_reject]:11 Check item "FreeRADIUS-Response-Delay-USec"       found in filter list for realm "DEFAULT". 
  # Instantiating module "attr_filter.access_challenge" from file /etc/raddb/mods-enabled/attr_filter
reading pairlist file /etc/raddb/mods-config/attr_filter/access_challenge
  # Instantiating module "attr_filter.accounting_response" from file /etc/raddb/mods-enabled/attr_filter
reading pairlist file /etc/raddb/mods-config/attr_filter/accounting_response
  # Instantiating module "cache_eap" from file /etc/raddb/mods-enabled/cache_eap
rlm_cache (cache_eap): Driver rlm_cache_rbtree (module rlm_cache_rbtree) loaded and linked
  # Instantiating module "detail" from file /etc/raddb/mods-enabled/detail
  # Instantiating module "auth_log" from file /etc/raddb/mods-enabled/detail.log
rlm_detail (auth_log): 'User-Password' suppressed, will not appear in detail output
  # Instantiating module "reply_log" from file /etc/raddb/mods-enabled/detail.log
  # Instantiating module "pre_proxy_log" from file /etc/raddb/mods-enabled/detail.log
  # Instantiating module "post_proxy_log" from file /etc/raddb/mods-enabled/detail.log
  # Instantiating module "eap" from file /etc/raddb/mods-enabled/eap
   # Linked to sub-module rlm_eap_md5
   # Linked to sub-module rlm_eap_leap
   # Linked to sub-module rlm_eap_gtc
   gtc {
        challenge = "Password: "
        auth_type = "PAP"
   }
   # Linked to sub-module rlm_eap_tls
   tls {
        tls = "tls-common"
   }
   tls-config tls-common {
        verify_depth = 0
        ca_path = "/etc/raddb/certs"
        pem_file_type = yes
        private_key_file = "/etc/raddb/certs/server.pem"
        certificate_file = "/etc/raddb/certs/server.pem"
        ca_file = "/etc/raddb/certs/ca.pem"
        private_key_password = <<< secret >>>
        dh_file = "/etc/raddb/certs/dh"
        fragment_size = 1024
        include_length = yes
        auto_chain = yes
        check_crl = no
        check_all_crl = no
        cipher_list = "DEFAULT"
        cipher_server_preference = no
        ecdh_curve = "prime256v1"
    cache {
        enable = no
        lifetime = 24
        max_entries = 255
    }
    verify {
        skip_if_ocsp_ok = no
    }
    ocsp {
        enable = no
        override_cert_url = yes
        url = "http://127.0.0.1/ocsp/"
        use_nonce = yes
        timeout = 0
        softfail = no
    }
   }
   # Linked to sub-module rlm_eap_ttls
   ttls {
        tls = "tls-common"
        default_eap_type = "md5"
        copy_request_to_tunnel = no
        use_tunneled_reply = no
        virtual_server = "inner-tunnel"
        include_length = yes
        require_client_cert = no
   }
tls: Using cached TLS configuration from previous invocation
   # Linked to sub-module rlm_eap_peap
   peap {
        tls = "tls-common"
        default_eap_type = "mschapv2"
        copy_request_to_tunnel = no
        use_tunneled_reply = no
        proxy_tunneled_request_as_eap = yes
        virtual_server = "inner-tunnel"
        soh = no
        require_client_cert = no
   }
tls: Using cached TLS configuration from previous invocation
   # Linked to sub-module rlm_eap_mschapv2
   mschapv2 {
        with_ntdomain_hack = no
        send_error = no
   }
  # Instantiating module "expiration" from file /etc/raddb/mods-enabled/expiration
  # Instantiating module "files" from file /etc/raddb/mods-enabled/files
reading pairlist file /etc/raddb/mods-config/files/authorize
reading pairlist file /etc/raddb/mods-config/files/accounting
reading pairlist file /etc/raddb/mods-config/files/pre-proxy
  # Instantiating module "linelog" from file /etc/raddb/mods-enabled/linelog
  # Instantiating module "log_accounting" from file /etc/raddb/mods-enabled/linelog
  # Instantiating module "logintime" from file /etc/raddb/mods-enabled/logintime
  # Instantiating module "mschap" from file /etc/raddb/mods-enabled/mschap
rlm_mschap (mschap): using internal authentication
  # Instantiating module "pap" from file /etc/raddb/mods-enabled/pap
  # Instantiating module "etc_passwd" from file /etc/raddb/mods-enabled/passwd
rlm_passwd: nfields: 3 keyfield 0(User-Name) listable: no
  # Instantiating module "preprocess" from file /etc/raddb/mods-enabled/preprocess
reading pairlist file /etc/raddb/mods-config/preprocess/huntgroups
reading pairlist file /etc/raddb/mods-config/preprocess/hints
  # Instantiating module "IPASS" from file /etc/raddb/mods-enabled/realm
  # Instantiating module "suffix" from file /etc/raddb/mods-enabled/realm
  # Instantiating module "realmpercent" from file /etc/raddb/mods-enabled/realm
  # Instantiating module "ntdomain" from file /etc/raddb/mods-enabled/realm
  # Instantiating module "sql" from file /etc/raddb/mods-enabled/sql
   postgresql {
        send_application_name = yes
   }
rlm_sql (sql): Attempting to connect to database "radius"
rlm_sql (sql): Initialising connection pool
   pool {
        start = 5
        min = 3
        max = 32
        spare = 10
        uses = 0
        lifetime = 0
        cleanup_interval = 30
        idle_timeout = 30
        retry_delay = 30
        spread = no
   }
rlm_sql (sql): Opening additional connection (0), 1 of 32 pending slots used
rlm_sql_postgresql: Connecting using parameters: dbname='radius' host='sessioninfo.c7igpa4atzlo.eu-central-1.rds.amazonaws.com' port=5432 user='radius' password='radpass' application_name='FreeRADIUS 3.0.13 - radiusd (sql)'
Connected to database 'radius' on 'sessioninfo.c7igpa4atzlo.eu-central-1.rds.amazonaws.com' server version 100006, protocol version 3, backend PID 17668 
rlm_sql (sql): Opening additional connection (1), 1 of 31 pending slots used
rlm_sql_postgresql: Connecting using parameters: dbname='radius' host='sessioninfo.c7igpa4atzlo.eu-central-1.rds.amazonaws.com' port=5432 user='radius' password='radpass' application_name='FreeRADIUS 3.0.13 - radiusd (sql)'
Connected to database 'radius' on 'sessioninfo.c7igpa4atzlo.eu-central-1.rds.amazonaws.com' server version 100006, protocol version 3, backend PID 17675 
rlm_sql (sql): Opening additional connection (2), 1 of 30 pending slots used
rlm_sql_postgresql: Connecting using parameters: dbname='radius' host='sessioninfo.c7igpa4atzlo.eu-central-1.rds.amazonaws.com' port=5432 user='radius' password='radpass' application_name='FreeRADIUS 3.0.13 - radiusd (sql)'
Connected to database 'radius' on 'sessioninfo.c7igpa4atzlo.eu-central-1.rds.amazonaws.com' server version 100006, protocol version 3, backend PID 17676 
rlm_sql (sql): Opening additional connection (3), 1 of 29 pending slots used
rlm_sql_postgresql: Connecting using parameters: dbname='radius' host='sessioninfo.c7igpa4atzlo.eu-central-1.rds.amazonaws.com' port=5432 user='radius' password='radpass' application_name='FreeRADIUS 3.0.13 - radiusd (sql)'
Connected to database 'radius' on 'sessioninfo.c7igpa4atzlo.eu-central-1.rds.amazonaws.com' server version 100006, protocol version 3, backend PID 17677 
rlm_sql (sql): Opening additional connection (4), 1 of 28 pending slots used
rlm_sql_postgresql: Connecting using parameters: dbname='radius' host='sessioninfo.c7igpa4atzlo.eu-central-1.rds.amazonaws.com' port=5432 user='radius' password='radpass' application_name='FreeRADIUS 3.0.13 - radiusd (sql)'
Connected to database 'radius' on 'sessioninfo.c7igpa4atzlo.eu-central-1.rds.amazonaws.com' server version 100006, protocol version 3, backend PID 17678 
 } # modules
radiusd: #### Loading Virtual Servers ####
server { # from file /etc/raddb/radiusd.conf
} # server
server default { # from file /etc/raddb/sites-enabled/rt-radius
 # Loading authenticate {...}
 # Loading authorize {...}
Ignoring "ldap" (see raddb/mods-available/README.rst)
 # Loading preacct {...}
 # Loading accounting {...}
 # Loading post-proxy {...}
 # Loading post-auth {...}
} # server default
radiusd: #### Opening IP addresses and Ports ####
listen {
        type = "acct"
        ipaddr = *
        port = 0
   limit {
        max_connections = 16
        lifetime = 0
        idle_timeout = 30
   }
}
Listening on acct address * port 1813 bound to server default
Ready to process requests

<After processing few request, we initiate switchover>

(190) Received Accounting-Request Id 17 from 10.128.133.49:64409 to 10.12.139.202:1813 length 437
(190)   User-Name = "awspreprod.movedata.mobi"
(190)   Acct-Status-Type = Start
(190)   NAS-IP-Address = 10.128.133.49
(190)   NAS-Identifier = "PUNE"
(190)   Service-Type = Framed-User
(190)   Framed-Protocol = GPRS-PDP-Context
(190)   Framed-IP-Address = 34.202.104.68
(190)   Called-Station-Id = "awspreprod.movedata.mobi"
(190)   Calling-Station-Id = "1234576977"
(190)   Acct-Delay-Time = 0
(190)   Acct-Session-Id = "0A0CE16405FF5CB02B8F62C5FF001F15A7"
(190)   Acct-Multi-Session-Id = "0A0CE16405FF5CB02B8F62C5FF001F15A7"
(190)   Acct-Authentic = Local
(190)   NAS-Port-Type = Wireless-Other
(190)   Event-Timestamp = "Apr 12 2019 06:09:19 UTC"
(190)   3GPP-IMSI = "333100001809088"
(190)   3GPP-Charging-ID = 34647696
(190)   3GPP-PDP-Type = 0
(190)   3GPP-Charging-Gateway-Address = 10.12.225.100
(190)   3GPP-GPRS-Negotiated-QoS-profile = "99-0911012901FEFE11050101"
(190)   3GPP-SGSN-Address = 10.10.53.100
(190)   3GPP-GGSN-Address = 10.12.225.100
(190)   3GPP-IMSI-MCC-MNC = "33310"
(190)   3GPP-GGSN-MCC-MNC = "111222"
(190)   3GPP-NSAPI = "5"
(190)   3GPP-Selection-Mode = "0"
(190)   3GPP-Charging-Characteristics = "0800"
(190)   3GPP-SGSN-MCC-MNC = "111222"
(190)   3GPP-RAT-Type = UTRAN
(190)   3GPP-Location-Info = 0x0011212200210004
(190)   3GPP-Negotiated-DSCP = 26
(190) # Executing section preacct from file /etc/raddb/sites-enabled/rt-radius
(190)   preacct {
(190)     [preprocess] = ok
(190)     policy acct_unique {
(190)       update request {
(190)         &Tmp-String-9 := "ai:"
(190)       } # update request = noop
(190)       if (("%{hex:&Class}" =~ /^%{hex:&Tmp-String-9}/) &&             ("%{string:&Class}" =~ /^ai:([0-9a-f]{32})/i)) {
(190)       EXPAND %{hex:&Class}
(190)          --> 
(190)       EXPAND ^%{hex:&Tmp-String-9}
(190)          --> ^61693a
(190)       if (("%{hex:&Class}" =~ /^%{hex:&Tmp-String-9}/) &&             ("%{string:&Class}" =~ /^ai:([0-9a-f]{32})/i))  -> FALSE
(190)       else {
(190)         update request {
(190)           EXPAND %{md5:%{User-Name},%{Acct-Session-ID},%{%{NAS-IPv6-Address}:-%{NAS-IP-Address}},%{NAS-Identifier},%{NAS-Port-ID},%{NAS-Port}}
(190)              --> 229070a5885df65e944e18f144f05ffb
(190)           &Acct-Unique-Session-Id := 229070a5885df65e944e18f144f05ffb
(190)         } # update request = noop
(190)       } # else = noop
(190)     } # policy acct_unique = noop
(190) suffix: Checking for suffix after "@"
(190) suffix: No '@' in User-Name = "awspreprod.movedata.mobi", looking up realm NULL
(190) suffix: No such realm "NULL"
(190)     [suffix] = noop
(190)     [files] = noop
(190)   } # preacct = ok
(190) # Executing section accounting from file /etc/raddb/sites-enabled/rt-radius
(190)   accounting {
(190) detail: EXPAND /var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/detail-%Y%m%d
(190) detail:    --> /var/log/radius/radacct/10.128.133.49/detail-20190412
(190) detail: /var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/detail-%Y%m%d expands to /var/log/radius/radacct/10.128.133.49/detail-20190412
(190) detail: EXPAND %t
(190) detail:    --> Fri Apr 12 06:10:13 2019
(190)     [detail] = ok
(190)     [unix] = ok
(190) sql: EXPAND %{tolower:type.%{%{Acct-Status-Type}:-none}.query}
(190) sql:    --> type.start.query
(190) sql: Using query template 'query'
rlm_sql (sql): Reserved connection (5)
(190) sql: EXPAND %{User-Name}
(190) sql:    --> awspreprod.movedata.mobi
(190) sql: SQL-User-Name set to 'awspreprod.movedata.mobi'
(190) sql: EXPAND INSERT INTO radacct (AcctSessionId,AcctUniqueId,AcctMultiSessionId,SessionState,StartEvent,StopEvent,UserName,GroupName,Realm,NASIPAddress,NASPortId,NASPortType,NASIdentifier,AcctAuthentic,AcctDelayTime,AcctStartTime,AcctUpdateTime,AcctStopTime,AcctInterval,AcctSessionTime,AcctInputOctets,AcctOutputOctets,AcctInputPackets,AcctOutputPackets,AcctTerminateCause,ConnectInfo_start,ConnectInfo_stop,Class,CalledStationId,CallingStationId,DelegatedIPv6Prefix,FramedInterfaceId,FramedIPaddress,FramedIPv6address,FramedProtocol,ServiceType,m3GPP_Camel_Charging_Info,m3GPP_Charging_Characteristics,m3GPP_Charging_Gateway_Address,m3GPP_Charging_Id,m3GPP_GGSN_Address,m3GPP_GGSN_MCC_MNC,m3GPP_GPRS_Negotiated_QoS_profile,m3GPP_IMEISV,m3GPP_IMSI,m3GPP_IMSI_MCC_MNC,m3GPP_IPv6_DNS_Servers,m3GPP_MS_Timezone,m3GPP_Negotiated_DSCP,m3GPP_NSAPI,m3GPP_Packet_Filter,m3GPP_PDP_Type,m3GPP_RAT_Type,m3GPP_Selection_Mode,m3GPP_Session_Stop_Indicator,m3GPP_SGSN_Address,m3GPP_SGSN_MCC_MNC,m3GPP_Teardown_Indicator,m3GPP_User_Location_Info) VALUES('%{Acct-Session-Id}','%{Acct-Unique-Session-Id}','%{Acct-Multi-Session-Id}','Active','START-A','','%{SQL-User-Name}',NULL,NULLIF('%{Realm}', ''),'%{%{NAS-IPv6-Address}:-%{NAS-IP-Address}}',NULLIF('%{%{NAS-Port-ID}:-%{NAS-Port}}',''),'%{NAS-Port-Type}','%{NAS-Identifier}','%{Acct-Authentic}','%{Acct-Delay-Time}',TO_TIMESTAMP(%{integer:Event-Timestamp}),TO_TIMESTAMP(%{integer:Event-Timestamp}),NULL,0,0,0,0,0,0,NULL,'%{Connect-Info}',NULL,'%{Class}','%{Called-Station-Id}','%{Calling-Station-Id}',NULLIF('%{Delegated-IPv6-Prefix}','')::inet,'%{Framed-Interface-Id}',NULLIF('%{Framed-IP-Address}','')::inet,NULLIF('%{Framed-IPv6-Prefix}','')::inet,'%{Framed-Protocol}','%{Service-Type}','%{3GPP-CAMEL-Charging-Info}','%{3GPP-Charging-Characteristics}','%{%{3GPP-Charging-Gateway-IPv6-Address}:-%{3GPP-Charging-Gateway-Address}}','%{3GPP-Charging-ID}','%{%{3GPP-GGSN-IPv6-Address}:-%{3GPP-GGSN-Address}}','%{3GPP-GGSN-MCC-MNC}','%{3GPP-GPRS-Negotiated-QoS-profile}','%{3GPP-IMEISV}','%{3GPP-IMSI}','%{3GPP-IMSI-MCC-MNC}',NULL,'%{3GPP-MS-Time-Zone}','%{3GPP-Negotiated-DSCP}','%{3GPP-NSAPI}','%{3GPP-Packet-Filter}','%{3GPP-PDP-Type}','%{3GPP-RAT-Type}','%{3GPP-Selection-Mode}',NULL,'%{%{3GPP-SGSN-IPv6-Address}:-%{3GPP-SGSN-Address}}','%{3GPP-SGSN-MCC-MNC}',NULL,'%{3GPP-Location-Info}')
(190) sql:    --> INSERT INTO radacct (AcctSessionId,AcctUniqueId,AcctMultiSessionId,SessionState,StartEvent,StopEvent,UserName,GroupName,Realm,NASIPAddress,NASPortId,NASPortType,NASIdentifier,AcctAuthentic,AcctDelayTime,AcctStartTime,AcctUpdateTime,AcctStopTime,AcctInterval,AcctSessionTime,AcctInputOctets,AcctOutputOctets,AcctInputPackets,AcctOutputPackets,AcctTerminateCause,ConnectInfo_start,ConnectInfo_stop,Class,CalledStationId,CallingStationId,DelegatedIPv6Prefix,FramedInterfaceId,FramedIPaddress,FramedIPv6address,FramedProtocol,ServiceType,m3GPP_Camel_Charging_Info,m3GPP_Charging_Characteristics,m3GPP_Charging_Gateway_Address,m3GPP_Charging_Id,m3GPP_GGSN_Address,m3GPP_GGSN_MCC_MNC,m3GPP_GPRS_Negotiated_QoS_profile,m3GPP_IMEISV,m3GPP_IMSI,m3GPP_IMSI_MCC_MNC,m3GPP_IPv6_DNS_Servers,m3GPP_MS_Timezone,m3GPP_Negotiated_DSCP,m3GPP_NSAPI,m3GPP_Packet_Filter,m3GPP_PDP_Type,m3GPP_RAT_Type,m3GPP_Selection_Mode,m3GPP_Session_Stop_Indicator,m3GPP_SGSN_Address,m3GPP_SGSN_MCC_MNC,m3GPP_Teardown_Indicator,m3GPP_User_Location_Info) VALUES('0A0CE16405FF5CB02B8F62C5FF001F15A7','229070a5885df65e944e18f144f05ffb','0A0CE16405FF5CB02B8F62C5FF001F15A7','Active','START-A','','awspreprod.movedata.mobi',NULL,NULLIF('', ''),'10.128.133.49',NULLIF('',''),'Wireless-Other','PUNE','Local','0',TO_TIMESTAMP(1555049359),TO_TIMESTAMP(1555049359),NULL,0,0,0,0,0,0,NULL,'',NULL,'','awspreprod.movedata.mobi','1234576977',NULLIF('','')::inet,'',NULLIF('34.202.104.68','')::inet,NULLIF('','')::inet,'GPRS-PDP-Context','Framed-User','','0800','10.12.225.100','34647696','10.12.225.100','111222','99-0911012901FEFE11050101','','333100001809088','33310',NULL,'','26','5','','0','UTRAN','0',NULL,'10.10.53.100','111222',NULL,'0x0011212200210004')
(190) sql: EXPAND /var/log/radius/accounting.sql
(190) sql:    --> /var/log/radius/accounting.sql
(190) sql: Executing query: INSERT INTO radacct (AcctSessionId,AcctUniqueId,AcctMultiSessionId,SessionState,StartEvent,StopEvent,UserName,GroupName,Realm,NASIPAddress,NASPortId,NASPortType,NASIdentifier,AcctAuthentic,AcctDelayTime,AcctStartTime,AcctUpdateTime,AcctStopTime,AcctInterval,AcctSessionTime,AcctInputOctets,AcctOutputOctets,AcctInputPackets,AcctOutputPackets,AcctTerminateCause,ConnectInfo_start,ConnectInfo_stop,Class,CalledStationId,CallingStationId,DelegatedIPv6Prefix,FramedInterfaceId,FramedIPaddress,FramedIPv6address,FramedProtocol,ServiceType,m3GPP_Camel_Charging_Info,m3GPP_Charging_Characteristics,m3GPP_Charging_Gateway_Address,m3GPP_Charging_Id,m3GPP_GGSN_Address,m3GPP_GGSN_MCC_MNC,m3GPP_GPRS_Negotiated_QoS_profile,m3GPP_IMEISV,m3GPP_IMSI,m3GPP_IMSI_MCC_MNC,m3GPP_IPv6_DNS_Servers,m3GPP_MS_Timezone,m3GPP_Negotiated_DSCP,m3GPP_NSAPI,m3GPP_Packet_Filter,m3GPP_PDP_Type,m3GPP_RAT_Type,m3GPP_Selection_Mode,m3GPP_Session_Stop_Indicator,m3GPP_SGSN_Address,m3GPP_SGSN_MCC_MNC,m3GPP_Teardown_Indicator,m3GPP_User_Location_Info) VALUES('0A0CE16405FF5CB02B8F62C5FF001F15A7','229070a5885df65e944e18f144f05ffb','0A0CE16405FF5CB02B8F62C5FF001F15A7','Active','START-A','','awspreprod.movedata.mobi',NULL,NULLIF('', ''),'10.128.133.49',NULLIF('',''),'Wireless-Other','PUNE','Local','0',TO_TIMESTAMP(1555049359),TO_TIMESTAMP(1555049359),NULL,0,0,0,0,0,0,NULL,'',NULL,'','awspreprod.movedata.mobi','1234576977',NULLIF('','')::inet,'',NULLIF('34.202.104.68','')::inet,NULLIF('','')::inet,'GPRS-PDP-Context','Framed-User','','0800','10.12.225.100','34647696','10.12.225.100','111222','99-0911012901FEFE11050101','','333100001809088','33310',NULL,'','26','5','','0','UTRAN','0',NULL,'10.10.53.100','111222',NULL,'0x0011212200210004')
alandekok commented 5 years ago

The database switchover duration is ~30sec, during this time database is not available

If you configure FreeRADIUS to use a database, then it tries to use the database. Taking the database down means that FreeRADIUS will have problems.

The PostgreSQL client APIs try to connect to the database, and block if the database is unavailable. This is due to underlying API / network issues.

i.e. if the database is unreachable, then the underlying TCP connections will quickly fail, and FreeRADIUS will not block. If the TCP connections are up but are unresponsive, then the PostgreSQL client library will block waiting for more data, and FreeRADIUS will block waiting for the PostgreSQL client library to return.

If you want FreeRADIUS to use a database, make sure that the database is responsive.

FreeRADIUS includes it's own failover mechanism, precisely because of problems like this. Most failover systems are broken and terrible. Whereas the FreeRADIUS failover system is under our own control.

The solution here is to rely on the FreeRADIUS failover mechanism. Don't play magic network games.