f5 user config change tracking

[etfeet]

HI,

what would it take to implement user config checking for F5 LTM's (TMOS)?

It would be awsome if we could do the following items for F5's as well. syslog udp+file example to catch config change event (ios/junos) and trigger config fetch will signal ios/junos user who made change, which output modules can use (via POST) The git output module uses this info - 'git blame' will for each line show who made the change and when

[ytti]

Hey,

I'm not sure what you mean by 'user config checking', I've never seen TMOS device myself. Perhaps @mikebryant or @danilopopeye knows.

When it comes to syslog udp+file, it should be relatively easy to add devices there. If you can show me syslog entry file LTM sends, I can give it a go.

[etfeet]

For user config tracking i mean:

User Fred changes manage IP to 4.4.4.4

Oxidized runs and tracks the changes and stores that Fred made the change in metadata,etc like it does with Junos/iOS. On Apr 8, 2016 2:09 AM, "ytti" notifications@github.com wrote:

Hey,

I'm not sure what you mean by 'user config checking', I've never seen TMOS device myself. Perhaps @mikebryant https://github.com/mikebryant or @danilopopeye https://github.com/danilopopeye knows.

When it comes to syslog udp+file, it should be relatively easy to add devices there. If you can show me syslog entry file LTM sends, I can give it a go.

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/ytti/oxidized/issues/381#issuecomment-207339851

[etfeet]

How exactly do you setup the following features that are on the wiki? It mentions them but it doesn't explain whats needed to configure them.

syslog udp+file example to catch config change event (ios/junos) and trigger config fetch
will signal ios/junos user who made change, which output modules can use (via POST)

[ytti]

There is example code, but likely some environment specific tuning is needed:https://github.com/ytti/oxidized/blob/master/extra/syslog.rb

[etfeet]

instead of having oxidized act as a syslog collector, would it be possible to have it watch a directory so that multiple applications can access the syslog messages?

I'm currently using syslog-ng to collect messages and sending them to librenms (run each message against a php script), preferably I could have syslog-ng send the messages to a shell script that pushes them to oxidized, or have oxidized watch a directory for messages.

[etfeet]

here are the syslog entries after changing the description on a VS to 1234

Apr 8 17:19:20 172.16.24.4 mcpd[5095]: 01070417:5: AUDIT - client tmui, user admin - transaction #17302664-3 - object 0 - modify { virtual_server { virtual_server_name "/common/redir_test" virtual_server_description "1234" virtual_server_enabled 1 virtual_server_conn_limit 0 virtual_server_eviction_policy "" virtual_server_rate_limit 0 virtual_server_rate_limit_mode 0 virtual_server_rclass "" virtual_server_bwcclass "" virtual_server_sf_flags 0 virtual_server_translate_addr 1 virtual_server_translate_port 1 virtual_server_service_policy "" virtual_server_nat64 0 virtual_server_srcport 0 virtual_server_auto_lasthop 0 virtual_server_type 0 virtual_server_source_address_translation_type 3 virtual_server_source_address_translation_pool "" virtual_server_lasthop_pool_name "" virtual_server_gtm_score 0 virtual_server_update_status 1 virtual_server_src_addr 0.0.0.0 virtual_server_addr 10.1.1.30 virtual_server_contribute_to_va_status 1 virtual_server_action_on_service_down 0 virtual_server_va_name "/common/10.1.1.30" virtual_server_wildmask 255.255.255.255 virtual_server_port http virtual_server_ip_proto 6 virtual_server_listed_enabled_vlans 0 } } [Status=Command OK] Apr 8 17:19:20 172.16.24.4 mcpd[5095]: 01070417:5: AUDIT - client tmui, user admin - transaction #17302664-4 - object 0 - obj_delete { virtual_server_profile { virtual_server_profile_vs_name "/common/redir_test" virtual_server_profile_profile_name "/Common/apache" virtual_server_profile_profile_type 1 } } [Status=Command OK] Apr 8 17:19:20 172.16.24.4 mcpd[5095]: 01070417:5: AUDIT - client tmui, user admin - transaction #17302664-5 - object 0 - obj_delete { virtual_server_profile { virtual_server_profile_vs_name "/common/redir_test" virtual_server_profile_profile_name "/Common/tcp" virtual_server_profile_profile_type 5 } } [Status=Command OK] Apr 8 17:19:20 172.16.24.4 mcpd[5095]: 01070417:5: AUDIT - client tmui, user admin - transaction #17302664-6 - object 0 - create { virtual_server_profile { virtual_server_profile_vs_name "/common/redir_test" virtual_server_profile_profile_name "/Common/apache" virtual_server_profile_profile_type 1 virtual_server_profile_profile_context 0 } } [Status=Command OK] Apr 8 17:19:20 172.16.24.4 mcpd[5095]: 01070417:5: AUDIT - client tmui, user admin - transaction #17302664-7 - object 0 - create { virtual_server_profile { virtual_server_profile_vs_name "/common/redir_test" virtual_server_profile_profile_name "/Common/tcp" virtual_server_profile_profile_type 5 virtual_server_profile_profile_context 0 } } [Status=Command OK]

[etfeet]

here is what it looks like if i do a config change from the cli on the LTM

Apr 8 17:28:09 172.16.24.4 mcpd[5095]: 01070417:5: AUDIT - client tmsh, tmsh-pid-6374, user admin - transaction #17314404-2 - object 0 - modify { ltcfg_instance { ltcfg_instance_container "" ltcfg_instance_name "/Common/syslog" ltcfg_instance_class_name "syslog" ltcfg_instance_instance_folder_name "/Common" ltcfg_instance_instance_leaf_name "syslog" ltcfg_instance_config_source 0 } } [Status=Command OK] Apr 8 17:28:09 172.16.24.4 mcpd[5095]: 01070417:5: AUDIT - client tmsh, tmsh-pid-6374, user admin - transaction #17314404-3 - object 0 - create_if { ltcfg_instance_field { ltcfg_instance_field_instance_name "/Common/syslog" ltcfg_instance_field_field_name "include" ltcfg_instance_field_class_name "syslog" ltcfg_instance_field_container "" ltcfg_instance_field_value "destination remote_server {udp(10.0.6.30 port (514));};filter f_alllogs {level (debug...emerg);};log {source(local);filter(f_alllogs);destination(remote_server);};" ltcfg_instance_field_userspec 1 ltcfg_instance_field_config_source 0 } } [Status=Command OK] Apr 8 17:28:09 172.16.24.4 tmsh[6374]: 01420002:5: AUDIT - pid=6374 user=admin folder=/Common module=(tmos)# status=[Command OK] cmd_data=modify /sys syslog include "destination remote_server {udp(10.0.6.30 port (514));};filter f_alllogs {level (debug...emerg);};log {source(local);filter(f_alllogs);destination(remote_server);};"

[ytti]

I'm sure you realise answer to your question must be 'yes'. Yes it's possible to create program which reads files and performs action (http post) conditionally based on what it read?

The example code has some efforts towards reading local file, but it's not complete. I do not currently have cycles for this myself, sorry. But I am happy to accept continuation on the example for reading files and triggering HTTP POST.

[etfeet]

How do you configure oxidized to start using syslog as a source with the above sample code? I've got some time I can spend to work on the filtering for catching change notifations from an f5 via syslog.

[ytti]

I'm not sure if this helps. But the general idea is that you run the script, the script either receives syslog (or reads file, not implemented completely) and upon finding appropriate string, it fires HTTP POST request to Oxidized, asking Oxidized to fetch config for the given box, with given commit message and commit author.

[etfeet]

What is the http post request that it sends to oxidized?

OK, I think i got the parsing figured out for the f5's. However, I'm using logstash to do it. How would I manually do an HTTP POST to oxidized to tell oxidized about the config change?

logstash filters and splits the data into fields, a change event looks like this after parsing. However, I need to know how to format the data for doing an HTTP POST.

syslog message:

Apr 14 17:14:02 bigip-ltm-a mcpd[5095]: 01070417:5: AUDIT - client tmui, user admin - transaction #62249766-3 - object 0 - modify { virtual_server { virtual_server_name "/common/redir_test" virtual_server_description "test123" virtual_server_enabled 1 virtual_server_conn_limit 0 virtual_server_eviction_policy "" virtual_server_rate_limit 0 virtual_server_rate_limit_mode 0 virtual_server_rclass "" virtual_server_bwcclass "" virtual_server_sf_flags 0 virtual_server_translate_addr 1 virtual_server_translate_port 1 virtual_server_service_policy "" virtual_server_nat64 0 virtual_server_srcport 0 virtual_server_auto_lasthop 0 virtual_server_type 0 virtual_server_source_address_translation_type 3 virtual_server_source_address_translation_pool "" virtual_server_lasthop_pool_name "" virtual_server_gtm_score 0 virtual_server_update_status 1 virtual_server_src_addr 0.0.0.0 virtual_server_addr 192.168.1.50 virtual_server_contribute_to_va_status 1 virtual_server_action_on_service_down 0 virtual_server_va_name "/common/192.168.1.50" virtual_server_wildmask 255.255.255.255 virtual_server_port http virtual_server_ip_proto 6 virtual_server_listed_enabled_vlans 0 } } [Status=Command OK]

after logstash parsing:

{
                 "message" => "Apr 14 17:14:02 bigip-ltm-a mcpd[5095]: 01070417:5: AUDIT - client tmui, user admin - transaction #62249766-3 - object 0 - modify { virtual_server { virtual_server_name \"/common/redir_test\" virtual_server_description \"test123\" virtual_server_enabled 1 virtual_server_conn_limit 0 virtual_server_eviction_policy \"\" virtual_server_rate_limit 0 virtual_server_rate_limit_mode 0 virtual_server_rclass \"\" virtual_server_bwcclass \"\" virtual_server_sf_flags 0 virtual_server_translate_addr 1 virtual_server_translate_port 1 virtual_server_service_policy \"\" virtual_server_nat64 0 virtual_server_srcport 0 virtual_server_auto_lasthop 0 virtual_server_type 0 virtual_server_source_address_translation_type 3 virtual_server_source_address_translation_pool \"\" virtual_server_lasthop_pool_name \"\" virtual_server_gtm_score 0 virtual_server_update_status 1 virtual_server_src_addr 0.0.0.0 virtual_server_addr 192.168.1.50 virtual_server_contribute_to_va_status 1 virtual_server_action_on_service_down 0 virtual_server_va_name \"/common/192.168.1.50\" virtual_server_wildmask 255.255.255.255 virtual_server_port http virtual_server_ip_proto 6 virtual_server_listed_enabled_vlans 0 } } [Status=Command OK]",
                "@version" => "1",
              "@timestamp" => "2016-04-15T00:14:02.000Z",
                    "path" => "/var/log/net/hosts/bigip-ltm-a.raw",
                    "host" => "ubuntu-librenms",
                    "type" => "syslog",
                    "tags" => [
        [0] "hosts-raw",
        [1] "bigip",
        [2] "f5-change"
    ],
        "syslog_timestamp" => "Apr 14 17:14:02",
         "syslog_hostname" => "bigip-ltm-a",
          "syslog_program" => "mcpd",
              "syslog_pid" => "5095",
          "syslog_message" => "01070417:5: AUDIT - client tmui, user admin - transaction #62249766-3 - object 0 - modify { virtual_server { virtual_server_name /common/redir_test virtual_server_description test123 virtual_server_enabled 1 virtual_server_conn_limit 0 virtual_server_eviction_policy  virtual_server_rate_limit 0 virtual_server_rate_limit_mode 0 virtual_server_rclass  virtual_server_bwcclass  virtual_server_sf_flags 0 virtual_server_translate_addr 1 virtual_server_translate_port 1 virtual_server_service_policy  virtual_server_nat64 0 virtual_server_srcport 0 virtual_server_auto_lasthop 0 virtual_server_type 0 virtual_server_source_address_translation_type 3 virtual_server_source_address_translation_pool  virtual_server_lasthop_pool_name  virtual_server_gtm_score 0 virtual_server_update_status 1 virtual_server_src_addr 0.0.0.0 virtual_server_addr 192.168.1.50 virtual_server_contribute_to_va_status 1 virtual_server_action_on_service_down 0 virtual_server_va_name /common/192.168.1.50 virtual_server_wildmask 255.255.255.255 virtual_server_port http virtual_server_ip_proto 6 virtual_server_listed_enabled_vlans 0 } } [Status=Command OK]",
             "received_at" => "2016-04-15T00:14:02.647Z",
           "received_from" => "ubuntu-librenms",
    "syslog_severity_code" => 5,
    "syslog_facility_code" => 1,
         "syslog_facility" => "user-level",
         "syslog_severity" => "notice",
      "syslog_fingerprint" => "1c710bad1e840775c5d60f84bb23e27de880c0c8",
                  "client" => "tmui",
                "username" => "admin",
          "transaction-id" => "62249766-3",
               "object-id" => "0",
              "event-type" => "modify",
             "change-data" => "{ virtual_server { virtual_server_name /common/redir_test virtual_server_description test123 virtual_server_enabled 1 virtual_server_conn_limit 0 virtual_server_eviction_policy  virtual_server_rate_limit 0 virtual_server_rate_limit_mode 0 virtual_server_rclass  virtual_server_bwcclass  virtual_server_sf_flags 0 virtual_server_translate_addr 1 virtual_server_translate_port 1 virtual_server_service_policy  virtual_server_nat64 0 virtual_server_srcport 0 virtual_server_auto_lasthop 0 virtual_server_type 0 virtual_server_source_address_translation_type 3 virtual_server_source_address_translation_pool  virtual_server_lasthop_pool_name  virtual_server_gtm_score 0 virtual_server_update_status 1 virtual_server_src_addr 0.0.0.0 virtual_server_addr 192.168.1.50 virtual_server_contribute_to_va_status 1 virtual_server_action_on_service_down 0 virtual_server_va_name /common/192.168.1.50 virtual_server_wildmask 255.255.255.255 virtual_server_port http virtual_server_ip_proto 6 virtual_server_listed_enabled_vlans 0 } }"
}

[ytti]

What do you mean manually? Like with 'curl'? You could do HTTP POST in shell script with curl to ask oxidized to fetch config or given box, you could attach in te POST request committer name and commit reason, and for example 'git' output would use these.

[etfeet]

yes. how do you do the http post with curl?

[ytti]

It's /next then you can give options about committer, commit message.

Examples here: https://github.com/ytti/oxidized/blob/master/extra/rest_client.rb#L19 and https://github.com/ytti/oxidized/blob/master/extra/syslog.rb#L89

[ytti]

I assume closed due to inactivity, please reopen if needed.