codeout / junoser

PEG parser for JUNOS configuration.
MIT License
69 stars 11 forks source link

Invalid syntax #24

Closed XioNoX closed 3 years ago

XioNoX commented 3 years ago

Hi, thanks for this tool, it seems like the perfect fit for our needs, and I'm still trying to understand how it works.

I followed the instruction to retrieve the schema with Netconf on a MX204 running Junos 18.2R3 and generated a new "tmp/rule.rb" file (and junoser) using bundle exec rake build:config build:rule.

However I'm then getting a few "Invalid syntax" on a config that loads fine on that router model:

Invalid syntax:  set interfaces interface-range external apply-groups-except vrrp
Invalid syntax:  set interfaces et-0/0/1 unit 501 apply-groups-except vrrp
Invalid syntax:  set interfaces et-0/0/1 unit 501 family inet rpf-check apply-groups-except external-links
Invalid syntax:  set interfaces et-0/0/1 unit 501 family inet6 rpf-check apply-groups-except external-links
Invalid syntax:  set forwarding-options rpf-loose-mode-discard family inet6
Invalid syntax:  set policy-options policy-statement BGP_Customer_out term aggregates6 from protocol ospf3
Invalid syntax:  set policy-options policy-statement BGP_aggregate_contributors term internal_only from protocol ospf3

So I'm wondering if I did something wrong or how to clear those errors?

As I understand it, a specific tmp/rule.rb and then "build:rule" is needed at least per platform type (eg. SRX vs. MX vs. QFX). Going forward, what are your recommendations for using this tool on a fleet of various Juniper configurations?

Thanks a lot!

codeout commented 3 years ago

Hi, thanks for your report!

That's a good point. We have some problems as you may know:

  1. There is platform-specific syntax in .xsd
  2. There is useful syntax that is not defined in .xsd
    • eg: apply-groups-except

So, (1) I did merge rules of various Juniper platforms manually on a per-request basis, and (2) manually implemented missing syntax.

Let me take a look into syntax errors by the way, some of them might be caused by junoser bug :(

codeout commented 3 years ago

Hi,

I've pushed fixes to this repo. Will publish it as a gem in a week.

codeout commented 3 years ago

v0.4.1 has been just published with the fixes.

XioNoX commented 3 years ago

Thanks! Following up on that thread.

The published GEM doesn't work out of the box with my MX204 configuration, but using the extracted .XSD works fine! No errors!

Using that same XSD on a EX4600 (14.1X53) config only returns the following error: Invalid syntax: set virtual-chassis vcp-snmp-statistics Which makes sens as a MX204 doesn't have VCP. However the same error is present with the provided junos-18.1R3-S9.rb.

What's surprising though, is that using the XSD extracted from that EX4600 on its own config returns a lot of errors on what seems to be basic features: For example (let me know if you need the full list):

Invalid syntax:  set interfaces xe-1/0/6 description "lvs4005 {#1044}"
Invalid syntax:  set system syslog host syslog.anycast.wmnet any info
Invalid syntax:  set interfaces xe-1/0/7 mtu 9192
Invalid syntax:  set interfaces xe-2/0/6 unit 0 family ethernet-switching interface-mode trunk
Invalid syntax:  set protocols lldp interface ge-1/0/13 disable
Invalid syntax:  set vlans private1-ulsfo forwarding-options dhcp-security option-82 circuit-id prefix host-name

Similarly using a XSD extracted from a SRX1500 (running 15.1) returns a lot of SRX related errors:

Invalid syntax:  set security policies global policy ntp_out match source-address foo
Invalid syntax:  set security policies global policy ntp_out match source-address bar
Invalid syntax:  set security policies from-zone foo to-zone bar policy XXXX

Using the provided vsrx-18.3R1.9.rb on that same SRX returns much less errors (full list):

Invalid syntax:  set groups node0 system backup-router 10.193.0.1 destination xxx/32
Invalid syntax:  set groups node1 system backup-router 10.193.0.1 destination xxx/32
Invalid syntax:  set system dump-on-panic
Invalid syntax:  set services ssl initiation profile syslog-tls-profile protocol-version tls12
Invalid syntax:  set security log stream syslog-tls-stream host port 6514
Invalid syntax:  set security log stream syslog-tls-stream-eqiad host port 6514
Invalid syntax:  set security ike proposal foo dh-group bar
Invalid syntax:  set security ipsec policy ipsec-policy-eqiad perfect-forward-secrecy keys foo
Invalid syntax:  set security address-book global address pypi.python.org dns-name pypi.python.org
Invalid syntax:  set security nat static rule-set static-nat rule foo match destination-address xxx/32
Invalid syntax:  set security nat source rule-set foo-nat rule foo match source-address-name bar
Invalid syntax:  set security nat source rule-set foo-nat rule foo match destination-address-name bar

Similarly, on a SRX300, only those are present: Invalid syntax: set applications application idrac-app1 term t1 protocol tcp destination-port 5900

As you can see I'm trying to see if we can get no errors what so ever on all of our Juniper devices :)

Thanks!

codeout commented 3 years ago

Thanks for your report! It's likely that Juniper's .XSD varies per platform as you mentioned, and unfortunately, it doesn't cover all configurations of the corresponding platform for some reason. We can't trust .XSD coming from EX for example.

Each .XSD is very huge and complicated to merge programmatically, but It's good for us if a single gem can cover all syntax we are using. All we can do is merge manually per-request basis.

I'll update mx-19.3R3-S1.3.rb to support your EX and SRX.

codeout commented 3 years ago

Hi @XioNoX,

I've pushed fixes to origin/master. If you have time, could you test again whether it supports the syntax configured on your devices? If it does, I'll publish a new version of the gem.

# configs that you reported here are all passed

junoser $ ruby -Ilib exe/junoser -c <<EOS
set interfaces xe-1/0/6 description "lvs4005 {#1044}"
set system syslog host syslog.anycast.wmnet any info
set interfaces xe-1/0/7 mtu 9192
set interfaces xe-2/0/6 unit 0 family ethernet-switching interface-mode trunk
set protocols lldp interface ge-1/0/13 disable
set vlans private1-ulsfo forwarding-options dhcp-security option-82 circuit-id prefix host-name

set security policies global policy ntp_out match source-address foo
set security policies global policy ntp_out match source-address bar
set security policies from-zone foo to-zone bar policy XXXX

set groups node0 system backup-router 10.193.0.1 destination xxx/32
set groups node1 system backup-router 10.193.0.1 destination xxx/32
set system dump-on-panic
set services ssl initiation profile syslog-tls-profile protocol-version tls12
set security log stream syslog-tls-stream host port 6514
set security log stream syslog-tls-stream-eqiad host port 6514
set security ike proposal foo dh-group group24
set security ipsec policy ipsec-policy-eqiad perfect-forward-secrecy keys group24
set security address-book global address pypi.python.org dns-name pypi.python.org
set security nat static rule-set static-nat rule foo match destination-address xxx/32
set security nat source rule-set foo-nat rule foo match source-address-name bar
set security nat source rule-set foo-nat rule foo match destination-address-name bar

set applications application idrac-app1 term t1 protocol tcp destination-port 5900
EOS
codeout commented 3 years ago

v0.4.2 has been published with the fix.