Cannot build terraform provider for junos-es-conf-logical-systems@2019-01-01.yang

[clintong]

I managed to build provider for all modules like interfaces, protocols and security however the logical systems is failing without error. Although the yin file is being generated, the xpath.txt is generated as an empty file. Can you kindly provide feedback.

[aburston]

Can you please provide more information on this.

For example, what did you do, step by step.

Ideally point at a repo where you have your makefile or shell script that does the build process, so I can reproduce your problem.

Once I have that I will be able to give you feedback or get someone to look at it.

[clintong]

Hi the below is the procedure I followed:

clone repositories

cd PATH git clone https://github.com/Juniper/junos-terraform.git git clone https://github.com/Juniper/yang.git

copy files to temp directory:

mkdir PATH/tmp cp yang/20.4/20.4R1/junos-es/conf/junos-es-conf-root@2019-01-01.yang PATH/tmp cp yang/20.4/20.4R1/common/junos-common-types@2019-01-01.yang PATH/tmp cp yang/20.4/20.4R1/junos-es/conf/junos-es-conf-logical-systems@2019-01-01.yang PATH/tmp

create config.toml file

vi config.toml

yangDir = "<PATH>/tmp"
providerDir = "<PATH>/terraform_providers"
xpathPath = "<PATH>/xpath_custom.xml"
providerName = "vsrx"
fileType = "both"

Generate YIN and xpath files

cd PATH/junos-terraform/cmd/processYang go build ./processYang -config PATH/config.toml

Output

                 ___ _____ ___  ______
                   |_  |_   _/ _ \ |  ___|
                     | | | |/ /_\ \| |_
                     | | | ||  _  ||  _|
                 /\__/ / | || | | || |
                 \____/  \_/\_| |_/\_|
                           0.1.5
--------------------------------------------------------------------------------------------------------------------------
- Creating Yin files from Yang file directory: <PATH>/tmp
--------------------------------------------------------------------------------------------------------------------------
Yin file for junos-common-types@2019-01-01 is generated
Yin file for junos-es-conf-logical-systems@2019-01-01 is generated
Yin file for junos-es-conf-root@2019-01-01 is generated
--------------------------------------------
- Creating _xpath files from the Yin files -
--------------------------------------------
Creating Xpath file: junos-common-types@2019-01-01_xpath.txt
Creating Xpath file: junos-common-types@2019-01-01_xpath.xml
Creating Xpath file: junos-es-conf-logical-systems@2019-01-01_xpath.txt
Creating Xpath file: junos-es-conf-logical-systems@2019-01-01_xpath.xml
Creating Xpath file: junos-es-conf-root@2019-01-01_xpath.txt
Creating Xpath file: junos-es-conf-root@2019-01-01_xpath.xml

Problem

cat junos-es-conf-logical-systems@2019-01-01_xpath.txt cat junos-es-conf-logical-systems@2019-01-01_xpath.xml

both files are empty

I also did the above procedure for junos-es-conf-interfaces@2019-01-01.yang and it works perfectly

[vinpatel24]

Hi, I have replicated this bug and it seems the issue is surrounding the junos-es-conf-logical-systems@2019-01-01.yang file being such a large file (over 130,000 lines) causing the 'pyang' command to be "killed" potentially resulting in no output. Below I have run the command manually to show the process being killed in real-time. In order to work around this issue, try increasing the memory on the device running the command for this file.

''' pyang -f yin junos-es-conf-logical-systems@2019-01-01.yang -o junos-es-conf-logical-systems@2019-01-01.yin -p /workspaces/junos-terraform/yang_files/ & [1] 14489

(venv) @vinpatel24 ➜ /workspaces/junos-terraform/yang_files (issue8) $ pmap 14489 | grep total total 1544920K (venv) @vinpatel24 ➜ /workspaces/junos-terraform/yang_files (issue8) $ pmap 14489 | grep total total 2002648K (venv) @vinpatel24 ➜ /workspaces/junos-terraform/yang_files (issue8) $ pmap 14489 | grep total total 2146008K (venv) @vinpatel24 ➜ /workspaces/junos-terraform/yang_files (issue8) $ pmap 14489 | grep total total 2293776K (venv) @vinpatel24 ➜ /workspaces/junos-terraform/yang_files (issue8) $ pmap 14489 | grep total total 2349072K (venv) @vinpatel24 ➜ /workspaces/junos-terraform/yang_files (issue8) $ pmap 14489 | grep total total 2389008K (venv) @vinpatel24 ➜ /workspaces/junos-terraform/yang_files (issue8) $ pmap 14489 | grep total total 2389008K (venv) @vinpatel24 ➜ /workspaces/junos-terraform/yang_files (issue8) $ pmap 14489 | grep total total 2389008K (venv) @vinpatel24 ➜ /workspaces/junos-terraform/yang_files (issue8) $ pmap 14489 | grep total total 2397200K (venv) @vinpatel24 ➜ /workspaces/junos-terraform/yang_files (issue8) $ pmap 14489 | grep total total 2425872K (venv) @vinpatel24 ➜ /workspaces/junos-terraform/yang_files (issue8) $ pmap 14489 | grep total total 2468880K (venv) @vinpatel24 ➜ /workspaces/junos-terraform/yang_files (issue8) $ pmap 14489 | grep total total 2518032K (venv) @vinpatel24 ➜ /workspaces/junos-terraform/yang_files (issue8) $ pmap 14489 | grep total [1]+ Killed pyang -f yin junos-es-conf-logical-systems@2019-01-01.yang -o junos-es-conf-logical-systems@2019-01-01.yin -p /workspaces/junos-terraform/yang_files/ '''

[vinpatel24]

Hi @clintong,

In addition to a possible memory space error mentioned earlier, another reason the xpath.xml and xpath.txt were not compiling were due to bugs in the Internal code which have been resolved in the following repo (https://github.com/vinpatel24/junos-terraform). Instead of forking/ cloning the juniper repo, fork/ clone this repo and run the commands in the README.md to complete the necessary output for ALL yang files. This has been tested for vSRX and vMX devices. Running the processYang files may take a while (~40 min) to run after building, but it will complete the way it is supposed to.

Going further, I want to make sure that the logical-systems provider you are trying to build is for the correct device. If you are trying to building a logical systems provider for a vSRX, it will fail to compile during the terraform apply stage because logical systems is NOT configurable for a vSRX. You can check this by accessing a vSRX device in configuration mode and running the command > show logical-systems which will fail to compile in the CLI meaning it is not a configurable. Hence, I have tested the logical-systems provider with a vMX device and was able to create, build, and configure a logical-systems providers on this device.

[clintong]

Hi @vinpatel24

Thanks for your support, really appreciate it. I have cloned your repo and following your code changes now the provider for logical system was correctly built. My intention is to use this to configure logical systems on physical firewalls. In order to help me use this module can you please provide me any documentation or example how to write a simple security policy from zone "trust" to zone "untrust" in logical system "foo" with application "bar".

[vinpatel24]

Hi @clintong,

Hopefully by next Monday or Tuesday we will be merging a pull request made by a Juniper Sales Engineer to the Juniper/junos-terraform MASTER branch which provides you with an example of security policy involving zones (trust and untrust). This can be found in the Samples directory in the "secutiy_ipsec", security_policy", and "security_routing_instance" folders.

In addition, I have now merged the changes from the vinpatel24/junos-terraform MASTER branch (the repo you forked to get logical systems to work) to the junos-terraform MASTER branch (to the official Juniper repo)

What this means is that the Juniper/junos-terraform MASTER branch is now updated with code that will process logical-systems along with all other files. This updated code also includes another merge request where the junos_helpers functionality from the /go-netconf repo is directly implemented into the project to reduce the number of NETCONF calls.

I will now be closing this issue as resolved. If any other questions arise please create a new issue. Thanks.

[aburston]

Works nows, closing