Issue with generate single Yin files from Yang file

[AndriiPysmenko]

Hi,

Is any known issue why it's not possible to generate single Yin files from Yang in the last version of that repo? Any plans to add some log for future troubleshooting to determine what exactly cause the problem?

./processYang -config ../../../jtaf/config.toml

                ___ _____ ___  ______ 
               |_  |_   _/ _ \ |  ___|
                 | | | |/ /_\ \| |_   
                 | | | ||  _  ||  _|  
             /\__/ / | || | | || |    
             \____/  \_/\_| |_/\_| 
                       0.1.5

---

• Creating Yin files from Yang file directory: /home/terraform/jtaf/yang -

  Yin file for junos-common-ddl-extensions@2019-01-01 is generated Yin file for junos-common-odl-extensions@2019-01-01 is generated Yin file for junos-common-types@2019-01-01 is generated error processing file: junos-es-conf-security@2019-01-01 output from pyang:
  panic: pyang error: exit status 1

goroutine 1 [running]: github.com/Juniper/junos-terraform/Internal/processYang.generateYinFile(0xc0000c8040, 0x19) /home/terraform/junos-terraform/Internal/processYang/processYang.go:173 +0x565 github.com/Juniper/junos-terraform/Internal/processYang.CreateYinFileAndXpath(0xc0000c8040, 0x19, 0xc0000a0060, 0x28, 0xc0000a0090, 0x1e, 0xc0000a4078, 0x4, 0x0, 0x0, ...) /home/terraform/junos-terraform/Internal/processYang/processYang.go:83 +0xa8 main.main() /home/terraform/junos-terraform/cmd/processYang/main.go:82 +0x498

[cruse1977]

in the code, the args for pyang are: similar to: pyang -f yin yangfile.yang -o yangfile.yin -p

I ended up running this manually to work out what was missing (my issue was - I also had to copy common yangs into the project dir which was the cause of pyang bombing out)

[JerradGit]

There is an outstanding pull request to fix the docs and make this more clear, as the poster above noted I prefer to check manually with pyang what isn't working as the errors are more clear. The golang code which calls pyang should probably have some better error checking instead of the panic it produces.

These dependant YANG files are referenced in the primary YANG files, for example the interfaces yang file junos-qfx-conf-interfaces@2019-01-01.yang has this at the top

module junos-qfx-conf-interfaces {
   namespace "http://yang.juniper.net/junos-qfx/conf/interfaces";

   prefix jc-interfaces;

   import junos-common-ddl-extensions {
     prefix junos;
     revision-date 2019-01-01;
   }

   import junos-common-types {
     prefix jt;
     revision-date 2019-01-01;
   }

   import junos-qfx-conf-root {
     prefix jc;
     revision-date 2019-01-01;
   }

So it's looking to import these other YANG files, which in the case of the Juniper YANG repo is located in the common folder e.g. common and the junos-qfx-conf-root is in the main folder with all the others conf-with-extensions.

So simply copy what is in common into the same folder as whatever other YANG models you are using. e.g

(.venv) ➜  fakebadsetup (master) ls -l                                                                                                  ✱
total 2768
-rw-r--r--  1 jerrad  staff  1413792  4 Feb 11:28 junos-qfx-conf-interfaces@2019-01-01.yang
(.venv) ➜  fakebadsetup (master) pyang *.yang                                                                                           ✱
junos-qfx-conf-interfaces@2019-01-01.yang:10: error: module "junos-common-ddl-extensions" not found in search path
junos-qfx-conf-interfaces@2019-01-01.yang:15: error: module "junos-common-types" not found in search path
junos-qfx-conf-interfaces@2019-01-01.yang:20: error: module "junos-qfx-conf-root" not found in search path

pyang makes it clear these modules are missing, so I copy them into this folder as well as junos-qfx-conf-root one

(.venv) ➜  fakebadsetup (master) ls -lt                                                                                                 ✱
total 2808
-rw-r--r--  1 jerrad  staff     6508  4 Feb 11:33 junos-qfx-conf-root@2019-01-01.yang
-rw-r--r--  1 jerrad  staff     3398  4 Feb 11:31 junos-common-types@2019-01-01.yang
-rw-r--r--  1 jerrad  staff     2346  4 Feb 11:31 junos-common-odl-extensions@2019-01-01.yang
-rw-r--r--  1 jerrad  staff     1806  4 Feb 11:31 junos-common-ddl-extensions@2019-01-01.yang
-rw-r--r--  1 jerrad  staff  1413792  4 Feb 11:28 junos-qfx-conf-interfaces@2019-01-01.yang

(.venv) ➜  fakebadsetup (master) pyang *.yang                                                                                           ✱
junos-qfx-conf-root@2019-01-01.yang:15: warning: imported module "junos-common-types" not used

Not used is fine in this case, it's not critical

Now that pyang is happy, I then do the processYang

(.venv) ➜  junos-terraform (master) ./processYang -config jerrad_test/config.toml                                                       ✱

                    ___ _____ ___  ______
                   |_  |_   _/ _ \ |  ___|
                     | | | |/ /_\ \| |_
                     | | | ||  _  ||  _|
                 /\__/ / | || | | || |
                 \____/  \_/\_| |_/\_|
                           0.1.5
--------------------------------------------------------------------------------------------------------------------------
- Creating Yin files from Yang file directory: /Users/jerrad/scratch/junos-terraform/jerrad_test/yang_files/fakebadsetup -
--------------------------------------------------------------------------------------------------------------------------
Yin file for junos-common-ddl-extensions@2019-01-01 is generated
Yin file for junos-common-odl-extensions@2019-01-01 is generated
Yin file for junos-common-types@2019-01-01 is generated
Yin file for junos-qfx-conf-interfaces@2019-01-01 is generated
Yin file for junos-qfx-conf-root@2019-01-01 is generated
--------------------------------------------
- Creating _xpath files from the Yin files -
--------------------------------------------
Creating Xpath file: junos-common-ddl-extensions@2019-01-01_xpath.txt
Creating Xpath file: junos-common-odl-extensions@2019-01-01_xpath.txt
Creating Xpath file: junos-common-types@2019-01-01_xpath.txt
Creating Xpath file: junos-qfx-conf-interfaces@2019-01-01_xpath.txt
Creating Xpath file: junos-qfx-conf-root@2019-01-01_xpath.txt

Sorry if this is overly verbose but it seems like people assume everyone just knows how this stuff works so I would rather be more explicit.

[davedotdev]

You are quite right. Too many assumptions are made and the documentation needs to be better. It will be better!

Marking this as a bug and enhancement.

Thanks for all of the detail. Superb stuff.

[vinpatel24]

For creating a junos-terraform provider from an existing JUNOS config, changes have been made to the internal code base and READme to ensure that pyang does not fail due to missing .yang files. We have added a check that if the .yang file is not needed (but causing the program to fail) it is subsequently removed.

For those MANUALLY adding yang files to the repo, users WILL NEED to make sure the correct yang files are added based on the expected capabilities (expected outcome of the provider) to avoid a failure since the program is unaware of the required .yang files needed.

This issue has been resolved and will be closed.