unit test failure on templates

[ktbyers]

@dbarrosop @mirceaulinic

I am testing the existing unit tests on napalm-ios and this fails:

DriverTemplateNotImplemented: Config template dir does not exist: /home/gituser/VENV/napalm/local/lib/python2.7/site-packages/napalm_ios-0.2.1-py2.7.egg/napalm_ios/templates. Please create it and add driver-specific templates.

Is this test expected to fail on napalm_ios (from what I see it is expected to fail)?

[dbarrosop]

Mmm, can you tell me what you did exactly? Couldn't reproduce it. I got this:

Test load_template method. ... ERROR
======================================================================
ERROR: Test load_template method.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/dbarroso/workspace/napalm/napalm-base/napalm_base/test/base.py", line 136, in test_load_template
    self.device.load_template('set_hostname', hostname='my-hostname')
  File "/Users/dbarroso/workspace/napalm/napalm-base/napalm_base/base.py", line 108, in load_template
    **template_vars)
  File "/Users/dbarroso/workspace/napalm/napalm-base/napalm_base/helpers.py", line 83, in load_template
    return cls.load_merge_candidate(config=configuration)
  File "/Users/dbarroso/workspace/napalm/napalm-ios/napalm_ios/ios.py", line 102, in load_merge_candidate
    raise NotImplementedError
NotImplementedError

[ktbyers]

I see the 'NotImplementedError' if I 'pip install napalm-ios'.

I see this error:

DriverTemplateNotImplemented: Config template dir does not exist: /home/gituser/VENV/napalm/local/lib/python2.7/site-packages/napalm_ios-0.2.1-py2.7.egg/napalm_ios/templates.

If I install using, 'python setup.py install', I get this error:

E
======================================================================
ERROR: test_load_template (__main__.TestConfigIOSDriver)
Test load_template method.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/gituser/VENV/napalm/local/lib/python2.7/site-packages/napalm_base/test/base.py", line 136, in test_load_template
    self.device.load_template('set_hostname', hostname='my-hostname')
  File "/home/gituser/VENV/napalm/local/lib/python2.7/site-packages/napalm_base/base.py", line 108, in load_template
    **template_vars)
  File "/home/gituser/VENV/napalm/local/lib/python2.7/site-packages/napalm_base/helpers.py", line 55, in load_template
    path=template_dir_path
DriverTemplateNotImplemented: Config template dir does not exist: /home/gituser/VENV/napalm/local/lib/python2.7/site-packages/napalm_ios-0.2.1-py2.7.egg/napalm_ios/templates.

Basically, there is nothing in setup.py that would cause ./templates to be included.

I have bypassed this in the local ios.py test since it will fail regardless, but it might be an issue for the other platforms.

[mirceaulinic]

I don't recall running the tests agains the class (e.g., nosetests -v TestIOSDriver:TestConfigIOSDriver) which implies the execution of setUpClass therefore to be run against a target device, after load_template was introduced. I will have a look for other drivers.

Basically, there is nothing in setup.py that would cause ./templates to be included.

It is specified in the manifest file: https://github.com/napalm-automation/napalm-ios/blob/develop/MANIFEST.in

Can you re-install also using pip install -e /path/to/napalm-ios and see if any improvement?

[ktbyers]

@mirceaulinic Yes, I don't think 'python setup.py install' uses MANIFEST.in.

I generally expect 'python setup.py install' to work, but it doesn't matter much..

I will retest with the pip install -e, but I expect that will work.

[mirceaulinic]

No, installing using setup.py install will not consider the manifest file, as this is for the sdist option (which should be executed when creating the package and upload to pypi). pip install -e works usually because will create an egg link to the source directory and the modules are not executed from lib/python2.7/site-packages but from your directory, e.g. /path/to/napalm-ios etc.

[ktbyers]

@mirceaulinic my point is a bit different...I view it as a bug if 'python setup.py install' doesn't work (as this is the default python install method if you clone the repo).

[mirceaulinic]

Totally agree, it's definitely a bug & python install must work!

[ktbyers]

Okay, sounds good :-)

[ktbyers]

FYI, pip -e worked fine.

[mirceaulinic]

@ktbyers

Can you please share the install output? Looks like the option we are looking for in setup.py is include_package_data=True which is already there.

Trying this out:

mirucha@Master-Roshi.local:~ git:master ✗
23:27:36 › git clone git@github.com:napalm-automation/napalm-ios.git
Cloning into 'napalm-ios'...
remote: Counting objects: 1168, done.
remote: Compressing objects: 100% (60/60), done.
remote: Total 1168 (delta 26), reused 0 (delta 0), pack-reused 1104
Receiving objects: 100% (1168/1168), 235.90 KiB | 233.00 KiB/s, done.
Resolving deltas: 100% (504/504), done.
Checking connectivity... done.
(napalm-test)
mirucha@Master-Roshi.local:~ git:master ✗
23:27:41 › cd napalm-ios
(napalm-test)
mirucha@Master-Roshi.local:~/napalm-ios git:develop ✔︎ Merge branch 'ktbyer
23:27:43 › python setup.py install
running install
running bdist_egg
running egg_info
creating napalm_ios.egg-info
writing requirements to napalm_ios.egg-info/requires.txt
writing napalm_ios.egg-info/PKG-INFO
writing top-level names to napalm_ios.egg-info/top_level.txt
writing dependency_links to napalm_ios.egg-info/dependency_links.txt
writing manifest file 'napalm_ios.egg-info/SOURCES.txt'
reading manifest file 'napalm_ios.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'napalm_ios.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.11-intel/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/napalm_ios
copying napalm_ios/__init__.py -> build/lib/napalm_ios
copying napalm_ios/ios.py -> build/lib/napalm_ios
creating build/lib/napalm_ios/templates
copying napalm_ios/templates/delete_ntp_peers.j2 -> build/lib/napalm_ios/templates
copying napalm_ios/templates/set_hostname.j2 -> build/lib/napalm_ios/templates
copying napalm_ios/templates/set_ntp_peers.j2 -> build/lib/napalm_ios/templates
...etc...

Looks like it does copy the files as specified in the manifest file.

I do not have any IOS box to test, but doing the same with napalm-junos:

› python setup.py install
running install
running bdist_egg
running egg_info
writing requirements to napalm_junos.egg-info/requires.txt
writing napalm_junos.egg-info/PKG-INFO
writing top-level names to napalm_junos.egg-info/top_level.txt
writing dependency_links to napalm_junos.egg-info/dependency_links.txt
reading manifest file 'napalm_junos.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'napalm_junos/utils/textfsm_templates/*.tpl'
writing manifest file 'napalm_junos.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.11-intel/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/napalm_junos
copying napalm_junos/__init__.py -> build/lib/napalm_junos
copying napalm_junos/junos.py -> build/lib/napalm_junos
creating build/lib/napalm_junos/utils
copying napalm_junos/utils/__init__.py -> build/lib/napalm_junos/utils
copying napalm_junos/utils/junos_views.py -> build/lib/napalm_junos/utils
creating build/lib/napalm_junos/templates
copying napalm_junos/templates/delete_ntp_peers.j2 -> build/lib/napalm_junos/templates
copying napalm_junos/templates/delete_ntp_servers.j2 -> build/lib/napalm_junos/templates
copying napalm_junos/templates/delete_prefix_list.j2 -> build/lib/napalm_junos/templates
copying napalm_junos/templates/delete_probes.j2 -> build/lib/napalm_junos/templates
...etc...

Which works as expected:

>>> from napalm_base import get_network_driver
>>> d = get_network_driver('junos')
>>> e = d('127.0.0.1', 'napalm', 'Napalm123', optional_args={'port': 12202, 'config_lock': False})
>>> e.open()
>>> e.load_template('set_hostname', hostname='abcd')
>>> print e.compare_config()
[edit system]
-  host-name vsrx;
+  host-name abcd;
>>> e.close()