ScriptModule matching query does not exist

[dietybright]

I am having issue when trying to discover device via netdoc

Server Error There was a problem with your request. Please contact an administrator.

The complete exception is provided below:

<class 'extras.models.scripts.ScriptModule.DoesNotExist'>

ScriptModule matching query does not exist.

Python version: 3.10.12 NetBox version: 3.5.9 Plugins: netdoc: 3.5.1

[ismailkalolwala]

I did the installation of the NETDOC over NETBOX and this is what i am getting when i am initiating device discovery

Server Error There was a problem with your request. Please contact an administrator.

The complete exception is provided below:

<class 'extras.models.scripts.ScriptModule.DoesNotExist'>

ScriptModule matching query does not exist.

Python version: 3.9.16 NetBox version: 3.5.8 Plugins: netbox_ipcalculator: 1.1 netdoc: 3.5.1 If further assistance is required, please post to the NetBox discussion forum on GitHub.

Email ID - kalolwalaismail@gmail.com

[dainok]

@dietybright @ismailkalolwala are you both using Docker?

[DubrovskyEI]

Hello! I don't use docker and got the same issue with netbox 3.5.0, 3.5.1, 3.5.8, 3.5.9 and netdoc 3.5.1 Now using netbox 3.4.9 and netdoc 0.10.33 and it's working.

[ismailkalolwala]

@dietybright @ismailkalolwala are you both using Docker?

No I am not using Docker. I have RedHat Enterprise Linux details are below

NAME="Red Hat Enterprise Linux" VERSION="8.8 (Ootpa)" ID="rhel" ID_LIKE="fedora" VERSION_ID="8.8" PLATFORM_ID="platform:el8" PRETTY_NAME="Red Hat Enterprise Linux 8.8 (Ootpa)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" HOME_URL="https://www.redhat.com/" DOCUMENTATION_URL="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8" BUG_REPORT_URL="https://bugzilla.redhat.com/"

I have installed the NETBOX initially and then i have installed NETDOC and now when i initiate a device discover and give me this error

Server Error There was a problem with your request. Please contact an administrator.

The complete exception is provided below:

<class 'extras.models.scripts.ScriptModule.DoesNotExist'>

ScriptModule matching query does not exist.

Python version: 3.9.16 NetBox version: 3.5.8 Plugins: netbox_ipcalculator: 1.1 netdoc: 3.5.1 If further assistance is required, please post to the NetBox discussion forum on GitHub.

From the error - i navigate from terminal - /opt/netbox/netbox/extras/models and tried running this

python3 scripts.py and it is giving me this error

[root@DC2-NETBOXPOC-SRV01 models]# python3 scripts.py Traceback (most recent call last): File "/opt/netbox/netbox/extras/models/scripts.py", line 8, in from core.choices import ManagedFileRootPathChoices ModuleNotFoundError: No module named 'core'

Requesting you to please assist.

Email Id : Kalolwalaismail@gmail.com

[dainok]

I replicated the issue. Can you test if you can discover using Customization -> Scripts?

[ismailkalolwala]

I replicated the issue. Can you test if you can discover using Customization -> Scripts?

I can add, but doesn't seems to be working.

[ketchup57]

What ever the fix will be for <class 'extras.models.scripts.ScriptModule.DoesNotExist'> on NetBox version: 3.5.8 will be the fix for the newest version. Im not that knowledgeable but looking to use this in the future and will hopefully help try and track a fix. To get a better Idea of the error from the gui. You have to upload the netdoc_scripts.py and netdoc_reports.py from jobs folder into netbox gui. Mind you this is a very generic build with the bare minimum added to the database to add a device to netbox.

I'll provide the specific error soon.

[ketchup57]

What ever the fix will be for <class 'extras.models.scripts.ScriptModule.DoesNotExist'> on NetBox version: 3.5.8 will be the fix for the newest version. Im not that knowledgeable but looking to use this in the future and will hopefully help try and track a fix. To get a better Idea of the error from the gui. You have to upload the netdoc_scripts.py and netdoc_reports.py from jobs folder into netbox gui. Mind you this is a very generic build with the bare minimum added to the database to add a device to netbox.

I'll provide the specific error soon.

Here is the error from the gui. This is netbox 3.6.2. Which I know isnt supported currently. Script Log Line Level Message 1 Info
Using existing discoverable with IP address 172.16.254.2

2 Info
Starting discovery on 172.16.254.2

3 Info
Norninr inventory includes 172.16.254.2

4 Warning No Netmiko Cisco IOS XE device found

5 Warning No Netmiko Cisco IOS XE (Telnet) device found

6 Warning No Netmiko Cisco NX-OS device found

7 Info
Starting discovery of Netmiko Cisco XR devices

8 Info
Nornir inventory includes 172.16.254.2

9 Warning No Netmiko HPE Comware device found

10 Warning No Netmiko HPE Procurve device found

11 Warning No Netmiko Linux device found

12 Warning No VMware vSphere device found

13 Warning No Palo Alto Networks NGFW device found

14 Info
Discovery completed

15 Info
Spawning ingest script

16 Failure An exception occurred: DoesNotExist: ScriptModule matching query does not exist.

Traceback (most recent call last): File "/opt/netbox/netbox/extras/scripts.py", line 504, in _run_script script.output = script.run(data=data, commit=commit) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/netbox/netbox/scripts/netdoc_scripts.py", line 180, in run output = discovery( ^^^^^^^^^^ File "/opt/netbox/venv/lib/python3.11/site-packages/netdoc/tasks.py", line 124, in discovery utils.spawn_script("Ingest") File "/opt/netbox/venv/lib/python3.11/site-packages/netdoc/utils.py", line 1160, in spawn_script module = ScriptModule.objects.get(data_path="netdoc_scripts.py") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/models/query.py", line 637, in get raise self.model.DoesNotExist( extras.models.scripts.ScriptModule.DoesNotExist: ScriptModule matching query does not exist.

17 Info
Database changes have been reverted due to error.

[dainok]

The issue is releated to the new script/report system implemented by netbox. https://github.com/dainok/netdoc/blob/master/netdoc/__init__.py should create scripts and reports automatically but it's failing in some cases.

[bobmarvel]

I do have similar issue: exception after trying to discover. Clean install 3.5.1 on ubuntu 20.04 (also tried 3.5.8). I have no scripts or reports in respective directories, therefore after trying discovery I get the following exception: <class 'extras.models.scripts.ScriptModule.DoesNotExist'>

ScriptModule matching query does not exist.

Python version: 3.10.12 NetBox version: 3.5.1

[ketchup57]

So I noticed that when I reloaded my containers and I had a saved device to be discovered. That during the boot up of all the containers, the worker was able to connect and loaded the running configuration. But nothing was saved and kicking off the discover has the same problem.

I’m not great at the development side of things but trying my best to grasp all of it.

On Mon, Sep 25, 2023 at 8:30 AM bobmarvel @.***> wrote:

I do have similar issue: exception after trying to discover. Clean install 3.5.1 on ubuntu 20.04 (also tried 3.5.8). I have no scripts or reports in respective directories, therefore after trying discovery I get the following exception: <class 'extras.models.scripts.ScriptModule.DoesNotExist'>

ScriptModule matching query does not exist.

Python version: 3.10.12 NetBox version: 3.5.1

— Reply to this email directly, view it on GitHub https://github.com/dainok/netdoc/issues/71#issuecomment-1733718351, or unsubscribe https://github.com/notifications/unsubscribe-auth/AONW7OQF666N4XOITW5Z7DLX4GBQ5ANCNFSM6AAAAAA5AZABBY . You are receiving this because you commented.Message ID: @.***>

[ketchup57]

Please tell me to shut up if none of this helps. I'm running some tests. Tried running the script from inside the container. I got the following error.

(venv) root@940b5661092b:/opt/netbox/netbox/scripts# python ./netdoc_scripts.py 
Traceback (most recent call last):
  File "/opt/netbox/netbox/scripts/netdoc_scripts.py", line 14, in <module>
    from dcim.models import Site, DeviceRole, Interface, Device
ModuleNotFoundError: No module named 'dcim'
(venv) root@940b5661092b:/opt/netbox/netbox/scripts# 

[dietybright]

Netbox 3.4.9 and Netdoc 0.10.33 is working but seem the discoverable information is not good enough, hope can fix in NetDoc 3.5.x

[dainok]

Netbox 3.4.9 and Netdoc 0.10.33 is working but seem the discoverable information is not good enough, hope can fix in NetDoc 3.5.x

@dietybright What do you mean? NetDoc 3.5 is compatibile (should be) with NetBox 3.5. Nothing changed about discovery, yet.

[dainok]

@dietybright @ketchup57 @bobmarvel @ismailkalolwala any chance you can test https://github.com/dainok/netdoc/pull/72 ?

[ketchup57]

@dietybright @ketchup57 @bobmarvel @ismailkalolwala any chance you can test #72 ?

I've tested those changes and it didn't auto create the scripts. When I manually upload the script the I got the output from the GUI from one of my first comments. If I run the script from the cli, I got the following.

(venv) root@940b5661092b:/opt/netbox/netbox/scripts# python ./netdoc_scripts.py 
Traceback (most recent call last):
  File "/opt/netbox/netbox/scripts/netdoc_scripts.py", line 14, in <module>
    from dcim.models import Site, DeviceRole, Interface, Device
ModuleNotFoundError: No module named 'dcim'
(venv) root@940b5661092b:/opt/netbox/netbox/scripts# 

I will keep running through it when I can to see if I can get any specific errors. Trying to compare from any other plugins to see if any of them could be used a reference.

[dainok]

Why are you uploading that script? I modified init.py as you can see from the PR.

[ketchup57]

Right, but it never created the script in the folder. I can remove and give it another try though.

[dainok]

If everything is working fine, there is no reason to create scripts or reports manually. If you can, test the PR. If you cannot, wait for the release.

[ketchup57]

It's not working correctly. So with just netdoc and one or two plugins installed. When I try to run a discovery of a device. I get the following.

<class 'extras.models.scripts.ScriptModule.DoesNotExist'>

The problem is part of the ingest of the discovery. Parts of the netdoc_script work like fixing the device types-names etc. I think its something more in line with the logs from the scripts not being ingested.

[dainok]

Please test 3.5.2, I will do that tomorrow.

[dainok]

Seems that DataSource is not populating script files. I'm investigating. You can manually create script and report files, but I prefer to avoid that.

[dainok]

Found the issue: the PIP package was missing report and script files. Fixed by https://github.com/dainok/netdoc/pull/73 Please test 3.5.3

[ismailkalolwala]

I am getting this now

Server Error There was a problem with your request. Please contact an administrator.

The complete exception is provided below:

<class 'KeyError'>

'Discover'

Python version: 3.10.12 NetBox version: 3.6.2 Plugins: netdoc: 3.5.3 If further assistance is required, please post to the NetBox discussion forum on GitHub.

[dietybright]

Hi,

I have upgarde netdoc to 3.5.3 but facing another issue

<class 'KeyError'> 'Discover' Python version: 3.10.12 NetBox version: 3.5.8 Plugins: netdoc: 3.5.3

Go to Customization > Scripts, there is an error message "Script file at: /opt/netbox/netbox/scripts/netdoc_scripts.py could not be loaded."

[dainok]

I think there is something dirty on your DB, maybe caused by previous bug. Could you open NetBox via web and delete all scripts, reports and data sources? Then restart.

[ismailkalolwala]

I got this working now and i can see discovery is working fine but i do not see any data under arp, route and mac address tables in netdoc gui sections

[dainok]

@ismailkalolwala it could depends by dozens of reasons. Start from job logs, then discovery log

[ismailkalolwala]

Discovery is finally working -- i can see netmiko library accessing my devices - in /var/log/syslog and also i can see from change log --- everything is good but ingested is not working.

[dainok]

What do you mean with not working? Can you post a screenshot?

[ismailkalolwala]

from /var/log/syslog - i can see netmiko library is used to get all information from Cisco IOS based devices - and same is seen on GUI in changelog

but no information in arp, routing table mac address.

[dainok]

Find the log with "hostname" as template, and check why they are not ingested.

[ismailkalolwala]

Sorry i am not getting you. Which logs - are u referring to /var/log/syslog or something else. or NTC Templates - can you please help me here with more details.

[ismailkalolwala]

This is what i got while running discovery on a single device and monitoring the syslog

Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: An exception occurred: TypeError: Device() got unexpected keyword arguments: 'device_role_id' Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: Traceback (most recent call last): Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: File "/opt/netbox/netbox/extras/scripts.py", line 504, in _run_script Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: script.output = script.run(data=data, commit=commit) Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: File "/opt/netbox/netbox/scripts/netdoc_scripts.py", line 353, in run Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: log_ingest(log) Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/utils.py", line 578, in log_ingest Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: module.ingest(log) Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/ingestors/netmiko_cisco_ios_hostname.py", line 36, in ingest Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: device_o = device.create(**data) Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/schemas/device.py", line 108, in create Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: obj = utils.object_create(Device, **kwargs) Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/utils.py", line 1055, in object_create Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: return model_o.objects.create(**kwargs) Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/manager.py", line 87, in manager_method Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: return getattr(self.get_queryset(), name)(*args, **kwargs) Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/query.py", line 656, in create Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: obj = self.model(**kwargs) Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: File "/opt/netbox/netbox/utilities/tracking.py", line 41, in __init__ Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: super().__init__(*args, **kwargs) Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 567, in __init__ Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: raise TypeError( Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: TypeError: Device() got unexpected keyword arguments: 'device_role_id' Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: Exception raised during script execution: Device() got unexpected keyword arguments: 'device_role_id' Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: 13:42:54 default: Job OK (1b80535e-28ed-4600-8627-bd82c3eb29aa) Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: 13:42:54 Result is kept for 500 seconds

[dainok]

Please let's discuss on https://github.com/dainok/netdoc/issues/74