search

netbox-community / netbox-demo-data

Demo data useful for populating demo instances of NetBox
MIT License
22 stars 10 forks source link

Import not successful #2

Open sw8899 opened 3 years ago

sw8899 commented 3 years ago

I have problems with the Ipmort:

(venv) sw@sw-VirtualBox:/opt/netbox/netbox$ ./manage.py loaddata -v 3 netbox-demo.json Loading 'netbox-demo' fixtures... Checking '/opt/netbox-2.11.7/netbox' for fixtures... Installing json fixture 'netbox-demo' from '/opt/netbox-2.11.7/netbox'. Traceback (most recent call last): File "/opt/netbox-2.11.7/venv/lib/python3.9/site-packages/django/core/serializers/json.py", line 69, in Deserializer objects = json.loads(stream_or_string) File "/usr/lib/python3.9/json/init.py", line 346, in loads return _default_decoder.decode(s) File "/usr/lib/python3.9/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.9/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 7 column 1 (char 6)

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/opt/netbox-2.11.7/netbox/./manage.py", line 10, in execute_from_command_line(sys.argv) File "/opt/netbox-2.11.7/venv/lib/python3.9/site-packages/django/core/management/init.py", line 419, in execute_from_command_line utility.execute() File "/opt/netbox-2.11.7/venv/lib/python3.9/site-packages/django/core/management/init.py", line 413, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/opt/netbox-2.11.7/venv/lib/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv self.execute(*args, *cmd_options) File "/opt/netbox-2.11.7/venv/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute output = self.handle(args, **options) File "/opt/netbox-2.11.7/venv/lib/python3.9/site-packages/django/core/management/commands/loaddata.py", line 78, in handle self.loaddata(fixture_labels) File "/opt/netbox-2.11.7/venv/lib/python3.9/site-packages/django/core/management/commands/loaddata.py", line 123, in loaddata self.load_label(fixture_label) File "/opt/netbox-2.11.7/venv/lib/python3.9/site-packages/django/core/management/commands/loaddata.py", line 181, in load_label for obj in objects: File "/opt/netbox-2.11.7/venv/lib/python3.9/site-packages/django/core/serializers/json.py", line 74, in Deserializer raise DeserializationError() from exc django.core.serializers.base.DeserializationError: Problem installing fixture '/opt/netbox-2.11.7/netbox/netbox-demo.json': (venv) sw@sw-VirtualBox:/opt/netbox/netbox$

madarsk commented 3 years ago

I've got the same problem, netbox v3.0.3

DanSheps commented 3 years ago

What data package are you using for import?

apishark commented 2 years ago

Instructions not working for netbox VERSION = '3.1.7'

ascension2020 commented 1 year ago

I'm sharing this for anyone else who has the same issue.

I had the same error (except the line and character were different) when importing the v3.4 demo data into a new install of netbox docker. I fixed it by opening the netbox-demo-v3.4.json file in VS Code and using the fix-json extension to correct it. I didn't have the time to find out exactly what was wrong with the original format.

abhi1693 commented 1 year ago

Are you sure that vscode is not doing something funny to the JSON? Because I use the demo data all the time and there is nothing wrong in the structure or the format.

ascension2020 commented 1 year ago

I don't understand your reasoning. I used VS Code to fix the JSON. VS Code had nothing to do with the original problem.

Message ID: @.*** com>

abhi1693 commented 1 year ago

If you did not modify the data yourself, the fix-json utility should not be needed IMO. I have not seen the error on 3.4 or 3.5 yet. Whatever issues were detected were already fixed for those versions last week

ascension2020 commented 1 year ago

If you'd like to work together to find a solution, I'm happy to assist. If you only want to gaslight me, let's just move on.

On Fri, Apr 21, 2023 at 3:18 PM Abhimanyu Saharan @.***> wrote:

If you did not modify the data yourself, the fix-json utility should not be needed IMO. I have not seen the error on 3.4 or 3.5 yet. Whatever issues were detected were already fixed for those versions last week

— Reply to this email directly, view it on GitHub https://github.com/netbox-community/netbox-demo-data/issues/2#issuecomment-1518243995, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADLIWNB7BTLDTFYCSUH3H3TXCLMRVANCNFSM47IBHSCQ . You are receiving this because you commented.Message ID: @.***>

abhi1693 commented 1 year ago

I did not intend to gaslight you at all. Not sure why you felt that. If you have the exact stackstrace, I can look into the issue but as I mentioned earlier, there is no issue as far as the latest data is concerned. We use the same to build nightly build the demo instances as well.

ascension2020 commented 1 year ago

I'll see if I can repro the issue tonight. If I can, I'll post the stack trace.

On Fri, Apr 21, 2023, 3:45 PM Abhimanyu Saharan @.***> wrote:

I did not intend to gaslight you at all. Not sure why you felt that. If you have the exact stackstrace, I can look into the issue but as I mentioned earlier, there is no issue as far as the latest data is concerned. We use the same to build nightly build the demo instances as well.

— Reply to this email directly, view it on GitHub https://github.com/netbox-community/netbox-demo-data/issues/2#issuecomment-1518264558, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADLIWNDOZOAG5QSKPYB7D5DXCLPV7ANCNFSM47IBHSCQ . You are receiving this because you commented. Message ID: @.*** com>

ascension2020 commented 1 year ago

OK, I think it had something to do with my docker installation. I don't know exactly what. I just know that it worked when I deleted all volumes, images and containers, pruned it, and updated docker to the latest version.

For anyone who comes across this in the future, here are the exact instructions to install netbox-docker and import the data, using version 3.4. This was done on a Mac, but the steps will probably work with Debian-based Linux distros, including WSL.

Install netbox-docker

bash-3.2$ git clone -b release https://github.com/netbox-community/netbox-docker.git
Cloning into 'netbox-docker'...
remote: Enumerating objects: 4200, done.
remote: Counting objects: 100% (53/53), done.
remote: Compressing objects: 100% (31/31), done.
remote: Total 4200 (delta 22), reused 43 (delta 20), pack-reused 4147
Receiving objects: 100% (4200/4200), 1.01 MiB | 1.37 MiB/s, done.
Resolving deltas: 100% (2428/2428), done.

bash-3.2$ cd netbox-docker
bash-3.2$ tee docker-compose.override.yml <<EOF
> version: '3.4'
> services:
>   netbox:
>     ports:
>       - 8000:8080
> EOF
version: '3.4'
services:
  netbox:
    ports:
      - 8000:8080

bash-3.2$ docker compose pull
[+] Running 14/33
 ✔ netbox-housekeeping Skipped - Image is already being pulled by netbox                                                                                                                                                                               0.0s 
 ✔ netbox-worker Skipped - Image is already being pulled by netbox                                                                                                                                                                                     0.0s 
 ✔ redis-cache Skipped - Image is already being pulled by redis                                                                                                                                                                                        0.0s 
 ⠏ netbox 14 layers [⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀]      0B/0B      Pulling                                                                                                                                                                                          12.0s
...

bash-3.2$ docker compose up
...

Clone the netbox-demo-data

bash-3.2$ cd../
bash-3.2$ git clone https://github.com/netbox-community/netbox-demo-data.git
Cloning into 'netbox-demo-data'...
remote: Enumerating objects: 92, done.
remote: Counting objects: 100% (92/92), done.
remote: Compressing objects: 100% (61/61), done.
remote: Total 92 (delta 56), reused 64 (delta 31), pack-reused 0
Receiving objects: 100% (92/92), 559.62 KiB | 899.00 KiB/s, done.
Resolving deltas: 100% (56/56), done.

Drop the database

bash-3.2$ docker-compose exec postgres sh -c 'psql -U $POSTGRES_USER postgres -c "DROP DATABASE netbox;"'
ERROR:  database "netbox" is being accessed by other users
DETAIL:  There are 2 other sessions using the database.

bash-3.2$ docker ps
CONTAINER ID   IMAGE                               COMMAND                  CREATED          STATUS                    PORTS                    NAMES
0168d79ed23e   netboxcommunity/netbox:v3.4-2.5.3   "/usr/bin/tini -- /o…"   27 minutes ago   Up 27 minutes (healthy)                            netbox-docker-netbox-worker-1
0cb3ae686494   netboxcommunity/netbox:v3.4-2.5.3   "/usr/bin/tini -- /o…"   27 minutes ago   Up 27 minutes (healthy)                            netbox-docker-netbox-housekeeping-1
ae9e39c3c84d   netboxcommunity/netbox:v3.4-2.5.3   "/usr/bin/tini -- /o…"   27 minutes ago   Up 27 minutes (healthy)   0.0.0.0:8000->8080/tcp   netbox-docker-netbox-1
9569c6ac6dad   redis:7-alpine                      "docker-entrypoint.s…"   27 minutes ago   Up 27 minutes             6379/tcp                 netbox-docker-redis-1
1c3c41c0f581   postgres:15-alpine                  "docker-entrypoint.s…"   27 minutes ago   Up 27 minutes             5432/tcp                 netbox-docker-postgres-1
874d073f43f9   redis:7-alpine                      "docker-entrypoint.s…"   27 minutes ago   Up 27 minutes             6379/tcp                 netbox-docker-redis-cache-1

bash-3.2$ docker stop netbox-docker-netbox-1
netbox-docker-netbox-1

bash-3.2$ docker-compose exec postgres sh -c 'psql -U $POSTGRES_USER postgres -c "DROP DATABASE netbox;"'
DROP DATABASE

Recreate the database and import the demo data

bash-3.2$ docker-compose exec postgres sh -c 'psql -U $POSTGRES_USER postgres -c "CREATE DATABASE netbox;"'
CREATE DATABASE

bash-3.2$ docker start netbox-docker-netbox-1

bash-3.2$ docker ps | grep netbox-1
ae9e39c3c84d   netboxcommunity/netbox:v3.4-2.5.3   "/usr/bin/tini -- /o…"   31 minutes ago   Up 2 minutes (healthy)    0.0.0.0:8000->8080/tcp   netbox-docker-netbox-1

bash-3.2$ docker-compose exec netbox bash -c "source /opt/netbox/venv/bin/activate && ./manage.py migrate"
🧬 loaded config '/etc/netbox/config/configuration.py'
🧬 loaded config '/etc/netbox/config/extra.py'
🧬 loaded config '/etc/netbox/config/logging.py'
🧬 loaded config '/etc/netbox/config/plugins.py'
Operations to perform:
  Apply all migrations: admin, auth, circuits, contenttypes, dcim, django_rq, extras, ipam, sessions, social_django, taggit, tenancy, users, virtualization, wireless
Running migrations:
  No migrations to apply.

bash-3.2$ export VERSION="v3.4"
bash-3.2$ docker cp ../netbox-demo-data/netbox-demo-$VERSION.json "$(docker-compose ps -q netbox)":/opt/netbox/netbox/netbox-demo.json

bash-3.2$ docker-compose exec netbox bash -c "source /opt/netbox/venv/bin/activate && ./manage.py loaddata netbox-demo.json"
🧬 loaded config '/etc/netbox/config/configuration.py'
🧬 loaded config '/etc/netbox/config/extra.py'
🧬 loaded config '/etc/netbox/config/logging.py'
🧬 loaded config '/etc/netbox/config/plugins.py'
Installed 6915 object(s) from 1 fixture(s)
Joshude commented 1 year ago

I have issues with importing the data, too. 

(venv) netboxadm@netbox:/opt/netbox/netbox$ ./manage.py loaddata -v 3 netbox-demo-v3.5.json
Loading 'netbox-demo-v3.5' fixtures...
Checking '/opt/netbox/netbox' for fixtures...
Installing json fixture 'netbox-demo-v3.5' from '/opt/netbox/netbox'.
Traceback (most recent call last):
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/serializers/json.py", line 70, in Deserializer
    yield from PythonDeserializer(objects, **options)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/serializers/python.py", line 103, in Deserializer
    Model = _get_model(d["model"])
TypeError: string indices must be integers

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/netbox/netbox/./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
    utility.execute()
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 440, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/base.py", line 402, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/base.py", line 448, in execute
    output = self.handle(*args, **options)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/commands/loaddata.py", line 102, in handle
    self.loaddata(fixture_labels)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/commands/loaddata.py", line 163, in loaddata
    self.load_label(fixture_label)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/commands/loaddata.py", line 251, in load_label
    for obj in objects:
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/serializers/json.py", line 74, in Deserializer
    raise DeserializationError() from exc
django.core.serializers.base.DeserializationError: Problem installing fixture '/opt/netbox/netbox/netbox-demo-v3.5.json':

The JSON seems to be fine.