search

networkop / cx

Containerised Cumulus VX
19 stars 4 forks source link

Unable to commit config on 5.x images #6

Closed jhlasnik closed 2 years ago

jhlasnik commented 2 years ago

In my initial testing of the 5.x images 5.0.0 and 5.0.1 I have been unable to apply any config.

root@6fd7fb5b48af1730:mgmt:~# nv set system hostname clab-sw1
root@6fd7fb5b48af1730:mgmt:~# nv config 
apply    detach   diff     history  patch    replace  save     show     
root@6fd7fb5b48af1730:mgmt:~# nv config save 
saved
root@6fd7fb5b48af1730:mgmt:~# nv config apply 
Failed to prepare to apply
  Unrecoverable internal error
root@6fd7fb5b48af1730:mgmt:~# nv config d
detach  diff    
root@6fd7fb5b48af1730:mgmt:~# nv config diff 
- set:
    system:
      hostname: clab-sw1
root@6fd7fb5b48af1730:mgmt:~# nv config apply 
Failed to prepare to apply
  Unrecoverable internal error
root@6fd7fb5b48af1730:mgmt:~# 

oot@6fd7fb5b48af1730:mgmt:~# nv set bridge domain br_default vlan 100 vni 100
root@6fd7fb5b48af1730:mgmt:~# nv config diff 
- set:
    bridge:
      domain:
        br_default:
          vlan:
            '100':
              vni:
                '100': {}
    nve:
      vxlan:
        enable: on
root@6fd7fb5b48af1730:mgmt:~# nv set interface lo ip address 1.1.1.1/32 
root@6fd7fb5b48af1730:mgmt:~# nv config apply 
Failed to prepare to apply
  Unrecoverable internal error
root@6fd7fb5b48af1730:mgmt:~#

I was also unable to build the 5.0.2 image but I'm still investigating if that's an issue on my machine.

networkop commented 2 years ago

Thanks for reporting @jhlasnik . Can you confirm how you're running cx? do you use containerlab? Can you also collect the logs from nvued when doing the apply, e.g.

journalctl -u nvued -f &
nv set system hostname clab-sw1
nv config apply -y
jhlasnik commented 2 years ago

Hi @networkop I am using containerlab and tried it on both 0.25.1 and 0.26.2

Here are the logs when trying to commit.

root@8622341e6ec0d73d:mgmt:~# journalctl -u nvued -f &
[1] 3078
root@8622341e6ec0d73d:mgmt:~# -- Logs begin at Mon 2022-05-23 13:21:32 UTC. --
May 23 13:21:45 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git rm nvue.json'
May 23 13:21:45 8622341e6ec0d73d nvued[1999]:    INFO:  Running: "/usr/bin/git commit -m 'Revision meta created'"
May 23 13:21:45 8622341e6ec0d73d nvued[1999]:    INFO:  Added API versions_v1
May 23 13:21:45 8622341e6ec0d73d nvued[1999]:    INFO:  Started the Job thread
May 23 13:21:45 8622341e6ec0d73d nvued[1999]:    INFO:  Started nvued
May 23 13:21:45 8622341e6ec0d73d nvued[1999]:    INFO:    configdir: /etc/nvue.d
May 23 13:21:45 8622341e6ec0d73d nvued[1999]:    INFO:   * Running on unix:///run/nvue/nvue.sock (Press CTRL+C to quit)
May 23 13:21:45 8622341e6ec0d73d nvued[1999]:    INFO:  Notifying systemd that NVUED is ready
May 23 13:21:45 8622341e6ec0d73d systemd[1]: Started NVIDIA User Experience Daemon.
May 23 13:21:46 8622341e6ec0d73d nvued[1999]:    INFO:  <local> - - [23/May/2022 13:21:46] "GET / HTTP/1.1" 200 -
root@8622341e6ec0d73d:mgmt:~# 
root@8622341e6ec0d73d:mgmt:~# 
root@8622341e6ec0d73d:mgmt:~# nv set system hostname clab-sw1
May 23 13:22:52 8622341e6ec0d73d nvued[1999]:    INFO:  <local> - - [23/May/2022 13:22:52] "GET /nvue_v1/system/hostname/clab-sw1?rev=operational HTTP/1.1" 404 -
May 23 13:22:52 8622341e6ec0d73d nvued[1999]:    INFO:  <local> - - [23/May/2022 13:22:52] "GET /nvue_v1/system/hostname?rev=operational HTTP/1.1" 404 -
May 23 13:22:52 8622341e6ec0d73d nvued[1999]:    INFO:  <local> - - [23/May/2022 13:22:52] "GET /nvue_v1/system?rev=operational HTTP/1.1" 200 -
May 23 13:22:52 8622341e6ec0d73d nvued[1999]:    INFO:  <local> - - [23/May/2022 13:22:52] "GET /nvue_v1/system?rev=applied HTTP/1.1" 200 -
root@8622341e6ec0d73d:mgmt:~# May 23 13:22:53 8622341e6ec0d73d nvued[1999]:    INFO:  <local> - - [23/May/2022 13:22:53] "GET /nvue_v1/system?pointers=%5B%22%2Fparameters%22%2C+%22%2Fpatch%2FrequestBody%2Fcontent%2Fapplication~1json%2Fschema%22%2C+%22%2Fpatch%2Fparameters%22%2C+%22%2Fpatch%2Fresponses%2F200%2Flinks%22%5D HTTP/1.1" 200 -
May 23 13:22:53 8622341e6ec0d73d nvued[1999]:    INFO:  <local> - - [23/May/2022 13:22:53] "PATCH /nvue_v1/?rev=applied HTTP/1.1" 409 -
May 23 13:22:53 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout applied'
May 23 13:22:53 8622341e6ec0d73d nvued[1999]:    INFO:  Creating pending revision: changeset/root/2022-05-23_13.22.53_CSHM
May 23 13:22:53 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout -b changeset/root/2022-05-23_13.22.53_CSHM'
May 23 13:22:53 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git cat-file -e changeset/root/2022-05-23_13.22.53_CSHM'
May 23 13:22:53 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add changeset/root/2022-05-23_13.22.53_CSHM'
May 23 13:22:53 8622341e6ec0d73d nvued[1999]:    INFO:  Running: "/usr/bin/git commit --allow-empty -m 'Updating changeset/root/2022-05-23_13.22.53_CSHM'"
May 23 13:22:53 8622341e6ec0d73d nvued[1999]:    INFO:  <local> - - [23/May/2022 13:22:53] "POST /nvue_v1/revision HTTP/1.1" 200 -
May 23 13:22:53 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout changeset/root/2022-05-23_13.22.53_CSHM'
May 23 13:22:53 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/nvue.json
May 23 13:22:53 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout changeset/root/2022-05-23_13.22.53_CSHM'
May 23 13:22:53 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:22:53 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout changeset/root/2022-05-23_13.22.53_CSHM'
May 23 13:22:53 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout changeset/root/2022-05-23_13.22.53_CSHM'
May 23 13:22:53 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/nvue.json
May 23 13:22:53 8622341e6ec0d73d nvued[1999]:    INFO:  <local> - - [23/May/2022 13:22:53] "PATCH /nvue_v1/system?rev=changeset%2Froot%2F2022-05-23_13.22.53_CSHM HTTP/1.1" 200 -

root@8622341e6ec0d73d:mgmt:~# nv config apply -y
verifyingMay 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  <local> - - [23/May/2022 13:23:10] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.22.53_CSHM HTTP/1.1" 200 -
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add changeset/root/2022-05-23_13.22.53_CSHM'
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Running: "/usr/bin/git commit --allow-empty -m 'Updating changeset/root/2022-05-23_13.22.53_CSHM'"
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Queued Job apply:changeset/root/2022-05-23_13.22.53_CSHM
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  <local> - - [23/May/2022 13:23:10] "PATCH /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.22.53_CSHM HTTP/1.1" 200 -
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Running Job apply:changeset/root/2022-05-23_13.22.53_CSHM
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Applying changeset/root/2022-05-23_13.22.53_CSHM: verifying
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add changeset/root/2022-05-23_13.22.53_CSHM'
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Running: "/usr/bin/git commit --allow-empty -m 'Updating changeset/root/2022-05-23_13.22.53_CSHM'"
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  <local> - - [23/May/2022 13:23:10] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.22.53_CSHM HTTP/1.1" 200 -
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/platform-detect'
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/platform-detect'
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/platform-detect'
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /usr/share/cumulus-platform/cumulus/vx/etc/cumulus/ports.conf
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/lib/python2.7/dist-packages/cumulus/portmodes.py'
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  <local> - - [23/May/2022 13:23:10] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.22.53_CSHM HTTP/1.1" 200 -
readyingMay 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  <local> - - [23/May/2022 13:23:10] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.22.53_CSHM HTTP/1.1" 200 -
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Running: "/bin/bash -c '/usr/cumulus/bin/validate-ports -f /tmp/ports.conf_20220523-132310'"
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Applying changeset/root/2022-05-23_13.22.53_CSHM: verified (Verified config)
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add changeset/root/2022-05-23_13.22.53_CSHM'
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Running: "/usr/bin/git commit --allow-empty -m 'Updating changeset/root/2022-05-23_13.22.53_CSHM'"
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Applying changeset/root/2022-05-23_13.22.53_CSHM: readying
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add changeset/root/2022-05-23_13.22.53_CSHM'
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Running: "/usr/bin/git commit --allow-empty -m 'Updating changeset/root/2022-05-23_13.22.53_CSHM'"
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout applied'
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git tag --force apply/2022-05-23_13.23.10_CSHN/start applied'
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Creating pending revision: apply/2022-05-23_13.23.10_CSHN
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout -b apply/2022-05-23_13.23.10_CSHN'
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add reload.yml apply_log.txt'
May 23 13:23:10 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.23.10_CSHN'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.23.10_CSHN'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.23.10_CSHN'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.23.10_CSHN'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/kernel_route_offload_flags.conf.orig
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/kernel_route_offload_flags.conf
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/sx_netdev.route_offload_flags.conf
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Schedule run of set_sx_netdev.sh
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/port_mirror.conf
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Schedule run of switchdctl.sh
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add kernel_route_offload_flags.conf.dst port_mirror.conf switchdctl.sh port_mirror.conf.dst kernel_route_offload_flags.conf sx_netdev.route_offload_flags.conf.dst kernel_route_o
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.23.10_CSHN'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add kernel_route_offload_flags.conf.dst port_mirror.conf switchdctl.sh port_mirror.conf.dst kernel_route_offload_flags.conf sx_netdev.route_offload_flags.conf.dst kernel_route_o
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  <local> - - [23/May/2022 13:23:11] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.22.53_CSHM HTTP/1.1" 200 -
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.23.10_CSHN'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add kernel_route_offload_flags.conf.dst port_mirror.conf switchdctl.sh port_mirror.conf.dst kernel_route_offload_flags.conf sx_netdev.route_offload_flags.conf.dst kernel_route_o
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.23.10_CSHN'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/platform-detect'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /usr/share/cumulus-platform/common/platform-info/sugon.json
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /usr/share/cumulus-platform/common/platform-info/mellanox.json
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /proc/meminfo
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/lsblk --json'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /run/system_mac
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/cumulus/bin/decode-syseeprom --json'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  <local> - - [23/May/2022 13:23:11] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.22.53_CSHM HTTP/1.1" 200 -
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add kernel_route_offload_flags.conf.dst port_mirror.conf switchdctl.sh port_mirror.conf.dst kernel_route_offload_flags.conf sx_netdev.route_offload_flags.conf.dst kernel_route_o
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.23.10_CSHN'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add kernel_route_offload_flags.conf.dst port_mirror.conf switchdctl.sh port_mirror.conf.dst kernel_route_offload_flags.conf sx_netdev.route_offload_flags.conf.dst kernel_route_o
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.23.10_CSHN'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/platform-detect'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/platform-detect'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /usr/share/cumulus-platform/cumulus/vx/etc/cumulus/ports.conf
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/ports.conf.orig
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/ports.conf
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add kernel_route_offload_flags.conf.dst ports.conf.dst ports.conf port_mirror.conf switchdctl.sh port_mirror.conf.dst kernel_route_offload_flags.conf sx_netdev.route_offload_fla
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.23.10_CSHN'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  <local> - - [23/May/2022 13:23:11] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.22.53_CSHM HTTP/1.1" 200 -
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Schedule run of run_chassismgrctl.sh
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add kernel_route_offload_flags.conf.dst ports.conf.dst ports.conf run_chassismgrctl.sh port_mirror.conf switchdctl.sh port_mirror.conf.dst kernel_route_offload_flags.conf sx_net
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.23.10_CSHN'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: "/bin/systemctl list-units 'ntp@*' --no-legend"
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/ntp.conf.orig
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/ntp.conf
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add kernel_route_offload_flags.conf.dst ports.conf.dst ntp.conf ports.conf run_chassismgrctl.sh port_mirror.conf switchdctl.sh port_mirror.conf.dst kernel_route_offload_flags.co
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.23.10_CSHN'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/lldp-nvue.conf
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add kernel_route_offload_flags.conf.dst ports.conf.dst ntp.conf lldp-nvue.conf ports.conf run_chassismgrctl.sh port_mirror.conf switchdctl.sh port_mirror.conf.dst kernel_route_o
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.23.10_CSHN'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/platform-detect'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /usr/share/cumulus-platform/common/platform-info/sugon.json
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /usr/share/cumulus-platform/common/platform-info/mellanox.json
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /proc/meminfo
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/lsblk --json'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /run/system_mac
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/cumulus/bin/decode-syseeprom --json'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/ptp4l.conf.orig
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/ptp4l.conf
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add ports.conf run_chassismgrctl.sh kernel_route_offload_flags.conf kernel_route_offload_flags.conf.dst ntp.conf lldp-nvue.conf port_mirror.conf.dst sx_netdev.route_offload_flag
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.23.10_CSHN'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add ports.conf run_chassismgrctl.sh kernel_route_offload_flags.conf kernel_route_offload_flags.conf.dst ntp.conf lldp-nvue.conf port_mirror.conf.dst sx_netdev.route_offload_flag
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  <local> - - [23/May/2022 13:23:11] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.22.53_CSHM HTTP/1.1" 200 -
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.23.10_CSHN'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/platform-detect'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /usr/share/cumulus-platform/common/platform-info/sugon.json
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /usr/share/cumulus-platform/common/platform-info/mellanox.json
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /proc/meminfo
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/lsblk --json'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /run/system_mac
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/cumulus/bin/decode-syseeprom --json'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/eni.orig
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/eni
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add ports.conf run_chassismgrctl.sh eni kernel_route_offload_flags.conf kernel_route_offload_flags.conf.dst eni.orig ntp.conf lldp-nvue.conf port_mirror.conf.dst sx_netdev.route
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.23.10_CSHN'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/sudo -S stat /etc/frr/frr.conf'
May 23 13:23:11 8622341e6ec0d73d sudo[3775]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/stat /etc/frr/frr.conf
May 23 13:23:11 8622341e6ec0d73d sudo[3775]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:23:11 8622341e6ec0d73d sudo[3775]: pam_unix(sudo:session): session closed for user root
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/sudo -S cp /etc/frr/frr.conf /var/lib/nvue/config/frr.conf.orig'
May 23 13:23:11 8622341e6ec0d73d sudo[3788]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/bin/cp /etc/frr/frr.conf /var/lib/nvue/config/frr.conf.orig
May 23 13:23:11 8622341e6ec0d73d sudo[3788]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:23:11 8622341e6ec0d73d sudo[3788]: pam_unix(sudo:session): session closed for user root
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/sudo -S chown nvue. /var/lib/nvue/config/frr.conf.orig'
May 23 13:23:11 8622341e6ec0d73d sudo[3801]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/bin/chown nvue. /var/lib/nvue/config/frr.conf.orig
May 23 13:23:11 8622341e6ec0d73d sudo[3801]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:23:11 8622341e6ec0d73d sudo[3801]: pam_unix(sudo:session): session closed for user root
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/sudo -S chmod 0644 /var/lib/nvue/config/frr.conf.orig'
May 23 13:23:11 8622341e6ec0d73d sudo[3814]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/bin/chmod 0644 /var/lib/nvue/config/frr.conf.orig
May 23 13:23:11 8622341e6ec0d73d sudo[3814]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:23:11 8622341e6ec0d73d sudo[3814]: pam_unix(sudo:session): session closed for user root
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/frr.conf.orig
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/frr.conf
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/sudo -S stat /etc/frr/daemons'
May 23 13:23:11 8622341e6ec0d73d sudo[3827]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/stat /etc/frr/daemons
May 23 13:23:11 8622341e6ec0d73d sudo[3827]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:23:11 8622341e6ec0d73d sudo[3827]: pam_unix(sudo:session): session closed for user root
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/sudo -S cp /etc/frr/daemons /var/lib/nvue/config/frr-daemons.orig'
May 23 13:23:11 8622341e6ec0d73d sudo[3840]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/bin/cp /etc/frr/daemons /var/lib/nvue/config/frr-daemons.orig
May 23 13:23:11 8622341e6ec0d73d sudo[3840]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:23:11 8622341e6ec0d73d sudo[3840]: pam_unix(sudo:session): session closed for user root
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/sudo -S chown nvue. /var/lib/nvue/config/frr-daemons.orig'
May 23 13:23:11 8622341e6ec0d73d sudo[3853]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/bin/chown nvue. /var/lib/nvue/config/frr-daemons.orig
May 23 13:23:11 8622341e6ec0d73d sudo[3853]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:23:11 8622341e6ec0d73d sudo[3853]: pam_unix(sudo:session): session closed for user root
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/sudo -S chmod 0644 /var/lib/nvue/config/frr-daemons.orig'
May 23 13:23:11 8622341e6ec0d73d sudo[3866]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/bin/chmod 0644 /var/lib/nvue/config/frr-daemons.orig
May 23 13:23:11 8622341e6ec0d73d sudo[3866]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:23:11 8622341e6ec0d73d sudo[3866]: pam_unix(sudo:session): session closed for user root
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/frr-daemons.orig
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/frr-daemons
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Exec vtysh: 'show run'
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  Running: "/usr/bin/sudo -S vtysh -c 'show run'"
May 23 13:23:11 8622341e6ec0d73d sudo[3879]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/vtysh -c show run
May 23 13:23:11 8622341e6ec0d73d sudo[3879]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:23:11 8622341e6ec0d73d nvued[1999]:    INFO:  <local> - - [23/May/2022 13:23:11] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.22.53_CSHM HTTP/1.1" 200 -
May 23 13:23:12 8622341e6ec0d73d sudo[3879]: pam_unix(sudo:session): session closed for user root
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/vtysh_run.conf
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  vtysh_run.conf could not be loaded.  This could be the first time running frr_v1 lifecycle.
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add frr-daemons ports.conf run_chassismgrctl.sh eni kernel_route_offload_flags.conf frr-daemons.dst kernel_route_offload_flags.conf.dst eni.orig ntp.conf lldp-nvue.conf frr.conf
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.23.10_CSHN'
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Running 'hostnamectl'
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/sudo -S hostnamectl'
May 23 13:23:12 8622341e6ec0d73d sudo[3910]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/hostnamectl
May 23 13:23:12 8622341e6ec0d73d sudo[3910]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:23:12 8622341e6ec0d73d sudo[3910]: pam_unix(sudo:session): session closed for user root
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Running 'hostnamectl'
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/sudo -S hostnamectl'
May 23 13:23:12 8622341e6ec0d73d sudo[3924]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/hostnamectl
May 23 13:23:12 8622341e6ec0d73d sudo[3924]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:23:12 8622341e6ec0d73d sudo[3924]: pam_unix(sudo:session): session closed for user root
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Running 'hostnamectl'
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/sudo -S hostnamectl'
May 23 13:23:12 8622341e6ec0d73d sudo[3937]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/hostnamectl
May 23 13:23:12 8622341e6ec0d73d sudo[3937]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:23:12 8622341e6ec0d73d sudo[3937]: pam_unix(sudo:session): session closed for user root
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Running 'hostnamectl'
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/sudo -S hostnamectl'
May 23 13:23:12 8622341e6ec0d73d sudo[3950]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/hostnamectl
May 23 13:23:12 8622341e6ec0d73d sudo[3950]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:23:12 8622341e6ec0d73d sudo[3950]: pam_unix(sudo:session): session closed for user root
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/etc_hosts.orig
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/etc_hosts
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/etc_dhcp_exit_hook_hostname.orig
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/etc_dhcp_exit_hook_hostname
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/etc_hostname.orig
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /var/lib/nvue/config/etc_hostname
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Schedule run of apply_hostname.sh
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add frr-daemons ports.conf run_chassismgrctl.sh eni etc_dhcp_exit_hook_hostname.dst kernel_route_offload_flags.conf etc_hosts.dst frr-daemons.dst kernel_route_offload_flags.conf
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.23.10_CSHN'
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Loading /etc/resolv.conf
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add frr-daemons ports.conf run_chassismgrctl.sh eni etc_dhcp_exit_hook_hostname.dst kernel_route_offload_flags.conf etc_hosts.dst frr-daemons.dst kernel_route_offload_flags.conf
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:   ERROR:  Uncaught exception during ready_apply
                                              Traceback (most recent call last):
                                                File "/usr/lib/python3/dist-packages/cue_config_v1/apply_config.py", line 1416, in _run_ready_apply
                                                  ctx.events.ready_apply(evt)
                                                File "/usr/lib/python3/dist-packages/cue/events.py", line 153, in __call__
                                                  handler.fn(evt)
                                                File "/usr/lib/python3/dist-packages/cue_dns_v1/lifecycle.py", line 126, in handle_ready_apply
                                                  if not dns_server_configs_present and not is_resolv_conf_owned_nvued():
                                                File "/usr/lib/python3/dist-packages/cue_dns_v1/lifecycle.py", line 90, in is_resolv_conf_owned_nvued
                                                  resolv_conf = fileops.load_file("/etc/resolv.conf")
                                                File "/usr/lib/python3/dist-packages/cue/fileops.py", line 91, in load_file
                                                  with open_file(filename, "rt") as f:
                                                File "/usr/lib/python3/dist-packages/cue/fileops.py", line 52, in open_file
                                                  return open(filename, mode)
                                              FileNotFoundError: [Errno 2] No such file or directory: '/etc/resolv.conf'
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Applying changeset/root/2022-05-23_13.22.53_CSHM: ready_error (Failed to prepare to apply)
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Apply Issues: (Unrecoverable internal error)
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Running: '/usr/bin/git add changeset/root/2022-05-23_13.22.53_CSHM'
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Running: "/usr/bin/git commit --allow-empty -m 'Updating changeset/root/2022-05-23_13.22.53_CSHM'"
May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  Ran Job apply:changeset/root/2022-05-23_13.22.53_CSHM
Failed to prepare to apply
  Unrecoverable internal error
root@8622341e6ec0d73d:mgmt:~# May 23 13:23:12 8622341e6ec0d73d nvued[1999]:    INFO:  <local> - - [23/May/2022 13:23:12] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.22.53_CSHM HTTP/1.1" 200 -
networkop commented 2 years ago

what if you try to run nv set system config apply ignore /etc/resolv.conf before the apply.

jhlasnik commented 2 years ago

Looks like I'm still hitting the same exception.

root@095f1d318944f917:mgmt:~# journalctl -u nvued -f &
[1] 16975
root@095f1d318944f917:mgmt:~# -- Logs begin at Mon 2022-05-23 13:31:52 UTC. --
May 23 13:58:26 095f1d318944f917 nvued[1997]:    INFO:  Loading /etc/resolv.conf
May 23 13:58:26 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add port_mirror.conf.dst eni.orig frr.conf.dst etc_dhcp_exit_hook_hostname.orig frr-daemons.dst etc_hosts etc_hosts.dst ports.conf.orig ports.conf.dst etc_hosts.orig kernel_rout
May 23 13:58:26 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:58:26 095f1d318944f917 nvued[1997]:   ERROR:  Uncaught exception during ready_apply
                                              Traceback (most recent call last):
                                                File "/usr/lib/python3/dist-packages/cue_config_v1/apply_config.py", line 1416, in _run_ready_apply
                                                  ctx.events.ready_apply(evt)
                                                File "/usr/lib/python3/dist-packages/cue/events.py", line 153, in __call__
                                                  handler.fn(evt)
                                                File "/usr/lib/python3/dist-packages/cue_dns_v1/lifecycle.py", line 126, in handle_ready_apply
                                                  if not dns_server_configs_present and not is_resolv_conf_owned_nvued():
                                                File "/usr/lib/python3/dist-packages/cue_dns_v1/lifecycle.py", line 90, in is_resolv_conf_owned_nvued
                                                  resolv_conf = fileops.load_file("/etc/resolv.conf")
                                                File "/usr/lib/python3/dist-packages/cue/fileops.py", line 91, in load_file
                                                  with open_file(filename, "rt") as f:
                                                File "/usr/lib/python3/dist-packages/cue/fileops.py", line 52, in open_file
                                                  return open(filename, mode)
                                              FileNotFoundError: [Errno 2] No such file or directory: '/etc/resolv.conf'
May 23 13:58:26 095f1d318944f917 nvued[1997]:    INFO:  Applying changeset/root/2022-05-23_13.57.45_8P1Z: ready_error (Failed to prepare to apply)
May 23 13:58:26 095f1d318944f917 nvued[1997]:    INFO:  Apply Issues: (Unrecoverable internal error)
May 23 13:58:26 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add changeset/root/2022-05-23_13.57.45_8P1Z'
May 23 13:58:26 095f1d318944f917 nvued[1997]:    INFO:  Running: "/usr/bin/git commit --allow-empty -m 'Updating changeset/root/2022-05-23_13.57.45_8P1Z'"
May 23 13:58:26 095f1d318944f917 nvued[1997]:    INFO:  Ran Job apply:changeset/root/2022-05-23_13.57.45_8P1Z
May 23 13:58:27 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:58:27] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.57.45_8P1Z HTTP/1.1" 200 -

root@095f1d318944f917:mgmt:~# 
root@095f1d318944f917:mgmt:~# 
root@095f1d318944f917:mgmt:~# 
root@095f1d318944f917:mgmt:~# 
root@095f1d318944f917:mgmt:~# nv  config detach 
root@095f1d318944f917:mgmt:~# nv set system hostname clab-sw1
May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:20] "GET /nvue_v1/system/hostname/clab-sw1?rev=operational HTTP/1.1" 404 -
May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:20] "GET /nvue_v1/system/hostname?rev=operational HTTP/1.1" 404 -
May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:20] "GET /nvue_v1/system?rev=operational HTTP/1.1" 200 -
May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:20] "GET /nvue_v1/system?rev=applied HTTP/1.1" 200 -
root@095f1d318944f917:mgmt:~# May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:20] "GET /nvue_v1/system?pointers=%5B%22%2Fparameters%22%2C+%22%2Fpatch%2FrequestBody%2Fcontent%2Fapplication~1json%2Fschema%22%2C+%22%2Fpatch%2Fparameters%22%2C+%22%2Fpatch%2Fresponses%2F200%2Flinks%22%5D HTTP/1.1" 200 -
May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:20] "PATCH /nvue_v1/?rev=applied HTTP/1.1" 409 -
May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout applied'
May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  Creating pending revision: changeset/root/2022-05-23_13.59.20_8P21
May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout -b changeset/root/2022-05-23_13.59.20_8P21'
May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git cat-file -e changeset/root/2022-05-23_13.59.20_8P21'
May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add changeset/root/2022-05-23_13.59.20_8P21'
May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  Running: "/usr/bin/git commit --allow-empty -m 'Updating changeset/root/2022-05-23_13.59.20_8P21'"
May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:20] "POST /nvue_v1/revision HTTP/1.1" 200 -
May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout changeset/root/2022-05-23_13.59.20_8P21'
May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/nvue.json
May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout changeset/root/2022-05-23_13.59.20_8P21'
May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout changeset/root/2022-05-23_13.59.20_8P21'
May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout changeset/root/2022-05-23_13.59.20_8P21'
May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/nvue.json
May 23 13:59:20 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:20] "PATCH /nvue_v1/system?rev=changeset%2Froot%2F2022-05-23_13.59.20_8P21 HTTP/1.1" 200 -

root@095f1d318944f917:mgmt:~# nv set system config apply ignore /etc/resolv.conf
May 23 13:59:27 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:27] "GET /nvue_v1/system/config/apply/ignore/%2Fetc%2Fresolv.conf?rev=operational HTTP/1.1" 200 -
May 23 13:59:27 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:27] "GET /nvue_v1/system/config/apply/ignore/%2Fetc%2Fresolv.conf?rev=applied HTTP/1.1" 200 -
May 23 13:59:27 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:27] "GET /nvue_v1/system/config/apply/ignore/%2Fetc%2Fresolv.conf?rev=changeset%2Froot%2F2022-05-23_13.59.20_8P21 HTTP/1.1" 200 -
May 23 13:59:27 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:27] "GET /nvue_v1/system/config/apply/ignore/%2Fetc%2Fresolv.conf?pointers=%5B%22%2Fparameters%22%2C+%22%2Fpatch%2FrequestBody%2Fcontent%2Fapplication~1json%2Fschema%22%2C+%22%2Fpatch%2Fparameters%22%2C+%22%2Fpatch%2Fresponses%2F200%2Flinks%22%5D HTTP/1.1" 200 -
May 23 13:59:28 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout changeset/root/2022-05-23_13.59.20_8P21'
May 23 13:59:28 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:59:28 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout changeset/root/2022-05-23_13.59.20_8P21'
May 23 13:59:28 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout changeset/root/2022-05-23_13.59.20_8P21'
May 23 13:59:28 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/nvue.json
May 23 13:59:28 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:28] "PATCH /nvue_v1/?rev=changeset%2Froot%2F2022-05-23_13.59.20_8P21 HTTP/1.1" 200 -
root@095f1d318944f917:mgmt:~# May 23 13:59:28 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:28] "GET /nvue_v1/system/config/apply?pointers=%5B%22%2Fparameters%22%2C+%22%2Fpatch%2FrequestBody%2Fcontent%2Fapplication~1json%2Fschema%22%2C+%22%2Fpatch%2Fparameters%22%2C+%22%2Fpatch%2Fresponses%2F200%2Flinks%22%5D HTTP/1.1" 200 -
May 23 13:59:28 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout changeset/root/2022-05-23_13.59.20_8P21'
May 23 13:59:28 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:59:28 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout changeset/root/2022-05-23_13.59.20_8P21'
May 23 13:59:28 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout changeset/root/2022-05-23_13.59.20_8P21'
May 23 13:59:28 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/nvue.json
May 23 13:59:28 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:28] "PATCH /nvue_v1/system/config/apply?rev=changeset%2Froot%2F2022-05-23_13.59.20_8P21 HTTP/1.1" 200 -

root@095f1d318944f917:mgmt:~# nv config apply -y
verifyingMay 23 13:59:35 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:35] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.59.20_8P21 HTTP/1.1" 200 -
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add changeset/root/2022-05-23_13.59.20_8P21'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: "/usr/bin/git commit --allow-empty -m 'Updating changeset/root/2022-05-23_13.59.20_8P21'"
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Queued Job apply:changeset/root/2022-05-23_13.59.20_8P21
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:36] "PATCH /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.59.20_8P21 HTTP/1.1" 200 -
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running Job apply:changeset/root/2022-05-23_13.59.20_8P21
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Applying changeset/root/2022-05-23_13.59.20_8P21: verifying
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add changeset/root/2022-05-23_13.59.20_8P21'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: "/usr/bin/git commit --allow-empty -m 'Updating changeset/root/2022-05-23_13.59.20_8P21'"
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:36] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.59.20_8P21 HTTP/1.1" 200 -
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/platform-detect'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/platform-detect'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/platform-detect'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Loading /usr/share/cumulus-platform/cumulus/vx/etc/cumulus/ports.conf
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:36] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.59.20_8P21 HTTP/1.1" 200 -
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: "/bin/bash -c '/usr/cumulus/bin/validate-ports -f /tmp/ports.conf_20220523-135936'"
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:36] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.59.20_8P21 HTTP/1.1" 200 -
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Applying changeset/root/2022-05-23_13.59.20_8P21: verified (Verified config)
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add changeset/root/2022-05-23_13.59.20_8P21'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: "/usr/bin/git commit --allow-empty -m 'Updating changeset/root/2022-05-23_13.59.20_8P21'"
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Applying changeset/root/2022-05-23_13.59.20_8P21: readying
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add changeset/root/2022-05-23_13.59.20_8P21'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: "/usr/bin/git commit --allow-empty -m 'Updating changeset/root/2022-05-23_13.59.20_8P21'"
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout applied'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git tag --force apply/2022-05-23_13.59.36_8P22/start applied'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Creating pending revision: apply/2022-05-23_13.59.36_8P22
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout -b apply/2022-05-23_13.59.36_8P22'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add apply_log.txt reload.yml'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.59.36_8P22'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.59.36_8P22'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.59.36_8P22'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.59.36_8P22'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/kernel_route_offload_flags.conf.orig
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/kernel_route_offload_flags.conf
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/sx_netdev.route_offload_flags.conf
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Schedule run of set_sx_netdev.sh
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/port_mirror.conf
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Schedule run of switchdctl.sh
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add kernel_route_offload_flags.conf.dst port_mirror.conf.dst kernel_route_offload_flags.conf.orig kernel_route_offload_flags.conf sx_netdev.route_offload_flags.conf.dst sx_netde
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.59.36_8P22'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add kernel_route_offload_flags.conf.dst port_mirror.conf.dst kernel_route_offload_flags.conf.orig kernel_route_offload_flags.conf sx_netdev.route_offload_flags.conf.dst sx_netde
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.59.36_8P22'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add kernel_route_offload_flags.conf.dst port_mirror.conf.dst kernel_route_offload_flags.conf.orig kernel_route_offload_flags.conf sx_netdev.route_offload_flags.conf.dst sx_netde
readyingMay 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:36] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.59.20_8P21 HTTP/1.1" 200 -
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.59.36_8P22'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/platform-detect'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Loading /usr/share/cumulus-platform/common/platform-info/sugon.json
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Loading /usr/share/cumulus-platform/common/platform-info/mellanox.json
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Loading /proc/meminfo
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/lsblk --json'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Loading /run/system_mac
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/cumulus/bin/decode-syseeprom --json'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:36] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.59.20_8P21 HTTP/1.1" 200 -
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add kernel_route_offload_flags.conf.dst port_mirror.conf.dst kernel_route_offload_flags.conf.orig kernel_route_offload_flags.conf sx_netdev.route_offload_flags.conf.dst sx_netde
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.59.36_8P22'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add kernel_route_offload_flags.conf.dst port_mirror.conf.dst kernel_route_offload_flags.conf.orig kernel_route_offload_flags.conf sx_netdev.route_offload_flags.conf.dst sx_netde
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.59.36_8P22'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/platform-detect'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/platform-detect'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Loading /usr/share/cumulus-platform/cumulus/vx/etc/cumulus/ports.conf
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/ports.conf.orig
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/ports.conf
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add kernel_route_offload_flags.conf.dst port_mirror.conf.dst kernel_route_offload_flags.conf.orig kernel_route_offload_flags.conf sx_netdev.route_offload_flags.conf.dst sx_netde
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.59.36_8P22'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:36] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.59.20_8P21 HTTP/1.1" 200 -
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Schedule run of run_chassismgrctl.sh
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add kernel_route_offload_flags.conf.dst port_mirror.conf.dst kernel_route_offload_flags.conf.orig kernel_route_offload_flags.conf sx_netdev.route_offload_flags.conf.dst sx_netde
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.59.36_8P22'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: "/bin/systemctl list-units 'ntp@*' --no-legend"
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/ntp.conf.orig
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/ntp.conf
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add kernel_route_offload_flags.conf.dst port_mirror.conf.dst kernel_route_offload_flags.conf.orig kernel_route_offload_flags.conf ntp.conf.dst ntp.conf sx_netdev.route_offload_f
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.59.36_8P22'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/lldp-nvue.conf
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add kernel_route_offload_flags.conf.dst port_mirror.conf.dst lldp-nvue.conf kernel_route_offload_flags.conf.orig kernel_route_offload_flags.conf ntp.conf.dst ntp.conf sx_netdev.
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:59:36 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.59.36_8P22'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/platform-detect'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /usr/share/cumulus-platform/common/platform-info/sugon.json
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /usr/share/cumulus-platform/common/platform-info/mellanox.json
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /proc/meminfo
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/lsblk --json'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /run/system_mac
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/cumulus/bin/decode-syseeprom --json'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/ptp4l.conf.orig
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/ptp4l.conf
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add port_mirror.conf.dst ports.conf.orig ports.conf.dst kernel_route_offload_flags.conf.orig ptp4l.conf ntp.conf.orig sx_netdev.route_offload_flags.conf ptp4l.conf.orig lldp-nvu
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.59.36_8P22'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:37] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.59.20_8P21 HTTP/1.1" 200 -
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add port_mirror.conf.dst ports.conf.orig ports.conf.dst kernel_route_offload_flags.conf.orig ptp4l.conf ntp.conf.orig sx_netdev.route_offload_flags.conf ptp4l.conf.orig lldp-nvu
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.59.36_8P22'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/platform-detect'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /usr/share/cumulus-platform/common/platform-info/sugon.json
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /usr/share/cumulus-platform/common/platform-info/mellanox.json
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /proc/meminfo
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/lsblk --json'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /run/system_mac
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/cumulus/bin/decode-syseeprom --json'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/eni.orig
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/eni
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add port_mirror.conf.dst eni.orig ports.conf.orig ports.conf.dst kernel_route_offload_flags.conf.orig ptp4l.conf ntp.conf.orig sx_netdev.route_offload_flags.conf ptp4l.conf.orig
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.59.36_8P22'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/sudo -S stat /etc/frr/frr.conf'
May 23 13:59:37 095f1d318944f917 sudo[17926]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/stat /etc/frr/frr.conf
May 23 13:59:37 095f1d318944f917 sudo[17926]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:59:37 095f1d318944f917 sudo[17926]: pam_unix(sudo:session): session closed for user root
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/sudo -S cp /etc/frr/frr.conf /var/lib/nvue/config/frr.conf.orig'
May 23 13:59:37 095f1d318944f917 sudo[17939]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/bin/cp /etc/frr/frr.conf /var/lib/nvue/config/frr.conf.orig
May 23 13:59:37 095f1d318944f917 sudo[17939]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:59:37 095f1d318944f917 sudo[17939]: pam_unix(sudo:session): session closed for user root
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/sudo -S chown nvue. /var/lib/nvue/config/frr.conf.orig'
May 23 13:59:37 095f1d318944f917 sudo[17952]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/bin/chown nvue. /var/lib/nvue/config/frr.conf.orig
May 23 13:59:37 095f1d318944f917 sudo[17952]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:59:37 095f1d318944f917 sudo[17952]: pam_unix(sudo:session): session closed for user root
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/sudo -S chmod 0644 /var/lib/nvue/config/frr.conf.orig'
May 23 13:59:37 095f1d318944f917 sudo[17965]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/bin/chmod 0644 /var/lib/nvue/config/frr.conf.orig
May 23 13:59:37 095f1d318944f917 sudo[17965]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:59:37 095f1d318944f917 sudo[17965]: pam_unix(sudo:session): session closed for user root
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/frr.conf.orig
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/frr.conf
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/sudo -S stat /etc/frr/daemons'
May 23 13:59:37 095f1d318944f917 sudo[17978]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/stat /etc/frr/daemons
May 23 13:59:37 095f1d318944f917 sudo[17978]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:59:37 095f1d318944f917 sudo[17978]: pam_unix(sudo:session): session closed for user root
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/sudo -S cp /etc/frr/daemons /var/lib/nvue/config/frr-daemons.orig'
May 23 13:59:37 095f1d318944f917 sudo[17991]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/bin/cp /etc/frr/daemons /var/lib/nvue/config/frr-daemons.orig
May 23 13:59:37 095f1d318944f917 sudo[17991]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:59:37 095f1d318944f917 sudo[17991]: pam_unix(sudo:session): session closed for user root
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/sudo -S chown nvue. /var/lib/nvue/config/frr-daemons.orig'
May 23 13:59:37 095f1d318944f917 sudo[18004]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/bin/chown nvue. /var/lib/nvue/config/frr-daemons.orig
May 23 13:59:37 095f1d318944f917 sudo[18004]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:59:37 095f1d318944f917 sudo[18004]: pam_unix(sudo:session): session closed for user root
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/sudo -S chmod 0644 /var/lib/nvue/config/frr-daemons.orig'
May 23 13:59:37 095f1d318944f917 sudo[18017]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/bin/chmod 0644 /var/lib/nvue/config/frr-daemons.orig
May 23 13:59:37 095f1d318944f917 sudo[18017]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:59:37 095f1d318944f917 sudo[18017]: pam_unix(sudo:session): session closed for user root
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/frr-daemons.orig
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/frr-daemons
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Exec vtysh: 'show run'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: "/usr/bin/sudo -S vtysh -c 'show run'"
May 23 13:59:37 095f1d318944f917 sudo[18030]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/vtysh -c show run
May 23 13:59:37 095f1d318944f917 sudo[18030]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:37] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.59.20_8P21 HTTP/1.1" 200 -
May 23 13:59:37 095f1d318944f917 sudo[18030]: pam_unix(sudo:session): session closed for user root
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/vtysh_run.conf
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  vtysh_run.conf could not be loaded.  This could be the first time running frr_v1 lifecycle.
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add port_mirror.conf.dst eni.orig frr.conf.dst frr-daemons.dst ports.conf.orig ports.conf.dst kernel_route_offload_flags.conf.orig ptp4l.conf ntp.conf.orig sx_netdev.route_offlo
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.59.36_8P22'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running 'hostnamectl'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/sudo -S hostnamectl'
May 23 13:59:37 095f1d318944f917 sudo[18061]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/hostnamectl
May 23 13:59:37 095f1d318944f917 sudo[18061]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:59:37 095f1d318944f917 sudo[18061]: pam_unix(sudo:session): session closed for user root
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running 'hostnamectl'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/sudo -S hostnamectl'
May 23 13:59:37 095f1d318944f917 sudo[18075]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/hostnamectl
May 23 13:59:37 095f1d318944f917 sudo[18075]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:59:37 095f1d318944f917 sudo[18075]: pam_unix(sudo:session): session closed for user root
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running 'hostnamectl'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/sudo -S hostnamectl'
May 23 13:59:37 095f1d318944f917 sudo[18088]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/hostnamectl
May 23 13:59:37 095f1d318944f917 sudo[18088]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:59:37 095f1d318944f917 sudo[18088]: pam_unix(sudo:session): session closed for user root
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running 'hostnamectl'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/sudo -S hostnamectl'
May 23 13:59:37 095f1d318944f917 sudo[18101]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/hostnamectl
May 23 13:59:37 095f1d318944f917 sudo[18101]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 23 13:59:37 095f1d318944f917 sudo[18101]: pam_unix(sudo:session): session closed for user root
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/etc_hosts.orig
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/etc_hosts
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/etc_dhcp_exit_hook_hostname.orig
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/etc_dhcp_exit_hook_hostname
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/etc_hostname.orig
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /var/lib/nvue/config/etc_hostname
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Schedule run of apply_hostname.sh
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add port_mirror.conf.dst eni.orig frr.conf.dst etc_dhcp_exit_hook_hostname.orig frr-daemons.dst etc_hosts etc_hosts.dst ports.conf.orig ports.conf.dst etc_hosts.orig kernel_rout
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-23_13.59.36_8P22'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Loading /etc/resolv.conf
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add port_mirror.conf.dst eni.orig frr.conf.dst etc_dhcp_exit_hook_hostname.orig frr-daemons.dst etc_hosts etc_hosts.dst ports.conf.orig ports.conf.dst etc_hosts.orig kernel_rout
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 23 13:59:37 095f1d318944f917 nvued[1997]:   ERROR:  Uncaught exception during ready_apply
                                              Traceback (most recent call last):
                                                File "/usr/lib/python3/dist-packages/cue_config_v1/apply_config.py", line 1416, in _run_ready_apply
                                                  ctx.events.ready_apply(evt)
                                                File "/usr/lib/python3/dist-packages/cue/events.py", line 153, in __call__
                                                  handler.fn(evt)
                                                File "/usr/lib/python3/dist-packages/cue_dns_v1/lifecycle.py", line 126, in handle_ready_apply
                                                  if not dns_server_configs_present and not is_resolv_conf_owned_nvued():
                                                File "/usr/lib/python3/dist-packages/cue_dns_v1/lifecycle.py", line 90, in is_resolv_conf_owned_nvued
                                                  resolv_conf = fileops.load_file("/etc/resolv.conf")
                                                File "/usr/lib/python3/dist-packages/cue/fileops.py", line 91, in load_file
                                                  with open_file(filename, "rt") as f:
                                                File "/usr/lib/python3/dist-packages/cue/fileops.py", line 52, in open_file
                                                  return open(filename, mode)
                                              FileNotFoundError: [Errno 2] No such file or directory: '/etc/resolv.conf'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Applying changeset/root/2022-05-23_13.59.20_8P21: ready_error (Failed to prepare to apply)
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Apply Issues: (Unrecoverable internal error)
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: '/usr/bin/git add changeset/root/2022-05-23_13.59.20_8P21'
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Running: "/usr/bin/git commit --allow-empty -m 'Updating changeset/root/2022-05-23_13.59.20_8P21'"
May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  Ran Job apply:changeset/root/2022-05-23_13.59.20_8P21
Failed to prepare to apply
  Unrecoverable internal error
root@095f1d318944f917:mgmt:~# May 23 13:59:37 095f1d318944f917 nvued[1997]:    INFO:  <local> - - [23/May/2022 13:59:37] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-23_13.59.20_8P21 HTTP/1.1" 200 -

root@095f1d318944f917:mgmt:~#
networkop commented 2 years ago

hm... ok. I'll try to build a test lab tomorrow and see what can be done.

networkop commented 2 years ago

I was able to reproduce this. This is due to a bug in ignite implementation where it tries to symlink /etc/resolv.conf to a non-existent /proc/net/pnp file, assuming that it exists but not actually checking it. Ther workaround is to just rm /etc/resolv.conf before doing any nv config apply command. The right way is to update ignite to check if the proc/net/pnp file exists before creating a symlink. https://github.com/weaveworks/ignite/blob/cc6fa4c9aba514e6d6e1030e3c35344f90061715/pkg/dmlegacy/image_format.go#L102

jhlasnik commented 2 years ago

So I tried that with and without the ignore still somehow seeing the same error.

root@dc2e06edfcaaad47:mgmt:~# ls -al /etc/resolv.conf 
lrwxrwxrwx 1 root root 15 May 22 17:12 /etc/resolv.conf -> ../proc/net/pnp
root@dc2e06edfcaaad47:mgmt:~# rm /etc/resolv.conf 
root@dc2e06edfcaaad47:mgmt:~# nv set system 
config         control-plane  global         hostname       message        port-mirror    timezone       
root@dc2e06edfcaaad47:mgmt:~# nv set system hostname clab-sw1
root@dc2e06edfcaaad47:mgmt:~# nv config apply -y
Failed to prepare to apply
  Unrecoverable internal error
root@dc2e06edfcaaad47:mgmt:~# ls -al /etc/resolv.conf 
ls: cannot access '/etc/resolv.conf': No such file or directory
root@dc2e06edfcaaad47:mgmt:~# journalctl -u nvued -f &
[1] 9261
root@dc2e06edfcaaad47:mgmt:~# -- Logs begin at Wed 2022-05-25 19:26:31 UTC. --
May 25 19:38:05 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /etc/resolv.conf
May 25 19:38:05 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add ports.conf.dst port_mirror.conf.dst run_chassismgrctl.sh ntp.conf.orig etc_hostname.orig vtysh_run.conf etc_dhcp_exit_hook_hostname eni.orig ptp4l.conf.orig ports.conf frr-d
May 25 19:38:05 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 25 19:38:05 dc2e06edfcaaad47 nvued[1995]:   ERROR:  Uncaught exception during ready_apply
                                              Traceback (most recent call last):
                                                File "/usr/lib/python3/dist-packages/cue_config_v1/apply_config.py", line 1416, in _run_ready_apply
                                                  ctx.events.ready_apply(evt)
                                                File "/usr/lib/python3/dist-packages/cue/events.py", line 153, in __call__
                                                  handler.fn(evt)
                                                File "/usr/lib/python3/dist-packages/cue_dns_v1/lifecycle.py", line 126, in handle_ready_apply
                                                  if not dns_server_configs_present and not is_resolv_conf_owned_nvued():
                                                File "/usr/lib/python3/dist-packages/cue_dns_v1/lifecycle.py", line 90, in is_resolv_conf_owned_nvued
                                                  resolv_conf = fileops.load_file("/etc/resolv.conf")
                                                File "/usr/lib/python3/dist-packages/cue/fileops.py", line 91, in load_file
                                                  with open_file(filename, "rt") as f:
                                                File "/usr/lib/python3/dist-packages/cue/fileops.py", line 52, in open_file
                                                  return open(filename, mode)
                                              FileNotFoundError: [Errno 2] No such file or directory: '/etc/resolv.conf'
May 25 19:38:05 dc2e06edfcaaad47 nvued[1995]:    INFO:  Applying changeset/root/2022-05-25_19.37.58_S2PG: ready_error (Failed to prepare to apply)
May 25 19:38:05 dc2e06edfcaaad47 nvued[1995]:    INFO:  Apply Issues: (Unrecoverable internal error)
May 25 19:38:05 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add changeset/root/2022-05-25_19.37.58_S2PG'
May 25 19:38:05 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: "/usr/bin/git commit --allow-empty -m 'Updating changeset/root/2022-05-25_19.37.58_S2PG'"
May 25 19:38:05 dc2e06edfcaaad47 nvued[1995]:    INFO:  Ran Job apply:changeset/root/2022-05-25_19.37.58_S2PG
May 25 19:38:05 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:38:05] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-25_19.37.58_S2PG HTTP/1.1" 200 -

root@dc2e06edfcaaad47:mgmt:~# nv set system config apply ignore /etc/resolv.conf
May 25 19:39:33 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:39:33] "GET /nvue_v1/system/config/apply/ignore/%2Fetc%2Fresolv.conf?rev=operational HTTP/1.1" 200 -
May 25 19:39:33 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:39:33] "GET /nvue_v1/system/config/apply/ignore/%2Fetc%2Fresolv.conf?rev=applied HTTP/1.1" 200 -
May 25 19:39:33 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:39:33] "GET /nvue_v1/system/config/apply/ignore/%2Fetc%2Fresolv.conf?rev=changeset%2Froot%2F2022-05-25_19.37.58_S2PG HTTP/1.1" 200 -
May 25 19:39:34 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:39:34] "GET /nvue_v1/system/config/apply/ignore/%2Fetc%2Fresolv.conf?pointers=%5B%22%2Fparameters%22%2C+%22%2Fpatch%2FrequestBody%2Fcontent%2Fapplication~1json%2Fschema%22%2C+%22%2Fpatch%2Fparameters%22%2C+%22%2Fpatch%2Fresponses%2F200%2Flinks%22%5D HTTP/1.1" 200 -
May 25 19:39:34 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add changeset/root/2022-05-25_19.37.58_S2PG'
May 25 19:39:34 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: "/usr/bin/git commit --allow-empty -m 'Updating changeset/root/2022-05-25_19.37.58_S2PG'"
May 25 19:39:34 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout changeset/root/2022-05-25_19.37.58_S2PG'
May 25 19:39:34 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 25 19:39:34 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout changeset/root/2022-05-25_19.37.58_S2PG'
May 25 19:39:34 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout changeset/root/2022-05-25_19.37.58_S2PG'
May 25 19:39:34 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/nvue.json
May 25 19:39:34 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:39:34] "PATCH /nvue_v1/?rev=changeset%2Froot%2F2022-05-25_19.37.58_S2PG HTTP/1.1" 200 -
root@dc2e06edfcaaad47:mgmt:~# May 25 19:39:34 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:39:34] "GET /nvue_v1/system/config/apply?pointers=%5B%22%2Fparameters%22%2C+%22%2Fpatch%2FrequestBody%2Fcontent%2Fapplication~1json%2Fschema%22%2C+%22%2Fpatch%2Fparameters%22%2C+%22%2Fpatch%2Fresponses%2F200%2Flinks%22%5D HTTP/1.1" 200 -
May 25 19:39:34 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout changeset/root/2022-05-25_19.37.58_S2PG'
May 25 19:39:34 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 25 19:39:34 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout changeset/root/2022-05-25_19.37.58_S2PG'
May 25 19:39:34 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout changeset/root/2022-05-25_19.37.58_S2PG'
May 25 19:39:34 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/nvue.json
May 25 19:39:34 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:39:34] "PATCH /nvue_v1/system/config/apply?rev=changeset%2Froot%2F2022-05-25_19.37.58_S2PG HTTP/1.1" 200 -

root@dc2e06edfcaaad47:mgmt:~# 
root@dc2e06edfcaaad47:mgmt:~# nv config diff 
- set:
    system:
      config:
        apply:
          ignore:
            /etc/resolv.conf: {}
      hostname: clab-sw1
root@dc2e06edfcaaad47:mgmt:~# May 25 19:39:41 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:39:41] "GET /config_v1/rev/changeset%2Froot%2F2022-05-25_19.37.58_S2PG?filled=false HTTP/1.1" 200 -
May 25 19:39:41 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout applied'
May 25 19:39:41 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/nvue.json
May 25 19:39:41 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:39:41] "GET /config_v1/rev/applied?filled=false HTTP/1.1" 200 -

root@dc2e06edfcaaad47:mgmt:~# nv config apply -y
verifyingMay 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:39:50] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-25_19.37.58_S2PG HTTP/1.1" 200 -
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add changeset/root/2022-05-25_19.37.58_S2PG'
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: "/usr/bin/git commit --allow-empty -m 'Updating changeset/root/2022-05-25_19.37.58_S2PG'"
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Queued Job apply:changeset/root/2022-05-25_19.37.58_S2PG
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:39:50] "PATCH /nvue_v1/revision/changeset%2Froot%2F2022-05-25_19.37.58_S2PG HTTP/1.1" 200 -
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running Job apply:changeset/root/2022-05-25_19.37.58_S2PG
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Applying changeset/root/2022-05-25_19.37.58_S2PG: verifying
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add changeset/root/2022-05-25_19.37.58_S2PG'
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: "/usr/bin/git commit --allow-empty -m 'Updating changeset/root/2022-05-25_19.37.58_S2PG'"
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:39:50] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-25_19.37.58_S2PG HTTP/1.1" 200 -
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/platform-detect'
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/platform-detect'
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/platform-detect'
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /usr/share/cumulus-platform/cumulus/vx/etc/cumulus/ports.conf
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:39:50] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-25_19.37.58_S2PG HTTP/1.1" 200 -
readyingMay 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: "/bin/bash -c '/usr/cumulus/bin/validate-ports -f /tmp/ports.conf_20220525-193950'"
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:39:50] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-25_19.37.58_S2PG HTTP/1.1" 200 -
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Applying changeset/root/2022-05-25_19.37.58_S2PG: verified (Verified config)
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add changeset/root/2022-05-25_19.37.58_S2PG'
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: "/usr/bin/git commit --allow-empty -m 'Updating changeset/root/2022-05-25_19.37.58_S2PG'"
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Applying changeset/root/2022-05-25_19.37.58_S2PG: readying
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add changeset/root/2022-05-25_19.37.58_S2PG'
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: "/usr/bin/git commit --allow-empty -m 'Updating changeset/root/2022-05-25_19.37.58_S2PG'"
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout applied'
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git tag --force apply/2022-05-25_19.39.50_S2PJ/start applied'
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Creating pending revision: apply/2022-05-25_19.39.50_S2PJ
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout -b apply/2022-05-25_19.39.50_S2PJ'
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add reload.yml apply_log.txt'
May 25 19:39:50 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-25_19.39.50_S2PJ'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-25_19.39.50_S2PJ'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-25_19.39.50_S2PJ'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-25_19.39.50_S2PJ'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/kernel_route_offload_flags.conf.orig
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/kernel_route_offload_flags.conf
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/sx_netdev.route_offload_flags.conf
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Schedule run of set_sx_netdev.sh
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/port_mirror.conf
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Schedule run of switchdctl.sh
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add sx_netdev.route_offload_flags.conf.dst port_mirror.conf.dst port_mirror.conf kernel_route_offload_flags.conf.dst kernel_route_offload_flags.conf.orig set_sx_netdev.sh kernel
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-25_19.39.50_S2PJ'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add sx_netdev.route_offload_flags.conf.dst port_mirror.conf.dst port_mirror.conf kernel_route_offload_flags.conf.dst kernel_route_offload_flags.conf.orig set_sx_netdev.sh kernel
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-25_19.39.50_S2PJ'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:39:51] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-25_19.37.58_S2PG HTTP/1.1" 200 -
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add sx_netdev.route_offload_flags.conf.dst port_mirror.conf.dst port_mirror.conf kernel_route_offload_flags.conf.dst kernel_route_offload_flags.conf.orig set_sx_netdev.sh kernel
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-25_19.39.50_S2PJ'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/platform-detect'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /usr/share/cumulus-platform/common/platform-info/sugon.json
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /usr/share/cumulus-platform/common/platform-info/mellanox.json
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /proc/meminfo
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/lsblk --json'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /run/system_mac
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/cumulus/bin/decode-syseeprom --json'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:39:51] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-25_19.37.58_S2PG HTTP/1.1" 200 -
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add sx_netdev.route_offload_flags.conf.dst port_mirror.conf.dst port_mirror.conf kernel_route_offload_flags.conf.dst kernel_route_offload_flags.conf.orig set_sx_netdev.sh kernel
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-25_19.39.50_S2PJ'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add sx_netdev.route_offload_flags.conf.dst port_mirror.conf.dst port_mirror.conf kernel_route_offload_flags.conf.dst kernel_route_offload_flags.conf.orig set_sx_netdev.sh kernel
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-25_19.39.50_S2PJ'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/platform-detect'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/platform-detect'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /usr/share/cumulus-platform/cumulus/vx/etc/cumulus/ports.conf
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/ports.conf.orig
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/ports.conf
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add sx_netdev.route_offload_flags.conf.dst ports.conf.orig ports.conf ports.conf.dst port_mirror.conf.dst port_mirror.conf kernel_route_offload_flags.conf.dst kernel_route_offlo
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:39:51] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-25_19.37.58_S2PG HTTP/1.1" 200 -
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-25_19.39.50_S2PJ'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Schedule run of run_chassismgrctl.sh
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add sx_netdev.route_offload_flags.conf.dst ports.conf.orig ports.conf ports.conf.dst port_mirror.conf.dst run_chassismgrctl.sh port_mirror.conf kernel_route_offload_flags.conf.d
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-25_19.39.50_S2PJ'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: "/bin/systemctl list-units 'ntp@*' --no-legend"
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/ntp.conf.orig
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/ntp.conf
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add sx_netdev.route_offload_flags.conf.dst ports.conf.orig ports.conf ports.conf.dst port_mirror.conf.dst run_chassismgrctl.sh ntp.conf.orig port_mirror.conf kernel_route_offloa
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-25_19.39.50_S2PJ'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/lldp-nvue.conf
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add sx_netdev.route_offload_flags.conf.dst ports.conf.orig ports.conf ports.conf.dst port_mirror.conf.dst run_chassismgrctl.sh lldp-nvue.conf ntp.conf.orig port_mirror.conf lldp
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-25_19.39.50_S2PJ'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/platform-detect'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /usr/share/cumulus-platform/common/platform-info/sugon.json
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /usr/share/cumulus-platform/common/platform-info/mellanox.json
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /proc/meminfo
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/lsblk --json'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /run/system_mac
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/cumulus/bin/decode-syseeprom --json'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/ptp4l.conf.orig
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/ptp4l.conf
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add ports.conf.dst port_mirror.conf.dst run_chassismgrctl.sh ntp.conf.orig ptp4l.conf.orig ports.conf lldp-nvue.conf ntp.conf lldp-nvue.conf.dst ptp4l.conf.dst kernel_route_offl
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:39:51] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-25_19.37.58_S2PG HTTP/1.1" 200 -
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-25_19.39.50_S2PJ'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add ports.conf.dst port_mirror.conf.dst run_chassismgrctl.sh ntp.conf.orig ptp4l.conf.orig ports.conf lldp-nvue.conf ntp.conf lldp-nvue.conf.dst ptp4l.conf.dst kernel_route_offl
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-25_19.39.50_S2PJ'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/platform-detect'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /usr/share/cumulus-platform/common/platform-info/sugon.json
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /usr/share/cumulus-platform/common/platform-info/mellanox.json
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /proc/meminfo
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/lsblk --json'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /run/system_mac
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/cumulus/bin/decode-syseeprom --json'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/eni.orig
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/eni
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add ports.conf.dst port_mirror.conf.dst run_chassismgrctl.sh ntp.conf.orig eni.orig ptp4l.conf.orig ports.conf lldp-nvue.conf ntp.conf lldp-nvue.conf.dst ptp4l.conf.dst kernel_r
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-25_19.39.50_S2PJ'
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/sudo -S stat /etc/frr/frr.conf'
May 25 19:39:51 dc2e06edfcaaad47 sudo[10114]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/stat /etc/frr/frr.conf
May 25 19:39:51 dc2e06edfcaaad47 sudo[10114]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 25 19:39:51 dc2e06edfcaaad47 sudo[10114]: pam_unix(sudo:session): session closed for user root
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/sudo -S cp /etc/frr/frr.conf /var/lib/nvue/config/frr.conf.orig'
May 25 19:39:51 dc2e06edfcaaad47 sudo[10127]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/bin/cp /etc/frr/frr.conf /var/lib/nvue/config/frr.conf.orig
May 25 19:39:51 dc2e06edfcaaad47 sudo[10127]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 25 19:39:51 dc2e06edfcaaad47 sudo[10127]: pam_unix(sudo:session): session closed for user root
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/sudo -S chown nvue. /var/lib/nvue/config/frr.conf.orig'
May 25 19:39:51 dc2e06edfcaaad47 sudo[10140]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/bin/chown nvue. /var/lib/nvue/config/frr.conf.orig
May 25 19:39:51 dc2e06edfcaaad47 sudo[10140]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 25 19:39:51 dc2e06edfcaaad47 sudo[10140]: pam_unix(sudo:session): session closed for user root
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/sudo -S chmod 0644 /var/lib/nvue/config/frr.conf.orig'
May 25 19:39:51 dc2e06edfcaaad47 sudo[10153]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/bin/chmod 0644 /var/lib/nvue/config/frr.conf.orig
May 25 19:39:51 dc2e06edfcaaad47 sudo[10153]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 25 19:39:51 dc2e06edfcaaad47 sudo[10153]: pam_unix(sudo:session): session closed for user root
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/frr.conf.orig
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/frr.conf
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/sudo -S stat /etc/frr/daemons'
May 25 19:39:51 dc2e06edfcaaad47 sudo[10166]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/stat /etc/frr/daemons
May 25 19:39:51 dc2e06edfcaaad47 sudo[10166]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 25 19:39:51 dc2e06edfcaaad47 sudo[10166]: pam_unix(sudo:session): session closed for user root
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/sudo -S cp /etc/frr/daemons /var/lib/nvue/config/frr-daemons.orig'
May 25 19:39:51 dc2e06edfcaaad47 sudo[10179]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/bin/cp /etc/frr/daemons /var/lib/nvue/config/frr-daemons.orig
May 25 19:39:51 dc2e06edfcaaad47 sudo[10179]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 25 19:39:51 dc2e06edfcaaad47 sudo[10179]: pam_unix(sudo:session): session closed for user root
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/sudo -S chown nvue. /var/lib/nvue/config/frr-daemons.orig'
May 25 19:39:51 dc2e06edfcaaad47 sudo[10192]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/bin/chown nvue. /var/lib/nvue/config/frr-daemons.orig
May 25 19:39:51 dc2e06edfcaaad47 sudo[10192]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 25 19:39:51 dc2e06edfcaaad47 sudo[10192]: pam_unix(sudo:session): session closed for user root
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/sudo -S chmod 0644 /var/lib/nvue/config/frr-daemons.orig'
May 25 19:39:51 dc2e06edfcaaad47 sudo[10205]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/bin/chmod 0644 /var/lib/nvue/config/frr-daemons.orig
May 25 19:39:51 dc2e06edfcaaad47 sudo[10205]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 25 19:39:51 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:39:51] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-25_19.37.58_S2PG HTTP/1.1" 200 -
May 25 19:39:51 dc2e06edfcaaad47 sudo[10205]: pam_unix(sudo:session): session closed for user root
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/frr-daemons.orig
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/frr-daemons
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Exec vtysh: 'show run'
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: "/usr/bin/sudo -S vtysh -c 'show run'"
May 25 19:39:52 dc2e06edfcaaad47 sudo[10222]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/vtysh -c show run
May 25 19:39:52 dc2e06edfcaaad47 sudo[10222]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 25 19:39:52 dc2e06edfcaaad47 sudo[10222]: pam_unix(sudo:session): session closed for user root
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/vtysh_run.conf
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  vtysh_run.conf could not be loaded.  This could be the first time running frr_v1 lifecycle.
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add ports.conf.dst port_mirror.conf.dst run_chassismgrctl.sh ntp.conf.orig vtysh_run.conf eni.orig ptp4l.conf.orig ports.conf frr-daemons.orig lldp-nvue.conf ntp.conf lldp-nvue.
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-25_19.39.50_S2PJ'
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running 'hostnamectl'
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/sudo -S hostnamectl'
May 25 19:39:52 dc2e06edfcaaad47 sudo[10249]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/hostnamectl
May 25 19:39:52 dc2e06edfcaaad47 sudo[10249]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 25 19:39:52 dc2e06edfcaaad47 sudo[10249]: pam_unix(sudo:session): session closed for user root
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running 'hostnamectl'
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/sudo -S hostnamectl'
May 25 19:39:52 dc2e06edfcaaad47 sudo[10263]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/hostnamectl
May 25 19:39:52 dc2e06edfcaaad47 sudo[10263]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 25 19:39:52 dc2e06edfcaaad47 sudo[10263]: pam_unix(sudo:session): session closed for user root
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running 'hostnamectl'
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/sudo -S hostnamectl'
May 25 19:39:52 dc2e06edfcaaad47 sudo[10276]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/hostnamectl
May 25 19:39:52 dc2e06edfcaaad47 sudo[10276]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 25 19:39:52 dc2e06edfcaaad47 sudo[10276]: pam_unix(sudo:session): session closed for user root
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running 'hostnamectl'
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/sudo -S hostnamectl'
May 25 19:39:52 dc2e06edfcaaad47 sudo[10289]:     nvue : TTY=unknown ; PWD=/var/lib/nvue ; USER=root ; COMMAND=/usr/bin/hostnamectl
May 25 19:39:52 dc2e06edfcaaad47 sudo[10289]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 25 19:39:52 dc2e06edfcaaad47 sudo[10289]: pam_unix(sudo:session): session closed for user root
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/etc_hosts.orig
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/etc_hosts
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/etc_dhcp_exit_hook_hostname.orig
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/etc_dhcp_exit_hook_hostname
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/etc_hostname.orig
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /var/lib/nvue/config/etc_hostname
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Schedule run of apply_hostname.sh
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add ports.conf.dst port_mirror.conf.dst run_chassismgrctl.sh ntp.conf.orig etc_hostname.orig vtysh_run.conf etc_dhcp_exit_hook_hostname eni.orig ptp4l.conf.orig ports.conf frr-d
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git checkout apply/2022-05-25_19.39.50_S2PJ'
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Loading /etc/resolv.conf
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add ports.conf.dst port_mirror.conf.dst run_chassismgrctl.sh ntp.conf.orig etc_hostname.orig vtysh_run.conf etc_dhcp_exit_hook_hostname eni.orig ptp4l.conf.orig ports.conf frr-d
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git commit -a --allow-empty -F -'
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:   ERROR:  Uncaught exception during ready_apply
                                              Traceback (most recent call last):
                                                File "/usr/lib/python3/dist-packages/cue_config_v1/apply_config.py", line 1416, in _run_ready_apply
                                                  ctx.events.ready_apply(evt)
                                                File "/usr/lib/python3/dist-packages/cue/events.py", line 153, in __call__
                                                  handler.fn(evt)
                                                File "/usr/lib/python3/dist-packages/cue_dns_v1/lifecycle.py", line 126, in handle_ready_apply
                                                  if not dns_server_configs_present and not is_resolv_conf_owned_nvued():
                                                File "/usr/lib/python3/dist-packages/cue_dns_v1/lifecycle.py", line 90, in is_resolv_conf_owned_nvued
                                                  resolv_conf = fileops.load_file("/etc/resolv.conf")
                                                File "/usr/lib/python3/dist-packages/cue/fileops.py", line 91, in load_file
                                                  with open_file(filename, "rt") as f:
                                                File "/usr/lib/python3/dist-packages/cue/fileops.py", line 52, in open_file
                                                  return open(filename, mode)
                                              FileNotFoundError: [Errno 2] No such file or directory: '/etc/resolv.conf'
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Applying changeset/root/2022-05-25_19.37.58_S2PG: ready_error (Failed to prepare to apply)
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Apply Issues: (Unrecoverable internal error)
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: '/usr/bin/git add changeset/root/2022-05-25_19.37.58_S2PG'
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Running: "/usr/bin/git commit --allow-empty -m 'Updating changeset/root/2022-05-25_19.37.58_S2PG'"
May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  Ran Job apply:changeset/root/2022-05-25_19.37.58_S2PG
Failed to prepare to apply
  Unrecoverable internal error
root@dc2e06edfcaaad47:mgmt:~# May 25 19:39:52 dc2e06edfcaaad47 nvued[1995]:    INFO:  <local> - - [25/May/2022 19:39:52] "GET /nvue_v1/revision/changeset%2Froot%2F2022-05-25_19.37.58_S2PG HTTP/1.1" 200 -
networkop commented 2 years ago

and what if you do something like nameserver 1.1.1.1 > /etc/resolv.conf after you've removed it?

jhlasnik commented 2 years ago

Yeah creating the new file did it I'm able to commit configs

jhlasnik commented 2 years ago

Tested and confirmed this is fixed with 5.0.3, closing issue