search

TritonDataCenter / smartos-live

For more information, please see http://smartos.org/ For any questions that aren't answered there, please join the SmartOS discussion list: https://smartos.topicbox.com/groups/smartos-discuss
1.57k stars 246 forks source link

Unable to restart metadata, vminfod and vmadmd after upgrade smartos #852

Closed rmaiquez closed 4 years ago

rmaiquez commented 5 years ago

Hello,

I upgraded yesterday the smartos server to latest version and when rebooted system these services not working, metadata and vmadmd are on status offline and vminfod is status manteinance.

I try get log but only have metadata

# svcs -pv vminfod vmadmd metadata
STATE          NSTATE        STIME    CTID   FMRI
offline        -             22:36:31    239 svc:/system/smartdc/metadata:default
offline        -             22:36:31    236 svc:/system/smartdc/vmadmd:default
maintenance    -             22:36:17      - svc:/system/smartdc/vminfod:default

# tail -25 /var/svc/log/system-smartdc-metadata:default.log
[ Oct 14 22:36:13 Method "start" exited with status 0. ]
{"name":"metadata","hostname":"smartos","pid":7478,"client":"Metadata Agent - VminfodWatcher","level":50,"err":{"message":"connect ECONNREFUSED","name":"Error","stack":"Error: connect ECONNREFUSED\n    at errnoException (net.js:907:11)\n    at Object.afterConnect [as oncomplete] (net.js:898:19)","code":"ECONNREFUSED"},"msg":"VminfodEventStream request error","time":"2019-10-14T22:36:13.975Z","v":0}
{"name":"metadata","hostname":"smartos","pid":7478,"client":"Metadata Agent - VminfodWatcher","level":20,"msg":"vminfod event stream stopped","time":"2019-10-14T22:36:13.977Z","v":0}
Uncaught Error: connect ECONNREFUSED

FROM
VminfodEventStream.EventEmitter.emit (events.js:72:15)
VminfodEventStream._vminfodEventStreamAbort [as _abort] (/usr/vm/node_modules/vminfod/client.js:581:10)
ClientRequest.reqError (/usr/vm/node_modules/vminfod/client.js:467:14)
ClientRequest.EventEmitter.emit (events.js:95:17)
Socket.socketErrorListener (http.js:1545:9)
Socket.EventEmitter.emit (events.js:95:17)
net.js:443:14
process._tickCallback (node.js:415:13)
[ Oct 14 22:36:31 Stopping because all processes in service exited. ]
[ Oct 14 22:36:31 Executing stop method (:kill). ]

for vmadmd and vminfod logs are empty.

I tried this https://github.com/joyent/smartos-live/issues/825 but not working for me.

can anybody help me?

rmaiquez commented 4 years ago

Ok, It's resolved to me.

My problem was that something has gone wrong with instance <>

I do:

zfs mount -a
svcadm clear vminfod

but that doesn't fixed it, so:

zfs get all zones/<<uuii-zone>>
find /zones/<<uuii-zone>>

that dataset doesn't existed, somehow a zone configuration remained after <> was destroyed. Thats instate was partially deleted, so I removed the zone configuration with:

zonecfg -z <<uuid-zone>>  delete

Then:

svcadm clear vminfod

And all working normal now