search

UniversalDevicesInc / polyglot-v2

Polyglot Version 2 for use with the ISY 994i 5.0+
MIT License
30 stars 18 forks source link

Nodes left in DB with profileNum:null #39

Open jimboca opened 6 years ago

jimboca commented 6 years ago

No idea how these got there, but this was from a user after deleting the nodeserver in slot 1.

pi@rpi3-2:~/development/tmp/postdump/polyglot $ bsondump nodes.bson | grep n001
{ "__v" : 0, "_id" : ObjectId( "5b5da3b749ce74acb11ca9d4" ), "added" : true, "address" : "n001_a34592746", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enabled" : true, "isprimary" : false, "name" : "Watch Tablo TV", "nodedef" : "HarmonyActivity", "primary" : "n001_h0eff39c215884", "profileNum" : null, "timeAdded" : "1532804000152" }
{ "__v" : 0, "_id" : ObjectId( "5b5da3b749ce74acb11ca9db" ), "added" : true, "address" : "n001_a34592964", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enabled" : true, "isprimary" : false, "name" : "Watch GameRm Tablo TV", "nodedef" : "HarmonyActivity", "primary" : "n001_h3f9c6482cc5e2", "profileNum" : null, "timeAdded" : "1532804000152" }
{ "__v" : 0, "_id" : ObjectId( "5b5da3b749ce74acb11ca9dd" ), "added" : true, "address" : "n001_a34556869", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enabled" : true, "isprimary" : false, "name" : "Watch GameRm Netflix", "nodedef" : "HarmonyActivity", "primary" : "n001_h3f9c6482cc5e2", "profileNum" : null, "timeAdded" : "1532804000152" }
{ "__v" : 0, "_id" : ObjectId( "5b5da3b749ce74acb11ca9df" ), "added" : true, "address" : "n001_a34556837", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enabled" : true, "isprimary" : false, "name" : "Listen to GameRm Amazon Music", "nodedef" : "HarmonyActivity", "primary" : "n001_h3f9c6482cc5e2", "profileNum" : null, "timeAdded" : "1532804000152" }
{ "__v" : 0, "_id" : ObjectId( "5b5da3b749ce74acb11ca9e0" ), "added" : true, "address" : "n001_a34556906", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enabled" : true, "isprimary" : false, "name" : "Listen to GameRm Pandora", "nodedef" : "HarmonyActivity", "primary" : "n001_h3f9c6482cc5e2", "profileNum" : null, "timeAdded" : "1532804000152" }
{ "__v" : 0, "_id" : ObjectId( "5b5da3b749ce74acb11ca9e1" ), "added" : true, "address" : "n001_a34556170", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enabled" : true, "isprimary" : false, "name" : "Watch GameRm Roku", "nodedef" : "HarmonyActivity", "primary" : "n001_h3f9c6482cc5e2", "profileNum" : null, "timeAdded" : "1532804000152" }
{ "__v" : 0, "_id" : ObjectId( "5b5da3b749ce74acb11ca9e2" ), "added" : true, "address" : "n001_d55985121", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enabled" : true, "isprimary" : false, "name" : "GameRm RokuExpress", "nodedef" : "d55985121", "primary" : "n001_h3f9c6482cc5e2", "profileNum" : null, "timeAdded" : "1532804000152" }
{ "__v" : 0, "_id" : ObjectId( "5b5da3b849ce74acb11ca9e4" ), "added" : true, "address" : "n001_a34641850", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enabled" : true, "isprimary" : false, "name" : "Watch KTVI", "nodedef" : "HarmonyActivity", "primary" : "n001_hccdb0ea2cc564", "profileNum" : null, "timeAdded" : "1532804000152" }
{ "__v" : 0, "_id" : ObjectId( "5b5da3b849ce74acb11ca9f2" ), "added" : true, "address" : "n001_d55981590", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "1" } }, "enabled" : true, "isprimary" : false, "name" : "LivingRm RokuUltra", "nodedef" : "d55981590", "primary" : "n001_hccdb0ea2cc564", "profileNum" : null, "timeAdded" : "1532804000152" }
jimboca commented 6 years ago

For reference, this should delete the original issue items:

mongo polyglot --eval "db.nodes.remove({'profileNum':null})"

I was having a similar problem with my slot 5, so dump shows this:

mongodump --db polyglot --out postdump
bsondump postdump/polyglot/nodes.bson | grep n005
{ "__v" : 0, "_id" : ObjectId( "5b078c34454df7a3f2383f11" ), "added" : true, "address" : "n005_hue-emu-cntl", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "1" } }, "enabled" : true, "isprimary" : false, "name" : "Hue Emulator Controller", "nodedef" : "controller", "primary" : "n005_controller", "profileNum" : "5", "timeAdded" : "1527220930028" }

So when I try to add a new nodeserver in this slot it makes the controller use this data instead of what my nodeserver tells it...

So I ran this.

mongo polyglot --eval "db.nodes.remove({'profileNum':'5'})"

Isn't that run when you delete a nodeserver from a slot??

I had looked at all nodes using robo3t and for some reason I didn't see those nodes but it could be my lack of knowledge about mongodb... That's why I did the dump and grep...

jimboca commented 5 years ago

This causes a lot of pain, just had another user with a corrupt slot. He deleted this nodeserver but all this old data is still there...

pi@raspberrypi:~ $ bsondump postdump/polyglot/nodes.bson | grep n008
{ "__v" : 0, "_id" : ObjectId( "5bd0b11d1be7af18a2e38e49" ), "added" : true, "address" : "n008_hb986629beb2a1", "controller" : "false", "drivers" : { "GV5" : { "uom" : 56, "value" : "0" }, "G                                              V4" : { "uom" : 56, "value" : "0" }, "GV3" : { "uom" : 25, "value" : "0" }, "GV2" : { "uom" : 56, "value" : "0" }, "GV1" : { "uom" : 56, "value" : "0" }, "ST" : { "uom" : 2, "value" : "0" } }                                              , "enabled" : true, "isprimary" : true, "name" : "Aubreys Room", "nodedef" : "hb986629beb2a1", "primary" : "n008_hb986629beb2a1", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b11d1be7af18a2e38e4a" ), "added" : true, "address" : "n008_h6cf5b471715d2", "controller" : "false", "drivers" : { "GV5" : { "uom" : 56, "value" : "0" }, "G                                              V4" : { "uom" : 56, "value" : "0" }, "GV3" : { "uom" : 25, "value" : "0" }, "GV2" : { "uom" : 56, "value" : "0" }, "GV1" : { "uom" : 56, "value" : "0" }, "ST" : { "uom" : 2, "value" : "0" } }                                              , "enabled" : true, "isprimary" : true, "name" : "Harmony Home Theather", "nodedef" : "h6cf5b471715d2", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1601be7af18a2e38e4b" ), "added" : true, "address" : "n008_a28359023", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Watch TV", "nodedef" : "HarmonyActivity", "primary" : "n008_hb986629beb2a1", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1601be7af18a2e38e4c" ), "added" : true, "address" : "n008_d47520777", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Aubreys Room", "nodedef" : "d47520777", "primary" : "n008_hb986629beb2a1", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1601be7af18a2e38e4d" ), "added" : true, "address" : "n008_d47520778", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Vizio TV", "nodedef" : "d47520778", "primary" : "n008_hb986629beb2a1", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e4e" ), "added" : true, "address" : "n008_a6723030", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enabled                                              " : true, "isprimary" : false, "name" : "Listen to Streaming Music", "nodedef" : "HarmonyActivity", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e4f" ), "added" : true, "address" : "n008_a21049561", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Watch PC", "nodedef" : "HarmonyActivity", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e50" ), "added" : true, "address" : "n008_a21082673", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Watch Roku", "nodedef" : "HarmonyActivity", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e51" ), "added" : true, "address" : "n008_a21733697", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Watch TV Family", "nodedef" : "HarmonyActivity", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e52" ), "added" : true, "address" : "n008_a30735294", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Switch", "nodedef" : "HarmonyActivity", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e53" ), "added" : true, "address" : "n008_a6720248", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enabled                                              " : true, "isprimary" : false, "name" : "Watch TV Xbox One", "nodedef" : "HarmonyActivity", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e54" ), "added" : true, "address" : "n008_a27402110", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Watch Family Fire TV", "nodedef" : "HarmonyActivity", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e55" ), "added" : true, "address" : "n008_a21109427", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Watch Netflix", "nodedef" : "HarmonyActivity", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e56" ), "added" : true, "address" : "n008_a6720244", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enabled                                              " : true, "isprimary" : false, "name" : "Watch TV", "nodedef" : "HarmonyActivity", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e57" ), "added" : true, "address" : "n008_a21034982", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Nintendo Wii U", "nodedef" : "HarmonyActivity", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e58" ), "added" : true, "address" : "n008_d38391867", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Samsung TV", "nodedef" : "d38391867", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e59" ), "added" : true, "address" : "n008_d38306246", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Onkyo AV Receiver 2", "nodedef" : "d38306246", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e5a" ), "added" : true, "address" : "n008_d46150275", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Media Room", "nodedef" : "d46150275", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e5b" ), "added" : true, "address" : "n008_d29165895", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Home Theater", "nodedef" : "d29165895", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e5c" ), "added" : true, "address" : "n008_d15790166", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Home Theater Roku", "nodedef" : "d15790166", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e5d" ), "added" : true, "address" : "n008_d15790161", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Sony Projector", "nodedef" : "d15790161", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e5e" ), "added" : true, "address" : "n008_d37357792", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Wii U", "nodedef" : "d37357792", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e5f" ), "added" : true, "address" : "n008_a6720245", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enabled                                              " : true, "isprimary" : false, "name" : "Watch Apple TV", "nodedef" : "HarmonyActivity", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e60" ), "added" : true, "address" : "n008_d38389577", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Bose AV Receiver", "nodedef" : "d38389577", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e61" ), "added" : true, "address" : "n008_d37356833", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Onkyo AV Receiver", "nodedef" : "d37356833", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e62" ), "added" : true, "address" : "n008_d15821876", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "G Air Mini Split", "nodedef" : "d15821876", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e63" ), "added" : true, "address" : "n008_d50771325", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Nintendo Game Console", "nodedef" : "d50771325", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e64" ), "added" : true, "address" : "n008_d38298833", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Bose Music Server", "nodedef" : "d38298833", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e65" ), "added" : true, "address" : "n008_d15790164", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Microsoft Xbox One", "nodedef" : "d15790164", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e66" ), "added" : true, "address" : "n008_d15790165", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Apple TV", "nodedef" : "d15790165", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
{ "__v" : 0, "_id" : ObjectId( "5bd0b1611be7af18a2e38e67" ), "added" : true, "address" : "n008_d37379220", "controller" : "false", "drivers" : { "ST" : { "uom" : 2, "value" : "0" } }, "enable                                              d" : true, "isprimary" : false, "name" : "Windows Computer", "nodedef" : "d37379220", "primary" : "n008_h6cf5b471715d2", "profileNum" : null, "timeAdded" : "1540347133421" }
400 objects found
bpaauwe commented 5 years ago

I believe I have a fix for this. When updateNode is called, it simply checks if the node exist by doing a query for one by address. If it find one of these records with the null profile number, it just uses it. My fix is to also check for a valid profile number, if it's null, then treat it as if the node record doesn't exist. That seems to make everything happy again.

jimboca commented 5 years ago

That sounds good! Also, seems like polyglot should do a sanity check on the db once in an while and cleanup these bad null entries or entries for a profile that is deleted?

jimboca commented 5 years ago

@bpaauwe & @Einstein42 Have you made this change? Can we please get it released :) It keeps causing pain.

bpaauwe commented 5 years ago

Good question, that was months ago. I'll have to check if that was part of the PR I submitted back on Nov. 8th

bpaauwe commented 5 years ago

Looks like it's on the edev branch.

Handle null profile number in updateNode. https://github.com/UniversalDevicesInc/polyglot-v2/commit/cf2949f338a6f351c4d20fc718331d9d228f5b7a

jimboca commented 5 years ago

Thanks! I've pinged everyone on slack to see what we can do.