redonkuless commented 5 years ago

Salt Master salt-master 2019.2.1 (Fluorine)

OS version: CentOS 7.7 SaltSatck version: salt-master 2019.2.1 (Fluorine)

Salt Minions 2019.2.1

OS version: Windows 2012R2 SaltStack version:

Issue description: Biosreleasedate does not exist in windows grains, failure to populate database with minion return due to error.

The minion function caused an exception: Traceback (most recent call last):
  File "c:\salt\bin\lib\site-packages\salt\minion.py", line 1663, in _thread_return
    return_data = minion_instance.executors[fname](opts, data, func, args, kwargs)
  File "c:\salt\bin\lib\site-packages\salt\executors\direct_call.py", line 12, in execute
    return func(*args, **kwargs)
  File "c:\salt\var\cache\salt\minion\extmods\modules\inventory.py", line 66, in audit
    properties[p] = grains[p]
  File "c:\salt\bin\lib\site-packages\salt\utils\context.py", line 236, in __getitem__
    return self._dict()[key]
KeyError: 'biosreleasedate'

neilmunday commented 5 years ago

Hi,

Thanks for reporting the issue. The Salt Minion Inventory hasn't been tried on Windows before, but hopefully between us we can get it to work.

I expect there might be other grain items that might not be present on a Windows minion. Just to check, can you send me the output of:

salt 'WINDOWS_HOST_HERE' grains.items

I can then check if I need to account for any other missing grains for Windows minions.

redonkuless commented 5 years ago

I've edited some return values to keep things private, but this should have everything your looking for. I appreciate the assistance with this.

24*:

SSDs:
biosversion:
    Intel IGD BDSM enabled at 0x%08x, size %lldMB, dev 00:02.0
cpu_model:
    Intel Core Processor (Haswell, no TSX)
cpuarch:
    AMD64
disks:
    - \\.\PhysicalDrive0
domain:
    openstacklocal
fqdn:
    WIN-B19999.openstacklocal
fqdn_ip4:
fqdn_ip6:
fqdns:
gpus:
groupname:
host:
    WIN-BKTA1599999
hwaddr_interfaces:
    ----------
    Red Hat VirtIO Ethernet Adapter:
        FA:16:
id:
    24A931A9-586E-4B00-9199-D99999999
init:
    Windows
ip4_interfaces:
    ----------
    Red Hat VirtIO Ethernet Adapter:
        - 192.99.151.159
ip6_interfaces:
    ----------
    Red Hat VirtIO Ethernet Adapter:
        - fe80::90c6:462:a2cc:4d71
ip_interfaces:
    ----------
    Red Hat VirtIO Ethernet Adapter:
        -  IP PRIVATE
        - fe80::90
ipv4:
    - IP PRIVATE (EDITED)
ipv6:
    - fe80::91
kernel:
    Windows
kernelrelease:
    6.3.9600
kernelversion:
    6.3.9600
locale_info:
    ----------
    defaultencoding:
        cp1252
    defaultlanguage:
        en_US
    detectedencoding:
        cp1252
    timezone:
        Pacific Daylight Time
localhost:
    WIN-BKTA15I9999
manufacturer:
    OpenStack Foundation
master:
    IP-PRIVATE
mem_total:
    1999
motherboard:
    ----------
    productname:
        None
    serialnumber:
        None
nodename:
    WIN-BKTA15I5999
num_cpus:
    1
num_gpus:
    0
os:
    Windows
os_family:
    Windows
osfinger:
    Windows-2012ServerR2
osfullname:
    Microsoft Windows Server 2012 R2 Standard
osmanufacturer:
    Microsoft Corporation
osrelease:
    2012ServerR2
osrelease_info:
    - 2012
    - 2
osservicepack:
    None
osversion:
    6.3.9600
path:
    C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\salt;c:\salt\bin\lib\site-packages\pywin32_system32;c:\salt\bin\lib\site-packages\pywin32_system32
pid:
    2172
productname:
    OpenStack Nova
ps:
    tasklist.exe
pythonexecutable:
    c:\salt\bin\python.exe
pythonpath:
    - c:\salt\bin\lib\site-packages\git\ext\gitdb
    - c:\salt\bin\Scripts
    - c:\salt\bin\python27.zip
    - c:\salt\bin\DLLs
    - c:\salt\bin\lib
    - c:\salt\bin\lib\plat-win
    - c:\salt\bin\lib\lib-tk
    - c:\salt\bin
    - c:\salt\bin\lib\site-packages
    - c:\salt\bin\lib\site-packages\win32
    - c:\salt\bin\lib\site-packages\win32\lib
    - c:\salt\bin\lib\site-packages\Pythonwin
    - c:\salt\bin\lib\site-packages\gitdb\ext\smmap
pythonversion:
    - 2
    - 7
    - 15
    - final
    - 0
saltpath:
    c:\salt\bin\lib\site-packages\salt
saltversion:
    2019.2.1
saltversioninfo:
    - 2019
    - 2
    - 1
    - 0
serialnumber:
    5b425103-2556-124f-1caf-5ffb569999
server_id:
    14562
shell:
    C:\Windows\system32\cmd.exe
timezone:
    (UTC-08:00) Pacific Time (US & Canada)
username:
    WIN-B$
virtual:
    physical
windowsdomain:
    PRIVATE
windowsdomaintype:
    Workgroup
zmqversion:
    4.3.1

redonkuless commented 5 years ago

This was gathered by doing salt 'windows-host' grains.ls

SSDs
- biosversion
- cpu_model
- cpuarch
- disks
- domain
- fqdn
- fqdn_ip4
- fqdn_ip6
- fqdns
- gpus
- groupname
- host
- hwaddr_interfaces
- id
- init
- ip4_interfaces
- ip6_interfaces
- ip_interfaces
- ipv4
- ipv6
- kernel
- kernelrelease
- kernelversion
- locale_info
- localhost
- manufacturer
- master
- mem_total
- motherboard
- nodename
- num_cpus
- num_gpus
- os
- os_family
- osfinger
- osfullname
- osmanufacturer
- osrelease
- osrelease_info
- osservicepack
- osversion
- path
- pid
- productname
- ps
- pythonexecutable
- pythonpath
- pythonversion
- saltpath
- saltversion
- saltversioninfo
- serialnumber
- server_id
- shell
- timezone
- username
- virtual
- windowsdomain
- windowsdomaintype
- zmqversion

neilmunday commented 5 years ago

Thanks for providing the output.

The inventory.py module also tries to run the Linux lsblk to find out extra information about disks attached to a minion. This will also need handling for non-Linux hosts.

I'll see what I can do.

Would you be willing to try out test versions for me?

redonkuless commented 5 years ago

Absolutely

redonkuless commented 5 years ago

do you have an email address I can get to communicate with you better?

neilmunday commented 5 years ago

Sure, you can use neil at mundayweb.com

neilmunday commented 5 years ago

I've created a new branch for version 1.7 to include support for Windows minions. Can you try downloading https://github.com/neilmunday/Salt-Minion-Inventory/blob/1.7/srv/salt/_modules/inventory.py and saving this to /srv/salt/_modules/inventory.py on your Salt Master?

You will need to restart your Salt Master and then run a state.apply on your Windows minion(s).

Can you let me know if this new version works for you?

redonkuless commented 5 years ago

salt 'EC' inventory.audit force=True EC: The minion function caused an exception: Traceback (most recent call last): File "c:\salt\bin\lib\site-packages\salt\minion.py", line 1663, in _thread_return return_data = minion_instance.executors[fname](opts, data, func, args, kwargs) File "c:\salt\bin\lib\site-packages\salt\executors\direct_call.py", line 12, in execute return func(args, **kwargs) File "c:\salt\var\cache\salt\minion\extmods\modules\inventory.py", line 75, in audit for u in salt['status.w'](): File "c:\salt\bin\lib\site-packages\salt\loader.py", line 1208, in getitem func = super(LazyLoader, self).getitem(item) File "c:\salt\bin\lib\site-packages\salt\utils\lazy.py", line 108, in getitem raise KeyError(key) KeyError: 'status.w'

neilmunday commented 5 years ago

Thanks for trying out the new version.

Can you also tell me if this works and if so, what the output is please?

salt WINDOWS_HOST status.uptime

redonkuless commented 5 years ago

salt 'EC*' status.uptime

EC: 6398.96

neilmunday commented 5 years ago

Thanks for the info.

I've updated https://github.com/neilmunday/Salt-Minion-Inventory/blob/1.7/srv/salt/_modules/inventory.py which I hope will work this time on your Windows minion(s).

You will need to restart your Salt Master and then run a state.apply on your Windows minion(s).

Can you let me know if it works for you?

redonkuless commented 5 years ago

salt 'EC*' -l debug inventory.audit force=True

[DEBUG ] Reading configuration from /etc/salt/master [DEBUG ] Including configuration from '/etc/salt/master.d/hubblestack-nova.conf' [DEBUG ] Reading configuration from /etc/salt/master.d/hubblestack-nova.conf [DEBUG ] Including configuration from '/etc/salt/master.d/reactor.conf' [DEBUG ] Reading configuration from /etc/salt/master.d/reactor.conf [DEBUG ] Including configuration from '/etc/salt/master.d/saltapi.conf' [DEBUG ] Reading configuration from /etc/salt/master.d/saltapi.conf [DEBUG ] Changed git to gitfs in master opts' fileserver_backend list [DEBUG ] Missing configuration file: /root/.saltrc [DEBUG ] Configuration file path: /etc/salt/master [WARNING ] Insecure logging configuration detected! Sensitive data may be logged. [DEBUG ] Reading configuration from /etc/salt/master [DEBUG ] Including configuration from '/etc/salt/master.d/hubblestack-nova.conf' [DEBUG ] Reading configuration from /etc/salt/master.d/hubblestack-nova.conf [DEBUG ] Including configuration from '/etc/salt/master.d/reactor.conf' [DEBUG ] Reading configuration from /etc/salt/master.d/reactor.conf [DEBUG ] Including configuration from '/etc/salt/master.d/saltapi.conf' [DEBUG ] Reading configuration from /etc/salt/master.d/saltapi.conf [DEBUG ] Changed git to gitfs in master opts' fileserver_backend list [DEBUG ] Missing configuration file: /root/.saltrc [DEBUG ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc [DEBUG ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc [DEBUG ] Initializing new AsyncZeroMQReqChannel for (u'/etc/salt/pki/master', u'vps257552.vps.ovh.ca_master', u'tcp://127.0.0.1:80', u'clear') [DEBUG ] Connecting the Minion to the Master URI (for the return server): tcp://127.0.0.1:80 [DEBUG ] Trying to connect to: tcp://127.0.0.1:80 [DEBUG ] Closing AsyncZeroMQReqChannel instance [DEBUG ] LazyLoaded local_cache.get_load [DEBUG ] Reading minion list from /var/cache/salt/master/jobs/93/00f71a582c1ccfd59f5544f62c6a02088fe79f24480998a9d28cef64985f95dda80691ae940b3f1c73a961ddf50e68a7ac82986d0f593fb9bbde2a3ba5456c/.minions.p [DEBUG ] get_iter_returns for jid 20191007143514775090 sent to set(['EC']) will timeout at 14:36:14.785801 [DEBUG ] jid 20191007143514775090 return from EC [DEBUG ] return event: {u'EC': {u'jid': u'20191007143514775090', u'retcode': 1, u'ret': u"Passed invalid arguments to inventory.audit: 'float' object has no attribute 'getitem'\n\n\tPerform an audit of this minion and return data via a salt\n\tevent for the master.\n\t", u'out': u'nested'}} [DEBUG ] LazyLoaded nested.output EC: Passed invalid arguments to inventory.audit: 'float' object has no attribute 'getitem'

    Perform an audit of this minion and return data via a salt
    event for the master.

[DEBUG ] jid 20191007143514775090 found all minions set([u'EC']) [DEBUG ] Closing IPCMessageSubscriber instance

neilmunday commented 5 years ago

Hmmmm....

Can you run this in one shell on your master:

salt-run state.event pretty=True

And then run in another shell:

salt WINDOWS_HOST inventory.audit force=True

I'm hoping that somewhere in the output is the line in the source code that is causing then error.

redonkuless commented 5 years ago

20191007145409643384 { "_stamp": "2019-10-07T18:54:09.644753", "minions": [ "EC" ] } salt/job/20191007145409643384/new { "_stamp": "2019-10-07T18:54:09.645615", "arg": [ { "kwarg": true, "force": true } ], "fun": "inventory.audit", "jid": "20191007145409643384", "minions": [ "EC" ], "missing": [], "tgt": "EC*", "tgt_type": "glob", "user": "sudo_lefteye" } salt/auth { "_stamp": "2019-10-07T18:54:13.137248", "act": "accept", "id": "EC", "pub": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAzl60s8gKCMTUgt2iBHvo\nB1Y66LXyojaiJZ/nqsPd8ljScdiBwiE5QR/pHPjw7aHv+g5DWCvMiP5WfzlfRIQ6\n05X2zW/80etsygQiFcStbq/yKjZkOBhClVcSv9ivrS/yuzVAvD0CQm9QeOk1a+O4\nbSJA6AdeuG86GYBNCwzm/goImxEFGnw05hi0mQSSdq1yjaiwhnZxXgwkeQfZd+ti\nwlACD3XbF4aoErzrAuRuDiNhz4yerg95alKJF9pbtWHIz3mBDyYf/es9R4mJI/9H\nON6wYVlxHGVxbfmvbnlcdvGUrtxwhB9jZNFqE1waDxlhQQKLnXKIbkLNvhB68//a\nAQIDAQAB\n-----END PUBLIC KEY-----", "result": true } minion/refresh/EC242CE3-86E8-CD99-D7B5-21837C6A5D49 { "Minion data cache refresh": "EC", "_stamp": "2019-10-07T18:54:13.357080" } salt/job/20191007145409643384/ret/EC { "_stamp": "2019-10-07T18:54:13.563540", "cmd": "_return", "fun": "inventory.audit", "fun_args": [ { "force": true } ], "id": "EC", "jid": "20191007145409643384", "out": "nested", "retcode": 1, "return": "Passed invalid arguments to inventory.audit: 'float' object has no attribute 'getitem'\n\n\tPerform an audit of this minion and return data via a salt\n\tevent for the master.\n\t", "success": false

neilmunday commented 4 years ago

1.7 branch merged into master.

Unable to test with Windows minions sadly - feedback and pull requests welcome!

neilmunday / Salt-Minion-Inventory

biosreleasedate Windows Does not exist, fails to audit windows #30

24*:

salt 'EC*' status.uptime

salt 'EC*' -l debug inventory.audit force=True