[BUG] ext_pillar for mongo & redis is unavailable

[bwaang]

Description Trying to setup ext_pillar for either mongo or redis yields this error:

[DEBUG   ] Could not LazyLoad mongo.ext_pillar: 'mongo' __virtual__ returned False
[CRITICAL] Specified ext_pillar interface mongo is unavailable
[DEBUG   ] Could not LazyLoad redis.ext_pillar: 'redis.ext_pillar' is not available.
[CRITICAL] Specified ext_pillar interface redis is unavailable
[INFO    ] Executing command 'echo' in directory '/root'
[DEBUG   ] stdout: {"test":"test_value"}
[DEBUG   ] output: {"test":"test_value"}

I've gotten this to work in previous much older versions of salt, but trying this now on the latest yielding the above errors

Setup These are the packages that I've installed (python3 is already available) running al2023 (Amazon Linux 2023):

yum install -y findutils python3-pip
yum install -y salt-master salt-minion salt-ssh salt-syndic salt-cloud salt-api
pip3 install pymongo
pip3 install redis

Here's my master configuration

#####         Redis settings         #####
##########################################
# NOTE: redis need to be installed or else connecting to redis will fail silently
# See: https://docs.saltproject.io/en/latest/ref/pillar/all/salt.pillar.redismod.html

redis.db: 0
redis.host: redis-stack-0.redis-service.default
redis.port: 6379
redis.password: ''

#####         Mongo settings         #####
##########################################
mongo.db: salt
mongo.host: mongod-0.mongodb-service.default,mongod-1.mongodb-service.default,mongod-2.mongodb-service.default
mongo.port: 27017
mongo.user: salt
mongo.password: [redacted]

# This will fetch pillar globbed on the first chunk of the minion names, e.g. 'dev-kubem-1' will match just 'dev'
# Eventually we can start matching on two globs like 'dev-sisense' to grab by dev & type, need to enforce this type of naming convention to split pillars
# To manage a global pillar alone can just hardcode a value in re_replace to something like 'global' and populate the mongo _id with 'global'
ext_pillar:
  - mongo: {collection: pillar, re_pattern: '([a-z0-9]+)-.*', re_replace: \1}
  - redis: {function: key_value}
  - cmd_json: 'echo {\"test\":\"test_value\"}'

# Enable auto_accept, this setting will automatically accept all incoming
# public keys from the minions. Note that this is insecure.
auto_accept: True

I have confirmed that from the salt-master I can connect to both mongo and redis which I've put some dummy data in:

bash-5.2# python3
Python 3.9.16 (main, Apr 24 2024, 00:00:00) 
[GCC 11.4.1 20230605 (Red Hat 11.4.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pymongo
>>> client = pymongo.MongoClient("mongodb://salt:[redacted]@mongod-0.mongodb-service.default,mongod-1.mongodb-service.default,mongod-2.mongodb-service.default:27017/salt")
>>> client["salt"]["pillar"].find_one()
{'_id': ObjectId('66b63e6d2a3d68e55ec191b7'), 'saltmaster': {'pillar': 'data-from-mongo'}, 'saltminion': {'pillar': 'data-from-mongo'}}

>>> import redis
>>> r = redis.Redis(host='redis-stack-0.redis-service.default', port='6379', db=0)
>>> r.keys('*')
[b'pillar', b'salt-*']

Please be as specific as possible and give set-up details.

• [ ] on-prem machine
• [ ] VM (Virtualbox, KVM, etc. please specify)
• [ ] VM running on a cloud service, please be explicit and add details
• [x] container (Kubernetes)
• [ ] or a combination, please be explicit
• [ ] jails if it is FreeBSD
• [ ] classic packaging
• [ ] onedir packaging
• [ ] used bootstrap to install

Steps to Reproduce the behavior

• Start-up salt-master & salt-minion on all containers in the cluster
• Call salt '*' pillar.items to query pillar
• see logs

Expected behavior That ext_pillar would be working and can pull external pillar data from mongo or redis

Screenshots N/A

Versions Report

salt --versions-report (both master & minion)

```yaml Salt Version: Salt: 3007.1 Python Version: Python: 3.10.14 (main, Apr 3 2024, 21:33:04) [GCC 11.2.0] Dependency Versions: cffi: 1.16.0 cherrypy: unknown dateutil: 2.8.2 docker-py: Not Installed gitdb: Not Installed gitpython: Not Installed Jinja2: 3.1.4 libgit2: Not Installed looseversion: 1.3.0 M2Crypto: Not Installed Mako: Not Installed msgpack: 1.0.7 msgpack-pure: Not Installed mysql-python: Not Installed packaging: 23.1 pycparser: 2.21 pycrypto: Not Installed pycryptodome: 3.19.1 pygit2: Not Installed python-gnupg: 0.5.2 PyYAML: 6.0.1 PyZMQ: 25.1.2 relenv: 0.16.0 smmap: Not Installed timelib: 0.3.0 Tornado: 6.3.3 ZMQ: 4.3.4 Salt Package Information: Package Type: onedir System Versions: dist: amzn 2023.5.20240730 locale: utf-8 machine: aarch64 release: 6.10.0-linuxkit system: Linux version: Amazon Linux 2023.5.20240730 ```

Additional context Testing this out locally first in a cluster of containers in kubernetes environment before pushing it to ec2 instances in AWS

[welcome[bot]]

Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey. Please be sure to review our Code of Conduct. Also, check out some of our community resources including:

• Community Wiki
• Salt’s Contributor Guide
• Join our Community Slack
• IRC on LiberaChat
• Salt Project YouTube channel
• Salt Project Twitch channel

There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar. If you have additional questions, email us at saltproject@vmware.com. We’re glad you’ve joined our community and look forward to doing awesome things with you!

[OrangeDog]

https://docs.saltproject.io/salt/install-guide/en/latest/topics/install-dependencies.html

[bwaang]

Oh nice, thanks did not know that salt now doesn't depend on the preinstalled python in the current version - good to know that change come in 3006 & later.

For anyone else debugging just ran this & it worked fine afterwards:

salt-pip install pymongo
salt-pip install redis

Calling salt '*' pillar.items yielded data from all ext_pillar sources now:

saltminion-5bc5b48465-58kc6:
    ----------
    _id:
        saltminion
    ext_pillar:
        json_text
    mongokey1:
        minion
    mongokey2:
        pillar
    redis_pillar:
        ----------
        redisminion1:
            minionval1
        redisminion2:
            minionval2
saltmaster-76596c588f-84sf7:
    ----------
    _id:
        saltmaster
    ext_pillar:
        json_text
    mongokey1:
        master
    mongokey2:
        pillar
    redis_pillar:
        ----------
        redis1:
            value1
        redis2:
            value2

Thanks for the help!