[BUG] boto_rds, and `boto3.assign_funcs`, requires boto.

[japtain-cack]

Description Custom execution module using boto3, specifically boto3.assign_funcs, requires boto, despite not having any boto dependencies.

Setup

Create a custom module, boto3_rds for example, and add the following code. Install only boto3 onto the minion.

"""
Connection module for Amazon RDS

.. versionadded:: 2015.8.0

:configuration: This module accepts explicit rds credentials but can also
    utilize IAM roles assigned to the instance through Instance Profiles.
    Dynamic credentials are then automatically obtained from AWS API and no
    further configuration is necessary. More Information available at:

    .. code-block:: text

        http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html

    If IAM roles are not used you need to specify them either in a pillar or
    in the minion's config file:

    .. code-block:: yaml

        rds.keyid: GKTADJGHEIQSXMKKRBJ08H
        rds.key: askdjghsdfjkghWupUjasdflkdfklgjsdfjajkghs

    A region may also be specified in the configuration:

    .. code-block:: yaml

        rds.region: us-east-1

    If a region is not specified, the default is us-east-1.

    It's also possible to specify key, keyid and region via a profile, either
    as a passed in dict, or as a string to pull from pillars or minion config:

    .. code-block:: yaml

        myprofile:
            keyid: GKTADJGHEIQSXMKKRBJ08H
            key: askdjghsdfjkghWupUjasdflkdfklgjsdfjajkghs
            region: us-east-1

:depends: boto3
"""
# keep lint from choking on _get_conn and _cache_id
# pylint: disable=E0602
# pylint whinging perfectly valid code
# pylint: disable=W0106

import logging

import salt.utils.versions
from salt.exceptions import SaltInvocationError

log = logging.getLogger(__name__)

# pylint: disable=import-error
try:
    # pylint: disable=unused-import
    import boto3

    # pylint: enable=unused-import
    from botocore.exceptions import ClientError

    logging.getLogger("boto3").setLevel(logging.CRITICAL)
    HAS_BOTO = True
except ImportError:
    HAS_BOTO = False
# pylint: enable=import-error

def __virtual__():
    """
    Only load if boto libraries exist and if boto libraries are greater than
    a given version.
    """

    return salt.utils.versions.check_boto_reqs(
        boto3_ver="1.3.1",
        check_boto=False,
        check_boto3=True,
    )

def __init__(opts):
    if HAS_BOTO:
        __utils__["boto3.assign_funcs"](__name__, "rds")

def describe_db_clusters(
    name=None,
    filters=None,
    jmespath="DBInstances",
    region=None,
    key=None,
    keyid=None,
    profile=None,
):
    """
    Return a detailed listing of some, or all, DB Clusters visible in the
    current scope.  Arbitrary subelements or subsections of the returned dataset
    can be selected by passing in a valid JMSEPath filter as well.

    CLI Example:

    .. code-block:: bash

        salt myminion boto_rds.describe_db_clusters jmespath='DBClusters[*].DBClusterIdentifier'

    """
    conn = _get_conn(region=region, key=key, keyid=keyid, profile=profile)
    pag = conn.get_paginator("describe_db_clusters")
    args = {}
    args.update({"DBClusterIdentifier": name}) if name else None
    args.update({"Filters": filters}) if filters else None
    pit = pag.paginate(**args)
    pit = pit.search(jmespath) if jmespath else pit
    try:
        return [p for p in pit]
    except ClientError as e:
        code = getattr(e, "response", {}).get("Error", {}).get("Code")
        if code != "DBClusterNotFound":
            log.error(__utils__["boto3.get_error"](e))
    return []

Run the example command in the docstring to test.

• [x] on-prem machine
• [ ] VM (Virtualbox, KVM, etc. please specify)
• [x] VM running on a cloud service, please be explicit and add details
• [ ] container (Kubernetes, Docker, containerd, etc. please specify)
• [ ] or a combination, please be explicit
• [ ] jails if it is FreeBSD
• [ ] classic packaging
• [x] onedir packaging
• [ ] used bootstrap to install

Steps to Reproduce the behavior Use above code blocks, add as custom module in /srv/salt/_modules, then run the example command in the describe_db_clusters function, targeting a minion with boto3 installed. Ensure no boto package is installed.

Expected behavior Salt is using Python3.x and python2.x packages/libraries should be deprecated and no longer needed, except in special circumstances.

Screenshots N/A

Versions Report

Salt Version:
          Salt: 3006.1

Python Version:
        Python: 3.10.11 (main, May  5 2023, 02:31:54) [GCC 11.2.0]

Dependency Versions:
          cffi: 1.14.6
      cherrypy: unknown
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.2
       libgit2: 1.6.4
  looseversion: 1.0.2
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 22.0
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.9.8
        pygit2: 1.12.1
  python-gnupg: 0.4.8
        PyYAML: 5.4.1
         PyZMQ: 23.2.0
        relenv: 0.12.3
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4

System Versions:
          dist: amzn 2
        locale: utf-8
       machine: x86_64
       release: 4.14.311-233.529.amzn2.x86_64
        system: Linux
       version: Amazon Linux 2

Salt master/minion are similar in OS/Platform, packages, and versions.

Additional context This module code is basically a copy of the boto_rds module, with a new function added. So I assume the issue is present in any boto_* module as well.

If you don't have boto installed, you will get the following error.

'boto3_rds' __virtual__ returned False: __init__ failed

As soon as you install boto, the custom module no longer produces an error.

[welcome[bot]]

Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey. Please be sure to review our Code of Conduct. Also, check out some of our community resources including:

• Community Wiki
• Salt’s Contributor Guide
• Join our Community Slack
• IRC on LiberaChat
• Salt Project YouTube channel
• Salt Project Twitch channel

There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar. If you have additional questions, email us at saltproject@vmware.com. We’re glad you’ve joined our community and look forward to doing awesome things with you!

[cmcmarrow]

boto3.assign_funcs seems like we are using to boto somewhere.

[japtain-cack]

I've also just discovered, boto3_route53 is requiring boto as well.

Unless you install boto, we get the following error:

boto3_route53' __virtual__ returned False: boto3_route53 module could not be loaded

Salt version is latest. 3006.5.