Frustrations with salt-extension

[darkpixel]

I thought I'd take another stab at turning a few modules I've written into a salt-ext.

I installed the latest version of salt-extension via pip install --upgrade salt-extension and it installed version 0.24.0.

I created followed the instructions...then realized 0.24.0 was old and there have been ~15 commits since then to fix bugs.

A new release needs to be cut or the instructions should be modified to tell users to just install from HEAD.

pip install git+https://github.com/saltstack/salt-extension.git

I then ran:

create-salt-extension --salt-version 3007.0 -l beacons -l grain -l module -l pillar cait-salt

Oops. I ran that from my 'code' directory thinking it would create 'cait-salt'. Nope. It littered my code directory with a ton of files and directories instead of warning that it was about to operate on a non-empty directory. zfs rollback to the rescue.

Created cait-salt, cd'd into it, and re-ran the command. It helpfully spit out some instructions for me:

Make sure you update the 'license' field and also the classifiers on the generated 'setup.cfg'.
Bare bones project is created.
You should now run the following commands:
  python3 -m venv .env --prompt 'cait-salt'
  source .env/bin/activate
  git init .
  python -m pip install -e .[dev,tests,docs]
  pre-commit install
  git add .
  git commit -a
The above command will fail because it's pinning the project dependencies.
Now run the following commands:
  git add .
  git commit -a -m 'Initial extension layout'
To run the included test suite, run the following command:
  nox -e tests-3 -- tests/
Please update these tests :)
Start Hacking!

...so I followed the instructions:

$ python3 -m venv .env --prompt 'cait-salt'
$ source .env/bin/activate
$ .env/bin/activate
$ git init .
Initialized empty Git repository in /tank/code/cait-salt/.git/
$python -m pip install -e .[dev,tests,docs]

Obtaining file:///tank/code/cait-salt
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing editable metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [37 lines of output]
      Traceback (most recent call last):
        File "/tank/code/cait-salt/.env/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/tank/code/cait-salt/.env/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/tank/code/cait-salt/.env/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 191, in prepare_metadata_for_build_editable
          return hook(metadata_directory, config_settings)
        File "/tmp/pip-build-env-0rjbz30q/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 453, in prepare_metadata_for_build_editable
          return self.prepare_metadata_for_build_wheel(
        File "/tmp/pip-build-env-0rjbz30q/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 366, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/tmp/pip-build-env-0rjbz30q/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 5, in <module>
        File "/tmp/pip-build-env-0rjbz30q/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
        File "/tmp/pip-build-env-0rjbz30q/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 147, in setup
          _setup_distribution = dist = klass(attrs)
        File "/tmp/pip-build-env-0rjbz30q/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 303, in __init__
          _Distribution.__init__(self, dist_attrs)
        File "/tmp/pip-build-env-0rjbz30q/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 283, in __init__
          self.finalize_options()
        File "/tmp/pip-build-env-0rjbz30q/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 654, in finalize_options
          ep(self)
        File "/tmp/pip-build-env-0rjbz30q/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 674, in _finalize_setup_keywords
          ep.load()(self, ep.name, value)
        File "/tmp/pip-build-env-0rjbz30q/overlay/lib/python3.10/site-packages/setuptools_scm/_integration/setuptools.py", line 101, in version_keyword
          _assign_version(dist, config)
        File "/tmp/pip-build-env-0rjbz30q/overlay/lib/python3.10/site-packages/setuptools_scm/_integration/setuptools.py", line 56, in _assign_version
          _version_missing(config)
        File "/tmp/pip-build-env-0rjbz30q/overlay/lib/python3.10/site-packages/setuptools_scm/_get_version_impl.py", line 112, in _version_missing
          raise LookupError(
      LookupError: setuptools-scm was unable to detect version for /tank/code/cait-salt.

      Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.

      For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=proj
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

$

It's frustrating that the tool released for de-cluttering salt core is so janky while stuff is already being tossed out of core in the new 3007.0 release.

[dmurphy18]

Stuff is being tossed out in the 3008 release, 3007.0 is similar to 3006 and was just released. Improvements to the tool gladly welcomed in a PR with tests, it is community driven after all.

[darkpixel]

Gotta love the Saltstack model. We're kicking a bunch of useful stuff out of core, go use this tool that's unsupported and hasn't worked properly in at least 6 months. If you don't like that it's broken, feel free to code it yourself.

I have a ton of code for managing large standardized networks of Windows machines and protecting virtualized environments from cryptolocker, and handling the security of those networks. I even built a module and web interface for gathering data from minions to generate reports, alert on out-of-date software and other security issues.

I'd love to release it (and support it) so the community can benefit from it, but I don't have time to constantly fight all the broken Salt infrastructure around releasing it. I don't have time to do both. I guess I'll take another look when 3008 lands.

[dmurphy18]

@darkpixel Rather than If you don't like that it's broken, feel free to code it yourself. , I was actually requesting that if there are bugs, please give us fixes, given the Salt Core Team got hit bad with the Broadcom acquisition, the team is stretched badly, and needs help from the community on plugging the holes when they are found. See https://github.com/salt-extensions and the Salt-Extension Working Group on https://saltproject.io/working-groups/

[darkpixel]

I get that things are crazy after the acquisition. I have zero clue about issues in setuptools-scm. Never used it. Never used setuptools. I'm trying to plug a hole for a group of users who manage Windows networks and virtualized environments, I don't have time to learn the intricacies of setuptools, so I can't give you a fix.

[dmurphy18]

@darkpixel Not a problem, just wanted to explain where the team was coming from, needing any help it can get. @twangboy will have to step in and resolve the windows issues. :)

[darkpixel]

No worries, but just to clarify, I'm not running this on Windows. I'm a Linux and BSD user, but I support a lot of people who make the bad decision to use Windows. ;)