FreeBSD pkg install fails

[ari]

Installing some packages fails without any clues in the logs at to why. On the command line, this succeeds:

pkg install  -y databases/py-MySQLdb

This is FreeBSD 10.1-RELEASE-p15.

However salt-call -l debug state.highstate fails with a state like this:

salt-mysql:
  pkg.installed:
    - name: databases/py-MySQLdb

Here's the relevant snippet from the logs. Everything looks like it worked perfectly, except for the ERROR on the last line.

[INFO    ] Running state [databases/py-MySQLdb] at time 11:02:07.503711
[INFO    ] Executing state pkg.installed for databases/py-MySQLdb
[DEBUG   ] Could not LazyLoad pkg.normalize_name
[DEBUG   ] Could not LazyLoad pkg.check_db
[DEBUG   ] Could not LazyLoad pkg.normalize_name
[INFO    ] Executing command 'pkg install  -y databases/py-MySQLdb' in directory '/root'
[INFO    ] Executing command 'pkg info -ao' in directory '/root'
[DEBUG   ] Could not LazyLoad pkg.hold
[ERROR   ] Installed Packages:
py27-MySQLdb changed from absent to 1.2.5

I get a very similar error if the package is already installed, so it appears to be an issue with the parsing of pkg info -ao but I can't see what:

# pkg info -ao | grep py-MySQLdb
py27-MySQLdb-1.2.5             databases/py-MySQLdb

This is the salt environment on the minion.

           Salt: 2015.5.3
         Python: 2.7.10 (default, Jul 10 2015, 21:46:13)
         Jinja2: 2.7.3
       M2Crypto: 0.22
 msgpack-python: 0.4.2
   msgpack-pure: Not Installed
       pycrypto: 2.6.1
        libnacl: Not Installed
         PyYAML: 3.11
          ioflo: Not Installed
          PyZMQ: 14.6.0
           RAET: Not Installed
            ZMQ: 4.1.2
           Mako: Not Installed
        Tornado: Not Installed

[jfindlay]

@ari, thanks for the report. @cedwards, @kim0, any ideas here?

[amontalban]

I'm facing the same issue, I think the problem is when you use / in the package name:

java:
  pkg.installed:
    - name: java/openjdk7

ID: java
Function: pkg.installed
Name: java/openjdk7
Result: False
Comment: The following packages failed to install/update: java/openjdk7
Started: 18:39:27.116051
Duration: 37042.663 ms
Changes:   
    ----------
    openjdk:
        ----------
        new:
            7.80.15_1,1
        old:

[INFO    ] Running state [java/openjdk7] at time 18:39:27.116051
[INFO    ] Executing state pkg.installed for java/openjdk7
[DEBUG   ] Could not LazyLoad pkg.normalize_name
[DEBUG   ] Could not LazyLoad pkg.check_db
[DEBUG   ] Could not LazyLoad pkg.normalize_name
[INFO    ] Executing command 'pkg install  -y java/openjdk7' in directory '/root'
[INFO    ] Executing command 'pkg info -ao' in directory '/root'
[DEBUG   ] Could not LazyLoad pkg.hold
[ERROR   ] Installed Packages:
openjdk changed from absent to 7.80.15_1,1

[DEBUG   ] Refreshing modules...
[INFO    ] Loading fresh modules for state activity
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[INFO    ] Completed state [java/openjdk7] at time 18:40:04.158714

root@devserver:/srv # salt-call --versions
           Salt: 2015.5.3
         Python: 2.7.10 (default, Jul 30 2015, 01:18:54)
         Jinja2: 2.7.3
       M2Crypto: 0.22
 msgpack-python: 0.4.2
   msgpack-pure: Not Installed
       pycrypto: 2.6.1
        libnacl: Not Installed
         PyYAML: 3.11
          ioflo: Not Installed
          PyZMQ: 14.6.0
           RAET: Not Installed
            ZMQ: 4.1.2
           Mako: Not Installed
        Tornado: Not Installed

This is the only package in my highstate that I need to use name java/openjdk7 because it's the only way to install it. For all other packages that I install I do not get this problem.

Thank you!

[amontalban]

I see that this issue is a duplicate of #3888 which is from 2013.

[amontalban]

I've been checking this issue and the problem is with the following lines in freebsdpkg.py:

https://github.com/saltstack/salt/blob/5f828882b86d556a72088dc8e88134e7cde7a744/salt/modules/freebsdpkg.py#L273-L283

Basically what it does is to parse the pkg info -ao output and extract the package name and package version from the left column of the command:

root@devserver:~ # pkg info -ao | grep -i java/openjdk7
openjdk-7.80.15_1,1            java/openjdk7

root@devserver:~ # pkg info -ao | grep -i databases/py-MySQLdb
py27-MySQLdb-1.2.5             databases/py-MySQLdb

So in this case it will be:

openjdk != java/openjdk7
py27-MySQLdb != databases/py-MySQLdb

I think that the best way to go is to use pkg query like this pkg query "%n %v %o" where:

• %n = Package name
• %v = Package version
• %o = Package origin

Depending on desired package we can either parse origin or package name.

Probably this will take a lot of work and I'm a Python newbie but hope this save some time for the one taking this issue.

Thanks!

[jfindlay]

@amontalban, your approach seems much better and you are very welcome to submit a pull request against the 2015.5 branch. Please feel welcome to contact me if you need help reviewing your code.

[ari]

I think your analysis is good except for the line of code you found. pkgng is here:

https://github.com/saltstack/salt/blob/5f828882b86d556a72088dc8e88134e7cde7a744/salt/modules/pkgng.py#L334

That splits the package name around the last '-' to strip off the version and then puts the output into

__salt__['pkg_resource.add_pkg'](ret, pkgname, pkgver)
origins[pkgname] = origin

I'm not 100% clear what happens next to verify that the right package is installed. But I'd love to see some more debug logging to show what salt was comparing to what. And your idea of using pkg query "%n %v %o" looks much more robust.

Unfortunately I can read python, but I've got little idea on how to work on a big project like this or run unit tests.

[amontalban]

@ari @jfindlay This has been fixed in #26133 and has been merged in 2015.5 branch.

[jfindlay]

Thanks, @amontalban.

[ari]

I don't understand the salt branching workflow (all that date driven naming that I thought died with Windows 98). I see the commit went to 'develop'. Will that land in 2015.5.x or will it go into the next major release (2016.x)?

Thanks for fixing this.

[rallytime]

@ari It looks like this issue was fixed with #26133, which was submitted against the 2015.5 branch. The fix will be available in the next release, which will be labeled 2015.5.4 (current stable is 2015.5.3).

We use a merge-forward process where bugs should be submitted against the the earliest branch that the bug occurred in, and then we'll merge that forward. So, the original pull request went into 2015.5. Then we merged 2015.5 into 2015.8 (which is the next feature-release branch - we're currently preparing the 2015.8.0 release as well) and then we merged 2015.8 into develop. Does that help clear things up?

[ari]

Thanks. Sort of clear. merge-forward is a bit unusual and I guess it works as long as you don't forget to merge and end up with regressions in future releases.

As a user of salt, I find the versioning confusing. Not only are users supposed to know that 2015.8 is bigger than 2015.5.9 (I get quite confused with USA style date formats), but that there is no 2015.6 and that 2015.8 is likely to be released in October by the time it clears testing, is packaged and ready for use. And then I need to know that Lithium is the same as 2015.5. Or something.

But thanks for sorting this bug and for a terrific product!

[rallytime]

@ari Good comments/questions. The merge-forwards are done every day or so, and it's the entire branch that moves forward, so we won't forget anything. The only tricky thing there can sometimes be merge conflicts, but @basepi handles them all very well and researches them thoroughly. :)

The date versioning is based on ... You are correct in that the release month should coincide with when we release the packaged version. We try to estimate when the actual release will hit, but sometimes we're off. (For example, for 2015.5.0, that branch was originally 2015.2, but we missed the deadline, so we needed to rename.) We're still shooting for an August release of 2015.8.0. This number is purely based on when we can get the next feature (as opposed to bug-fix) release ready, which is why there was not a 2015.6 or 2015.7 release.

As far as the names from the Periodic Table, we should only be referring to the "Lithium" or "Hydrogen" name as a place-holder for the upcoming feature-release until the release has actually been given a number, such as 2015.5.0. Once the number is assigned, references to that release should disappear. (Though sometimes it's hard to drop that right away from our vernacular.) Hopefully that helps a little more. ;)

[ari]

And what benefits do you get from this versioning scheme? Most projects either go for short continuous development cycles (like Jenkins) with a single monotonically increasing number, or more commonly a major/minor/bugfix numbering which reflects whether a user is likely to need to update their configuration.

With salt I need to know when reading some mailing list messages that "fixed in 2015.2" actually means 2015.5. And googling for comments or bug reports is way more difficult than it would be otherwise.

As a user I need to understand that salt versions are:

• an element from the periodic table when the release is far in the future
• a year and a month (which change) when the release is nearer in the future, telling me roughly when to expect that release to be finished. Unless it isn't ready
• a year and a month and a bug fix version (which looks like a date, but isn't) when the release is in the past

I can cope, but it is confusing.

[jfindlay]

Salt's versioning scheme isn't my favorite, but the concept is arguably a reasonable alternative to semver and a single, incremented number. It is the same as that used by ubuntu. Sometimes I wish we had dropped the 20 part of the versions because saying or typing 2015.5.3 is a little more than I want, but in the end I think it is less confusing than ubuntu's 14.04.4 for many of the same reasons you mentioned.

As for 2015.2 vs 2015.5, I also agree that it is vexing trying to understand what happened with that release and why, but our release pattern is getting tighter and more predictable. The most important thing I think relating to our major releases is that our latest releases are becoming incrementally more stable, which is an accomplishment for a project with over a thousand total contributors and a QA team a little over a year old.

Unlike ubuntu, which selects codenames that cannot be ordered, chemical element names are intrinsically ordered, so there is at least a chance you don't have to go to wikipedia to figure out what that order is. In the future, as our release pattern converges on something stable, we may even designify the codenames in favor of the release dates, but that is speculative opinion on my part.