:Author: Gustavo Niemeyer :Contact: niemeyer@conectiva.com :Revision: $Rev$ :Date: $Date$
.. contents::
The Smart Package Manager project has the ambitious objective of creating smart and portable algorithms for solving adequately the problem of managing software upgrading and installation. This tool works in all major distributions, and will bring notable advantages over native tools currently in use (APT, APT-RPM, YUM, URPMI, etc).
From The Free On-line Dictionary of Computing::
smart
1. <programming> Said of a program that does the {Right Thing}
in a wide variety of complicated circumstances. (...)The development of Smart Package Manager started on May 4th, 2004, and version 1.0 was released on Aug 14th, 2008, after extended beta testing.
Smart has been developed with modularity and flexibility in mind. It's completely backend-based, and package-manager-agnostic. Support is currently implemented for RPM, DPKG, and Slackware package management systems, and porting it to new systems should be very easy.
That's one of the most interesting aspects of Smart Package Manager,
and the one who has motivated calling it smart. Computing
transactions respecting the relations involved in the package
management world may become an unpleasant task when thousands of
packages and relations are being considered, or even when just
a few complex relations turn the most obvious choice into the
unwanted one.
While other applications try to find a possible solution to satisfy the relations involved in some user-requested operation, and sometimes even fail to do so [1]_, Smart goes beyond it. In the kernel of Smart Package Manager lives an algorithm that will not only find a solution, if one is available, but will find the best solution. This is done by quickly weighting every possible solution with a pluggable policy, which redefines the term "best" depending on the operation goal (install, remove, upgrade, etc).
This behavior has many interesting consequences. In upgrades, for instance, while precedence is given to newer versions, intermediate versions may get selected if they bring a better global result for the system. Packages may even be reinstalled, if different packages with the same name-version pair have different relations, and the one not installed is considered a better option.
Another important goal achieved with the transaction algorithm is that, even though it is able to check and fix relations in the whole system, it will work even when there are broken relations in installed packages. Only relations related to the operation being made are checked for correctness.
.. [1] Check Case Studies_ for real cases where the algorithm
works better than what is implemented in other applications.
Smart has multiple native and completely integrated interfaces:
Command line interface, with several useful subcommands: update, install, reinstall, upgrade, remove, check, fix, download, search, and more.
Shell interface, with command and argument completion, making it easy to perform multiple operations quickly using a local or remote terminal.
Graphic interface, offering the friendliness of visual user interaction.
Command line interface with graphic feedback, allowing one to integrate the power of command line with graphic environments.
Besides these interfaces, ksmarttray is also included in the Smart package. It notifies users about available updates using a KDE tray icon.
Channels are the way Smart becomes aware about external repositories of information. Many different channel types are supported, depending on the backend and kind of information desired:
Priorities are a powerful way to easily handle integration of multiple channels and explicit user setups regarding preferred package versions.
Basically, packages with higher priorities are considered a better option to be installed in the system, even when package versions state otherwise. Priorities may be individually assigned to all packages in given channels, to all packages with given names, and to packages with given names inside given channels.
With custom priority setups, it becomes possible to avoid unwanted upgrades, force downgrades, select packages in given channels as preferential, and other kinds of interesting setups.
Smart offers a very flexible mirror support. Mirrors are URLs that supposedly provide the same contents as are available in other URLs, named origins. There is no internal restriction on the kind of information which is mirrored. Once an origin URL is provided, and one or more mirror URLs are provided, these mirrors will be considered for any file which is going to be fetched from an URL starting with the origin URL.
Mirror precedence is dynamically computed based on the history of downloads of all mirrors available for a given origin URL (including the origin site itself). The fastest mirrors and with less errors are chosen. When errors occur, the next mirror in the queue is tried.
For instance, if a mirror http://mirror.url/path/ is provided
for the origin ftp://origin.url/other/path/, and a file in
ftp://origin.url/other/path/subpath/somefile is going to be
fetched, the mirror will be considered for being used, and the
URL http://mirror.url/path/subpath/somefile will be used if
the mirror is chosen. Notice that strings are compared and
replaced without any pre-processing, so that it's possible to
use different schemes (ftp, http, etc) in mirror entries, and
even URLs ending in prefixes of directory entries.
Smart has a fast parallel downloading mechanism, allowing multiple connections to be used for one or more sites. The mechanism supports:
and more.
At the moment, the following schemes are nativelly supported:
Additionally, the following schemes are supported when pycurl is available:
Smart Package Manager implements builtin support for removable media (CDROMs, DVDs, etc) in most of the supported channel types. The following features are currently implemented:
Smart Package Manager may be run in many different ways, depending on the interface in use and on the intended goal.
The following command would install the foobar package, for instance::
smart install foobarWhile the following command would install the foobar package, but with
graphic output::
smart --gui install foobarTo open the graphic interface in interactive mode, one may simply run::
smart --guiSimilarly, the following command would open the shell interface::
smart --shellExtensive help is available for all commands, by using the --help
switch::
smart --help
smart install --help
smart channel --help
...:Core: Smart is written in Python, with some core modules rewritten as C extensions for memory savings and performance gains. With that in mind, the core system of Smart depends on Python 2.3 or higher, and a C compiler to build the extensions.
:Graphic Interface:
The "gtk" graphic interface depends on pygtk 2.4 or higher.
The "qt" graphic interface depends on pyqt 3.3 (not 4.x).
:RPM backend:
The RPM backend depends on the Python rpm module of RPM 4.4 or
higher, due to a limitation which was present in previous versions
of the ts.dbMatch() method, and the availability of the
readHeaderFromFD() function.
In the `contrib/patches/` subdirectory there are patches for
previous RPM versions including the missing functionality. There
are also pre-packaged binary versions which include the patched
module without requiring changes in other tools.:DPKG backend: There are no extra dependencies besides DPKG itself.
:Slackware backend:
There are no extra dependencies besides the packaging scripts
installpkg, upgradepkg and removepkg.
In this section will be described real cases showing Smart behavior
in comparison with other tools, or handling unusual situations.
Notice that Smart was not tuned to work in any of these cases, and the reason it works is because handling unusual situations was the initial project goal.
This case happened in a real world environment where a weakness in
the algorithm used by APT (which is the same used in APT-RPM)
turned a simple operation into a problem of obscure results.
Smart Package Manager was used in the same environment to show
its results.
The problem starts when an installation of xscreensaver is tried::
[root@damien:/root] apt-get install xscreensaver Reading Package Lists... Done Building Dependency Tree... Done Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming.
Since you only requested a single operation it is extremely likely that the package is simply not installable and a bug report against that package should be filed. The following information may help to resolve the situation:
The following packages have unmet dependencies: xscreensaver: Depends: libglade-2.0.so.0 Depends: libxml2.so.2 E: Broken packages
The error shown makes the user believe that libglade-2.0.so.0 and
libxml2.so.2 are not available. That's not the case::
[root@damien:/root] apt-get install libxml2 Reading Package Lists... Done Building Dependency Tree... Done Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming.
Since you only requested a single operation it is extremely likely that the package is simply not installable and a bug report against that package should be filed. The following information may help to resolve the situation:
The following packages have unmet dependencies: libxml2: Depends: glibc-iconv but it is not going to be installed E: Broken packages
Another misguiding error message. Let's go further::
[root@damien:/root] apt-get install glibc-iconv Reading Package Lists... Done Building Dependency Tree... Done Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming.
Since you only requested a single operation it is extremely likely that the package is simply not installable and a bug report against that package should be filed. The following information may help to resolve the situation:
The following packages have unmet dependencies: glibc-iconv: Depends: glibc-gconvdata (= 2.3.3) but 1:2.3.2-586_1cl is to be installed E: Broken packages
Version 2.3.3 is needed, but 1:2.3.2-586_1cl is to be installed. This
message is mostly correct. The only problem is, "1:2.3.2-586_1cl" is
already installed::
[root@damien:/root] apt-cache policy glibc-gconvdata glibc-gconvdata: Installed: 1:2.3.2-586_1cl Candidate: 1:2.3.2-586_1cl Version Table: *** 1:2.3.2-586_1cl 0 100 RPM Database 0:2.3.3-69473cl 0 500 file: conectiva/all pkglist
The problem was found. A package from another repository (586_1cl shows it's not native, in that specific case) has a higher epoch than the one available in the usual repository. This clearly shows that the APT algorithm marks a single version as candidate, and when this is not the wanted version for some operation, the whole operation is compromised.
When testing Smart Package Manager in the same environment, the
expected result is obtained::
[root@damien:/root] smart install xscreensaver Updating cache... ######################################## [100%]
Computing transaction...
Downgrading packages (1): glibc-gconvdata-0:2.3.3-69473cl.i386
Installing packages (4): glibc-iconv-0:2.3.3-69473cl.i386 libglade2-2.4.0-68154cl.i386 libxml2-2:2.6.13-67598cl.i386 xscreensaver-4.15-69825cl.i386
Confirm changes (Y/n)?
Smart correctly selected glibc-gconvdata for downgrading as the only
possibility of performing the user requested operation.
This is another real case, and is being reproduced in a controlled environment for tests with YUM, APT-RPM, and Smart.
The issue is, a package named A requires package BCD explicitly, and
RPM detects implicit dependencies between A and libB, libC, and libD.
Package BCD provides libB, libC, and libD, but additionally there
is a package B providing libB, a package C providing libC, and
a package D providing libD.
In other words, there's a package A which requires four different symbols,
and one of these symbols is provided by a single package BCD, which happens
to provide all symbols needed by A. There are also packages B, C, and D,
that provide some of the symbols required by A, but can't satisfy all
dependencies without BCD.
The expected behavior for an operation asking to install A is obviously
selecting BCD to satisfy A's dependencies, on the other hand, YUM and
APT fail to deliver that as a guaranteed operation, as is shown
below.
First, let's see how YUM deals with the problem::
[root@burma ~]% yum install A Setting up Install Process Setting up Repo: localpub repomd.xml 100% |=========================| 951 B 00:00 Reading repository metadata in from local files localpub : ################################################## 5/5 Resolving Dependencies --> Populating transaction set with selected packages. Please wait. ---> Downloading header for A to pack into transaction set. A-1.0-1cl.i386.rpm 100% |=========================| 1.0 kB 00:00 ---> Package A.i386 0:1.0-1cl set to be installed --> Running transaction check --> Processing Dependency: libD for package: A --> Processing Dependency: libC for package: A --> Processing Dependency: libB for package: A --> Processing Dependency: BCD for package: A --> Restarting Dependency Resolution with new changes. --> Populating transaction set with selected packages. Please wait. ---> Downloading header for D to pack into transaction set. D-1.0-1cl.i386.rpm 100% |=========================| 1.0 kB 00:00 ---> Package D.i386 0:1.0-1cl set to be installed ---> Downloading header for C to pack into transaction set. C-1.0-1cl.i386.rpm 100% |=========================| 1.0 kB 00:00 ---> Package C.i386 0:1.0-1cl set to be installed ---> Downloading header for B to pack into transaction set. B-1.0-1cl.i386.rpm 100% |=========================| 1.0 kB 00:00 ---> Package B.i386 0:1.0-1cl set to be installed ---> Downloading header for BCD to pack into transaction set. BCD-1.0-1cl.i386.rpm 100% |=========================| 1.0 kB 00:00 ---> Package BCD.i386 0:1.0-1cl set to be installed --> Running transaction check
Dependencies Resolved Transaction Listing: Install: A.i386 0:1.0-1cl
Performing the following to resolve dependencies: Install: B.i386 0:1.0-1cl Install: BCD.i386 0:1.0-1cl Install: C.i386 0:1.0-1cl Install: D.i386 0:1.0-1cl Is this ok [y/N]:
YUM selected all packages for installation, even though BCD
alone would satisfy A's dependencies.
Let's see how APT deals with that::
[root@burma ~]% apt-get install A Reading Package Lists... Done Building Dependency Tree... Done The following extra packages will be installed: B BCD The following NEW packages will be installed: A B BCD 0 upgraded, 3 newly installed, 0 removed and 0 not upgraded. Need to get 0B/4055B of archives. After unpacking 0B of additional disk space will be used. Do you want to continue? [Y/n] n
As a coincidence, APT did a better job, and selected only B and
BCD to satisfy A's dependency, which is still not right.
Now, let's see how Smart would solve the problem::
[root@burma ~]% smart install A Updating cache... ######################################## [100%]
Computing transaction...
Installing packages (2): A-1.0-1cl@i386 BCD-1.0-1cl@i386
2.7kb of package files are needed.
Confirm changes (Y/n)?
Smart correctly selected only BCD, since it's necessary anyway, and
solves all dependencies.
That's another interesting case which was tested with APT-RPM and YUM.
In this case, there's a package A version 1.0 installed in the
system, and there are two versions available for upgrading: 1.5 and 2.0.
Version 1.5 may be installed without problems, but version 2.0 has a
dependency on B, which is not available anywhere.
In this case, the best possibility is upgrading to 1.5, since upgrading to 2.0 is not an option.
Let's see how APT reacts to this situation::
[root@burma ~]% apt-get upgrade A Reading Package Lists... Done Building Dependency Tree... Done The following packages have been kept back A 0 upgraded, 0 newly installed, 0 removed and 1 not upgraded.
APT seems to refuse to upgrade A, even though version 1.5 might be
installed without problems.
What happens when forcing APT to install A::
[root@burma ~]% apt-get install A Reading Package Lists... Done Building Dependency Tree... Done Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming.
Since you only requested a single operation it is extremely likely that the package is simply not installable and a bug report against that package should be filed. The following information may help to resolve the situation:
The following packages have unmet dependencies: A: Depends: B but it is not installable E: Broken packages
It really refuses to install the newest version, and doesn't consider the possibility of using version 1.5.
Now, let's see how YUM would handle it.
:Update: This test case was showing the wrong results for YUM, since it was using a cached version of the package header that didn't present the missing dependency. I apologise for showing invalid results.
::
[root@burma ~]% yum update Setting up Update Process Setting up Repo: localpub repomd.xml 100% |=========================| 951 B 00:00 Reading repository metadata in from local files primary.xml.gz 100% |=========================| 809 B 00:00 MD Read : ################################################## 3/3 localpub : ################################################## 3/3 Resolving Dependencies --> Populating transaction set with selected packages. Please wait. ---> Downloading header for A to pack into transaction set. A-2.0-1cl.i386.rpm 100% |=========================| 1.3 kB 00:00 ---> Package A.i386 0:2.0-1cl set to be updated --> Running transaction check --> Processing Dependency: B for package: A --> Finished Dependency Resolution Error: missing dep: B for pkg A
Just like APT, YUM selected version 2.0 and didn't consider the availability of an intermediate version.
Now, let's see how Smart would behave in the same situation::
[root@burma ~]% smart upgrade Loading cache... Updating cache... ######################################## [100%]
Computing transaction...
Upgrading packages (1): A-1.5-1cl@i386
1.3kb of package files are needed.
Confirm changes (Y/n)?
Smart correctly selects the intermediate version 1.5, which is the only viable possibility given the current options.
This case presents the following situation: there's a package A,
installed in the system, which depends on libfoo, currently
being provided by B 1.0. What happens if B is upgraded to
version 2.0, but libfoo is moved to be provided by package C?
The expected behavior would be to upgrade B to version 2.0,
and install C to satisfy A's dependency.
That's not what happens with APT::
[root@burma ~]% apt-get dist-upgrade Reading Package Lists... Done Building Dependency Tree... Done Calculating Upgrade... Done The following packages will be upgraded B The following packages will be REMOVED: A 1 upgraded, 0 newly installed, 1 removed and 0 not upgraded. Need to get 0B/1321B of archives. After unpacking 0B of additional disk space will be used. Do you want to continue? [Y/n]
Let's see Smart in the same situation::
[root@burma ~]% smart upgrade Loading cache... Updating cache... ######################################## [100%]
Computing transaction...
Upgrading packages (1): B-2.0-1cl@i386
Installing packages (1): C-2.0-1cl@i386
2.6kB of package files are needed.
Confirm changes (Y/n)?
Smart correctly selected package C for installation as a viable
possibility of leaving A installed in the system while upgrading
B.
This is the credit section, where people and institutions that have somehow contributed to the project are mentioned.
:Conectiva, Inc.: Funded the creation of Smart, and its development up to August of 2005.
:Canonical Ltd: Funded Smart development up to November of 2009.
:Unity Linux: Smart development and deployment support.
:Wanderlei Cavassin: Conectiva's research & development coordinator, who believed the project was viable and encouraged the author to work on it.
:Ednilson Miura & Herton Ronaldo Krzesinski: Conectiva employees, helped setting up many distributions for tests whenever necessary.
:Andreas Hasenack: Conectiva employee, helped as being the first brave pre-alpha tester, and contributed with many ideas, discussions, etc.
:Arnaldo Carvalho de Melo: Conectiva board member, helped with the "channel of mirrors" idea and by encouraging the author to build a generic channel information method.
:Others @ Conectiva: Many other people in Conectiva helped with ideas and alpha-testing in general during the pre-release period of Smart development.
:Guilerme Manika & Ruda Moura: Ancient Conectiva employees, now board members of the Haxent company, helped by testing Smart extensively in Fedora, reporting many bugs and suggesting changes. They have also created the Smart FAQ_.
:APT-RPM & Debian:
Experience on packaging and ideas for a better framework were
developed while the author of Smart worked as the APT-RPM
maintainer.
:Jeff Johnson: Contributed as being the RPM maintainer itself, and in many discussions regarding packaging theory in general.
:Seth Vidal:
YUM author, and member of the Duke University, contributed
to Smart with the development of the XML MetaData repository
format and discussions about it.
:Michael Vogt: Currently the maintainer of the Synaptic, used to co-maintain it with the author of Smart. Many of his ideas ended up being adopted in Smart as a consequence.
:Sebastian Heinlein: Author of the package icons for Synaptic, that were mercilessly stolen to be used in Smart's graphic interface.
:TaQ/PiterPunk at #slackware-br: These guys helped Smart development by explaining details of Slackware practices regarding packaging.
:Matt Zimmerman:
Debian/Ubuntu developer and co-maintainer of the APT software,
helped by shining some light regarding details of the DPKG
pre-depends ordering expectations.
:Mauricio Teixeira: FAQ maintenance, YaST2 channel maintainer, "tracker cleaner", general suggestions and code contributions.
:Jonathan Rocker: Documentation help.
.. _FAQ: http://zorked.net/smart/FAQ.html