search

rpm-software-management / createrepo_c

C implementation of the createrepo.
http://rpm-software-management.github.io/createrepo_c
GNU General Public License v2.0
102 stars 93 forks source link

readme

createrepo_c

C implementation of createrepo

Run createrepo -h for usage syntax.

Devel tips

Building

Package build requires - Pkg name in Fedora/Ubuntu:

From your checkout dir:

mkdir build
cd build/
cmake ..
make

To build the documentation, from the build/ directory:

make doc

Note: For build with debugging symbols you could use (from the build/ directory):

cmake -DCMAKE_BUILD_TYPE:STRING=DEBUG .. && make

Building from an rpm checkout

E.g. when you want to try weak and rich dependencies.

cmake .. && make

Note: The RPM must be built in that directory

Commands I am using for building the RPM:

cd /home/tmlcoch/git/rpm
CPPFLAGS='-I/usr/include/nss3/ -I/usr/include/nspr4/' ./autogen.sh --rpmconfigure --with-vendor=redhat --with-external-db --with-lua --with-selinux --with-cap --with-acl --enable-python
make clean && make

Other build options

-DENABLE_LEGACY_WEAKDEPS=ON

Enable legacy SUSE/Mageia/Mandriva weakdeps support (Default: ON)

-DENABLE_THREADED_XZ_ENCODER=ON

Threaded XZ encoding (Default: OFF)

Note: This option is disabled by default, because Createrepo_c parallelizes a lot of tasks (including compression) by default; this only adds extra threads on XZ library level which causes thread bloat and for most usecases doesn't bring any performance boost. On regular hardware (e.g. less-or-equal 4 cores) this option may even cause degradation of performance.

-DENABLE_DRPM=ON

Enable DeltaRPM support using drpm library (Default: ON)

Adds support for creating DeltaRPMs and incorporating them into the repository.

-DWITH_ZCHUNK=ON

Build with zchunk support (Default: ON)

-DWITH_LIBMODULEMD=ON

Build with libmodulemd support (Default: ON)

Adds support for working with repos containing Fedora Modularity metadata.

Build tarball

utils/make_tarball.sh [git revision]

Without git revision specified HEAD is used.

Build Python package

To create a binary "wheel" distribution, use:

python setup.py bdist_wheel

To create a source distribution, use:

python setup.py sdist

Installing source distributions require the installer of the package to have all of the build dependencies installed on their system, since they compile the code during installation. Binary distributions are pre-compiled, but they are likely not portable between substantially different systems, e.g. Fedora and Ubuntu.

Note: if you are building a bdist or installing the sdist on a system with an older version of Pip, you may need to install the scikit-build Python package first.

To install either of these packages, use:

pip install dist/{{ package name }}

To create an "editable" install of createrepo_c, use:

python setup.py develop

Note: To recompile the libraries and binaries, you muse re-run this command.

Build RPM package

Modify createrepo_c.spec and run:

utils/make_rpm.sh

Note: Current .spec for Fedora rawhide

Testing

All unit tests run from librepo checkout dir

Build C tests && run c and python tests

make tests && make test

Note: For a verbose output of testing use: make ARGS="-V" test

Run only C unittests (from your checkout dir):

build/tests/run_tests.sh

Note: The C tests have to be built by make tests)!

Run only Python unittests (from your checkout dir):

PYTHONPATH=`readlink -f ./build/src/python/` python3 -m unittest discover -bs tests/python/

Note: When compiling createrepo_c without libmodulemd support add WITH_LIBMODULEMD=OFF

Links

Bugzilla

Important notes

In original createrepo sha is a nickname for the sha1 checksum. Createrepo_c mimics this behaviour.

Contribution

Here's the most direct way to get your work merged into the project.

  1. Fork the project

  2. Clone down your fork

  3. Implement your feature or bug fix and commit changes

  4. If the change fixes a bug at Red Hat bugzilla, or if it is important to the end user, add the following block to the commit message:

    = changelog = msg: message to be included in the changelog type: one of: bugfix/enhancement/security (this field is required when message is present) resolves: URLs to bugs or issues resolved by this commit (can be specified multiple times) related: URLs to any related bugs or issues (can be specified multiple times)

    • For example::

      = changelog = msg: Enhance error handling when locating repositories type: bugfix resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1762697

    • For your convenience, you can also use git commit template by running the following command in the top-level directory of this project:

      git config commit.template ./.git-commit-template

  5. In a separate commit, add your name and email into the authors file as a reward for your generosity

  6. Push the branch to your fork

  7. Send a pull request for your branch

Differences in behavior between createrepo_c and createrepo

Checksums after update

Use case:

createrepo_c result:

original createrepo result:

Skip symlinks param

Use case:

createrepo_c result:

original createrepo result:

Base path from update-md-path repo

Use case:

createrepo_c results:

original createrepo result:

Crippled paths in filelists.xml after update

Use case:

createrepo_c results:

original createrepo result:

--update leaves behind some old repodata files

Use case:

createrepo_c results:

original createrepo result:

Mergerepo_c

Default merge method

Modifyrepo_c

Modifyrepo_c is compatible with classical Modifyrepo except some misbehaviour:

Batch file

When there is need to do several modification to repository (repomd.xml) a batch file could be used.

Batch file is Modifyrepo_c specific. It is not supported by the classical Modifyrepo - at least not yet.

Example

# Add:
#   [<path/to/file>]
#   <options>

# Metadata that use a bunch of config options
[some/path/comps.xml]
type=group
compress=true
compress-type=gz
unique-md-filenames=true
checksum=sha256
new-name=group.xml

# Metadata that use default settings
[some/path/bar.xml]

# Remove:
#   [<metadata name>]
#   remove=true

[updateinfo]
remove=true

Supported options

Option name Description Supported value(s) Default
path Path to the file. When specified it override the path specified in group name (name between [] parenthesis) Any string group name (string between '[' ']')
type Type of the metadata Any string Based on filename
remove Remove specified file/type from repodata true or false | false
compress Compress the new metadata before adding it to repo true or false | true
compress-type Compression format to use gz, bz2, xz | gz
checksum Checksum type to use md5, sha, sha1, sha224, sha256, sha384, sha512 | sha256
unique-md-filenames Include the file's checksum in the filename true or false | true
new-name New name for the file. If compress is true, then compression suffix will be appended. If unique-md-filenames is true, then checksum will be prepended. Any string Original source filename

Notes