Katello 3.0 RC5

[pressplay21]

Does the script work with Katello 3.0 RC5?

[rdrgmnzs]

It does load the data into pulp 2.8 which is what Katello 3.0 RC5 ships with, unfortunately with the 30min I've been able to put into testing it I was not able to make Katello "see" that the errata is in the pulp repo.

[bmerjil]

Hope this helps. It seems that pulp-admin only updates the mongdb units_erratum in the pulp_database collection, but does not update the katello_errata table in foreman database on postgres. I have tried to sync the repositories but nothing yet. I don't know the cause yet as I'm kind of new to debugging.

[pescobar]

I tried this script with Katello 3.0 and I get this in the STDOUT

INFO: Loading errata XML
INFO: Getting server inventory
sh: pulp-admin: command not found
NOTICE: Skipping errata CEBA-2005:169 (CentOS and up2date - bugfix update) -- No packages found
NOTICE: Skipping errata CEBA-2005:842 (Important CentOS shadow-utils - bugfix update) -- No packages found
NOTICE: Skipping errata CEBA-2006:0613 (Important CentOS freetype - bugfix update) -- No packages found
.
.
.
NOTICE: Skipping errata CEBA-2007:0080 (CentOS gcc - bugfix update) -- No packages found
NOTICE: Skipping errata CEBA-2007:0101 (CentOS gcc4 - bugfix update) -- No packages found
NOTICE: Skipping errata CEBA-2007:0159 (CentOS rgmanager Update) -- No packages found
NOTICE: Skipping errata CEBA-2007:0160 (CentOS gfs-utils Update) -- No packages found

[bmerjil]

Hi Pescobar,

It looks like you might be missing the pulp-admin-client package which has the pulp-admin binary.

The Notice lines are normal as most of those would be for older CentOS version (e.g.: 3, 4), unless you are syncing those repositories, they can safely be ignored.

[pescobar]

Hi @bmerjil I noticed the error about pulp-admin and installed it. Now I don't ghet the command not found error but still the errata is not visible in the foreman webui after running the script and syncing the repo.

The step I am following are:

wget -N http://cefs.steve-meier.de/errata.latest.xml
grep -i ^default_password /etc/pulp/server.conf
./errata_import.pl --errata=errata.latest.xml --user=<admin> --password=<password>

then I go to the foreman webui and sync the repository but still no errata info available in my centos product. Am I doing something wrong?

thanks for your help.

[bmerjil]

@pescobar Ya that is the same issue I'm still running in to as well. Still trying to figure that out.

[rdrgmnzs]

@pescobar @bmerjil Yeah, that is the same issue as I'm seeing as well. Unfortunately since I no longer work at a place using Katello I only have time to debug on the weekends which has made finding what the issue is difficult. I'll try and give it some more time soon, but in the meantime if you find what the issue is please let me know.

[demueller]

Getting this issue in Katello Version 3.0 as well

[eduardohki]

I got the same "package not found" problem in Katello 3.0.

After some tests I realized I was using the Foreman/Katello admin password - and Pulp has its own admin account/password, which can be found in: /etc/pulp/server.conf

Using the correct username/password I was able to upload errata information in Pulp.

Maybe this needs to be added into setup procedures in README.md

[demueller]

Hello eduardohki, thank you for advice. After providing the right password from /etc/pulp/server.conf the script started properly, but after waiting for abount 2 hours, after the script went to the end i updated the CentOS repos again with fortunately no result, no errata packages avaliable on the foreman-gui.

Is it the right end of the Skcript, no finish message or like that?

DEBUG: Package: xen-ocaml-devel-4.2.5-38.el6.centos.alt.x86_64.rpm not found DEBUG: Package: xen-runtime-4.2.5-38.el6.centos.alt.x86_64.rpm not found NOTICE: Skipping errata CESA-2105:X001 (Important xen kernel libvirt python-virtinst e1000e (Xen4CentOS Only) Security Update) -- No packages found DEBUG: Skipping meta

[eduardohki]

Hi demueller,

I got the same issue, and this last log message isn't related to the this problem...

I found out that the issue now is about how Katello 3.0 syncs their repository and generate applicability.

In the Katello 2.x, after every update, Katello launches a new task to regenerate the applicability of the referenced repo.

In Katello 3.0, I found out that this procedure is bundled within the repo sync, and it doesn't run if there is no new updates.

So this script has no problem itself, just the Katello's funcionality changed. :/

I'm looking forward to understand how to run a manual "regenerate applicability" for the related repo.

[bmerjil]

So I have been trying to dig down the rabbit hole of katello and have found that most of the calls, made during a repository sync and metadata build, are in the python{ver}/site-pacakges/pulp* directories. There is a call to generate a temp db (pulp_rpm/plugins/importers/yum/repomd/metadata.py) with the metadata that is located on the server. In my case the official EPEL repository that I sync from has this available so it works fine. If there was a way to implement the pulp* python scripts on local files that could possibly work.

[eduardohki]

I found how to make the "applicability generation", from the Foreman IRC: [14:50] eduardohki: so, you should be able to go into Settings > Katello [14:50] eduardohki: change force_post_sync_actions to true [14:50] eduardohki: and that should force on your next sync, those actions to occur

I made this and It worked again :)

[pescobar]

I tried to follow @eduardohki instructions and I enabled the change force_post_sync_actions to true, after this I did:

wget -N http://cefs.steve-meier.de/errata.latest.xml
pulp_pass=`grep -i ^default_password /etc/pulp/server.conf |awk {'print $2'}`
./errata_import.pl --errata=errata.latest.xml --user=admin --password=$pulp_pass

Then I synced the repository but still I get no errata info.

I attach a screenshot of my yum repository setup.

Any suggestion why it's not working?

[eduardohki]

@pescobar only "updates" repo have errata information. You are trying to sync the "os" repo.

In CentOS, there are 3 main repos:

• os: the base system (static, like the installation media)
• updates: updates provided by centos group (which can have errata info)
• extras: extra tools for centos.

You should sync the "updates" repo to have Errata information.

***Update: I was wrong... I have errata information in os repo too...

[pescobar]

@eduardohki thanks for your help. I tried to sync the "updates" repository, execute errata_import.pl and resync it again but still I get no errata info :? I am puzzled...

I get many errors like these ones but I think this should not be a problem, isn't it?

NOTICE: Skipping errata CESA-2016:1406 (Important CentOS kernel Security Update) -- No packages found
NOTICE: Skipping errata CESA-2016:C001 (ipa and glusterfs Update) -- No packages found
NOTICE: Skipping errata CESA-2105:X001 (Important xen kernel libvirt python-virtinst e1000e (Xen4CentOS Only) Security Update) -- No packages found

I attach a screenshot of my "centos updates" repository config in case anyone can point what I am doing wrong

[rdrgmnzs]

@pescobar Could you run the following manually to get a list of your repos: pulp-admin -u admin -p [Pulp Password] repo list

I believe you are running into an issue, because this script isn't smart enough to figure out which are the "base" repos and Katello creates a new repo every time you publish a content view.

The work around for this is to use the --include-repo flag and specify exactly which repos the errata should be inserted into.

[rdrgmnzs]

@eduardohki You are the man! Thanks for figuring out the force_post_sync_actions flag. Just tried that on a katello-deploy instance and it worked beautifully.

[demueller]

@eduardohki, force_post_sync_actions worked best for me. Thank you guys. I have got my new Errata Information/Packages from the OS repo. Is there a way to automate the whole process? I mean starting a skript is no problem, but what about the gui, can u automate the sync process?

[eduardohki]

@demueller @brdude I'm glad it worked for you guys.

We can automate the sync process using hammer (cli), and I'm looking to embed it to this script.

I ask to reinforce the feature request below, to build an API in Katello in order to import these errata information in a more elegant and integrated manner. What do you think?

http://projects.theforeman.org/issues/8656

[rdrgmnzs]

@eduardohki That would be great and I do hope they add it someday.

[petr22]

Hello everyone, I did a fresh install of katello 3.0 and tried to import errata into "Centos 6 updates" repository with this script but it didnt work. I think the problem is that you compare rpm package names in repository with package names in the errata.latest.xml file. But in the repository there are only the newest packages which dont have any errata obviously. For older packages there are only DRPMs and those arent compared with the errata xml file. As you can see in the official repository http://mirror.centos.org/centos/6/updates/x86_64/ . I am not sure if its supposed to work this way but if you sync a new repository, you wont get any errata imported. Or am I wrong?

Another problem is that pulp doesnt even sync all drpms. I reported it to the pulp guys on IRC, who created a ticket https://pulp.plan.io/issues/2094 .

[friendlypenguin]

@petr22: do you see the errata under Content > Errata ? Be sure to Publish a new version of your content view and promote it to the desired environment in order to be able to install the errate on the content hosts.

I can confirm, that I was able to use this script on a katello server with the following specs:

- CentOS: Linux release 7.2.1511 (Core)
- katello: noarch Version     : 3.0.0, Release     : 9.el7
- pulp-server: noarch Version     : 2.8.4 Release     : 1.el7

However, after importing errata, resyncing repos, publish and promote new content views to environments, I only see one single errata: CESA-2016:1486 CentOS samba Security Update Security Advisory - Moderate 3 Applicable, 3 Installable 7/26/16

Investigating https://lists.centos.org/pipermail/centos-announce/2016-July/date.html it seems that this is the newest one. no others are shown. (as of today, 27. July 2016)

Does anyone know what could be the problem here?

[friendlypenguin]

I was able to sort the problem out: If a package is already in a content view, then the downloaded errata goes in the repository of that content-view and not in the "base"-repo. The single errata I saw was exactly the one that was put in the "base"-repo. All other erratas where put in the repo of the latest version of the content-view of one of the environments (cannot say why testenv and not prodenv...):

--> Code (the codeblock functionality breaks the backticks) for i in [backtick here]>pulp-admin -uadmin -p[pulppassword] rpm repo list | grep ^Id: | awk '{ print $2 }'|sort [backtick here]; do echo *$i***** ; pulp-admin -uadmin -p[pulppassword] rpm repo content errata --repo-id=$i; done ***_Org-Centos7-extras_x8664*** *_Org-Centos7-os_x8664******* *_Org-Centos7-updates_x8664******* Description: Not available Id: CESA-2016:1486 Severity: Moderate Title: CentOS samba Security Update Type: security

*_Org-Library-ProdEnvView-Centos7-extras_x8664******* *_Org-Library-ProdEnvView-Centos7-os_x8664******* *_Org-Library-ProdEnvView-Centos7-updates_x8664**** *_Org-ProdEnvView-1_0-Centos7-updates_x8664****** _Org-ProdEnvView-2_0-Centos7-extras_x8664****** _Org-ProdEnvView-2_0-Centos7-os_x8664****** _Org-ProdEnvView-2_0-Centos7-updates_x8664****** _Org-TestEnv-TestEnvView-Centos7-extras_x8664****** _Org-TestEnv-TestEnvView-Centos7-os_x8664****** _Org-TestEnv-TestEnvView-Centos7-updates_x8664****** _Org-TestEnvView-1_0-Centos7-extras_x8664****** _Org-TestEnvView-1_0-Centos7-os_x8664****** _Org-TestEnvView-1_0-Centos7-updates_x8664****** _Org-TestEnvView-2_0-Centos7-extras_x8664***** Description: Not available Id: CEBA-2014:2014 Severity: Title: CentOS golang BugFix Update Type: bugfix

Description: Not available Id: CEBA-2015:0081 Severity: Title: CentOS golang-github-gorilla-mux BugFix Update Type: bugfix

*_Org-TestEnvView-2_0-Centos7-katello-client_x8664******* *_Org-TestEnvView-2_0-Centos7-os_x8664******* Description: Not available Id: CEBA-2015:2182 Severity: Title: CentOS kexec-tools BugFix Update Type: bugfix

Description: Not available Id: CEBA-2014:1252 Severity: Title: CentOS gutenprint FASTTRACK BugFix Update Type: bugfix

Description: Not available Id: CESA-2015:2393 Severity: Moderate Title: CentOS wireshark Security Update Type: security ... .. .

So i guess I have to let the script put the errata in the repos Org-Centos7-extras_x86_64 Org-Centos7-os_x86_64 Org-Centos7-updates_x86_64 And not in the other repos...

I have to learn perl this afternoon... :-/

[rdrgmnzs]

@friendlypenguin you can already specify which repos the script inserts errata into. This makes it so that only the listed repos have their existing errata checked and new errata inserted into.

/errata_import.pl --errata=/tmp/errata.latest.xml --rhsa-oval=/tmp/com.redhat.rhsa-all.xml --user=admin --password=[PASSWORD] --debug --include-repo=Org-Centos7-extras_x86_64 Org-Centos7-os_x86_64 Org-Centos7-updates_x86_64

Guess adding this to the documentation will make it easier on people. Let me do that now.

[friendlypenguin]

@brdude that's exactly what solves the problem. I used the command slightly different with multiple times the --include-repo option as mentioned in the source code. I cannot confirm (and neither dispute) that your version above will work, but certainly the following does:

./errata_import.pl --errata=/tmp/errata.latest.xml --user=admin --password=[PASSWORD] --include-repo=Org-Centos7-extras_x86_64 --include-repo=Org-Centos7-os_x86_64 --include-repo=Org-Centos7-updates_x86_64

I was able to import the errata into katello (after syncing they were visible under content > errata). For beeing able to apply them to (content) hosts you have to create a new version of the corresponding content view an then promote it to the needed environment.

So I can confirm, that your script works on CentOS: Linux release 7.2.1511 (Core), katello: noarch Version : 3.0.0, Release : 9.el7 pulp-server: noarch Version : 2.8.4 Release : 1.el7.

I guess this issue can be closed.

Many thanks for your script, it's pure awesomeness!

[bmerjil]

So an update on my situation. Thanks to everyone here finding the flag and the configurations. I have now setup a cron job with a user that only has permission to manage the packages, errata, and content views. This is using the errata_import script with hammer so now I have a limited automation for this.