search

CentOS / container-pipeline-service

Code, infrastructure and deployment backend for the CentOS Container Pipeline backing up build system for registry.centos.org
https://registry.centos.org
GNU General Public License v3.0
49 stars 27 forks source link

Continues processing index entries upon errors, updates buildconfig names to lower case #617

Closed navidshaikh closed 6 years ago

navidshaikh commented 6 years ago

  1. Updated the PR to change the behavior of seed-job as well. Now the seed-job will continue to process rest of the entries in index if it encountered errors while processing index. It will print the error and move on.

  2. Fix OSIO 1522 Since OpenShift doesn't allow the value of metadata.name to have uppercase chars. Added a regular expression to validate the pipeline name generated for using in template's metadata.name field. oc imposes certain guidelines for same and provides the regular expression in error/help message to be used for validation in case it found some invalid string.

We are using same regular expression as provided by oc to validate the pipeline name while processing index and generating build configs from it.

Added a custom exception, which will be raised if the populated string is invalid.

Added a unit test as well to test the functionality.

navidshaikh commented 6 years ago

Updated the PR to change the behavior of seed-job as well. Now the seed-job will continue to process rest of the entries in index if it encountered errors while processing index. It will print the error and move on.

navidshaikh commented 6 years ago

Added 3 new unit tests to test the functionality. Sample run

nosetests -w . -v tests 
IndexReader: Tests processing pipeline_name ... ok
IndexReader: Tests processing pipeline_name for upper case params ... ok
IndexReader: Tests exception while processing pipeline name ... ok
Tests creating Project class object which loads entries ... ok
IndexReader: Tests exception while loading invalid index entry ... ok
Test processing pre_build_context ... ok
Test processing pre_build_script ... ok
Tests depends_on field processing and defaults ... ok
Tests desired tag processing and defaults ... ok
Test the helper method to replace . / : _ with hyphen ... ok
Test __str__ representation of Project object ... ok
Notifications: Tests processing body of email ... ok
Notitications: Tests processing subject of email ... ok

----------------------------------------------------------------------
Ran 13 tests in 0.002s

OK
navidshaikh commented 6 years ago

Tested the functionality by explicitly malforming an index entry (removed target-file here). Sample output of seed-job run

+ python ccp/index_reader.py /tmp/container-index/index.d 172.29.33.53:5000 cccp nshaikh@redhat.com smtp://mail.centos.org
Error processing index entry {u'app-id': u'centos', u'git-url': u'https://github.com/sclorg/s2i-perl-container.git', u'git-path': u'5.20/', u'build-context': u'./', u'depends-on': [u'centos/centos:latest', u'centos/s2i-core-centos7:latest', u'centos/s2i-base-centos7:latest'], u'notify-email': u'shaikhnavid14@gmail.com', u'desired-tag': u'latest', u'git-branch': u'master', u'job-id': u'perl-520-centos7', u'id': u'1'}. Moving on.
Error: 'target-file'
Number of projects in index 9
Number of projects in OpenShift 0
Number of projects to be updated/created: 9