search

hpcsi / losf

LosF: A Linux operating system Framework for managing HPC clusters
Other
13 stars 4 forks source link

addrpm fails to add/replace rpm with existing filename #59

Closed hilmiller closed 9 years ago

hilmiller commented 10 years ago

To summarize, losf addrpm doesn't copy an RPM into the losf repository if there is already a package of the same filename in there, however addrpm does update the md5sum in the packages.config which I think is unexpected behavior. There should probably be a warning or something, to at least notify me that this package was not added to the losf repository because of an existing filename. Or, it shouldn't update the md5sum in the packages.config, instead ignoring my request. Although I imagine most people would prefer to have the existing package overwritten entirely.

Here is an existing package in losf, installed previously with losf addrpm.

[root@master.cluster]# pwd             
/admin/losf/config/cluster/rpms/x86_64 

[root@master.cluster]# md5sum mvapich2-intel14-2.0b-3.x86_64.rpm     
cb4e65b9e3741e9aa4c696d8cc66dc26  mvapich2-intel14-2.0b-3.x86_64.rpm 

[root@master.cluster]# rpm -qip mvapich2-intel14-2.0b-3.x86_64.rpm                     
Name        : mvapich2-intel14             Relocations: (not relocatable)              
Version     : 2.0b                              Vendor: (none)                         
Release     : 3                             Build Date: Tue 11 Mar 2014 04:32:15 PM CDT
Install Date: (not installed)               Build Host: build.stampede.tacc.utexas.edu 
Group       : Development/Libraries         Source RPM: mvapich2-2.0b-3.src.rpm        
Size        : 86848063                         License: freely distributable           
Signature   : (none)                                                                   
Packager    : TACC - karl@tacc.utexas.edu, mclay@tacc.utexas.edu                       
Summary     : OSU IB MPI-2 implementation                                              
Description :                                                                          
MPICH is an open-source and portable implementation of the Message-Passing             
Interface (MPI, www.mpi-forum.org).  MPI is a library for parallel programming,        
and is available on a wide range of parallel machines, from single laptops to          
massively parallel vector parallel processors.                                         
MPICH includes all of the routines in MPI 1.2, along with the I/O routines             
from MPI-2 and some additional routines from MPI-2, including those supporting         
MPI Info and some of the additional datatype constructors.  MPICH  was                 
developed by Argonne National Laboratory. See www.mcs.anl.gov/mpi/mpich for            
more information.

Here is a new and different package, which happens to have the same file name name.

[root@node01.cluster]# pwd
/admin/build/rpms/node/RPMS/x86_64

[root@node01.cluster]# md5sum mvapich2-intel14-2.0b-3.x86_64.rpm 
ab8cde0035a6888115ae188af7523de7  mvapich2-intel14-2.0b-3.x86_64.rpm

[root@node01.cluster]# rpm -qip mvapich2-intel14-2.0b-3.x86_64.rpm 
Name        : mvapich2-intel14             Relocations: (not relocatable)
Version     : 2.0b                              Vendor: (none)
Release     : 3                             Build Date: Mon 03 Nov 2014 10:06:07 AM CST
Install Date: (not installed)               Build Host: node01.cluster.tacc.utexas.edu
Group       : Development/Libraries         Source RPM: mvapich2-2.0b-3.src.rpm
Size        : 85046610                         License: freely distributable
Signature   : (none)
Packager    : TACC - hil@tacc.utexas.edu
Summary     : OSU IB MPI-2 implementation
Description :
MPICH is an open-source and portable implementation of the Message-Passing
Interface (MPI, www.mpi-forum.org).  MPI is a library for parallel programming,
and is available on a wide range of parallel machines, from single laptops to
massively parallel vector parallel processors.
MPICH includes all of the routines in MPI 1.2, along with the I/O routines
from MPI-2 and some additional routines from MPI-2, including those supporting
MPI Info and some of the additional datatype constructors.  MPICH  was
developed by Argonne National Laboratory. See www.mcs.anl.gov/mpi/mpich for
more information.

When I attempt to add my new package, everything looks fine.

[root@node01.cluster]# losf addrpm mvapich2-intel14-2.0b-3.x86_64.rpm 

** Checking on possible addition of custom RPM package: mvapich2-intel14-2.0b-3.x86_64.rpm
   --> md5sum = ab8cde0035a6888115ae188af7523de7
   --> Cluster = cluster, Node Type = node

   --> Would you like to add mvapich2-intel14-2.0b-3.x86_64.rpm 
       to your local LosF config for cluster:node nodes?

[LosF] Enter yes/no to confirm (or -1 to add to multiple node types): yes

   --> Reading Custom package config file:
       --> /admin/losf/config/custom-packages/cluster/packages.config
   --> Attempting to add mvapich2-intel14-2.0b-3.x86_64
       --> mvapich2-intel14 not previously configured - registering for addition/upgrade
   --> Updating Custom RPM config file...

Custom RPM config update complete; you can now run "update" to make changes take effect

However when I run update, I get an md5sum mismatch.

[root@node01.cluster]# update
** Config dir -> /admin/losf/config
** Checking on OS packages to remove (cluster:node)
** Checking on Custom packages to remove (cluster:node)
** Syncing OS packages (cluster:node)
   --> OK: OS packages in sync (168 rpms checked)
** Syncing Custom packages (cluster:node)
   --> RPM md5sums do not match for /admin/losf/config/cluster/rpms/x86_64/mvapich2-intel14-2.0b-3.x86_64.rpm
       --> desired    = ab8cde0035a6888115ae188af7523de7
       --> local      = cb4e65b9e3741e9aa4c696d8cc66dc26

[ERROR]: FAILED: Aborting install

At this point, my packages.config has the correct md5sum.

node=mvapich2-intel14-2.0b-3.x86_64 name=mvapich2-intel14 version=2.0b release=3 arch=x86_64 ab8cde0035a6888115ae
188af7523de7 NODEPS IGNORESIZE

However the new package's file didn't make it into the repository.

[root@master.cluster]# pwd                                          
/admin/losf/config/cluster/rpms/x86_64                              
[root@master.cluster]# md5sum mvapich2-intel14-2.0b-3.x86_64.rpm    
cb4e65b9e3741e9aa4c696d8cc66dc26  mvapich2-intel14-2.0b-3.x86_64.rpm
koomie commented 10 years ago

I agree, this is not desired behavior. You shouldn't add an rpm with the exact same name//version/release and it should have error'ed off to avoid updating the config with the new md5sum.

Thanks for the report. Will get a fix scheduled for the next release.

hilmiller commented 10 years ago

Glad I could help!

koomie commented 9 years ago

The fix for this has been pushed to the devel branch. Going forward, the user will be notified if a cached copy of the new package being requested for addition is already present. In that case, the add rpm request will fail with a message like the following:

[ERROR]: Cached foo-1.0-1.x86_64.rpm exists already in rpm_topdir (/tmp/X2BhEKj5Qb/test/rpms). Consider increasing the
[ERROR]: RPM release number or remove the cached copy if you are confident this RPM has not been
[ERROR]: propagated previously.