search

cern-it-sdc-id / lcgdm-puppet-examples

the repository contains manifest examples for DPM/LFC configuration
1 stars 2 forks source link

BDII #1

Open masoodmmt opened 3 years ago

masoodmmt commented 3 years ago

Hi Admin, I configured my DPM 1.14.2 with use of your preferred below link: https://github.com/cern-it-sdc-id/lcgdm-puppet-examples/blob/master/dpm/head/head%2Bdisknode.pp

Everything works fine except BDII: Error: Execution of '/usr/bin/yum -d 0 -e 0 -y install glite-info-provider-service' returned 1: Error: Nothing to do Error: /Stage[main]/Lcgdm::Bdii::Dpm/Package[glite-info-provider-service]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/yum -d 0 -e 0 -y install glite-info-provider-service' returned 1: Error: Nothing to do Notice: /Stage[main]/Bdii::Service/Service[bdii]: Dependency Package[glite-info-provider-service] has failures: true Warning: /Stage[main]/Bdii::Service/Service[bdii]: Skipping because of failed dependencies

And so:

ldapsearch -LLL -x -h localhost:2170 -b o=glue | perl -p0e 's/\n //g' | wc ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1) 0 0 0

Installed packages are as below:

[root@se1 ~]# rpm -qa | grep bdii bdii-5.2.25-2.el7.noarch

[root@se1 ~]# rpm -qa | grep dmlite dmlite-plugins-memcache-1.14.2-1.el7.x86_64 dmlite-dome-1.14.2-1.el7.x86_64 dmlite-docs-1.14.2-1.el7.x86_64 dmlite-puppet-dpm-1.14.2-1.el7.noarch dmlite-dpmhead-1.14.2-1.el7.x86_64 dmlite-dpm-tester-1.14.2-1.el7.x86_64 dmlite-plugins-domeadapter-1.14.2-1.el7.x86_64 dmlite-plugins-profiler-1.14.2-1.el7.x86_64 dmlite-dpm-dsi-1.14.2-1.el7.x86_64 dmlite-libs-1.14.2-1.el7.x86_64 dmlite-shell-1.14.2-1.el7.x86_64 dmlite-dpmdisk-1.14.2-1.el7.x86_64 dmlite-devel-1.14.2-1.el7.x86_64 dmlite-private-devel-1.14.2-1.el7.x86_64 python-dmlite-1.14.2-1.el7.x86_64 dmlite-apache-httpd-1.14.2-1.el7.x86_64 dmlite-plugins-librarian-1.14.2-1.el7.x86_64 dmlite-dpm-xrootd-1.14.2-1.el7.x86_64 dmlite-plugins-mysql-1.14.2-1.el7.x86_64 dmlite-plugins-adapter-1.14.2-1.el7.x86_64

[root@se1 ~]# rpm -qa | grep dpm dpm-server-mysql-1.13.0-1.el7.x86_64 dpm-perl-1.13.0-1.el7.x86_64 dpm-rfio-server-1.13.0-1.el7.x86_64 python2-dpm-1.13.0-1.el7.x86_64 dmlite-puppet-dpm-1.14.2-1.el7.noarch dmlite-dpmhead-1.14.2-1.el7.x86_64 dmlite-dpm-tester-1.14.2-1.el7.x86_64 dpm-1.13.0-1.el7.x86_64 dmlite-dpm-dsi-1.14.2-1.el7.x86_64 dpm-devel-1.13.0-1.el7.x86_64 dpm-contrib-admintools-0.2.5-1.el7.x86_64 dpm-libs-1.13.0-1.el7.x86_64 dmlite-dpmdisk-1.14.2-1.el7.x86_64 dmlite-dpm-xrootd-1.14.2-1.el7.x86_64 dpm-copy-server-mysql-1.13.0-1.el7.x86_64 dpm-srm-server-mysql-1.13.0-1.el7.x86_64 dpm-name-server-mysql-1.13.0-1.el7.x86_64

masoodmmt commented 3 years ago

It is solved: yum install dmlite-plugins-s3

from the UMD-4 repos

masoodmmt commented 3 years ago

BDII is corrected but GSIFTP interrupted by that package (dmlite-plugins-s3)

gfal-ls error: 13 (Permission denied) - globus_ftp_client: the server responded with an error 530 Login incorrect. : Internal server error

andrea-manzi commented 3 years ago

Hello, this is quite an old way to configure DPM via puppet and it's not supported now. please follow the guide at https://twiki.cern.ch/twiki/bin/view/DPM/DpmSetupPuppetInstallation cheers Andrea

masoodmmt commented 3 years ago

Hello Andrea, For your information, We have made a new test-bed SE (Storage Element) for this new upgrade(dpm-1.14.2) besides our old SE1 (dpm-1.13 on SLC6) Our configuration file (like twiki of cern as you have mentioned) is the same as down below puppet file.

Before I make use of your GITLAB as you can see the puppet file is as same as Twiki of Cern. The command (puppet parser validate) has no error but when we APPLY our SE puppet in the dpm-14 and puppet 5.5.22 it shows SYNTAX error as below.

Error: Syntax error at end of input

(file: /usr/share/dmlite/puppet/modules/dpm/manifests/head_disknode.pp) on node se1.hep.ipm.ir

However this puppet file works fine in dpm-13 via puppet 5.5.17 If it needs new changes please inform me.

our puppet file (same as twiki of cern) is:

class{'dpm::head_disknode': localdomain => 'hep.ipm.ir', db_user => 'dpmmgr', db_pass => 'XXXXXXXX', db_host => 'localhost', disk_nodes => ['$::fqdn'], mysql_root_pass => 'XXXXXXXX', token_password => 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX', xrootd_sharedkey => 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX', http_macaroon_secret => 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX', site_name => 'IR-IPM-HEP', volist => [dteam, ops, cms], new_installation => true, mountpoints => ['/srv/dpm','/srv/dpm/01'], configure_dome => true, configure_domeadapter => true, host_dn => '/C=IR/O=IRAN-GRID/OU=GCG/CN=se1.hep.ipm.ir' }

In addition, the dpm-1.14(on centos7) just works for me with YOUR gitlab puppet fromat (head_disknode)!

Bests, Masood

andrea-manzi commented 3 years ago

Hi Masood i have found the problem ( introduced in dmlite 1.14.0) and opened a PR to fix it

https://gitlab.cern.ch/lcgdm/dmlite/-/merge_requests/26

you should just modify the file /usr/share/dmlite/puppet/modules/dpm/manifests/head_disknode.pp and add 2 missing parenthesis after line 392,

if $configure_dome { cron { 'dpm-argus-banned-users': ensure => 'absent'; 'dpm-argus-banned-groups': ensure => 'absent'; } }

can you give it a try?

masoodmmt commented 3 years ago

Hi Andrea, I added 2 missing parenthesis as you mentioned:

 class{'lcgdm::argus':
    dpmhost   => $::fqdn,
    argus_url => $argus_banning,
  }
}

} else { if $configure_dome { cron { 'dpm-argus-banned-users': ensure => 'absent'; 'dpm-argus-banned-groups': ensure => 'absent'; } } }

pools configuration

And puppet validation is with no error: puppet parser validate /usr/share/dmlite/puppet/modules/dpm/manifests/head_disknode.pp But after that APPLYING our dpm puppet file shows new errors:

[root@se1 ~]# puppet parser validate /usr/share/dmlite/puppet/modules/dpm/manifests/head_disknode.pp [root@se1 ~]# puppet apply -v --modulepath /usr/share/dmlite/puppet/modules dpm-13.pp
Info: Loading facts Info: Loading facts Info: Loading facts Info: Loading facts Info: Loading facts Info: Loading facts Warning: The function 'hiera' is deprecated in favor of using 'lookup'. See https://puppet.com/docs/puppet/5.5/deprecated_language.html\n (file & line not available) Error: Evaluation Error: Error while evaluating a Resource Statement, Class[Dpm::Head_disknode]: parameter 'disk_nodes' index 0 expects a Stdlib::Host = Variant[Stdlib::Fqdn = Pattern[/\A(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9-][a-zA-Z0-9]).)([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9-][A-Za-z0-9])\z/], Stdlib::Compat::Ip_address = Variant[Stdlib::Compat::Ipv4 = Pattern[/^(((0-9|[1-9]0-9|1[0-9]{2}(?!\d)|2[0-4]0-9|250-5)[.]){3}(0-9|[1-9]0-9|1[0-9]{2}(?!\d)|2[0-4]0-9|250-5))(\/(((0-9|[1-9]0-9|1[0-9]{2}(?!\d)|2[0-4]0-9|250-5)[.]){3}(0-9|[1-9]0-9|1[0-9]{2}(?!\d)|2[0-4]0-9|250-5)|[0-9]+))?$/], Stdlib::Compat::Ipv6 = Pattern[/\s((([0-9A-Fa-f]{1,4}:){7}([0-9A-Fa-f]{1,4}|:))|(([0-9A-Fa-f]{1,4}:){6}(:[0-9A-Fa-f]{1,4}|((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){5}(((:[0-9A-Fa-f]{1,4}){1,2})|:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){4}(((:[0-9A-Fa-f]{1,4}){1,3})|((:[0-9A-Fa-f]{1,4})?:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){3}(((:[0-9A-Fa-f]{1,4}){1,4})|((:[0-9A-Fa-f]{1,4}){0,2}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){2}(((:[0-9A-Fa-f]{1,4}){1,5})|((:[0-9A-Fa-f]{1,4}){0,3}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){1}(((:[0-9A-Fa-f]{1,4}){1,6})|((:[0-9A-Fa-f]{1,4}){0,4}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(:(((:[0-9A-Fa-f]{1,4}){1,7})|((:[0-9A-Fa-f]{1,4}){0,5}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:)))(%.+)?\s*$/]]] value, got String (file: /root/dpm-13.pp, line: 1, column: 1) on node se1.hep.ipm.ir [root@se1 ~]#

andrea-manzi commented 3 years ago

hello, looks like the value you are passing to the disk_nodes is not validated. are you using disk_nodes => ['$::fqdn']? from this version they have introduced type checking on the puppet modules so it seems that this input is not more valid. please use disk_nodes => [$::fqdn] cheers Andrea

masoodmmt commented 3 years ago

Yes the solution was what you said. Thanks for the cooperation.