search

vmware-archive / cfops

This is simply an automation that is based on the supported way to back up Pivotal Cloud Foundry
http://www.cfops.io
Apache License 2.0
35 stars 24 forks source link

cfops elastic-runtime backup of PCF 1.7 ommits postgres DBs #88

Closed rguilme1 closed 8 years ago

rguilme1 commented 8 years ago

I have a PCF 1.7 environment migrated from 1.6 that uses Postgres for the DBs. cfops is not backing them up after the upgrade.

cfops version v2.2.18

2016/05/27 04:09:26 D0527 04:09:26.500544 5994 init.go:62] registering plugin: &{./plugins/cfops-mysql-plugin {mysql-tile map[]} 0x55f330} 2016/05/27 04:09:26 D0527 04:09:26.50071 5994 init.go:34] loading plugin from: &{cfops-mysql-plugin 13929160 488 {63598654920 292118353 0xda6100} {64772 3407903 1 33256 0 4900784 0 0 13929160 4096 27208 {1464250091 462185043} {1463058120 292118353} {1463058120 373120723} [0 0 0]}} 2016/05/27 04:09:26 D0527 04:09:26.815555 5994 init.go:62] registering plugin: &{./plugins/cfops-rabbitmq-plugin {rabbitmq backup-and-restore-rabbitmq map[]} 0x55f330} 2016/05/27 04:09:26 D0527 04:09:26.815672 5994 init.go:34] loading plugin from: &{cfops-rabbitmq-plugin 12252824 488 {63598654920 315119026 0xda6100} {64772 3407905 1 33256 0 4900784 0 0 12252824 4096 23936 {1464250878 494157278} {1463058120 315119026} {1463058120 373120723} [0 0 0]}} 2016/05/27 04:09:27 D0527 04:09:27.240156 5994 init.go:62] registering plugin: &{./plugins/cfops-redis-plugin {redis-tile map[]} 0x55f330} 2016/05/27 04:09:27 D0527 04:09:27.240293 5994 init.go:34] loading plugin from: &{cfops-redis-plugin 13957432 488 {63598654920 343119844 0xda6100} {64772 3407906 1 33256 0 4900784 0 0 13957432 4096 27264 {1464250091 626189597} {1463058120 343119844} {1463058120 373120723} [0 0 0]}} 2016/05/27 04:09:27 D0527 04:09:27.24074 5994 createCliCommand.go:80] checking registry for '%s' tile elastic-runtime 2016/05/27 04:09:27 D0527 04:09:27.240768 5994 createCliCommand.go:83] found tile in registry 2016/05/27 04:09:27 D0527 04:09:27.240787 5994 createCliCommand.go:86] we have all required flags and a proper builder 2016/05/27 04:09:27 D0527 04:09:27.240835 5994 remote_execute.go:26] using password for authn 2016/05/27 04:09:27 D0527 04:09:27.2409 5994 opsmanager.go:76] Exporting url 'https://cflom101.travp.net/api/installation_settings' 2016/05/27 04:09:27 D0527 04:09:27.240926 5994 opsmanager.go:130] attempting to auth against https://cflom101.travp.net/api/installation_settings 2016/05/27 04:09:27 D0527 04:09:27.240952 5994 opsmanager.go:171] aquiring your token from: https://cflom101.travp.net/api/installation_settings https://cflom101.travp.net/api/installation_settings 2016/05/27 04:09:30 D0527 04:09:30.041096 5994 opsmanager.go:174] your token eyJhbGciOiJSUzI1NiJ9.eyJqdGkiOiIzOGJiYzhkNy01MDJiLTQ5YjgtODUxMS1kMTkxZjJiZTYxZTIiLCJzdWIiOiI0NzE2M2I4Ni0wZmI0LTQ3NDctOTc0NC0yMGRlOGEzYWRmZTgiLCJzY29wZSI6WyJjbGllbnRzLnJlYWQiLCJvcHNtYW4udXNlciIsInVhYS5hZG1pbiIsInNjaW0ucmVhZCIsIm9wc21hbi5hZG1pbiIsImNsaWVudHMud3JpdGUiLCJzY2ltLndyaXRlIl0sImNsaWVudF9pZCI6Im9wc21hbiIsImNpZCI6Im9wc21hbiIsImF6cCI6Im9wc21hbiIsImdyYW50X3R5cGUiOiJwYXNzd29yZCIsInVzZXJfaWQiOiI0NzE2M2I4Ni0wZmI0LTQ3NDctOTc0NC0yMGRlOGEzYWRmZTgiLCJvcmlnaW4iOiJ1YWEiLCJ1c2VyX25hbWUiOiJhZG1pbiIsImVtYWlsIjoiYWRtaW5AdGVzdC5vcmciLCJhdXRoX3RpbWUiOjE0NjQzMzY1NjksInJldl9zaWciOiI0MjFiYTIwMiIsImlhdCI6MTQ2NDMzNjU2OSwiZXhwIjoxNDY0Mzc5NzY5LCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjgwODAvdWFhL29hdXRoL3Rva2VuIiwiemlkIjoidWFhIiwiYXVkIjpbIm9wc21hbiIsImNsaWVudHMiLCJ1YWEiLCJzY2ltIl19.h97LELtPMrnPnQXCQr9fdilsk2bTQ1fuXzDvLa6hY74upnhHZvLY034s11Wab6aBQsNm_ESQ5-SKtwzO5LR3azprLBV7_mRzLvfrj8dENN8s3jtpLE3zzLX9YotxbgH6U7vQwwYOS6v_TCz8pw2P8JZHv8w0VMt0fowNMEzS8WZAFbz0pARIUcjvopzaSwydXaAgT5emIqO8Cet_PHXm7fMAdTUsKhEqMz3B3kS58cKlhYBg_x5L4-4YX7db1eX2u3ht-40xGemyLHAk3r6Hbk3Zlu7c2m760ry6Zv8ZkdthQRnG-D6N8hdeHeW7cnzAVtgFv8yjteUmTdmIsnZjXA https://cflom101.travp.net/uaa 2016/05/27 04:09:31 D0527 04:09:31.793438 5994 createCliCommand.go:69] Running %s for tile: %+v backup {0xc820194b40 0xdc3a20} 2016/05/27 04:09:31 D0527 04:09:31.827844 5994 elasticruntime.go:47] Retrieving All CC VMs 2016/05/27 04:09:31 D0527 04:09:31.836897 5994 elasticruntime.go:77] Entering getAllCloudControllerVMs() function 2016/05/27 04:09:31 D0527 04:09:31.836987 5994 elasticruntime.go:80] getAllCloudControllerVMs() function map[connectionURL:https://10.47.104.2:25555/deployments/cf-30c5f1a2dc10c1a5fc44/vms directorInfo:0xc8201b06e0] 2016/05/27 04:09:31 D0527 04:09:31.837045 5994 elasticruntime.go:85] Retrieving CC vms 2016/05/27 04:09:31 D0527 04:09:31.844393 5994 elasticruntime.go:94] Unmarshalling CC vms 2016/05/27 04:09:31 D0527 04:09:31.844473 5994 elasticruntime.go:55] Setting up CC jobs 2016/05/27 04:09:31 D0527 04:09:31.844509 5994 elasticruntime.go:59] Running db action 2016/05/27 04:09:31 D0527 04:09:31.844555 5994 elasticruntime.go:108] RunDbAction info: &{SystemInfo:{GetSet:{} systemInfo:map[] Product:cf Component:nfs_server Identifier:vm_credentials Ip:10.47.104.6 User:vcap Pass:b2adc52ecaa57537 VcapUser:vcap VcapPass:b2adc52ecaa57537 SSHPrivateKey: RemoteArchivePath:/var/vcap/store/shared/archive.backup}} 2016/05/27 04:09:31 D0527 04:09:31.8446 5994 remote_execute.go:26] using password for authn 2016/05/27 04:09:31 I0527 04:09:31.844623 5994 elasticruntime.go:138] Exporting %s nfs_server 2016/05/27 04:22:49 D0527 04:22:49.082727 5994 elasticruntime.go:143] Done backing up %s nfs_server 2016/05/27 04:22:49 D0527 04:22:49.128202 5994 elasticruntime.go:108] RunDbAction info: &{SystemInfo:{GetSet:{} systemInfo:map[] Product:cf Component:mysql Identifier:mysql_admin_credentials Ip:10.47.104.18 User:root Pass:c87fd3997b699534b988 VcapUser:vcap VcapPass:5f83d974e1e18653 SSHPrivateKey: RemoteArchivePath:/var/vcap/store/mysql/archive.backup} Database:mysql} 2016/05/27 04:22:49 D0527 04:22:49.136519 5994 mysqldump.go:26] setting up a new remote MyslDump object 2016/05/27 04:22:49 D0527 04:22:49.137702 5994 remote_execute.go:26] using password for authn 2016/05/27 04:22:49 I0527 04:22:49.160789 5994 elasticruntime.go:138] Exporting %s mysql 2016/05/27 04:22:49 D0527 04:22:49.826191 5994 mysqldump.go:68] mysqldump Dump called: /var/vcap/packages/mariadb/bin/mysqldump -u root -h localhost --password=c87fd3997b699534b988 --all-databases 2016/05/27 04:22:49 D0527 04:22:49.826268 5994 elasticruntime.go:143] Done backing up %s mysql 2016/05/27 04:22:49 D0527 04:22:49.834054 5994 createCliCommand.go:56] Tile action completed successfully

kelapure commented 8 years ago

The gist here is that - After upgrading to PCF 1.7 cfops does not detect that we use postgres DBs and does not back up the cloud controller, uaa, etc.

It is a critical issue with us to migrate to PCF 1.7.

xchapter7x commented 8 years ago

@kelapure https://github.com/pivotalservices/cfbackup/commit/73f473d6ef2853935a18daa7e97c528567ca79ce

should fix the issue. Release to follow shortly.

xchapter7x commented 8 years ago

should be fixed as of https://github.com/pivotalservices/cfops/releases/latest