search

syndicate-storage / syndicate-core

Scalable Software-define Storage System
Apache License 2.0
4 stars 1 forks source link

Cannot load cert bundle version. #34

Closed iychoi closed 7 years ago

iychoi commented 7 years ago

When a volume/gateway is exported and imported to a new system, there's no cert bundle version file in the system. This causes a fail when running update_gateway command. 

Attaching to containers_production_imicrobe_1
PRINT DEBUG MESSAGES
Found existing gateway and volume
AG RESTART
Registering Syndicate...
Registering Syndicate... Done!
Importing a Volume...
volumes/imicrobe.cert
volumes/3459028955753786832.cert
Importing a Volume... Done!
Preparing driver code...
--2017-07-28 01:54:26--  https://raw.githubusercontent.com/syndicate-storage/syndicate-fs-driver/master/src/sgfsdriver/ag_driver/driver
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12090 (12K) [text/plain]
Saving to: '/home/syndicate/ag_driver/driver'

     0K .......... .                                          100% 7.37M=0.002s

2017-07-28 01:54:26 (7.37 MB/s) - '/home/syndicate/ag_driver/driver' saved [12090/12090]

Preparing driver code... Done!
Importing an AG...
gateways/ag_imicrobe.pkey
gateways/ag_imicrobe.cert
gateways/8814559010125371945.cert
Reloading certs...
[2017-07-28 01:54:28,475] [ERROR] [certs:358] (75) No such file or directory: /home/syndicate/.syndicate/certs/imicrobe/ag_imicrobe/bundle.version
[2017-07-28 01:54:28,476] [WARNING] [certs:964] (75) No cached cert bundle version for volume 'imicrobe'
[2017-07-28 01:54:28,477] [WARNING] [certs:971] (75) No cached volume cert for volume 'imicrobe'
[2017-07-28 01:54:29,398] [DEBUG] [certs:990] (75) Got valid cert bundle for imicrobe (volume_version=1, cert_bundle_version=1497268308) from cyverse@opencloud.us
[2017-07-28 01:54:29,401] [DEBUG] [certs:1012] (75) Got valid volume certificate for imicrobe
[2017-07-28 01:54:30,458] [DEBUG] [syndicate:1125] (100) ipc = /home/syndicate/.syndicate/ipc
[2017-07-28 01:54:30,458] [DEBUG] [syndicate:1125] (100) logs = /home/syndicate/.syndicate/logs
[2017-07-28 01:54:30,458] [DEBUG] [syndicate:1125] (100) __from_main__ = True
[2017-07-28 01:54:30,458] [DEBUG] [syndicate:1125] (100) syndicate_host = syndicate-ms-datasets-prod.appspot.com
[2017-07-28 01:54:30,459] [DEBUG] [syndicate:1125] (100) certs = /home/syndicate/.syndicate/certs
[2017-07-28 01:54:30,459] [DEBUG] [syndicate:1125] (100) syndicate_public_key_pem = -----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAq+fr4XcgYfUNxdX/W/WA
joTuy3Mvl0oXbp2w9rkzGHGeD3CNhI18Xms0/F2ZHklvTLqRf8/61rJINyy6ErAz
/7GJ8GXWjdYCNAwgVGWr57R0gDf+wasG4kyoukRsLe7Fi8iljpEXSQzcBhquMNKG
IexxkVzxXorQOf3Qgk0wJC2GaXUMsYnUnO/urLBQviN6Zp+gq0sXCGbJNFBFjHlp
sfJuoGbER9XxyxXBYYztjcHny5UlyNpjrblE/mSNip67HBxelgJpPnzcCvb/qR7Z
kLjJQ3lr8THc6BJocqNA2eqRntxD8xSI91gAj1tQ/pfpVFzsAewIRhhwcphW2DTL
fOj+3KfEKOqO3es9uu1zrGyxAsgAB2hMgsCh5MbFegX0L/i/43wI73TdcZoHLEw3
bPI/SU9bd1/brMkwhkyakRhdfy1cRpW27qGyjLI2Lm0wh5Kbq/s4fAf5Mka9QTP1
nDioHqqr9quEsH5E5xQE27DVS1fmVdfviI9+mEEhxdQkoFmoGoTD3Mddr1DDYb18
SbGbbiuwzJMvIpVEPrIXTGip3GlSG6087GMnhg0oayDvCDMPoBW4MxQnt1MLTQPw
47Gdvv4WXyWV591bNlny3zwOa1f2K2PUv28p3uSWopn85tj+ZQiFT5VhQkqYj3AJ
UINLyI2q7UmIuilVw+7ZJSECAwEAAQ==
-----END PUBLIC KEY-----
[2017-07-28 01:54:30,459] [DEBUG] [syndicate:1125] (100) config_dir = /home/syndicate/.syndicate
[2017-07-28 01:54:30,459] [DEBUG] [syndicate:1125] (100) trust_public_key = False
[2017-07-28 01:54:30,459] [DEBUG] [syndicate:1125] (100) config_path = /home/syndicate/.syndicate/syndicate.conf
[2017-07-28 01:54:30,459] [DEBUG] [syndicate:1125] (100) drivers = /home/syndicate/.syndicate/drivers
[2017-07-28 01:54:30,459] [DEBUG] [syndicate:1125] (100) params = ['update_gateway', 'ag_imicrobe', 'driver=/home/syndicate/ag_driver']
[2017-07-28 01:54:30,459] [DEBUG] [syndicate:1125] (100) gateways = /home/syndicate/.syndicate/gateways
[2017-07-28 01:54:30,459] [DEBUG] [syndicate:1125] (100) MS_url = http://syndicate-ms-datasets-prod.appspot.com:80
[2017-07-28 01:54:30,459] [DEBUG] [syndicate:1125] (100) username = cyverse@opencloud.us
[2017-07-28 01:54:30,459] [DEBUG] [syndicate:1125] (100) users = /home/syndicate/.syndicate/users
[2017-07-28 01:54:30,459] [DEBUG] [syndicate:1125] (100) amd = /home/syndicate/.syndicate/amd
[2017-07-28 01:54:30,459] [DEBUG] [syndicate:1125] (100) no_reload = False
[2017-07-28 01:54:30,459] [DEBUG] [syndicate:1125] (100) syndicate = /home/syndicate/.syndicate/syndicate
[2017-07-28 01:54:30,459] [DEBUG] [syndicate:1125] (100) data = /home/syndicate/.syndicate/data
[2017-07-28 01:54:30,459] [DEBUG] [syndicate:1125] (100) syndicate_port = 80
[2017-07-28 01:54:30,459] [DEBUG] [syndicate:1125] (100) syndicate_public_key = <_RSAobj @0x7f667e19ce60 n(4096),e>
[2017-07-28 01:54:30,460] [DEBUG] [syndicate:1125] (100) _in_argv = ['no_reload', 'debug', 'trust_public_key', 'params']
[2017-07-28 01:54:30,460] [DEBUG] [syndicate:1125] (100) _in_config = ['username', 'MS_url']
[2017-07-28 01:54:30,460] [DEBUG] [syndicate:1125] (100) helpers = {'ipc': '/home/syndicate/.syndicate/ipc', 'users': '/home/syndicate/.syndicate/users', 'fetch_volume_cert': '/usr/lib/syndicate/fetch_volume_cert', 'amd': '/home/syndicate/.syndicate/amd', 'fetch_user_cert': '/usr/lib/syndicate/fetch_user_cert', 'driver_reload': '/usr/lib/syndicate/driver_reload', 'certs': '/home/syndicate/.syndicate/certs', 'fetch_gateway_cert': '/usr/lib/syndicate/fetch_gateway_cert', 'syndicate': '/home/syndicate/.syndicate/syndicate', 'config_dir': '/home/syndicate/.syndicate', 'logs': '/home/syndicate/.syndicate/logs', 'validate_user_cert': '/usr/lib/syndicate/validate_user_cert', '_in_argv': [], '_in_config': [], 'drivers': '/home/syndicate/.syndicate/drivers', 'config_path': '/home/syndicate/.syndicate/syndicate.conf', 'fetch_cert_bundle': '/usr/lib/syndicate/fetch_cert_bundle', 'data': '/home/syndicate/.syndicate/data', 'certs_reload': '/usr/lib/syndicate/certs_reload', 'fetch_syndicate_pubkey': '/usr/lib/syndicate/fetch_syndicate_pubkey', 'gateways': '/home/syndicate/.syndicate/gateways', 'volumes': '/home/syndicate/.syndicate/volumes', 'fetch_driver': '/usr/lib/syndicate/fetch_driver'}
[2017-07-28 01:54:30,460] [DEBUG] [syndicate:1125] (100) no_tls = True
[2017-07-28 01:54:30,460] [DEBUG] [syndicate:1125] (100) volumes = /home/syndicate/.syndicate/volumes
[2017-07-28 01:54:30,460] [DEBUG] [syndicate:1125] (100) debug = True
[2017-07-28 01:54:30,462] [DEBUG] [client:186] (100) MS URL http://syndicate-ms-datasets-prod.appspot.com:80//API is NOT confidential!
[2017-07-28 01:54:30,510] [DEBUG] [client:263] (100) As cyverse@opencloud.us, call update_gateway(8814559010125371945 gateway_cert_b64=CAkQAxiptPfR/dbmqXogh8u4z5bngIoVK..., driver_text={"config": "ewogICAiREFUQVNFVF9ESVIiOi...)
[2017-07-28 01:54:32,373] [ERROR] [syndicate:1138] (100) No cert bundle version information for volume 'imicrobe'
Traceback (most recent call last):
  File "/usr/bin/syndicate", line 1135, in main
Help for 'update_gateway':

   Update a gateway.

   Positional arguments:
      name (str):
         The name of the gateway to update.

   Optional keyword arguments:
      host=str:
         The hostname that should be resolved to contact this gateway.

      port=int:
         The port number this gateway should listen on.

      cert_expires=int:
         Date when this gateway's certificate expires, in seconds
    result = client.ms_rpc( rpc_client, method_name, *args, **kw )
  File "/usr/lib/python2.7/dist-packages/syndicate/util/client.py", line 269, in ms_rpc
    object_cls.PostProcessResult(extras, config, method_name, args, kw, ret)
  File "/usr/lib/python2.7/dist-packages/syndicate/util/objects.py", line 2666, in PostProcessResult
    gateway_status = reloader.send_reload(config, reload_owner_id, volume_id, gateway_id)
  File "/usr/lib/python2.7/dist-packages/syndicate/util/reload.py", line 320, in send_reload
    msg = make_reload_request(config, user_id, volume_id, gateway_id=gateway_id)
  File "/usr/lib/python2.7/dist-packages/syndicate/util/reload.py", line 250, in make_reload_request
    raise MissingCertException("No cert bundle version information for volume '%s'" % volume_cert.name)
MissingCertException: No cert bundle version information for volume 'imicrobe'
         since the epoch.

      driver=str:
         This is serialized JSON string that contains this gateway's 
         driver logic.  The contents are specific to the gateway 
         implementation. 

      private_key=str:
         This is a PEM-encoded private key for the gateway.  Pass "auto"
         to generate one automatically.

      caps=str|int:
         This is the capabilities string (or value) for this gateway.
         Capabilities are a bit-field of the following:

         GATEWAY_CAP_READ_DATA          Gateway can read data
         GATEWAY_CAP_WRITE_DATA         Gateway can write data
         GATEWAY_CAP_READ_METADATA      Gateway can read metadata
         GATEWAY_CAP_WRITE_METADATA     Gateway can write metadta
         GATEWAY_CAP_COORDINATE         Gateway can coordinate writes

         The volume owner sets a whitelist of allowed capabilities in
         create_gateway.  The user can only enable these white-listed 
         capabilities.  Only the volume owner can change the capability
         white-list.

   Returns:
      On success, this method returns True.
      Raises an exception on error.

   Authorization:
      An administrator can update any gateway.
      A volume owner can update any gateway in the volumes (s)he owns.
      A user cannot update the gateway.

   Remarks:
      Syndicate sends 'gateway_cert_b64' as a keyword argument that contains 
      the user-signed base64-encoded gateway certificate which contains all of the 
      keyword arguments.  Syndicate will merge the keyword arguments with the 
      gateway certificate stored locally.  If the gateway certificate is not 
      stored locally, it will try to use the cached copy on the MS.

      Syndicate may optionally send 'cert_bundle_b64' as a keyword argument as well,
      which will contain a serialized, volume owner-signed certificate bundle 
      for all gateways in the volume.  This will only be passed if the volume 
      owner is altering a gateway's capability whitelist.

      In addition, kw may contain 'driver_text'--the JSON-encoded driver for 
      the gateway--as well as 'cert_bundle_b64' (the new volume cert 
      bundle version vector, base64-encoded).

Updating an AG... Failed!
iychoi commented 7 years ago

Addressed at a commit 5e89ba7e42a092b4fa3f42e03b981452cebc1539 The commit let an exported volume cert file include volume cert bundle version.