RuntimeError: VM has no ip assignments in zerotier network

[serboctor]

S3 installation is throwing this error:

Traceback (most recent call last):
  File "/opt/code/github/threefoldtech/0-robot/zerorobot/task/task.py", line 78, in execute
    self._result = self._func()
  File "/opt/code/github/threefoldtech/0-templates/templates/s3/s3.py", line 295, in install
    self._deploy_minio(namespaces_connections, tlog_connection, master)
  File "/opt/code/github/threefoldtech/0-templates/templates/s3/s3.py", line 682, in _deploy_minio
    vm_robot, _ = self._vm_robot_and_ip()
  File "/opt/code/github/threefoldtech/0-templates/templates/s3/s3.py", line 431, in _vm_robot_and_ip
    raise RuntimeError('VM has no ip assignments in zerotier network')
RuntimeError: VM has no ip assignments in zerotier network

[serboctor]

According to my dm_vm service the vm is on node ac1f6b47a294 The zerotier_client and vm services seem to have to correct data

In [7]: zt = robot.services.get(template_name='zerotier_client', name='6f86ba4e-dc55-4008-aaad-be7a6714ddb5')

In [8]: zt.data
Out[8]: 
{'data': {'token': 'UvpmfbQRrGmxM7xYuFg8yvZuraquuPC3'},
 'guid': 'e985718c-f7b2-41dc-84aa-9e219978e2ed',
 'name': '6f86ba4e-dc55-4008-aaad-be7a6714ddb5',
 'public': False,
 'state': [],
 'template': 'github.com/threefoldtech/0-templates/zerotier_client/0.0.1',
 'version': '0.0.1'}

In [9]:  vm = robot.services.get(template_name='vm', name='6f86ba4e-dc55-4008-aaad-be7a6714ddb5_vm')
   ...: 
   ...: 

In [10]: vm.data
Out[10]: 
{'data': {'ipxeUrl': '',
  'cpu': 2,
  'nics': [{'type': 'zerotier',
    'name': 'mgmt_nic',
    'id': '159924d630750aad',
    'ztClient': '6f86ba4e-dc55-4008-aaad-be7a6714ddb5'},
   {'type': 'default', 'name': 'nat0'}],
  'disks': [{'filesystem': None,
    'url': 'zdb://172.18.0.14:9900?size=10G&blocksize=4096&namespace=44b41f92-4abc-4bee-a05b-ba8579d66e67&password=znbyws41ee8flng1j5hpnehm272eqdk8',
    'name': '6f86ba4e-dc55-4008-aaad-be7a6714ddb5_s3vm',
    'label': 's3vm',
    'mountPoint': None}],
  'vnc': -1,
  'tags': [],
  'ztIdentity': '9773614afa:0:a3f7fbefee89f1a2427fe8e64fd76a21ad499e9201390bfa546e356178405d76c23585c1b83fc2ead58e698ecfc902b776b0e62e3c23a1e510a6f64cd2b521b9:29b155a4ffdb7ed5f92f592a4f51b092b678f8b1e9775dd85156f9a84c82e0fe13341fab607a8767c409917fb590ed1127ebb7548b1a08466eb8b0ff031a0ef1',
  'memory': 4000,
  'uuid': 'ed31bf75-ff3f-4c05-b730-1d66b003e482',
  'flist': 'https://hub.grid.tf/tf-autobuilder/zero-os-development.flist',
  'kernelArgs': [{'key': 'development', 'name': 'development'},
   {'key': 'zerotier', 'value': '159924d630750aad', 'name': 'zerotier'},
   {'key': 'ztid',
    'value': '9773614afa:0:a3f7fbefee89f1a2427fe8e64fd76a21ad499e9201390bfa546e356178405d76c23585c1b83fc2ead58e698ecfc902b776b0e62e3c23a1e510a6f64cd2b521b9:29b155a4ffdb7ed5f92f592a4f51b092b678f8b1e9775dd85156f9a84c82e0fe13341fab607a8767c409917fb590ed1127ebb7548b1a08466eb8b0ff031a0ef1',
    'name': 'ztid'}],
  'configs': [],
  'info': None,
  'mounts': [],
  'ports': []},
 'guid': 'c8c0ed33-aa92-4c37-89c3-76f5cb547ea4',
 'name': '6f86ba4e-dc55-4008-aaad-be7a6714ddb5_vm',
 'public': False,
 'state': [{'category': 'status', 'state': 'ok', 'tag': 'running'},
  {'category': 'actions', 'state': 'ok', 'tag': 'start'},
  {'category': 'actions', 'state': 'ok', 'tag': 'install'}],
 'template': 'github.com/threefoldtech/0-templates/vm/0.0.1',
 'version': '0.0.1'}

And the zerodb backend for the node running the vm has the data for the zerotier instance:

ser@wonderwoman:~$ redis-cli -h  172.30.192.1 -p 10011
172.30.192.1:10011> select ac1f6b47a294
OK
172.30.192.1:10011> get  "cfg::myconfig::6f86ba4e-dc55-4008-aaad-be7a6714ddb5::j.clients.zerotier"
"{\"networkid\": \"\", \"token_\": \"653935663465396561363436346162383739653831383635109c392738e17e9de706d3c77c74a5a276d1370627c585ec27de7540e45e33efbd8b2686143450ff1ff11985b6f2d6c5\", \"nodeids\": \"\"}"
172.30.192.1:10011> 

[serboctor]

and to double check that the value of the token is stored correctly in the zdb backend, I ran a script that gets the instance on the node and prints the token. All seems fine.

In [41]: containter.client.system('python3 zt.py').get()
Out[41]: 
STATE: 0 SUCCESS
STDOUT:
************
UvpmfbQRrGmxM7xYuFg8yvZuraquuPC3

STDERR:

[serboctor]

This seems to be an issue with using zdb as a backend, sometimes it returns False when doing the exists check even though the zerotier was already created. I did some manual and unittests, to test the configmanager when file/db backends are used and I didnt find any issues. https://github.com/threefoldtech/jumpscale_core/tree/development/Jumpscale/tools/configmanager/test

[zaibon]

We're not using that anymore.