search

threefoldtecharchive / 0-templates

0-robot templates
Apache License 2.0
1 stars 1 forks source link

`node service not found` error while deploying s3 #207

Closed 0xIslamTaha closed 5 years ago

0xIslamTaha commented 5 years ago

Description I tried to deploy s3 by doing demo.deploy_n(1,'kristof-farm', 1000, 1, 1) then I checked my zrobot logs and I found this error. I dont knw from where it is comming and when I listed the node templates, I could found that there is node template.

zrobot templates

In [6]: robot = j.clients.zrobot.get('ac1f6b457488', data={'url':'http://10.102.113.188:6600'})

In [7]: robot = j.clients.zrobot.robots['ac1f6b457488']

In [8]: robot.services.guids
Out[8]: {'9e3f1b89-e1ef-4450-915c-228470de4977': robot://ac1f6b457488/github.com/threefoldtech/0-templates/node_capacity/0.0.1?name=_node_capacity&guid=9e3f1b89-e1ef-4450-915c-228470de4977}

In [9]: robot.templates.uids
Out[9]: 
{github.com/threefoldtech/0-templates/zerotier_client/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c322702e8>,
 github.com/threefoldtech/0-templates/zeroboot_client/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32270ef0>,
 github.com/threefoldtech/0-templates/bridge/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c322708d0>,
 github.com/threefoldtech/0-templates/ipmi_client/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c322702b0>,
 github.com/threefoldtech/0-templates/zeromd_cluster/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32270eb8>,
 github.com/threefoldtech/0-templates/traefik/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32270898>,
 github.com/threefoldtech/0-templates/minio/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32270278>,
 github.com/threefoldtech/0-templates/ardb/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32270e80>,
 github.com/threefoldtech/0-templates/zeroboot_ipmi_host/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32270860>,
 github.com/threefoldtech/0-templates/reverse_proxy/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32270240>,
 github.com/threefoldtech/0-templates/coredns/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32270e48>,
 github.com/threefoldtech/0-templates/node_port_manager/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32270828>,
 github.com/threefoldtech/0-templates/zeroboot_racktivity_host/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32270208>,
 github.com/threefoldtech/0-templates/network/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32270e10>,
 github.com/threefoldtech/0-templates/s3/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c322707f0>,
 github.com/threefoldtech/0-templates/gateway/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c322701d0>,
 github.com/threefoldtech/0-templates/statistics/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32270dd8>,
 github.com/threefoldtech/0-templates/racktivity_client/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c322707b8>,
 github.com/threefoldtech/0-templates/dm_gateway/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32270198>,
 github.com/threefoldtech/0-templates/zeromd/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267048>,
 github.com/threefoldtech/0-templates/node/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267080>,
 github.com/threefoldtech/0-templates/vm/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c322670b8>,
 github.com/threefoldtech/0-templates/explorer/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c322670f0>,
 github.com/threefoldtech/0-templates/healthcheck/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267128>,
 github.com/threefoldtech/0-templates/s3_redundant/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267160>,
 github.com/threefoldtech/0-templates/peer_discovery/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267198>,
 github.com/threefoldtech/0-templates/namespace/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c322671d0>,
 github.com/threefoldtech/0-templates/node_ip/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267208>,
 github.com/threefoldtech/0-templates/zrobot/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267240>,
 github.com/threefoldtech/0-templates/block_creator/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267278>,
 github.com/threefoldtech/0-templates/ssh_client/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c322672b0>,
 github.com/threefoldtech/0-templates/node_capacity/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c322672e8>,
 github.com/threefoldtech/0-templates/web_gateway/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267320>,
 github.com/threefoldtech/0-templates/dm_vm/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267358>,
 github.com/threefoldtech/0-templates/zeroos_client/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267390>,
 github.com/threefoldtech/0-templates/etcd/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c322673c8>,
 github.com/threefoldtech/0-templates/portal_connection/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267400>,
 github.com/threefoldtech/0-templates/zeroboot_pool/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267438>,
 github.com/threefoldtech/0-templates/zeroboot_reservation/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267470>,
 github.com/threefoldtech/0-templates/alerta/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c322674a8>,
 github.com/threefoldtech/0-templates/container/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c322674e0>,
 github.com/threefoldtech/0-templates/zrobot_config/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267518>,
 github.com/threefoldtech/0-templates/public_gateway/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267550>,
 github.com/threefoldtech/0-templates/vdisk/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267588>,
 github.com/threefoldtech/0-templates/rtinfo/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c322675c0>,
 github.com/threefoldtech/0-templates/erp_registeration/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c322675f8>,
 github.com/threefoldtech/0-templates/ardb_cluster/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267630>,
 github.com/threefoldtech/0-templates/etcd_cluster/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267668>,
 github.com/threefoldtech/0-templates/block_creator_status_reporter/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c322676a0>,
 github.com/threefoldtech/0-templates/hardware_check/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c322676d8>,
 github.com/threefoldtech/0-templates/zerodb/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267710>,
 github.com/threefoldtech/0-templates/influxdb_client/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267748>,
 github.com/threefoldtech/0-templates/zeroos_bootstrap/0.0.1: <JumpscaleZrobot.clients.zerorobot.client.Template.Template at 0x7f3c32267780>}

zrobot node service there r two node services 

In [7]: robot.services.find(template_name='node')
Out[7]: 
[robot://ac1f6b2723a0/github.com/threefoldtech/0-templates/node/0.0.1?name=local&guid=32f9cfe0-7a21-42c1-9127-154ddcbf32d9,
 robot://ac1f6b2723a0/github.com/threefoldtech/0-templates/node/0.0.1?name=node&guid=3193a1a0-5335-4419-8d83-ab39eaa64209]

Logs

[Mon12 10:28] - s3.py             :556 :s3_demo_0-a4e0af32-6711-428a-b314-787b336761f7 (s3) - INFO     - number of possible nodes to use for namespace deployments 22
[Mon12 10:28] - s3.py             :563 :s3_demo_0-a4e0af32-6711-428a-b314-787b336761f7 (s3) - INFO     - try to install namespace namespace_tlog on node ac1f6b2723a0
[Mon12 10:28] - s3.py             :556 :s3_demo_0-a4e0af32-6711-428a-b314-787b336761f7 (s3) - INFO     - number of possible nodes to use for namespace deployments 22
[Mon12 10:28] - s3.py             :563 :s3_demo_0-a4e0af32-6711-428a-b314-787b336761f7 (s3) - INFO     - try to install namespace namespace on node ac1f6b457344
[Mon12 10:28] - s3.py             :563 :s3_demo_0-a4e0af32-6711-428a-b314-787b336761f7 (s3) - INFO     - try to install namespace namespace on node ac1f6b456cf4
[Mon12 10:28] - s3.py             :563 :s3_demo_0-a4e0af32-6711-428a-b314-787b336761f7 (s3) - INFO     - try to install namespace namespace on node ac1f6b457338
[Mon12 10:28] - dm_vm.py          :82  :a4e0af32-6711-428a-b314-787b336761f7-f24bb0cb-464a-4814-8b74-8282543a4363 (dm_vm) - INFO     - Installing vm a4e0af32-6711-428a-b314-787b336761f7
[Mon12 10:28] - ZeroRobotManager.py:170 :j.zerorobot          - ERROR    - fail to create service: fail to create service None github.com/threefoldtech/0-templates/namespace/0.0.1: Node service not found, can't install the namespace
Traceback (most recent call last):
  File "/opt/code/github/threefoldtech/0-robot/zerorobot/dsl/ZeroRobotManager.py", line 155, in create
    new_service, resp = self._client.api.services.createService(req)
  File "/opt/code/github/threefoldtech/0-robot/JumpscaleZrobot/clients/zerorobot/client/services_service.py", line 213, in createService
    resp = self.client.post(uri, data, headers, query_params, content_type)
  File "/opt/code/github/threefoldtech/0-robot/JumpscaleZrobot/clients/zerorobot/client/http_client.py", line 62, in post
    return self._handle_data(uri, data, headers, params, content_type, self.session.post)
  File "/opt/code/github/threefoldtech/0-robot/JumpscaleZrobot/clients/zerorobot/client/http_client.py", line 58, in _handle_data
    res.raise_for_status()
  File "/usr/local/lib/python3.6/dist-packages/requests/models.py", line 939, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: BAD REQUEST for url: http://10.102.113.188:6600/services

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/code/github/threefoldtech/0-templates/templates/s3/s3.py", line 595, in _install_namespace
    namespace = robot.services.create(template_uid=NS_TEMPLATE_UID, data=data)
  File "/opt/code/github/threefoldtech/0-robot/zerorobot/dsl/ZeroRobotManager.py", line 171, in create
    raise ServiceCreateError(e['message'], err)
zerorobot.dsl.ZeroRobotManager.ServiceCreateError: fail to create service None github.com/threefoldtech/0-templates/namespace/0.0.1: Node service not found, can't install the namespace: 400 Client Error: BAD REQUEST for url: http:/
/10.102.113.188:6600/services

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "src/gevent/greenlet.py", line 716, in gevent._greenlet.Greenlet.run
  File "/opt/code/github/threefoldtech/0-templates/templates/s3/s3.py", line 603, in _install_namespace
    raise NamespaceDeployError(str(err), node)
s3.NamespaceDeployError: (NamespaceDeployError(...), "fail to create service None github.com/threefoldtech/0-templates/namespace/0.0.1: Node service not found, can't install the namespace: 400 Client Error: BAD REQUEST for url: htt
p://10.102.113.188:6600/services")
2018-11-12T10:28:42Z <Greenlet "Greenlet-0" at 0x7fb374163748: <bound method S3._install_namespace of <s3.S3 object at 0x7fb374cc2dd8>>(node={'farmer_id': 'kristof-farm', 'location': {'city':, name='namespace_tlog', disk_type='ssd'
, size=10, password='qmr2bq71erlcc7y0fjdxeow8ny051ss3')> failed with NamespaceDeployError

env zrobot: dev zos: 

In [10]: node.client.info.version()
Out[10]: 
{'branch': 'development',
 'revision': '06091505e87a2b6b44e0ee49541acf630a175678',
 'dirty': False}
zaibon commented 5 years ago

I did found 2 service node installed, this should not happens. I remove the service too much

0xIslamTaha commented 5 years ago

@zaibon we should prevent creation more than one node service per node.

hossnys commented 5 years ago

verified with creating both s3 and s3_redundant successfully jumbscale_core : 74c2b1ff 0-templates : e03497e