search

coreos / kpm

KPM is a tool to deploy and manage application stacks on Kubernetes.
Apache License 2.0
124 stars 29 forks source link

Instantiate registry without beta.kpm.sh? / registry @ beta.kpm.sh is apparently down. #148

Closed dougbtv closed 7 years ago

dougbtv commented 7 years ago

I've been deploying a kpm registry with:

kpm deploy coreos/kpm-registry --namespace kpm -H https://beta.kpm.sh

But, I'm currently out of luck, it's apparently down. (See "Logs 1" below)

Is there any documentation on how to spin up a kpm registry without it?

I'm also trying to run a registry locally and then push the kpm registry manifest, but.... then I don't have the etcd package pushed, and I can't find the manifest for it.

like:

$ docker run --name tempetcd -dt -p 2379:2379 -p 2380:2380 quay.io/coreos/etcd:v3.0.6 /usr/local/bin/etcd -listen-client-urls http://0.0.0.0:2379,http://0.0.0.0:4001 -advertise-client-urls http://$127.0.0.1:2379,http://127.0.0.1:4001
$ gunicorn kpm.api.wsgi:app -b :5555
$ pwd
/usr/src/kpm/deploy/kpm-registry
$ kpm push -H http://localhost:5555 -f
$ kpm deploy coreos/kpm-registry --namespace kpm -H http://localhost:5555
[ ... snip ... ]
requests.exceptions.HTTPError: 404 Client Error: NOT FOUND for url: http://localhost:5555/api/v1/packages/coreos/etcd/pull
$ curl http://localhost:5555/api/v1/packages/coreos/etcd/pull
{
  "error": {
    "code": "package-not-found", 
    "details": {
      "package": "coreos/etcd", 
      "version": null
    }, 
    "message": "package coreos/etcd doesn't exist"
  }
}

Logs 1: beta.kpm.sh apparently down

[root@demos9s kpm-registry]# kpm deploy coreos/kpm-registry --namespace kpm -H https://beta.kpm.sh
Traceback (most recent call last):
  File "/usr/bin/kpm", line 4, in <module>
    __import__('pkg_resources').run_script('kpm==0.21.2rc0', 'kpm')
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 739, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1494, in run_script
    exec(code, namespace, namespace)
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/EGG-INFO/scripts/kpm", line 6, in <module>
    cli()
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/command.py", line 64, in cli
    args.func(args)
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/commands/command_base.py", line 18, in call
    self(options)()
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/commands/command_base.py", line 21, in __call__
    self._call()
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/commands/deploy.py", line 74, in _call
    fmt=self.output)
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/formats/kub.py", line 182, in deploy
    return self._process_deploy(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/formats/kub.py", line 130, in _process_deploy
    dest = os.path.join(dest, self.name, self.version)
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/formats/kub_base.py", line 99, in name
    return self.manifest.package['name']
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/formats/kub_base.py", line 70, in manifest
    self._manifest = ManifestJsonnet(self.package,
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/formats/kub_base.py", line 63, in package
    result = self._fetch_package()
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/formats/kub_base.py", line 125, in _fetch_package
    return self._registry.pull(self._deploy_name, self._deploy_version)
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/registry.py", line 51, in pull
    r = requests.get(path, params=params, headers=self.headers)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 70, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='beta.kpm.sh', port=443): Max retries exceeded with url: /api/v1/packages/coreos/kpm-registry/pull (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x268fa90>: Failed to establish a new connection: [Errno 111] Connection refused',))

[root@demos9s kpm-registry]# curl https://beta.kpm.sh
curl: (7) Failed connect to beta.kpm.sh:443; Connection refused
dougbtv commented 7 years ago

Alright, I think I have a recipe that works to get a kpm registry bootstrapped with gunicorn... And then a kpm registry deployed from a kpm registry ;)

Get up etcd first

$ docker run --name tempetcd -dt -p 2379:2379 -p 2380:2380 quay.io/coreos/etcd:v3.0.6 /usr/local/bin/etcd -listen-client-urls http://0.0.0.0:2379,http://0.0.0.0:4001 -advertise-client-urls http://$127.0.0.1:2379,http://127.0.0.1:4001

Now you can get a registry up with gunicorn

$ pwd
/usr/src
$ gunicorn kpm.api.wsgi:app -b :5555

And then you can push it. But make sure you have a valid image tag in the manifest with a grep v0 manifest.jsonnet (I personally am working on the v0.21.2 tag of this git repo)

$ pwd
/usr/src/kpm/deploy/kpm-registry
$ kpm push -H http://localhost:5555 -f
package: coreos/kpm-registry (0.21.2-4) pushed

Can we deploy it now? Not quite... we get an error package coreos/etcd doesn't exist if we try at this point. And I found the manifest for it in the kubespray/kpm-packages repo.

$ cd /usr/src/
$ git clone https://github.com/kubespray/kpm-packages.git
$ cd kpm-packages/
$ cd coreos/etcdv3
$ pwd
/usr/src/kpm-packages/coreos/etcdv3
$ kpm push -H http://localhost:5555 -f
$ kpm list -H http://localhost:5555
app                  version    downloads
-------------------  ---------  -----------
coreos/etcd          3.0.6-1    -
coreos/kpm-registry  0.21.2-4   -

Now you should be able to deploy a kpm registry via:

$ kpm deploy coreos/kpm-registry --namespace kpm -H http://localhost:5555
ant31 commented 7 years ago

@dougbtv Thx to report Yes, this how I boostrap/deploy the registry from a local instance.

I've moved most of the packages to https://cnr.kubespray.io. I will point beta.kpm.sh to it.

dougbtv commented 7 years ago

Thanks @ant31 I went ahead and took the liberty of opening a PR with this sequence of operations in a markdown doc to outline how to get that up and running, it's in #149

dougbtv commented 7 years ago

Thanks again on the pointer to the URL. It's not entirely necessary for me now that I can bootstrap my registry by running it locally, but, just an FYI that it doesn't appear that the kpm-registry package is up on cnr.kubespray.io

[root@demos9s kpm-registry]# curl https://cnr.kubespray.io/api/v1/packages/coreos/kpm-registry/pull
<html>
<head><title>404 Not Found</title></head>
<body bgcolor="white">
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.11.6</center>
</body>
</html>
[root@demos9s kpm-registry]# kpm deploy coreos/kpm-registry --namespace kpm -H  https://cnr.kubespray.io
Traceback (most recent call last):
  File "/usr/bin/kpm", line 4, in <module>
    __import__('pkg_resources').run_script('kpm==0.21.2rc0', 'kpm')
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 739, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1494, in run_script
    exec(code, namespace, namespace)
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/EGG-INFO/scripts/kpm", line 6, in <module>
    cli()
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/command.py", line 64, in cli
    args.func(args)
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/commands/command_base.py", line 18, in call
    self(options)()
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/commands/command_base.py", line 21, in __call__
    self._call()
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/commands/deploy.py", line 74, in _call
    fmt=self.output)
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/formats/kub.py", line 182, in deploy
    return self._process_deploy(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/formats/kub.py", line 130, in _process_deploy
    dest = os.path.join(dest, self.name, self.version)
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/formats/kub_base.py", line 99, in name
    return self.manifest.package['name']
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/formats/kub_base.py", line 70, in manifest
    self._manifest = ManifestJsonnet(self.package,
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/formats/kub_base.py", line 63, in package
    result = self._fetch_package()
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/formats/kub_base.py", line 125, in _fetch_package
    return self._registry.pull(self._deploy_name, self._deploy_version)
  File "/usr/lib/python2.7/site-packages/kpm-0.21.2rc0-py2.7.egg/kpm/registry.py", line 52, in pull
    r.raise_for_status()
  File "/usr/lib/python2.7/site-packages/requests/models.py", line 893, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://cnr.kubespray.io/api/v1/packages/coreos/kpm-registry/pull
ant31 commented 7 years ago

Yes my bad, cnr.kubespray.io is the UI. beta.kpm.sh or kpm.sh are working again (same app).

You have to use latest version (>=0.24.2) and it improves speed on stackanetes deployment (it starts immediately). Also the cli changed a little: