contrib of reference engine by nginx

[xiekeyang]

@wking You seem have set up a reference engine by nginx and store some static page of ref-engine and cas-engine objects and debugged it. could you please submit them commit to contrib folder, or send to me? I'd debug on my local and implement golang client.

[wking]

You seem have set up a reference engine by nginx and store some static page of ref-engine and cas-engine objects and debugged it. could you please submit them commit to contrib folder...

The README has the full Nginx config, and tells you what to write where. Is that not working for you?

[xiekeyang]

All curl requests for each object have worked OK. I'm HTTPS connection on my local, because your implementation seems only support HTTPS. Below is my HTTP result:

$ python3 -m oci_discovery.ref_engine_discovery -l debug 127.0.0.1:8080/app#1.0
DEBUG:oci_discovery.ref_engine_discovery:discovering ref engines via https://127.0.0.1:8080/.well-known/oci-host-ref-engines
WARNING:oci_discovery.ref_engine_discovery:failed to fetch https://127.0.0.1:8080/.well-known/oci-host-ref-engines (<urlopen error [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:645)>)
DEBUG:oci_discovery.ref_engine_discovery:discovering ref engines via https://0.0.1:8080/.well-known/oci-host-ref-engines
WARNING:oci_discovery.ref_engine_discovery:failed to fetch https://0.0.1:8080/.well-known/oci-host-ref-engines (<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)>)
DEBUG:oci_discovery.ref_engine_discovery:discovering ref engines via https://0.1:8080/.well-known/oci-host-ref-engines
WARNING:oci_discovery.ref_engine_discovery:failed to fetch https://0.1:8080/.well-known/oci-host-ref-engines (<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)>)
DEBUG:oci_discovery.ref_engine_discovery:discovering ref engines via http://127.0.0.1:8080/.well-known/oci-host-ref-engines
DEBUG:oci_discovery.ref_engine_discovery:received ref-engine discovery object:
{'refEngines': [{'protocol': 'oci-index-template-v1',
                 'uri': 'https://{host}/oci-index/{path}'}]}
DEBUG:oci_discovery.ref_engine.oci_index_template:fetching an OCI index for 127.0.0.1:8080/app#1.0 from https://127.0.0.1:8080/oci-index/app
Traceback (most recent call last):
  File "/usr/lib/python3.5/urllib/request.py", line 1254, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "/usr/lib/python3.5/http/client.py", line 1106, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.5/http/client.py", line 1151, in _send_request
    self.endheaders(body)
  File "/usr/lib/python3.5/http/client.py", line 1102, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python3.5/http/client.py", line 934, in _send_output
    self.send(msg)
  File "/usr/lib/python3.5/http/client.py", line 877, in send
    self.connect()
  File "/usr/lib/python3.5/http/client.py", line 1260, in connect
    server_hostname=server_hostname)
  File "/usr/lib/python3.5/ssl.py", line 377, in wrap_socket
    _context=self)
  File "/usr/lib/python3.5/ssl.py", line 752, in __init__
    self.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 988, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 633, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:645)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/kxie/work/src/github.com/xiekeyang/oci-discovery/oci_discovery/ref_engine_discovery/__init__.py", line 61, in resolve
    roots = list(ref_engine.resolve(name=name))
  File "/home/kxie/work/src/github.com/xiekeyang/oci-discovery/oci_discovery/ref_engine/oci_index_template.py", line 46, in resolve
    media_type='application/vnd.oci.image.index.v1+json')
  File "/home/kxie/work/src/github.com/xiekeyang/oci-discovery/oci_discovery/fetch_json/__init__.py", line 21, in fetch
    response = _urllib_request.urlopen(uri)
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 466, in open
    response = self._open(req, data)
  File "/usr/lib/python3.5/urllib/request.py", line 484, in _open
    '_open', req)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 1297, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/usr/lib/python3.5/urllib/request.py", line 1256, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:645)>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/kxie/work/src/github.com/xiekeyang/oci-discovery/oci_discovery/ref_engine_discovery/__main__.py", line 57, in <module>
    resolved[name] = resolve(name=name, protocols=protocols)
  File "/home/kxie/work/src/github.com/xiekeyang/oci-discovery/oci_discovery/ref_engine_discovery/__init__.py", line 64, in resolve
    error.geturl(), error))
AttributeError: 'URLError' object has no attribute 'geturl'

[wking]

I emailed this in a few hours back, but GitHub seems to have ignored the email. Re-posting via the web UI:

On Mon, Sep 11, 2017 at 04:25:20AM +0000, xiekeyang wrote:

I'm HTTPS connection on my local, because your implementation seems only support HTTPS.

To support HTTP as well, you can drop the 301 from the example and use the example's HTTPS configuration instead.

$ python3 -m oci_discovery.ref_engine_discovery -l debug 127.0.0.1:8080/app#1.0

This is not a valid image name, because 127.0.0.1:8080 is an authority, not a host. More on why I used host here. For local testing, you'll want a server listinging on the usual ports (80 and/or 443) somewhere. You can use a container with it's own network namespace if you want to do that without leaving your box.

Alternatively, you can use authority-based names like your 127.0.0.1:8080, but you'll want to patch your _IP_V4_REGEXP to:

_IP_V4_REGEXP = _re.compile(
    '^' + _DEC_OCTET + '(\.' + _DEC_OCTET + '){3}(:[0-9]+)?$')

DEBUG:oci_discovery.ref_engine_discovery:discovering ref engines via https://127.0.0.1:8080/.well-known/oci-host-ref-engines WARNING:oci_discovery.ref_engine_discovery:failed to fetch https://127.0.0.1:8080/.well-known/oci-host-ref-engines (<urlopen error [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:645)>)

unknown protocol suggests you're not actually serving valid TLS on port 8080.

DEBUG:oci_discovery.ref_engine_discovery:discovering ref engines via https://0.0.1:8080/.well-known/oci-host-ref-engines WARNING:oci_discovery.ref_engine_discovery:failed to fetch https://0.0.1:8080/.well-known/oci-host-ref-engines (<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)>)

This is the DNS ancestor walk incorrectly treating your authority part as a FQDN. The _IP_V4_REGEXP patch I suggest earlier in this comment avoids this issue.

DEBUG:oci_discovery.ref_engine_discovery:discovering ref engines via http://127.0.0.1:8080/.well-known/oci-host-ref-engines DEBUG:oci_discovery.ref_engine_discovery:received ref-engine discovery object: {'refEngines': [{'protocol': 'oci-index-template-v1', 'uri': 'https://{host}/oci-index/{path}'}]} DEBUG:oci_discovery.ref_engine.oci_index_template:fetching an OCI index for 127.0.0.1:8080/app#1.0 from https://127.0.0.1:8080/oci-index/app

This far looks good to me, but if you aren't serving HTTPS on 8080 (see the UNKNOWN_PROTOCOL bit earlier) so you probably didn't want to use https in your URI entry.

ssl.SSLError: [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:645)

And this is your ref-engine hit dying because the URI said to use HTTPS, but your 8080 server doesn't support it.

[xiekeyang]

@wking , Yes it is my problem in configuration. It works correctly now on my local PC.

[wking]

It works correctly now on my local PC.

So this issue is closeable?