search

HDFGroup / h5serv

Reference service implementation of the HDF5 REST API
Other
168 stars 35 forks source link

Tests fail with connection error but running the server works. #104

Closed jbants closed 7 years ago

jbants commented 7 years ago

I've set up a vanilla Ubuntu 16.04 machine on AWS and followed the instruction tutorials here. I've set the security group with access to port 5000. The conda install command failed because it wasn't able to find watchdog so I installed watchdog using pip. I can run the server as outlined in the documentation and can reach the api endpoint. However, when i run the tests i get a connection error.

Any Ideas? Test output is below.

Thanks.

1
timeUtilTest
2016-11-19T23:47:44Z
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK
fileUtilTest
....
----------------------------------------------------------------------
Ran 4 tests in 0.000s

OK
('>filename:', 'passwd.h5')
('>username:', 'test_user1')
('>password:', 'test')
('>email:', None)
user already exists
('remove files', 'test_user1')
('cwd:', '/home/ubuntu/h5serv/data/home/test_user1')
create symlink
('>filename:', 'passwd.h5')
('>username:', 'test_user2')
('>password:', 'test')
('>email:', None)
user already exists
('remove files', 'test_user2')
('cwd:', '/home/ubuntu/h5serv/data/home/test_user2')
create symlink
('remove files', '../../data/test')
('remove files', '../../data/test/subdir')
('remove files', '../../data/test/subdir/subdir')
copyfile("array_dset.h5", "../../data/test/array_dset.h5")
copyfile("h5ex_d_gzip.h5", "../../data/test/h5ex_d_gzip.h5")
copyfile("null_objref_dset.h5", "../../data/test/null_objref_dset.h5")
copyfile("h5ex_d_shuffle.h5", "../../data/test/h5ex_d_shuffle.h5")
copyfile("h5ex_d_transform.h5", "../../data/test/h5ex_d_transform.h5")
copyfile("objref_dset.h5", "../../data/test/objref_dset.h5")
copyfile("objref_dset.h5", "../../data/test/objref_dset_updated.h5")
copyfile("h5ex_d_unlimadd.h5", "../../data/test/h5ex_d_unlimadd.h5")
copyfile("compound_attr.h5", "../../data/test/compound_attr.h5")
copyfile("scalar.h5", "../../data/test/scalar.h5")
copyfile("scalar.h5", "../../data/test/scalar_1d_deleted.h5")
copyfile("tstr.h5", "../../data/test/tstr.h5")
copyfile("compound.h5", "../../data/test/compound.h5")
copyfile("dset_gzip.h5", "../../data/test/dset_gzip.h5")
copyfile("dim_scale.h5", "../../data/test/dim_scale.h5")
copyfile("group1k.h5", "../../data/test/group1k.h5")
copyfile("group1k.h5", "../../data/test/group1k_updated.h5")
copyfile("fixed_string_dset.h5", "../../data/test/fixed_string_dset.h5")
copyfile("dim_scale_data.h5", "../../data/test/dim_scale_data.h5")
copyfile("dim_scale_data.h5", "../../data/test/dim_scale_updated.h5")
copyfile("vlen_attr.h5", "../../data/test/vlen_attr.h5")
copyfile("h5ex_d_checksum.h5", "../../data/test/h5ex_d_checksum.h5")
copyfile("opaque_dset.h5", "../../data/test/opaque_dset.h5")
copyfile("objref_attr.h5", "../../data/test/objref_attr.h5")
copyfile("bitfield_attr.h5", "../../data/test/bitfield_attr.h5")
copyfile("tall_with_udlink.h5", "../../data/test/tall_with_udlink.h5")
copyfile("tall.h5", "../../data/test/tall.h5")
copyfile("tall.h5", "../../data/test/tall_updated.h5")
copyfile("tall.h5", "../../data/test/tall_ro.h5")
copyfile("tall.h5", "../../data/test/tall_g2_deleted.h5")
copyfile("tall.h5", "../../data/test/tall_dset112_deleted.h5")
copyfile("tall.h5", "../../data/test/tall_dset22_deleted.h5")
copyfile("tall.h5", "../../data/test/tall_acl.h5")
copyfile("tall.h5", "../../data/test/tall_acl_delete.h5")
copyfile("tall.h5", "../../data/test/tall.dots.need.to.be.encoded.h5")
copyfile("compound_array_attr.h5", "../../data/test/compound_array_attr.h5")
copyfile("h5ex_d_rdwr.h5", "../../data/test/h5ex_d_rdwr.h5")
copyfile("vlen_string_dset.h5", "../../data/test/vlen_string_dset.h5")
copyfile("h5ex_d_alloc.h5", "../../data/test/h5ex_d_alloc.h5")
copyfile("null_space_dset.h5", "../../data/test/null_space_dset.h5")
copyfile("h5ex_d_compact.h5", "../../data/test/h5ex_d_compact.h5")
copyfile("vlen_string_attr.h5", "../../data/test/vlen_string_attr.h5")
copyfile("regionref_dset.h5", "../../data/test/regionref_dset.h5")
copyfile("regionref_dset.h5", "../../data/test/regionref_dset_updated.h5")
copyfile("h5ex_d_nbit.h5", "../../data/test/h5ex_d_nbit.h5")
copyfile("h5ex_d_fillval.h5", "../../data/test/h5ex_d_fillval.h5")
copyfile("zerodim.h5", "../../data/test/filename with space.h5")
copyfile("zerodim.h5", "../../data/test/deleteme.h5")
copyfile("zerodim.h5", "../../data/test/readonly.h5")
copyfile("zerodim.h5", "../../data/test/subdir/zerodim.h5")
copyfile("zerodim.h5", "../../data/test/subdir/deleteme.h5")
copyfile("zerodim.h5", "../../data/test/subdir/subdir/deleteme.h5")
copyfile("fixed_string_attr.h5", "../../data/test/fixed_string_attr.h5")
copyfile("enum_attr.h5", "../../data/test/enum_attr.h5")
copyfile("resizable.h5", "../../data/test/resizable.h5")
copyfile("resizable.h5", "../../data/test/resized.h5")
copyfile("h5ex_d_chunk.h5", "../../data/test/h5ex_d_chunk.h5")
copyfile("h5ex_d_sofloat.h5", "../../data/test/h5ex_d_sofloat.h5")
copyfile("compound_array_dset.h5", "../../data/test/compound_array_dset.h5")
copyfile("opaque_attr.h5", "../../data/test/opaque_attr.h5")
copyfile("h5ex_d_unlimgzip.h5", "../../data/test/h5ex_d_unlimgzip.h5")
copyfile("notahdf5file.h5", "../../data/test/notahdf5file.h5")
copyfile("committed_type.h5", "../../data/test/committed_type.h5")
copyfile("enum_dset.h5", "../../data/test/enum_dset.h5")
copyfile("bitfield_dset.h5", "../../data/test/bitfield_dset.h5")
copyfile("compound_committed.h5", "../../data/test/compound_committed.h5")
copyfile("array_attr.h5", "../../data/test/array_attr.h5")
copyfile("regionref_attr.h5", "../../data/test/regionref_attr.h5")
copyfile("h5ex_d_soint.h5", "../../data/test/h5ex_d_soint.h5")
copyfile("dset1k.h5", "../../data/test/dset1k.h5")
copyfile("namedtype.h5", "../../data/test/namedtype.h5")
copyfile("namedtype.h5", "../../data/test/namedtype_deleted.h5")
copyfile("vlen_dset.h5", "../../data/test/vlen_dset.h5")
copyfile("h5ex_d_extern.h5", "../../data/test/h5ex_d_extern.h5")
copyfile("h5ex_d_hyper.h5", "../../data/test/h5ex_d_hyper.h5")
copyfile("fillvalue.h5", "../../data/test/fillvalue.h5")
copyfile("arraytype.h5", "../../data/test/arraytype.h5")
copyfile("attr1k.h5", "../../data/test/attr1k.h5")
copyfile("type1k.h5", "../../data/test/type1k.h5")
copyfile("null_space_attr.h5", "../../data/test/null_space_attr.h5")
('chmod', '../../data/test/tall_ro.h5')
('chmod', '../../data/test/readonly.h5')
roottest
EEEEEEEEEEEEEEEEEE
======================================================================
ERROR: testDelete (__main__.RootTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "roottest.py", line 166, in testDelete
    rsp = requests.delete(req, headers=headers)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 150, in delete
    return request('delete', url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc57f1cd1d0>: Failed to establish a new connection: [Errno 111] Connection refused',))

======================================================================
ERROR: testDeleteNotFound (__main__.RootTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "roottest.py", line 181, in testDeleteNotFound
    rsp = requests.delete(req, headers=headers)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 150, in delete
    return request('delete', url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc57f12ffd0>: Failed to establish a new connection: [Errno 111] Connection refused',))

======================================================================
ERROR: testDeleteReadonly (__main__.RootTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "roottest.py", line 174, in testDeleteReadonly
    rsp = requests.delete(req, headers=headers)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 150, in delete
    return request('delete', url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc57f0c00d0>: Failed to establish a new connection: [Errno 111] Connection refused',))

======================================================================
ERROR: testDeleteSubSubdomain (__main__.RootTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "roottest.py", line 188, in testDeleteSubSubdomain
    rsp = requests.delete(req, headers=headers)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 150, in delete
    return request('delete', url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc57f12ff50>: Failed to establish a new connection: [Errno 111] Connection refused',))

======================================================================
ERROR: testDomainWithSpaces (__main__.RootTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "roottest.py", line 131, in testDomainWithSpaces
    rsp = requests.get(req, headers=headers)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 70, in get
    return request('get', url, params=params, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc57f12fed0>: Failed to establish a new connection: [Errno 111] Connection refused',))

======================================================================
ERROR: testGetDomain (__main__.RootTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "roottest.py", line 37, in testGetDomain
    rsp = requests.get(req, headers=headers)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 70, in get
    return request('get', url, params=params, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc57f0c0190>: Failed to establish a new connection: [Errno 111] Connection refused',))

======================================================================
ERROR: testGetDomainWithDot (__main__.RootTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "roottest.py", line 210, in testGetDomainWithDot
    rsp = requests.get(req, headers=headers)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 70, in get
    return request('get', url, params=params, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc57f12ffd0>: Failed to establish a new connection: [Errno 111] Connection refused',))

======================================================================
ERROR: testGetInfo (__main__.RootTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "roottest.py", line 27, in testGetInfo
    rsp = requests.get(req)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 70, in get
    return request('get', url, params=params, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: /info (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc57f12ff50>: Failed to establish a new connection: [Errno 111] Connection refused',))

======================================================================
ERROR: testGetNotFound (__main__.RootTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "roottest.py", line 78, in testGetNotFound
    rsp = requests.get(req, headers=headers)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 70, in get
    return request('get', url, params=params, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc57f0c0190>: Failed to establish a new connection: [Errno 111] Connection refused',))

======================================================================
ERROR: testGetReadOnly (__main__.RootTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "roottest.py", line 57, in testGetReadOnly
    rsp = requests.get(req, headers=headers)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 70, in get
    return request('get', url, params=params, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc57f0c02d0>: Failed to establish a new connection: [Errno 111] Connection refused',))

======================================================================
ERROR: testGetSubdomain (__main__.RootTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "roottest.py", line 139, in testGetSubdomain
    rsp = requests.get(req, headers=headers)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 70, in get
    return request('get', url, params=params, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc57f0c02d0>: Failed to establish a new connection: [Errno 111] Connection refused',))

======================================================================
ERROR: testGetToc (__main__.RootTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "roottest.py", line 68, in testGetToc
    rsp = requests.get(req, headers=headers)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 70, in get
    return request('get', url, params=params, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc57f0c02d0>: Failed to establish a new connection: [Errno 111] Connection refused',))

======================================================================
ERROR: testInvalidDomain (__main__.RootTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "roottest.py", line 98, in testInvalidDomain
    rsp = requests.get(req, headers=headers)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 70, in get
    return request('get', url, params=params, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc57f0c02d0>: Failed to establish a new connection: [Errno 111] Connection refused',))

======================================================================
ERROR: testPut (__main__.RootTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "roottest.py", line 196, in testPut
    rsp = requests.put(req, headers=headers)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 124, in put
    return request('put', url, data=data, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc57f0c03d0>: Failed to establish a new connection: [Errno 111] Connection refused',))

======================================================================
ERROR: testPutNameWithDot (__main__.RootTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "roottest.py", line 230, in testPutNameWithDot
    rsp = requests.put(req, headers=headers)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 124, in put
    return request('put', url, data=data, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc57f0c03d0>: Failed to establish a new connection: [Errno 111] Connection refused',))

======================================================================
ERROR: testPutSubSubdomain (__main__.RootTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "roottest.py", line 155, in testPutSubSubdomain
    rsp = requests.put(req, headers=headers)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 124, in put
    return request('put', url, data=data, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc57f0c03d0>: Failed to establish a new connection: [Errno 111] Connection refused',))

======================================================================
ERROR: testPutSubdomain (__main__.RootTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "roottest.py", line 147, in testPutSubdomain
    rsp = requests.put(req, headers=headers)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 124, in put
    return request('put', url, data=data, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc57f0c03d0>: Failed to establish a new connection: [Errno 111] Connection refused',))

======================================================================
ERROR: testWrongTopLevelDomain (__main__.RootTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "roottest.py", line 85, in testWrongTopLevelDomain
    rsp = requests.get(req, headers=headers)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 70, in get
    return request('get', url, params=params, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc57f0c03d0>: Failed to establish a new connection: [Errno 111] Connection refused',))

----------------------------------------------------------------------
Ran 18 tests in 0.025s

FAILED (errors=18)
server log...
2016-11-19 23:46:50,714:INFO:tocUtil.py:219::<type 'str'>
2016-11-19 23:46:50,714:INFO:tocUtil.py:220::data_dir: [../data]
2016-11-19 23:46:50,714:INFO:tocUtil.py:221::<type 'str'>
2016-11-19 23:46:50,714:INFO:tocUtil.py:224::../data/../data/home
2016-11-19 23:46:50,714:INFO:tocUtil.py:234::grppath: /test/subdir
2016-11-19 23:46:50,714:INFO:tocUtil.py:243::grppath: /test/subdir
2016-11-19 23:46:50,714:INFO:tocUtil.py:244::base_domain: hdfgroup.org
2016-11-19 23:46:50,714:INFO:tocUtil.py:245::domainpath: subdir.test.hdfgroup.org
2016-11-19 23:46:50,714:INFO:tocUtil.py:247::walk, file: zerodim.h5
2016-11-19 23:46:50,714:INFO:tocUtil.py:253::walk, filepath: ../data/test/subdir/zerodim.h5
2016-11-19 23:46:50,714:INFO:tocUtil.py:273::filename (noext): zerodim
2016-11-19 23:46:50,714:INFO:tocUtil.py:281::tocfile - create_group: /test/subdir
2016-11-19 23:46:50,715:INFO:tocUtil.py:285::filedomain: zerodim.subdir.test.hdfgroup.org
2016-11-19 23:46:50,715:INFO:tocUtil.py:289::tocFile - ExternalLink: subdir.test.hdfgroup.org
2016-11-19 23:46:50,715:INFO:tocUtil.py:247::walk, file: subdir
2016-11-19 23:46:50,715:INFO:tocUtil.py:253::walk, filepath: ../data/test/subdir/subdir
2016-11-19 23:46:50,715:INFO:tocUtil.py:265::skip non-hdf5 extension
2016-11-19 23:46:50,715:INFO:tocUtil.py:247::walk, file: deleteme.h5
2016-11-19 23:46:50,715:INFO:tocUtil.py:253::walk, filepath: ../data/test/subdir/deleteme.h5
2016-11-19 23:46:50,715:INFO:tocUtil.py:273::filename (noext): deleteme
2016-11-19 23:46:50,715:INFO:tocUtil.py:285::filedomain: deleteme.subdir.test.hdfgroup.org
2016-11-19 23:46:50,716:INFO:tocUtil.py:289::tocFile - ExternalLink: subdir.test.hdfgroup.org
2016-11-19 23:46:50,716:INFO:tocUtil.py:217::root: ../data/test/subdir/subdir
2016-11-19 23:46:50,716:INFO:tocUtil.py:218::toc_dir: [../data]
2016-11-19 23:46:50,716:INFO:tocUtil.py:219::<type 'str'>
2016-11-19 23:46:50,716:INFO:tocUtil.py:220::data_dir: [../data]
2016-11-19 23:46:50,716:INFO:tocUtil.py:221::<type 'str'>
2016-11-19 23:46:50,716:INFO:tocUtil.py:224::../data/../data/home
2016-11-19 23:46:50,716:INFO:tocUtil.py:234::grppath: /test/subdir/subdir
2016-11-19 23:46:50,716:INFO:tocUtil.py:243::grppath: /test/subdir/subdir
2016-11-19 23:46:50,716:INFO:tocUtil.py:244::base_domain: hdfgroup.org
2016-11-19 23:46:50,716:INFO:tocUtil.py:245::domainpath: subdir.subdir.test.hdfgroup.org
2016-11-19 23:46:50,716:INFO:tocUtil.py:247::walk, file: deleteme.h5
2016-11-19 23:46:50,717:INFO:tocUtil.py:253::walk, filepath: ../data/test/subdir/subdir/deleteme.h5
2016-11-19 23:46:50,717:INFO:tocUtil.py:273::filename (noext): deleteme
2016-11-19 23:46:50,717:INFO:tocUtil.py:281::tocfile - create_group: /test/subdir/subdir
2016-11-19 23:46:50,717:INFO:tocUtil.py:285::filedomain: deleteme.subdir.subdir.test.hdfgroup.org
2016-11-19 23:46:50,717:INFO:tocUtil.py:289::tocFile - ExternalLink: subdir.subdir.test.hdfgroup.org
2016-11-19 23:46:50,717:INFO:tocUtil.py:217::root: ../data/home
2016-11-19 23:46:50,717:INFO:tocUtil.py:218::toc_dir: [../data]
2016-11-19 23:46:50,717:INFO:tocUtil.py:219::<type 'str'>
2016-11-19 23:46:50,717:INFO:tocUtil.py:220::data_dir: [../data]
2016-11-19 23:46:50,717:INFO:tocUtil.py:221::<type 'str'>
2016-11-19 23:46:50,718:INFO:tocUtil.py:224::../data/../data/home
2016-11-19 23:46:50,718:INFO:tocUtil.py:226::skipping home dir
2016-11-19 23:46:50,718:INFO:tocUtil.py:217::root: ../data/home/test_user2
2016-11-19 23:46:50,718:INFO:tocUtil.py:218::toc_dir: [../data]
2016-11-19 23:46:50,718:INFO:tocUtil.py:219::<type 'str'>
2016-11-19 23:46:50,718:INFO:tocUtil.py:220::data_dir: [../data]
2016-11-19 23:46:50,718:INFO:tocUtil.py:221::<type 'str'>
2016-11-19 23:46:50,718:INFO:tocUtil.py:224::../data/../data/home
2016-11-19 23:46:50,718:INFO:tocUtil.py:226::skipping home dir
2016-11-19 23:46:50,718:INFO:tocUtil.py:217::root: ../data/home/test_user1
2016-11-19 23:46:50,718:INFO:tocUtil.py:218::toc_dir: [../data]
2016-11-19 23:46:50,718:INFO:tocUtil.py:219::<type 'str'>
2016-11-19 23:46:50,718:INFO:tocUtil.py:220::data_dir: [../data]
2016-11-19 23:46:50,718:INFO:tocUtil.py:221::<type 'str'>
2016-11-19 23:46:50,719:INFO:tocUtil.py:224::../data/../data/home
2016-11-19 23:46:50,719:INFO:tocUtil.py:226::skipping home dir
2016-11-19 23:46:50,719:INFO:tocUtil.py:217::root: ../data/public
2016-11-19 23:46:50,719:INFO:tocUtil.py:218::toc_dir: [../data]
2016-11-19 23:46:50,719:INFO:tocUtil.py:219::<type 'str'>
2016-11-19 23:46:50,719:INFO:tocUtil.py:220::data_dir: [../data]
2016-11-19 23:46:50,719:INFO:tocUtil.py:221::<type 'str'>
2016-11-19 23:46:50,719:INFO:tocUtil.py:224::../data/../data/home
2016-11-19 23:46:50,719:INFO:tocUtil.py:234::grppath: /public
2016-11-19 23:46:50,719:INFO:tocUtil.py:243::grppath: /public
2016-11-19 23:46:50,719:INFO:tocUtil.py:244::base_domain: hdfgroup.org
2016-11-19 23:46:50,719:INFO:tocUtil.py:245::domainpath: public.hdfgroup.org
2016-11-19 23:46:50,720:INFO:tocUtil.py:247::walk, file: tall.h5
2016-11-19 23:46:50,720:INFO:tocUtil.py:253::walk, filepath: ../data/public/tall.h5
2016-11-19 23:46:50,720:INFO:tocUtil.py:273::filename (noext): tall
2016-11-19 23:46:50,720:INFO:tocUtil.py:281::tocfile - create_group: /public
2016-11-19 23:46:50,720:INFO:tocUtil.py:285::filedomain: tall.public.hdfgroup.org
2016-11-19 23:46:50,720:INFO:tocUtil.py:289::tocFile - ExternalLink: public.hdfgroup.org
2016-11-19 23:46:50,721:INFO:app.py:321::verifyFile: ../data/.toc.h5
2016-11-19 23:46:50,721:INFO:fileUtil.py:266::verifyFile('../data/.toc.h5', False)
2016-11-19 23:46:50,721:INFO:app.py:188::baseHandler, href: http://54.186.250.126:5000
2016-11-19 23:46:50,721:INFO:app.py:194::REQ GET http://54.186.250.126:5000/ {remote_ip: 50.66.101.142}
2016-11-19 23:46:50,721:INFO:hdf5db.py:163::init -- filePath: ../data/.toc.h5 mode: r+
2016-11-19 23:46:50,722:INFO:hdf5db.py:194::Hdf5db __enter
2016-11-19 23:46:50,722:INFO:hdf5db.py:548::initializing file
2016-11-19 23:46:50,853:INFO:hdf5db.py:570::visit: public name: Group
2016-11-19 23:46:50,854:INFO:hdf5db.py:570::visit: test name: Group
2016-11-19 23:46:50,855:INFO:hdf5db.py:570::visit: test/subdir name: Group
2016-11-19 23:46:50,856:INFO:hdf5db.py:570::visit: test/subdir/subdir name: Group
2016-11-19 23:46:50,856:INFO:hdf5db.py:713::getUUIDByPath: [/]
2016-11-19 23:46:50,856:INFO:hdf5db.py:198::Hdf5db __exit
2016-11-19 23:46:51,209:INFO:h5watchdog.py:27::H5EventHandler -- Created file: ../data/.toc.h5
2016-11-19 23:46:51,210:INFO:h5watchdog.py:45::H5EventHandler -- Modified directory: ../data
2016-11-19 23:46:51,225:INFO:h5watchdog.py:45::H5EventHandler -- Modified file: ../data/.toc.h5
2016-11-19 23:46:51,390:INFO:h5watchdog.py:45::H5EventHandler -- Modified file: ../data/.toc.h5
2016-11-19 23:46:51,744:INFO:app.py:3091::process_queue, got: ../data/.toc.h5
2016-11-19 23:46:51,744:INFO:app.py:3056::updateToc(../data/.toc.h5)
2016-11-19 23:46:51,744:INFO:app.py:3065::ignore toc file creation
2016-11-19 23:47:25,853:WARNING:app.py:2961::Caught signal: 2
2016-11-19 23:47:25,853:INFO:app.py:2968::Stopping http server
2016-11-19 23:47:25,853:INFO:app.py:2971::Will shutdown in 2 seconds ...
2016-11-19 23:47:25,853:INFO:app.py:2985::closing db
2016-11-19 23:47:27,853:INFO:app.py:2982::Shutdown
Failed
jreadey commented 7 years ago

Hi,

It looks like the test cases are hitting the endpoint. What output do you see when you first start the server? E.g. On my system I get:

$ python app.py
Using logfile:  ../log/h5serv.log
password_uri config: ../util/admin/passwd.h5
Setting log level to: INFO
INFO:app.py:3164::log test
dirname path: 
favicon_path: favicon.ico
Static content in the path:../static will be displayed via the url: /views/(.*)
isdebug: True
domain: hdfgroup.org
ssl_port: 6050
Setting watchdog on:  ../data/
INFO:passwordUtil.py:56::getAuthClient
INFO:passwordUtil.py:58::password_uri:../util/admin/passwd.h5
INFO:authFile.py:36::AuthFile class init(../util/admin/passwd.h5)
INFO:app.py:3216::INITIALIZING...
INFO:app.py:3217::Starting event loop on port: 5000
Starting event loop on port: 5000

Once the server is started, what does curl return? E.g.:

$ curl http://127.0.0.1:5000/
{"lastModified": "2016-11-20T02:15:20Z", "created": "2016-11-20T02:15:20Z", "root": "2ff0219c-aec7-11e6-b6c9-3c15c2da029e", "hrefs": [{"href": "http://127.0.0.1:5000/", "rel": "self"}, {"href": "http://127.0.0.1:5000/datasets", "rel": "database"}, {"href": "http://127.0.0.1:5000/groups", "rel": "groupbase"}, {"href": "http://127.0.0.1:5000/datatypes", "rel": "typebase"}, {"href": "http://127.0.0.1:5000/groups/2ff0219c-aec7-11e6-b6c9-3c15c2da029e", "rel": "root"}]}
jreadey commented 7 years ago

BTW, I've updated the readthedocs instructions (to use pip for watchdog).

Also, Are you running testall.py from the AWS machine or externally? If externally, the testscript won't work since it's trying to do things like copy files to the server data directory.

jbants commented 7 years ago

Hi @jreadey, thanks for looking into this and updating the docs.

When i run python app.py --datapath=/data I get the following:

('Using logfile: ', '../log/h5serv.log')
('password_uri config:', '../util/admin/passwd.h5')
Setting log level to: INFO
INFO:app.py:3164::log test
('dirname path:', '')
('favicon_path:', 'favicon.ico')
Static content in the path:../static will be displayed via the url: /views/(.*)
('isdebug:', True)
('domain:', 'hdfgroup.org')
('ssl_port:', 6050)
('Setting watchdog on: ', '/data')
INFO:passwordUtil.py:56::getAuthClient
INFO:passwordUtil.py:58::password_uri:../util/admin/passwd.h5
INFO:authFile.py:36::AuthFile class init(../util/admin/passwd.h5)
INFO:app.py:3216::INITIALIZING...
INFO:app.py:3217::Starting event loop on port: 5000
Starting event loop on port: 5000

curl yields:

curl http://127.0.0.1:5000/
{"lastModified": "2016-11-20T00:18:29Z", "hrefs": [{"href": "http://127.0.0.1:5000/", "rel": "self"}, {"href": "http://127.0.0.1:5000/datasets", "rel": "database"}, {"href": "http://127.0.0.1:5000/groups", "rel": "groupbase"}, {"href": "http://127.0.0.1:5000/datatypes", "rel": "typebase"}, {"href": "http://127.0.0.1:5000/groups/a0d1564e-aeb6-11e6-9dd1-0670fd626c9f", "rel": "root"}], "root": "a0d1564e-aeb6-11e6-9dd1-0670fd626c9f", "created": "2016-11-20T00:18:29Z"}

Im running these directly on the AWS machine.

jreadey commented 7 years ago

That's odd.

Try running just one of the failing test cases. Cd to h5serv/test/integ and run:

$ python roottest.py RootTest.testGetDomain

Does the test case pass and what output do you see from the h5serv log?

jbants commented 7 years ago

Running the roottest

python roottest.py RootTest.testGetDomain
E
======================================================================
ERROR: testGetDomain (__main__.RootTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "roottest.py", line 37, in testGetDomain
    rsp = requests.get(req, headers=headers)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 70, in get
    return request('get', url, params=params, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f3f3a2bb910>: Failed to establish a new connection: [Errno 111] Connection refused',))

----------------------------------------------------------------------
Ran 1 test in 0.005s

FAILED (errors=1)

I rotated the log files and run the above. Nothing was written to the new log file and this is the output in the log file after running the server again. (assuming it's writing to /h5serv/log/h5serv.log)

2016-11-20 05:36:02,609:INFO:app.py:3164::log test
2016-11-20 05:36:02,614:INFO:passwordUtil.py:56::getAuthClient
2016-11-20 05:36:02,614:INFO:passwordUtil.py:58::password_uri:../util/admin/passwd.h5
2016-11-20 05:36:02,615:INFO:authFile.py:36::AuthFile class init(../util/admin/passwd.h5)
2016-11-20 05:36:02,615:INFO:app.py:3216::INITIALIZING...
2016-11-20 05:36:02,615:INFO:app.py:3217::Starting event loop on port: 5000
2016-11-20 05:36:24,851:INFO:app.py:284::getFilePath: 54.186.250.126:5000 checkExists: True
2016-11-20 05:36:24,851:INFO:app.py:286::tocFilePath: /data/.toc.h5
2016-11-20 05:36:24,851:INFO:app.py:321::verifyFile: /data/.toc.h5
2016-11-20 05:36:24,852:INFO:fileUtil.py:266::verifyFile('/data/.toc.h5', False)
2016-11-20 05:36:24,852:INFO:app.py:188::baseHandler, href: http://54.186.250.126:5000
2016-11-20 05:36:24,852:INFO:app.py:194::REQ GET http://54.186.250.126:5000/ {remote_ip: 50.66.101.142}
2016-11-20 05:36:24,852:INFO:hdf5db.py:163::init -- filePath: /data/.toc.h5 mode: r+
2016-11-20 05:36:24,853:INFO:hdf5db.py:194::Hdf5db __enter
2016-11-20 05:36:24,853:INFO:hdf5db.py:713::getUUIDByPath: [/]
2016-11-20 05:36:24,854:INFO:hdf5db.py:198::Hdf5db __exit
2016-11-20 05:36:27,986:WARNING:app.py:2961::Caught signal: 2
2016-11-20 05:36:27,987:INFO:app.py:2968::Stopping http server
2016-11-20 05:36:27,987:INFO:app.py:2971::Will shutdown in 2 seconds ...
2016-11-20 05:36:27,987:INFO:app.py:2985::closing db
2016-11-20 05:36:29,988:INFO:app.py:2982::Shutdown
jreadey commented 7 years ago

And your sure you are running the test from the same host? :)

What does:

$ lsof -i :5000

Return?

Could there be some firewall software running on the host?

jbants commented 7 years ago

Yup, im sshing into the ubuntu machine and running everything from there.

lsof -i :5000
COMMAND  PID   USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
python  1510 ubuntu    9u  IPv4  17146      0t0  TCP *:5000 (LISTEN)
python  1510 ubuntu   10u  IPv6  17147      0t0  TCP *:5000 (LISTEN)

If i run the server in one console and the root test in another, i get this error printed to console for the server.

ERROR:tornado.application:Uncaught exception GET / (127.0.0.1)
HTTPServerRequest(protocol='http', host='tall.test.hdfgroup.org', method='GET', uri='/', version='HTTP/1.1', remote_ip='127.0.0.1', headers={'Connection': 'keep-alive', 'User-Agent': 'python-request/2.12.1', 'Host': 'tall.test.hdfgroup.org', 'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate'})
Traceback (most recent call last):
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/tornado/web.py", line 1467, in _execute
    result = method(*self.path_args, **self.path_kwargs)
  File "app.py", line 2801, in get
    self.baseHandler()
  File "app.py", line 185, in baseHandler
    self.filePath = self.getFilePath(self.domain, checkExists)
  File "app.py", line 288, in getFilePath
    tocUtil.createTocFile(tocFilePath)
  File "/home/ubuntu/h5serv/server/tocUtil.py", line 212, in createTocFile
    f = h5py.File(toc_file, 'w')
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/h5py/_hl/files.py", line 272, in __init__
    fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
  File "/home/ubuntu/anaconda2/envs/bluegeo/lib/python2.7/site-packages/h5py/_hl/files.py", line 98, in make_fid
    fid = h5f.create(name, h5f.ACC_TRUNC, fapl=fapl, fcpl=fcpl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/home/ilan/minonda/conda-bld/work/h5py/_objects.c:2696)
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/home/ilan/minonda/conda-bld/work/h5py/_objects.c:2654)
  File "h5py/h5f.pyx", line 96, in h5py.h5f.create (/home/ilan/minonda/conda-bld/work/h5py/h5f.c:2109)
IOError: Unable to create file (Unable to open file: name = '/data/.toc.h5', errno = 13, error message = 'permission denied', flags = 13, o_flags = 242)
ERROR:tornado.access:500 GET / (127.0.0.1) 4.28ms

The same user that installed hd5py is running the test and server and i have verified that they are the owner of /data.toc.h5.

jreadey commented 7 years ago

My usual setup is to have the server running in one console and the test code running in another. That way it's easy to compare the test output with what the server log is generating.

I noticed the data directory is "/data". The default (which you can verify by looking at the datapath key of h5serv/server/config.py) is "../data". Did you override the default location?

On the first request the server receives it will create a file: ".toc.h5" in the data directory. Do you see these file? Verify that the user can create files in this directory by doing something like: "touch foo".

jbants commented 7 years ago

Ah ok. Yea, i set up a different data directory on another volume just from more storage space.

Running the server without the --datapath flag in one console and the tests in another console results in all tests passing. I assume i can run the server with the datapath pointing to another volume in 'production' mode without issues. It's just the tests that need the default server parameters.

Thanks for your help in clearing this up. Really looking forward to using this!

jreadey commented 7 years ago

Great, glad we got things sorted out.

If you'd really like to run the test case with data in the /data directory, you can modify line 20 of h5serv/test/integ/setupdata.py. Just change DES = "../../data" to the location you want.