BTS connected but there's no telemetry data

[i-am-nut]

I managed to connect a SysmoBTS to CCM. I'm running osmo-nitb on osmocom vm client and osmo-bts on sysmo. Issuing show bts 0 on osmo-nitb vty from vm client shows that it's connected.

Also, i've registrated snowflake ID of osmocom vm client on the cloud and checking the dashboard it says "active".

However, when clicking on "Tower Monitoring" to see the stats, it says: This tower is running old software (version "None") and is not sending telemetry data in checkins.

I'm reading the scripts to understand what's going on but could get nothing so far. What might be wrong? Any thoughts?

[kheimerl]

Here's the html for that particular page: https://github.com/facebookincubator/CommunityCellularManager/blob/cae14433d28b42b97b8e4c5b1f52e3e0813d1a92/cloud/endagaweb/templates/dashboard/staff/tower-monitoring.html

You'll see the "old software" check is the endaga_version which is apparently either not being set ( https://github.com/facebookincubator/CommunityCellularManager/blob/91d78f1a8994a9eec181a10ee5a06eecb4952bc6/client/scripts/update_installed_versions#L33) or not being sent in the checkin ( https://github.com/facebookincubator/CommunityCellularManager/blob/afcb0c460d7c537f04b9bbb656d6df252699f0fc/cloud/endagaweb/checkin.py#L185 ).

Does that make sense?

On Wed, Sep 6, 2017 at 8:57 AM, Emerson Junior notifications@github.com wrote:

I managed to connect a SysmoBTS to CCM. I'm running osmo-nitb on osmocom vm client and osmo-bts on sysmo. Issuing show bts 0 on osmo-nitb vty from vm client shows that it's connected.

Also, i've registrated snowflake ID of osmocom vm client on the cloud and checking the dashboard it says "active".

However, when clicking on "Tower Monitoring" to see the stats, it says: This tower is running old software (version "None") and is not sending telemetry data in checkins.

I'm reading the scripts to understand what's going on but could get nothing so far. What might be wrong? Any thoughts?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/facebookincubator/CommunityCellularManager/issues/79, or mute the thread https://github.com/notifications/unsubscribe-auth/AA_rnwwnIoceelrh6uetkIFPC6x4asU4ks5sfsD0gaJpZM4POmXm .

[i-am-nut]

Makes sense, but i still can't understand why endaga_version is not being sent to the checkin. May you clarify it a bit more?

[kheimerl]

My guess it's not being set correctly. I'd look on the BTS and see if the 'endaga_version' is set in the database (run 'endaga_db_get endaga_version')

On Thu, Sep 21, 2017 at 6:59 AM, Emerson Junior notifications@github.com wrote:

Makes sense, but i still can't understand why endaga_version is not being sent to the checkin. May you clarify it a bit more?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/facebookincubator/CommunityCellularManager/issues/79#issuecomment-331165169, or mute the thread https://github.com/notifications/unsubscribe-auth/AA_rn4KiuMfQ7BkijrHmYmFgna3_6d6pks5skmvYgaJpZM4POmXm .

[i-am-nut]

I did run 'endaga_db_get endaga_version'

vagrant@endaga-client-osmocom:~$ endaga_db_get endaga_version
0.8.1
vagrant@endaga-client-osmocom:~$ 

How may it not being sent in the checkin as it is correctly set?

[i-am-nut]

I'm not sure if it helps in something, but on Web VM, output of python manage.py runserver 192.168.40.10:8000 shows the following:

[21/Sep/2017 15:24:02] "GET /dashboard/towers/68700135-6342-4821-9b9f-31254fd8cac5 HTTP/1.1" 200 8323
Error handling checkin (BTS 68700135-6342-4821-9b9f-31254fd8cac5): list index out of range
BTS status: {u'system_utilization': {u'memory_percent': 75.3, u'disk_percent': 41.6, u'bytes_sent_delta': 156942, u'bytes_received_delta': 327168, u'cpu_percent': 0.0}, u'uptime': 4051, u'versions': {u'openbts-public': None, u'python-gsm': u'', u'type': u'osmocom', u'osmocom-public': u'0.15.1.20170703', u'python-osmocom': u'1.0', u'python-openbts': None, u'endaga': u'0.8.1', u'freeswitch': u'1.6.16~33~e6d643b-1~jessie+1', u'python-endaga-core': u'0.6.1', u'gsm': u'0.15.1.20170703'}, u'camped_subscribers': [], u'openbts_load': {u'checkin.process_lat': 0.006463766098022461, u'checkin.raw_rsp_sz': 130982, u'checkin.rsp_sz': 130982, u'checkin.req_sz': 494, u'checkin.lat': 1.9099562168121338, u'checkin.post_lat': 0.868234395980835, u'checkin.raw_req_sz': 920}, u'openbts_noise': {u'some_noise_stats_here_tbd': 0}, u'radio': {u'band': u'GSM900', u'c0': u'63'}, u'subscribers': {}, u'usage': {u'events': []}}
Internal Server Error: /api/v1/checkin
Traceback (most recent call last):
  File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/exception.py", line 39, in inner
    response = get_response(request)
  File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
    response = self._get_response(request)
  File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view
    return view_func(*args, **kwargs)
  File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/views/generic/base.py", line 68, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/rest_framework/views.py", line 466, in dispatch
    response = self.handle_exception(exc)
  File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/rest_framework/views.py", line 463, in dispatch
    response = handler(request, *args, **kwargs)
  File "/home/vagrant/cloud/endagaweb/views/api.py", line 592, in post
    resp = checkin.CheckinResponder(bts).process(bts_status)
  File "/home/vagrant/cloud/endagaweb/checkin.py", line 161, in process
    resp['config'] = self._optimize('config', self.gen_config())
  File "/home/vagrant/cloud/endagaweb/checkin.py", line 394, in gen_config
    channel='stable').order_by('-date')[0].version
  File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/db/models/query.py", line 296, in __getitem__
    return list(qs)[0]
IndexError: list index out of range
[21/Sep/2017 15:26:42] ERROR [exception.py:124] Internal Server Error: /api/v1/checkin
Traceback (most recent call last):
  File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/exception.py", line 39, in inner
    response = get_response(request)
  File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
    response = self._get_response(request)
  File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view
    return view_func(*args, **kwargs)
  File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/views/generic/base.py", line 68, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/rest_framework/views.py", line 466, in dispatch
    response = self.handle_exception(exc)
  File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/rest_framework/views.py", line 463, in dispatch
    response = handler(request, *args, **kwargs)
  File "/home/vagrant/cloud/endagaweb/views/api.py", line 592, in post
    resp = checkin.CheckinResponder(bts).process(bts_status)
  File "/home/vagrant/cloud/endagaweb/checkin.py", line 161, in process
    resp['config'] = self._optimize('config', self.gen_config())
  File "/home/vagrant/cloud/endagaweb/checkin.py", line 394, in gen_config
    channel='stable').order_by('-date')[0].version
  File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/db/models/query.py", line 296, in __getitem__
    return list(qs)[0]
IndexError: list index out of range
[21/Sep/2017 15:26:43] "POST /api/v1/checkin?id=68700135 HTTP/1.1" 500 130978

Any ideas?

[kheimerl]

Ok yes! If I remember correctly the problem is there aren't versions defined in the cloud, which should be part of the test_db setup. Did you setup the fake database?

On Thu, Sep 21, 2017 at 11:35 AM, Emerson Junior notifications@github.com wrote:

I'm not sure if it helps in something, but on Web VM, output of python manage.py runserver 192.168.40.10:8000 shows the following:

[21/Sep/2017 15:24:02] "GET /dashboard/towers/68700135-6342-4821-9b9f-31254fd8cac5 HTTP/1.1" 200 8323 Error handling checkin (BTS 68700135-6342-4821-9b9f-31254fd8cac5): list index out of range BTS status: {u'system_utilization': {u'memory_percent': 75.3, u'disk_percent': 41.6, u'bytes_sent_delta': 156942, u'bytes_received_delta': 327168, u'cpu_percent': 0.0}, u'uptime': 4051, u'versions': {u'openbts-public': None, u'python-gsm': u'', u'type': u'osmocom', u'osmocom-public': u'0.15.1.20170703', u'python-osmocom': u'1.0', u'python-openbts': None, u'endaga': u'0.8.1', u'freeswitch': u'1.6.16~33~e6d643b-1~jessie+1', u'python-endaga-core': u'0.6.1', u'gsm': u'0.15.1.20170703'}, u'camped_subscribers': [], u'openbts_load': {u'checkin.process_lat': 0.006463766098022461, u'checkin.raw_rsp_sz': 130982, u'checkin.rsp_sz': 130982, u'checkin.req_sz': 494, u'checkin.lat': 1.9099562168121338, u'checkin.post_lat': 0.868234395980835, u'checkin.raw_req_sz': 920}, u'openbts_noise': {u'some_noise_stats_here_tbd': 0}, u'radio': {u'band': u'GSM900', u'c0': u'63'}, u'subscribers': {}, u'usage': {u'events': []}} Internal Server Error: /api/v1/checkin Traceback (most recent call last): File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response response = self._get_response(request) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, callback_kwargs) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view return view_func(*args, *kwargs) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/views/generic/base.py", line 68, in view return self.dispatch(request, args, kwargs) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/rest_framework/views.py", line 466, in dispatch response = self.handle_exception(exc) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/rest_framework/views.py", line 463, in dispatch response = handler(request, *args, kwargs) File "/home/vagrant/cloud/endagaweb/views/api.py", line 592, in post resp = checkin.CheckinResponder(bts).process(bts_status) File "/home/vagrant/cloud/endagaweb/checkin.py", line 161, in process resp['config'] = self._optimize('config', self.gen_config()) File "/home/vagrant/cloud/endagaweb/checkin.py", line 394, in gen_config channel='stable').order_by('-date')[0].version File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/db/models/query.py", line 296, in getitem return list(qs)[0] IndexError: list index out of range [21/Sep/2017 15:26:42] ERROR [exception.py:124] Internal Server Error: /api/v1/checkin Traceback (most recent call last): File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response response = self._get_response(request) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, *callback_kwargs) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view return view_func(args, kwargs) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/views/generic/base.py", line 68, in view return self.dispatch(request, *args, *kwargs) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/rest_framework/views.py", line 466, in dispatch response = self.handle_exception(exc) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/rest_framework/views.py", line 463, in dispatch response = handler(request, args, **kwargs) File "/home/vagrant/cloud/endagaweb/views/api.py", line 592, in post resp = checkin.CheckinResponder(bts).process(bts_status) File "/home/vagrant/cloud/endagaweb/checkin.py", line 161, in process resp['config'] = self._optimize('config', self.gen_config()) File "/home/vagrant/cloud/endagaweb/checkin.py", line 394, in gen_config channel='stable').order_by('-date')[0].version File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/db/models/query.py", line 296, in getitem return list(qs)[0] IndexError: list index out of range [21/Sep/2017 15:26:43] "POST /api/v1/checkin?id=68700135 HTTP/1.1" 500 130978

Any ideas?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/facebookincubator/CommunityCellularManager/issues/79#issuecomment-331244184, or mute the thread https://github.com/notifications/unsubscribe-auth/AA_rn9dZgUHUwdLFDJq_EHHg7qkcR-aoks5skqyPgaJpZM4POmXm .

[kheimerl]

Specifically this code: https://github.com/facebookincubator/CommunityCellularManager/blob/50c47d264ebd9c0c82c4c46d43a606805f4e0bcb/cloud/endagaweb/management/commands/setup_test_db.py#L71

Does that.

On Thu, Sep 21, 2017 at 6:09 PM, Kurtis Heimerl kheimerl@cs.washington.edu wrote:

Ok yes! If I remember correctly the problem is there aren't versions defined in the cloud, which should be part of the test_db setup. Did you setup the fake database?

On Thu, Sep 21, 2017 at 11:35 AM, Emerson Junior <notifications@github.com

wrote:

I'm not sure if it helps in something, but on Web VM, output of python manage.py runserver 192.168.40.10:8000 shows the following:

[21/Sep/2017 15:24:02] "GET /dashboard/towers/68700135-6342-4821-9b9f-31254fd8cac5 HTTP/1.1" 200 8323 Error handling checkin (BTS 68700135-6342-4821-9b9f-31254fd8cac5): list index out of range BTS status: {u'system_utilization': {u'memory_percent': 75.3, u'disk_percent': 41.6, u'bytes_sent_delta': 156942, u'bytes_received_delta': 327168, u'cpu_percent': 0.0}, u'uptime': 4051, u'versions': {u'openbts-public': None, u'python-gsm': u'', u'type': u'osmocom', u'osmocom-public': u'0.15.1.20170703', u'python-osmocom': u'1.0', u'python-openbts': None, u'endaga': u'0.8.1', u'freeswitch': u'1.6.16~33~e6d643b-1~jessie+1', u'python-endaga-core': u'0.6.1', u'gsm': u'0.15.1.20170703'}, u'camped_subscribers': [], u'openbts_load': {u'checkin.process_lat': 0.006463766098022461, u'checkin.raw_rsp_sz': 130982, u'checkin.rsp_sz': 130982, u'checkin.req_sz': 494, u'checkin.lat': 1.9099562168121338, u'checkin.post_lat': 0.868234395980835, u'checkin.raw_req_sz': 920}, u'openbts_noise': {u'some_noise_stats_here_tbd': 0}, u'radio': {u'band': u'GSM900', u'c0': u'63'}, u'subscribers': {}, u'usage': {u'events': []}} Internal Server Error: /api/v1/checkin Traceback (most recent call last): File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response response = self._get_response(request) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, callback_kwargs) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view return view_func(*args, *kwargs) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/views/generic/base.py", line 68, in view return self.dispatch(request, args, kwargs) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/rest_framework/views.py", line 466, in dispatch response = self.handle_exception(exc) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/rest_framework/views.py", line 463, in dispatch response = handler(request, *args, kwargs) File "/home/vagrant/cloud/endagaweb/views/api.py", line 592, in post resp = checkin.CheckinResponder(bts).process(bts_status) File "/home/vagrant/cloud/endagaweb/checkin.py", line 161, in process resp['config'] = self._optimize('config', self.gen_config()) File "/home/vagrant/cloud/endagaweb/checkin.py", line 394, in gen_config channel='stable').order_by('-date')[0].version File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/db/models/query.py", line 296, in getitem return list(qs)[0] IndexError: list index out of range [21/Sep/2017 15:26:42] ERROR [exception.py:124] Internal Server Error: /api/v1/checkin Traceback (most recent call last): File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response response = self._get_response(request) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, *callback_kwargs) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view return view_func(args, kwargs) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/views/generic/base.py", line 68, in view return self.dispatch(request, *args, *kwargs) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/rest_framework/views.py", line 466, in dispatch response = self.handle_exception(exc) File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/rest_framework/views.py", line 463, in dispatch response = handler(request, args, **kwargs) File "/home/vagrant/cloud/endagaweb/views/api.py", line 592, in post resp = checkin.CheckinResponder(bts).process(bts_status) File "/home/vagrant/cloud/endagaweb/checkin.py", line 161, in process resp['config'] = self._optimize('config', self.gen_config()) File "/home/vagrant/cloud/endagaweb/checkin.py", line 394, in gen_config channel='stable').order_by('-date')[0].version File "/home/vagrant/.virtualenvs/endaga/local/lib/python2.7/site-packages/django/db/models/query.py", line 296, in getitem return list(qs)[0] IndexError: list index out of range [21/Sep/2017 15:26:43] "POST /api/v1/checkin?id=68700135 HTTP/1.1" 500 130978

Any ideas?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/facebookincubator/CommunityCellularManager/issues/79#issuecomment-331244184, or mute the thread https://github.com/notifications/unsubscribe-auth/AA_rn9dZgUHUwdLFDJq_EHHg7qkcR-aoks5skqyPgaJpZM4POmXm .

[i-am-nut]

Nice, you've got it!

It seems the issue is solved now, but before closing it i'm wondering what's the meaning in setting first a test db with some fictional data in order to use the application properly? If you hadn't told me, i'd never realize it. Sorry if i'm not getting the point.

[kheimerl]

It's in the readme!

Version numbers need to be generated, we probably should have a better error handler for that case though.

On Mon, Sep 25, 2017 at 6:59 AM, Emerson Junior notifications@github.com wrote:

Nice, you've got it!

It seems the issue is solved now, but before closing it i'm wondering what's the meaning in setting first a test db with some fictional data in order to use the application properly? If you hadn't told me, i'd never realize it.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/facebookincubator/CommunityCellularManager/issues/79#issuecomment-331889973, or mute the thread https://github.com/notifications/unsubscribe-auth/AA_rn-bHknIsZB79ReAtJcevX69Tns6iks5sl7HbgaJpZM4POmXm .

[marvdias]

Hi, I have a similar problem, I can see the tower status as "Active" however there is no telemetry data. The message on the monitor tab says "There are no monitoring stats available. This tower should send some data over soon.", I guess the check-in is not working as well. On web cloud I have messages like these:

[20/Oct/2017 18:45:35] "GET /api/v1/bts/register?vpn_ip=10.64.0.18&bts_uuid=346eae3b-e2d4-402e-aeed-f6c38cd73be9&federer_port=80&vpn_status=up HTTP/1.1" 200 112
[20/Oct/2017 18:45:37] "GET /api/v1/bts/register?vpn_ip=10.64.0.18&bts_uuid=346eae3b-e2d4-402e-aeed-f6c38cd73be9&vpn_status=up&federer_port=80 HTTP/1.1" 200 112
[20/Oct/2017 18:45:40] "GET /api/v1/bts/register?vpn_status=up&vpn_ip=10.64.0.18&federer_port=80&bts_uuid=346eae3b-e2d4-402e-aeed-f6c38cd73be9 HTTP/1.1" 200 112
[20/Oct/2017 18:45:44] "GET /api/v1/bts/register?federer_port=80&bts_uuid=346eae3b-e2d4-402e-aeed-f6c38cd73be9&vpn_ip=10.64.0.18&vpn_status=up HTTP/1.1" 200 112
[20/Oct/2017 18:45:55] "GET /api/v1/bts/register?federer_port=80&vpn_ip=10.64.0.18&bts_uuid=346eae3b-e2d4-402e-aeed-f6c38cd73be9&vpn_status=up HTTP/1.1" 200 112
[20/Oct/2017 18:45:58] "GET /api/v1/bts/register?vpn_ip=10.64.0.18&bts_uuid=346eae3b-e2d4-402e-aeed-f6c38cd73be9&federer_port=80&vpn_status=up HTTP/1.1" 200 112
[20/Oct/2017 18:46:02] "GET /api/v1/bts/register?vpn_ip=10.64.0.18&bts_uuid=346eae3b-e2d4-402e-aeed-f6c38cd73be9&vpn_status=up&federer_port=80 HTTP/1.1" 200 112
[20/Oct/2017 18:46:06] "GET /api/v1/bts/register?federer_port=80&vpn_status=up&bts_uuid=346eae3b-e2d4-402e-aeed-f6c38cd73be9&vpn_ip=10.64.0.18 HTTP/1.1" 200 112

The endaga log shows a critical error :

2017-10-20T18:49:00.409213+00:00 [CRITICAL] endaga: endagad:22:<module>: caught unhandled exception: 'python-gsm_version', Traceback (most recent call last):#012  File "/usr/local/lib/python3.4/dist-packages/core/db/kvstore.py", line 79, in __getitem__#012    (key, ))#012  File "/usr/local/lib/python3.4/dist-packages/core/db/connector.py", line 132, in exec_and_fetch_one#012    raise IndexError("no value generated by '%s'" % (stmt, ))#012IndexError: no value generated by 'SELECT value FROM endaga_config WHERE key = %s;'#012#012During handling of the above exception, another exception occurred:#012#012Traceback (most recent call last):#012  File "/usr/local/bin/endagad", line 19, in <module>#012    EndagaD().run()#012  File "/usr/local/lib/python3.4/dist-packages/core/endagad.py", line 112, in run#012    checkin_data = eapi.checkin(timeout=30)#012  File "/usr/local/lib/python3.4/dist-packages/core/interconnect.py", line 129, in checkin#012    status['versions'] = bts.get_versions()#012  File "/usr/local/lib/python3.4/dist-packages/core/bts/_osmocom.py", line 243, in get_versions#012    versions['python-osmocom'] = self.conf['python-gsm_version']#012  File "/usr/local/lib/python3.4/dist-packages/core/config_database.py", line 55, in __getitem__#012    return self._ducktype(super(ConfigDB, self).__getitem__(key))#012  File "/usr/local/lib/python3.4/dist-packages/core/db/kvstore.py", line 82, in __getitem__#012    raise KeyError(key)#012KeyError: 'python-gsm_version'

Is it normal? What should I do?

[kheimerl]

Did you try creating the test database?

On Fri, Oct 20, 2017 at 11:52 AM, marvdias notifications@github.com wrote:

Hi, I have a similar problem, I can see the tower status as "Active" however there is no telemetry data. The message on the monitor tab says "There are no monitoring stats available. This tower should send some data over soon.", I guess the check-in is not working as well. On web cloud I have messages like these:

[20/Oct/2017 18:45:35] "GET /api/v1/bts/register?vpn_ip=10.64.0.18&bts_uuid=346eae3b-e2d4-402e-aeed-f6c38cd73be9&federer_port=80&vpn_status=up HTTP/1.1" 200 112 [20/Oct/2017 18:45:37] "GET /api/v1/bts/register?vpn_ip=10.64.0.18&bts_uuid=346eae3b-e2d4-402e-aeed-f6c38cd73be9&vpn_status=up&federer_port=80 HTTP/1.1" 200 112 [20/Oct/2017 18:45:40] "GET /api/v1/bts/register?vpn_status=up&vpn_ip=10.64.0.18&federer_port=80&bts_uuid=346eae3b-e2d4-402e-aeed-f6c38cd73be9 HTTP/1.1" 200 112 [20/Oct/2017 18:45:44] "GET /api/v1/bts/register?federer_port=80&bts_uuid=346eae3b-e2d4-402e-aeed-f6c38cd73be9&vpn_ip=10.64.0.18&vpn_status=up HTTP/1.1" 200 112 [20/Oct/2017 18:45:55] "GET /api/v1/bts/register?federer_port=80&vpn_ip=10.64.0.18&bts_uuid=346eae3b-e2d4-402e-aeed-f6c38cd73be9&vpn_status=up HTTP/1.1" 200 112 [20/Oct/2017 18:45:58] "GET /api/v1/bts/register?vpn_ip=10.64.0.18&bts_uuid=346eae3b-e2d4-402e-aeed-f6c38cd73be9&federer_port=80&vpn_status=up HTTP/1.1" 200 112 [20/Oct/2017 18:46:02] "GET /api/v1/bts/register?vpn_ip=10.64.0.18&bts_uuid=346eae3b-e2d4-402e-aeed-f6c38cd73be9&vpn_status=up&federer_port=80 HTTP/1.1" 200 112 [20/Oct/2017 18:46:06] "GET /api/v1/bts/register?federer_port=80&vpn_status=up&bts_uuid=346eae3b-e2d4-402e-aeed-f6c38cd73be9&vpn_ip=10.64.0.18 HTTP/1.1" 200 112

The endaga log shows a critical error :

2017-10-20T18:49:00.409213+00:00 [CRITICAL] endaga: endagad:22:: caught unhandled exception: 'python-gsm_version', Traceback (most recent call last):#012 File "/usr/local/lib/python3.4/dist-packages/core/db/kvstore.py", line 79, in getitem#012 (key, ))#012 File "/usr/local/lib/python3.4/dist-packages/core/db/connector.py", line 132, in exec_and_fetch_one#012 raise IndexError("no value generated by '%s'" % (stmt, ))#012IndexError: no value generated by 'SELECT value FROM endaga_config WHERE key = %s;'#012#012During handling of the above exception, another exception occurred:#012#012Traceback (most recent call last):#012 File "/usr/local/bin/endagad", line 19, in #012 EndagaD().run()#012 File "/usr/local/lib/python3.4/dist-packages/core/endagad.py", line 112, in run#012 checkin_data = eapi.checkin(timeout=30)#012 File "/usr/local/lib/python3.4/dist-packages/core/interconnect.py", line 129, in checkin#012 status['versions'] = bts.get_versions()#012 File "/usr/local/lib/python3.4/dist-packages/core/bts/_osmocom.py", line 243, in get_versions#012 versions['python-osmocom'] = self.conf['python-gsm_version']#012 File "/usr/local/lib/python3.4/dist-packages/core/config_database.py", line 55, in getitem#012 return self._ducktype(super(ConfigDB, self).getitem(key))#012 File "/usr/local/lib/python3.4/dist-packages/core/db/kvstore.py", line 82, in getitem#012 raise KeyError(key)#012KeyError: 'python-gsm_version'

Is it normal? What should I do?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/facebookincubator/CommunityCellularManager/issues/79#issuecomment-338292991, or mute the thread https://github.com/notifications/unsubscribe-auth/AA_rnzblcIJofmWAn3mTk0zmPAC386Rhks5suOvzgaJpZM4POmXm .

[marvdias]

Yes, before it I ran "python manage.py setup_test_db"

[kheimerl]

And there's dummy data in the web portal?

Does the client have ['python-gsm_version'] in the local database ('endaga_db_get')?

On Fri, Oct 20, 2017 at 12:15 PM, marvdias notifications@github.com wrote:

Yes, before it I ran "python manage.py setup_test_db"

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/facebookincubator/CommunityCellularManager/issues/79#issuecomment-338298417, or mute the thread https://github.com/notifications/unsubscribe-auth/AA_rn3ZC0Lf7KIStAcwfO6FzMeud05E4ks5suPFggaJpZM4POmXm .

[marvdias]

Yes there is dummy data, for the other towers randomly created. Does the client have ['python-gsm_version'] in the local database ('endaga_db_get')? This message was shown after the command:

endaga_db_get python-gsm_version
no such key: python-gsm_version

[kheimerl]

that is a problem! Try running update_installed_versions https://github.com/facebookincubator/CommunityCellularManager/blob/master/client/scripts/update_installed_versions

This should be run when you install the endaga metapackage, so I bet there was an error in doing that.

On Fri, Oct 20, 2017 at 12:30 PM, marvdias notifications@github.com wrote:

Yes there is dummy data, for the other towers randomly created. Does the client have ['python-gsm_version'] in the local database ('endaga_db_get')? This message was shown after the command:

endaga_db_get python-gsm_version no such key: python-gsm_version

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/facebookincubator/CommunityCellularManager/issues/79#issuecomment-338301697, or mute the thread https://github.com/notifications/unsubscribe-auth/AA_rn8ZZlfwo2AhDK_po7rkJO8uyLTFxks5suPTTgaJpZM4POmXm .

[marvdias]

Well, I ran update_installed_versions, and it only updates the following:

vagrant@endaga-client-osmocom:~$ sudo update_installed_versions
endaga:0.8.1
freeswitch:1.6.16~33~e6d643b-1~jessie+1
gsm:0.15.1.20171001

Is there anything else I can do?

[kheimerl]

I feel like you definitely messed up the package installs. Is python3-osmocom installed at all?

On Fri, Oct 20, 2017 at 1:11 PM, marvdias notifications@github.com wrote:

Well, I ran update_installed_versions, and it only updates the following:

vagrant@endaga-client-osmocom:~$ sudo update_installed_versions endaga:0.8.1 freeswitch:1.6.16~33~e6d643b-1~jessie+1 gsm:0.15.1.20171001

Is there anything else I can do?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/facebookincubator/CommunityCellularManager/issues/79#issuecomment-338310682, or mute the thread https://github.com/notifications/unsubscribe-auth/AA_rn0VDvbalSB4YSY6vzB9DzDUuQxRKks5suP58gaJpZM4POmXm .

[marvdias]

Well, I don't know what could have gone wrong.. I just installed the endaga-osmocom package. I looked for the python3-osmocom and:

vagrant@endaga-client-osmocom:~$ sudo apt-cache search python3-osmocom
python3-osmocom - Osmocom VTY client
vagrant@endaga-client-osmocom:~$ sudo apt-get install python3-osmocom
Reading package lists... Done
Building dependency tree
Reading state information... Done
python3-osmocom is already the newest version.
python3-osmocom set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
20 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n]
Setting up libcurl3:amd64 (7.38.0-4+deb8u6) ...
Setting up libcurl3-gnutls:amd64 (7.38.0-4+deb8u6) ...
Setting up libosmocore8:amd64 (0.9.6.20171019) ...
Setting up libosmocodec0:amd64 (0.9.6.20171019) ...
Setting up libosmogsm7:amd64 (0.9.6.20171019) ...
Setting up libosmocoding0:amd64 (0.9.6.20171019) ...
Setting up libosmovty3:amd64 (0.9.6.20171019) ...
Setting up libosmoctrl0:amd64 (0.9.6.20171019) ...
Setting up libosmogb4:amd64 (0.9.6.20171019) ...
Setting up libsmpp0:amd64 (1.12.0.20171019) ...
Setting up curl (7.38.0-4+deb8u6) ...
Setting up libgtp1:amd64 (1.0.0.20171019) ...
Setting up libosmoabis5:amd64 (0.4.0.20171019) ...
Setting up libosmotrau1:amd64 (0.4.0.20171019) ...
Setting up libsmpp34-dev:amd64 (1.12.0.20171019) ...
Setting up osmo-bts-trx (0.6.0.20171019) ...
Setting up osmo-pcu (0.3.0.20171019) ...
Setting up osmo-sip-connector (1.20171019) ...
Setting up osmo-trx (0.1.9.20171019) ...
Setting up osmocom-sgsn (0.15.1.20171019) ...
Processing triggers for libc-bin (2.19-18+deb8u10) ...

[marvdias]

Is there anyway to proceed with another clean osmocom environment without destroying this one?

[kheimerl]

yes but it's a bit complicated.

Given that the package is installed fine but the setup_installed_versions script isn't grabbing it, you might have to step down into the get_versions() function, it's apparently not getting the versions for some of your packages. Alternatively you can just set it manually with endaga_db_set

On Fri, Oct 20, 2017 at 1:39 PM, marvdias notifications@github.com wrote:

Is there anyway to proceed with another clean osmocom environment without destroying this one?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/facebookincubator/CommunityCellularManager/issues/79#issuecomment-338316609, or mute the thread https://github.com/notifications/unsubscribe-auth/AA_rn7O6XLPAPecdVby6mDUXUgAyfSnrks5suQTsgaJpZM4POmXm .

[marvdias]

Alternatively you can just set it manually with endaga_db_set

I just did it and set to '1.0' version. Now in the web cloud :

[20/Oct/2017 20:38:18] "GET /api/v1/bts/register?vpn_status=up&federer_port=80&vpn_ip=10.64.0.18&bts_uuid=346eae3b-e2d4-402e-aeed-f6c38cd73be9 HTTP/1.1" 200 112
[20/Oct/2017 20:38:19] INFO [checkin.py:80] got delta section 'config' from BTS 346eae3b-e2d4-402e-aeed-f6c38cd73be9: {u'alg': u'md5', u'sig': u'9ede14274f99566cea24c9bba126f31a'}
[20/Oct/2017 20:38:19] INFO [checkin.py:80] got delta section 'subscribers' from BTS 346eae3b-e2d4-402e-aeed-f6c38cd73be9: {u'alg': u'md5', u'sig': u'f5166904a4c424eb6f171dd8c66c03ea'}
[20/Oct/2017 20:38:20] INFO [connectionpool.py:257] Starting new HTTP connection (1): 192.168.40.10
[20/Oct/2017 20:38:20] "POST /sason/acquire/ HTTP/1.1" 200 9
[20/Oct/2017 20:38:20] "POST /api/v1/checkin?id=346eae3b HTTP/1.1" 200 282

and the endaga log :

2017-10-20T20:36:01.699820+00:00 [ERROR] endaga: checkin.py:63:process: Unexpected checkin section: sas
2017-10-20T20:36:01.727910+00:00 [ERROR] endaga: base.py:312:process_bts_settings: Failed to process openbts setting(): Control.LUR.OpenRegistration = ^90155

2017-10-20T20:37:09.017253+00:00 [ERROR] endaga: servicecontrol.py:106:startProcess: Supervisor: failed to start osmo-trx, fault code: 40

2017-10-20T20:37:12.821034+00:00 [ERROR] endaga: checkin.py:63:process: Unexpected checkin section: sas
2017-10-20T20:38:16.746848+00:00 [ERROR] endaga: servicecontrol.py:106:startProcess: Supervisor: failed to start osmo-trx, fault code: 40
2017-10-20T20:38:20.362358+00:00 [ERROR] endaga: checkin.py:63:process: Unexpected checkin section: sas
2017-10-20T20:39:28.085085+00:00 [ERROR] endaga: servicecontrol.py:106:startProcess: Supervisor: failed to start osmo-trx, fault code: 40
2017-10-20T20:39:30.065419+00:00 [ERROR] endaga: checkin.py:63:process: Unexpected checkin section: sas

[marvdias]

Just looked into the monitor tab, and it is showing some data!

[kheimerl]

Oh! I think you might just have an old version of the client. Did you build it yourself? The one in the repo is older than the cloud.

On Fri, Oct 20, 2017 at 1:52 PM, marvdias notifications@github.com wrote:

Alternatively you can just set it manually with endaga_db_set

I just did it and set to '1.0' version. Now in the web cloud :

[20/Oct/2017 20:38:18] "GET /api/v1/bts/register?vpn_status=up&federer_port=80&vpn_ip=10.64.0.18&bts_uuid=346eae3b-e2d4-402e-aeed-f6c38cd73be9 HTTP/1.1" 200 112 [20/Oct/2017 20:38:19] INFO [checkin.py:80] got delta section 'config' from BTS 346eae3b-e2d4-402e-aeed-f6c38cd73be9: {u'alg': u'md5', u'sig': u'9ede14274f99566cea24c9bba126f31a'} [20/Oct/2017 20:38:19] INFO [checkin.py:80] got delta section 'subscribers' from BTS 346eae3b-e2d4-402e-aeed-f6c38cd73be9: {u'alg': u'md5', u'sig': u'f5166904a4c424eb6f171dd8c66c03ea'} [20/Oct/2017 20:38:20] INFO [connectionpool.py:257] Starting new HTTP connection (1): 192.168.40.10 [20/Oct/2017 20:38:20] "POST /sason/acquire/ HTTP/1.1" 200 9 [20/Oct/2017 20:38:20] "POST /api/v1/checkin?id=346eae3b HTTP/1.1" 200 282

and the endaga log :

2017-10-20T20:36:01.699820+00:00 [ERROR] endaga: checkin.py:63:process: Unexpected checkin section: sas 2017-10-20T20:36:01.727910+00:00 [ERROR] endaga: base.py:312:process_bts_settings: Failed to process openbts setting(): Control.LUR.OpenRegistration = ^90155

2017-10-20T20:37:09.017253+00:00 [ERROR] endaga: servicecontrol.py:106:startProcess: Supervisor: failed to start osmo-trx, fault code: 40

2017-10-20T20:37:12.821034+00:00 [ERROR] endaga: checkin.py:63:process: Unexpected checkin section: sas 2017-10-20T20:38:16.746848+00:00 [ERROR] endaga: servicecontrol.py:106:startProcess: Supervisor: failed to start osmo-trx, fault code: 40 2017-10-20T20:38:20.362358+00:00 [ERROR] endaga: checkin.py:63:process: Unexpected checkin section: sas 2017-10-20T20:39:28.085085+00:00 [ERROR] endaga: servicecontrol.py:106:startProcess: Supervisor: failed to start osmo-trx, fault code: 40 2017-10-20T20:39:30.065419+00:00 [ERROR] endaga: checkin.py:63:process: Unexpected checkin section: sas

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/facebookincubator/CommunityCellularManager/issues/79#issuecomment-338319372, or mute the thread https://github.com/notifications/unsubscribe-auth/AA_rn46ld6ND3JFqGAEpQrW1H-bhTs7Yks5suQf8gaJpZM4POmXm .

[marvdias]

No, I just cloned the master. Did the vagrant up (and provision) for the cloud. In the client used the apt-get to install the endaga-osmocom packages

[kheimerl]

Yep! So i'm glad it's checking in but you will probably see smaller errors like that for stuff that isn't in the old client. It's ok. Sounds like the installed_packages bit broke though with the old version.

On Fri, Oct 20, 2017 at 2:12 PM, marvdias notifications@github.com wrote:

No, I just cloned the master. Did the vagrant up (and provision) for the cloud. In the client used the apt-get to install the endaga-osmocom packages

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/facebookincubator/CommunityCellularManager/issues/79#issuecomment-338323461, or mute the thread https://github.com/notifications/unsubscribe-auth/AA_rn_Je4u530EKBYeZHknOEJ-58I-qAks5suQzGgaJpZM4POmXm .

[marvdias]

Another thing that i had to do is in the cert cloud. I had to install the fpm manually (ssh to cert and use the folllowing command )- it does not skip this step otherwise.

sudo apt-get install ruby-dev gcc
sudo gem install fpm

[kheimerl]

Feel free to suggest patches!

On Fri, Oct 20, 2017 at 2:21 PM, marvdias notifications@github.com wrote:

Another thing that i had to do is in the cert cloud. I had to install the fpm manually (ssh to cert and use the folllowing command )- it does not skip this step otherwise.

sudo apt-get install ruby-dev gcc sudo gem install fpm

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/facebookincubator/CommunityCellularManager/issues/79#issuecomment-338325280, or mute the thread https://github.com/notifications/unsubscribe-auth/AA_rnzq7aqDZ-VDSTgaCjvWhfZw--WcMks5suQ7egaJpZM4POmXm .

[9muir]

There's an Ansible role, setup_fpm, that should take care of installing all the dependencies for FPM. It seems to work for me, in particular the 'Build certifier package' step of the Ansible certifier_base role, which is what uses FPM, succeeded and the playbook was executed without errors.

@marvdias if this is still a problem for you please post the output of vagrant up cert in the cloud directory (after deleting the exist certifier VM), and reopen the task.