wlanslovenija / nodewatcher

A modular open networks growing platform.
http://nodewatcher.net/
Other
63 stars 20 forks source link

Export survey module data #26

Closed mitar closed 8 years ago

mitar commented 8 years ago

A followup from #23 with branch made in this repository.

mitar commented 8 years ago

@CdavM: Please test if it works. I have not run the code with my changes. If you notice anything, feel free to just fix it.

CdavM commented 8 years ago

@mitar : The code doesn't work anymore :( When I try running your command to export the data, I get the following error:

Traceback (most recent call last):
  File "manage.py", line 9, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 353, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 345, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 348, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 399, in execute
    output = self.handle(*args, **options)
  File "/code/nodewatcher/modules/monitor/http/survey/management/commands/export_survey_data.py", line 68, in handle
    json.dump(graph, f, ensure_ascii=False, cls=django_json.DjangoJSONEncoder)
  File "/usr/lib/python2.7/json/__init__.py", line 190, in dump
    fp.write(chunk)
TypeError: must be unicode, not str

After reverting to f.write(json.dumps(...)), I get the following error:

Traceback (most recent call last):
  File "manage.py", line 9, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 353, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 345, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 348, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 399, in execute
    output = self.handle(*args, **options)
  File "/code/nodewatcher/modules/monitor/http/survey/management/commands/export_survey_data.py", line 68, in handle
    f.write(json.dumps(graph, ensure_ascii=False))
  File "/usr/lib/python2.7/json/__init__.py", line 250, in dumps
    sort_keys=sort_keys, **kw).encode(obj)
  File "/usr/lib/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: datetime.datetime(2016, 7, 25, 4, 7, 48, tzinfo=<bson.tz_util.FixedOffset object at 0x7fb1a377bf90>) is not JSON serializable

So I'd just go back to my way of exporting. I have no idea how to track down these unicode errors. Or do you have any ideas on fixing this?

CdavM commented 8 years ago

OK I fixed writing the output and I reverted to using 'v' and 'e' as keys, as that is done elsewhere in the code.

mitar commented 8 years ago

Above you mentioned that you had issues exporting dates, but then I do not see that you reverted to your old way of exporting dates as strings. So it works for you as it is now?