search

doableware / djongo

Django and MongoDB database connector
https://www.djongomapper.com
GNU Affero General Public License v3.0
1.89k stars 355 forks source link

ENFORCE_SCHEMA does not work with inconsistent data types #158

Open radzhome opened 6 years ago

radzhome commented 6 years ago

One line description of the issue

ENFORCE_SCHEMA does not work with inconsistent data types

Python script

Say your model looks something like this

class License(models.Model):
    _id = models.TextField(primary_key=True)
    some_date = models.DateTimeField(auto_now_add=True, editable=True)

You create an instance of it in admin and all is good. In mongo the some_date field is saved as "some_date" : ISODate("2018-07-17T16:18:04.503Z"). Now lets change this to a string in the db, "some_date" : "2018-07-17T16:18:04.503Z". When you try to open this now in admin, there is an error. ENFORCE_SCHEMA works great if the data is completely missing, but what about cases where its the wrong type? Would be nice to allow some data casting on the model if its not in the matching data type. I'm trying to do this with middleware but no luck yet.

Traceback

Internal Server Error: /admin/client_app/client/f4c64540-89dc-11e8-80f5-6476ba8e4cb4/change/
Traceback (most recent call last):
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/core/handlers/exception.py", line 35, in inner
    response = get_response(request)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/core/handlers/base.py", line 158, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/core/handlers/base.py", line 156, in _get_response
    response = response.render()
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/response.py", line 106, in render
    self.content = self.rendered_content
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/response.py", line 83, in rendered_content
    content = template.render(context, self._request)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/backends/django.py", line 61, in render
    return self.template.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 175, in render
    return self._render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 167, in _render
    return self.nodelist.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 943, in render
    bit = node.render_annotated(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 910, in render_annotated
    return self.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/loader_tags.py", line 155, in render
    return compiled_parent._render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 167, in _render
    return self.nodelist.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 943, in render
    bit = node.render_annotated(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 910, in render_annotated
    return self.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/loader_tags.py", line 155, in render
    return compiled_parent._render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 167, in _render
    return self.nodelist.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 943, in render
    bit = node.render_annotated(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 910, in render_annotated
    return self.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/loader_tags.py", line 67, in render
    result = block.nodelist.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 943, in render
    bit = node.render_annotated(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 910, in render_annotated
    return self.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/loader_tags.py", line 67, in render
    result = block.nodelist.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 943, in render
    bit = node.render_annotated(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 910, in render_annotated
    return self.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/defaulttags.py", line 211, in render
    nodelist.append(node.render_annotated(context))
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 910, in render_annotated
    return self.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/loader_tags.py", line 194, in render
    return template.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 177, in render
    return self._render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 167, in _render
    return self.nodelist.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 943, in render
    bit = node.render_annotated(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 910, in render_annotated
    return self.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/defaulttags.py", line 211, in render
    nodelist.append(node.render_annotated(context))
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 910, in render_annotated
    return self.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/defaulttags.py", line 211, in render
    nodelist.append(node.render_annotated(context))
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 910, in render_annotated
    return self.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/defaulttags.py", line 314, in render
    return nodelist.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 943, in render
    bit = node.render_annotated(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 910, in render_annotated
    return self.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/defaulttags.py", line 314, in render
    return nodelist.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 943, in render
    bit = node.render_annotated(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 910, in render_annotated
    return self.render(context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 999, in render
    return render_value_in_context(output, context)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/template/base.py", line 978, in render_value_in_context
    value = str(value)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/utils/html.py", line 380, in <lambda>
    klass.__str__ = lambda self: mark_safe(klass_str(self))
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/forms/boundfield.py", line 36, in __str__
    return self.as_widget()
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/forms/boundfield.py", line 118, in as_widget
    **kwargs
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/forms/widgets.py", line 234, in render
    context = self.get_context(name, value, attrs)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/contrib/admin/widgets.py", line 104, in get_context
    context = super().get_context(name, value, attrs)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/forms/widgets.py", line 808, in get_context
    value = self.decompress(value)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/forms/widgets.py", line 896, in decompress
    value = to_current_timezone(value)
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/forms/utils.py", line 176, in to_current_timezone
    if settings.USE_TZ and value is not None and timezone.is_aware(value):
  File "/Users/rwojcik/.local/share/virtualenvs/content-services-djongo-32J-jGSB/lib/python3.7/site-packages/django/utils/timezone.py", line 247, in is_aware
    return value.utcoffset() is not None
AttributeError: 'str' object has no attribute 'utcoffset'
xeor commented 5 years ago

This might be related to when a model coming from an ArrayModelField is changed.. I'm ending up getting an FieldDoesNotExist because I changed a model that is I use as ArrayModelField, even tho I have ENFORCE_SCHEMA to False