diegojromerolopez
commented
6 years ago Hi Erik.
I would say that error is an userialization error. Could you try a debug and check if the sent serialized list of object is right?
Closed eriktelepovsky closed 6 years ago
diegojromerolopez
commented
6 years ago Hi Erik.
I would say that error is an userialization error. Could you try a debug and check if the sent serialized list of object is right?
eriktelepovsky
commented
6 years ago Hi Diego.
Thank you for your reply. It works fine, when I use Django's native include:
{% include 'statistics/widgets/commissions.html' with commission_list=commission_list %}How exactly can I help you with the debugging? Let me know what you need to understand the issue better, please.
diegojromerolopez
commented
6 years ago Hello Erik.
Is commission_list of type "list"? Try using a QuerySet instead.
If you can, send me the model and the content of the HTTP request and will try it.
eriktelepovsky
commented
6 years ago Hi. commission_list is actually a QuerySet from ListView. I also tried object_list, the result is same. I also tried to truncate the content of the commissions.html file, didn't help.
My model:
class Commission(models.Model):
ROLE_SALESMAN = 'SALESMAN'
ROLE_DISPATCHER = 'DISPATCHER'
ROLE_OTHER = 'OTHER'
ROLES = (
(ROLE_SALESMAN, _('salesman')),
(ROLE_DISPATCHER, _('dispatcher')),
(ROLE_OTHER, _('other')),
)
order = models.ForeignKey(Offer, on_delete=models.CASCADE)
role = models.CharField(verbose_name=_('role'), choices=ROLES, max_length=10)
user = models.ForeignKey(settings.AUTH_USER_MODEL, verbose_name=_('user'))
rate = models.DecimalField(_('rate'), help_text='%', max_digits=4, decimal_places=1, validators=[MinValueValidator(0), MaxValueValidator(100)], default=100)
amount = models.DecimalField(_('amount'), help_text='€', max_digits=7, decimal_places=2, db_index=True)
extra = models.DecimalField(_('extra'), help_text='€', max_digits=7, decimal_places=2, db_index=True)
income = models.DecimalField(_('income'), help_text='€', max_digits=7, decimal_places=2, db_index=True, default=0)
quota = models.DecimalField(_('quota'), help_text='€', max_digits=7, decimal_places=2, validators=[MinValueValidator(0)], default=0)
quota_fulfillment = models.DecimalField(_('quota'), help_text='€', max_digits=8, decimal_places=2, validators=[MinValueValidator(0)], default=0)
wallet = models.DecimalField(_('wallet'), help_text='€', max_digits=7, decimal_places=2, db_index=True, default=0)
created = models.DateTimeField(_('created'), auto_now_add=True)
modified = models.DateTimeField(_('modified'), auto_now=True)
objects = CommissionQuerySet.as_manager()
history = AuditlogHistoryField()
class Meta:
verbose_name = _('commission')
verbose_name_plural = _('commissions')
ordering = ('created',)
unique_together = (('order', 'user'),)
def __str__(self):
return '[{}] {}: {}€'.format(self.order.order_number, self.user, self.total)
def get_absolute_url(self):
return self.order.get_absolute_url()Hello again.
It seems that the (un)serializer is broken. Try to do the following:
Create a test case in your project with the following (pseudo)code. The idea of this test is to check that the serialization is working well for the queryset Comission.
Other idea that I'm thinking is checking by hand if the serialization is done right.
from __future__ import unicode_literals
import jsonpickle
import uuid
from django.conf import settings
from django import template
from django.contrib.contenttypes.models import ContentType
from django.db.models.query import QuerySet
from django.template import loader, Context
from django.utils.text import slugify
from __future__ import unicode_literals
from Crypto.Cipher import AES
def encrypt(key, data):
cipher = AES.new(key.encode("utf-8"), AES.MODE_EAX)
encrypted_data, tag = cipher.encrypt_and_digest(data.encode("utf-8"))
return cipher.nonce, encrypted_data, tag
def decrypt(key, nonce, encrypted_data, tag):
cipher = AES.new(key.encode("utf-8"), AES.MODE_EAX, nonce.encode("latin-1"))
data = cipher.decrypt_and_verify(encrypted_data.encode("latin-1"), tag.encode("latin-1"))
return data.encode("utf-8")
class PickleTest(unittest.TestCase):
"""
Tests the pickle for several objects.
"""
def __init__(self, *args, **kwargs):
super(PickleTest, self).__init__(*args, **kwargs)
def test_pickle(self):
# QuerySet that is not working
comissions = Commission.objects.filter(**your_condition)
model = Comission
model_name = model.__name__
app_name = ContentType.objects.get_for_model(model).app_label
sql_query, params = comissions.query.sql_with_params()
nonce, encrypted_sql, tag = crypto.encrypt(key="test_passwd", data=sql_query)
comission_context = {
"type": "QuerySet",
"query": encrypted_sql.decode("latin-1"),
"params": params,
"nonce": nonce.decode("latin-1"),
"tag": tag.decode("latin-1"),
"app_name": app_name,
"model": model_name,
}
self.assertEqual(
jsonpickle.loads(jsonpickle.dumps(comission_context)), comission_context
)Try this and let's see if it works.
eriktelepovsky
commented
6 years ago Thanks for reply.
Here is my tests.py file:
from __future__ import unicode_literals
import unittest
import django
import jsonpickle
from async_include import crypto
from django.conf import settings
from django.contrib.contenttypes.models import ContentType
from swida.core.logistics.models import Commission
class PickleTest(unittest.TestCase):
"""
Tests the pickle for several objects.
"""
def __init__(self, *args, **kwargs):
super(PickleTest, self).__init__(*args, **kwargs)
def test_pickle(self):
# QuerySet that is not working
commissions = Commission.objects.all()
model = Commission
model_name = model.__name__
app_name = ContentType.objects.get_for_model(model).app_label
sql_query, params = commissions.query.sql_with_params()
nonce, encrypted_sql, tag = crypto.encrypt(key=settings.SECRET_KEY[:16], data=sql_query)
commission_context = {
"type": "QuerySet",
"query": encrypted_sql.decode("latin-1"),
"params": params,
"nonce": nonce.decode("latin-1"),
"tag": tag.decode("latin-1"),
"app_name": app_name,
"model": model_name,
}
self.assertEqual(
jsonpickle.loads(jsonpickle.dumps(commission_context)), commission_context
)Test passed OK :/
(swida) Eriks-MacBook-Pro:swida erik$ python ../manage.py test swida.core.logistics --keepdb --settings=swida.settings.dev
Using existing test database for alias 'default'...
System check identified no issues (0 silenced).
.
----------------------------------------------------------------------
Ran 1 test in 0.003s
OK
Preserving test database for alias 'default'...I confirm the issue is same for any Commission queryset (regardless the filter condition).
diegojromerolopez
commented
6 years ago Hello Erik.
Try removing the manager (objects field), I don't know if it's related, but I'm at loss. If that doesn't work, try leaving only one field and adding field by field trying to check in what field it doesn't work.
Forget it, it is not related because only the primary key is sent through the AJAX request.
Other test you can do is unserialize the AJAX request. That is:
Please, keep me posted with your progress.
eriktelepovsky
commented
6 years ago Actually, when I remove the manager, I get a different exception:
Internal Server Error: /async_include/get
Traceback (most recent call last):
File "/Users/erik/env/swida/lib/python3.5/site-packages/django/core/handlers/exception.py", line 41, in inner
response = get_response(request)
File "/Users/erik/env/swida/lib/python3.5/site-packages/django/core/handlers/base.py", line 187, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/Users/erik/env/swida/lib/python3.5/site-packages/django/core/handlers/base.py", line 185, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/Users/erik/env/swida/lib/python3.5/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view
return view_func(*args, **kwargs)
File "/Users/erik/src/django/swida/source/async_include/views.py", line 69, in get_template
tag=tag
File "/Users/erik/src/django/swida/source/async_include/crypto.py", line 17, in decrypt
return data.encode("utf-8")
AttributeError: 'bytes' object has no attribute 'encode'When I change line 17 in crypto.py file from
return data.encode("utf-8")to:
return datait works for me. It seems to be Python 3 compatibility issue.
eriktelepovsky
commented
6 years ago But the previous change fixes AttributeError: 'bytes' object has no attribute 'encode' exception only. I am going to debug the IndexError: list index out of range more deeply.
eriktelepovsky
commented
6 years ago It seems I found the issue. I updated jsonpickle library from 0.9.4 to 0.9.5 and it started working. Even with return data.encode("utf-8") in the crypto.py file which is weird to me, but it finally works! Thank you for your help.
diegojromerolopez
commented
6 years ago Excellent! Thank you for your error reports.
I just upgraded the requirements of this package in 0.5.3 version.
You can upgrade your project dependencies and check if everything works as intended, please confirm it indeed works in the 0.5.3 version so I can close this issue.
eriktelepovsky
commented
6 years ago I confirm. It works correctly in 0.5.3 version ;)
vibe78
commented
5 years ago not helpful
Any idea why do I get this? :/