Django 2.0 support - Githubissues

fijter commented 6 years ago

It won't take long before Django 2.0 is released which is currently in beta. I've tried out HVAD with it and it doesn't seem compatible, at least not the admin part; I seem to be getting this error on a list overview:

_clone() got an unexpected keyword argument '_forced_unique_fields'

It would be nice if I could continue using HVAD after upgrading to 2.0 once it gets released.

spectras commented 6 years ago

Hello!

You are perfectly right, it's not compatible right now. Due to lack of time, I only work on hvad to keep it compatible those days. I did not follow Django release cycle lately, if Django 2.0 is due soon, it's high time I put compatibility on my todo list.

No ETA yet, but I push this to the top of my list. Thanks for reminding me!

claudep commented 6 years ago

For information, this is due to this Django commit: https://github.com/django/django/commit/66933a6619be386248ea9329c81b257d5e2e5990 So I guess _chain should be used instead of _clone for Django >= 2.0

spectras commented 6 years ago

Okay, it seems we reached a turning point in hvad history.

On the one hand, there have been many changes in Django 2.0 internals. They break a lot if things in current hvad codebase.
On the other hand, hvad2, while still incomplete, works well with Django 2.0. This is because it uses a completely different approach from hvad1.

Thus the dilemma I now face: should I spend lengthy hours fixing hvad1 compatibility, or should I release hvad2 despite it being incomplete?

Would you be willing to try hvad2 in its current state? You may install it through pip install https://github.com/KristianOellegaard/django-hvad/archive/dev/hvad2.tar.gz. It requires Django 2.0 and python 3.

Most notable changes from hvad 1 are:

AUTOLOAD_TRANSLATIONS setting is now False by default.
Instanciating a translatable model now always creates a translation, unless explicitly told not to.
Accessing the language_code attribute never loads a translation. It returns None if no translation is loaded.
A new introspection API has replaced the incoherent set of methods and functions:
- instance.translations.active replaces get_cached_translation(instance).
- instance.translations.all_languages() replaces instance.get_available_languages().
- getattr(instance.translations.active, name) replaces instance.safe_translation_getter(name).
- lazy_translation_getter is gone.
Under the hood, the queries generated by hvad2 have changed to use a true JOIN clause with the joining conditions directly embedded. This dramatically improves performance, but causes issues when using several separate filter clauses on the same table. This is the part that still needs work, though it's only triggered by edge cases.

spectras commented 6 years ago

I have been looking some more, the ramifications of Django 2 changes run even deeper than I thought.

This means given the little time I have, making hvad1 work with Django2 is close to unfeasible without some external contribution. So I guess I'm heading towards this approach:

Consider hvad 1.8 the final hvad1 release, supporting Django1 until Django 1.11 LTS expires.
Introduce hvad 2.0 starting from Django 2.0.

I am worried because the change is somewhat disruptive to existing code bases. I hoped it could be introduced in a softer manner, slowly phasing out deprecated methods and gathering feedback on the new API instead of forcing it onto user projects.

claudep commented 6 years ago

I guess you're going to merge the hvad2 branch into master soon? Are they blocker to make a pre-release?

spectras commented 6 years ago

Hello,

It's a good question, I am still worried it is not as polished I wished it to be. Yet I don't have time to work on it at the moment. Perhaps a good compromise would be releasing it as is and tagging it as an alpha release.

As for points that still need work:

Finding all the corner cases that will generate different results and documenting them.
Relevant work is on #196. It wasn't merged because I couldn't find a satisfying solution to the issues that this created.

Won't make it to v2: most of #249. Those were the actual reasons the refactor was initiated for, so it's a pity they won't make it. But the refactor in itself brings many improvements to the core, most notably the change to actual join filters instead of regular filters.

At least this will lay the ground that will allow those things to be added incrementally at a later point.

claudep commented 6 years ago

Yes, I think that with a pre-release you'll more likely get the expected reports for corner cases…

claudep commented 6 years ago

I tried the dev/hvad2 branch with Django 2.0, but still got an error: TypeError: _clone() got an unexpected keyword argument '_local_field_names'

Any idea?

elenaoat commented 6 years ago

Hello, I am getting an assertion error assert django.VERSION < (1, 10) on django-hvad-1.8.0 with django 1.10 So I guess not even this version of Django is supported?

rasca commented 6 years ago

Hi @spectras . I'm updating a big app to the latest Django.

We are deciding which way to take for translations given that hvad doesn't support Dango>=2 yet.

Could you please explain us what the current status is? Both for the master branch and the hvad2 branch. We might put in the work to finish hvad2 but we need some guidance. #273 talks about some design decisions needed and corner cases. Where can I read about these?

tback commented 5 years ago

Hey @rasca: I fixed the tests for django 2.1 postgres on https://github.com/tback/django-hvad/tree/dev/hvad2 . I don't know if that works or where it will take me, I just thought you might be interested in any work that is going on.

chakmear commented 5 years ago

Hi @spectras I just started updating several apps of mine to Django 2, most of them using hvad, all of them with a lot of translated data but I'm getting to the same dead end as others, hvad not supporting Django 2... Can you tell us about the current status? And is an update to "hvadDjango2Compatible" possible without loosing data? Thank you very much!

spectras commented 5 years ago

Hello everyone,

I am really sorry hvad has gotten out of focus lately. As a matter of fact, maintaining hvad as django changes is quite some work, and I got discouraged by the sheer amount of things that broke with Django 2.0. Especially as I had hvad almost working with the beta release, and multiple breaking changes were added right before the final release, at a time I had no free time at all.

Now, I understand many large and smaller projects depend on hvad and I don't want hvad to be yet another argument against open-source dependencies. Here is what I intend to do:

I will devote a fair share of my free time to hvad in the next few weeks.
By mid-November, say 18th, depending on my findings, either have hvad running on Django 2.1, or publish a clear end-of-life.

Thanks for bearing the lack of news. I will look what people posted here and the other issues, especially @tback 's passing tests, which look really promising.

Also, if someone is interested in stepping up and becoming an official hvad maintainer, there is definitely a gap to fill :)

spectras commented 5 years ago

Good news, I have a version that passes all tests on Django 2.1 on mysql, postgres and sqlite.

As it turns out, the errors were scary, but they stemmed from a smaller-than-expected number of roots. Always frustrating when so many hours end up in so little actual changes in the code.

As hvad2.0 introduces breaking changes in the API, I need to write up on those. Especially as no solution was found for a few problematic corner cases identified in #273, so in the meantime they will need to be thoroughly documented. There is also some cleaning to do.

So it is unlikely I can release today, but expect it within the upcoming week.

bahattincinic commented 5 years ago

@spectras When will you release new stable version? I saw that you released a beta version. But I'm not sure that is it works correctly. Do you think should I wait for the stable version?

bahattincinic commented 5 years ago

I tested the beta version. When I try to use python manage.py dumpdata with the translatable model. I got the following error;

CommandError: Unable to serialize database: 'Model' object has no attribute '_hvad_query'

i think, it related to the Django serializers. i got the same error.

from django.core import serializers
serializers.serialize(queryset=Model.objects.all(), format='json')

Very Hacky Solution:

https://github.com/django/django/blob/master/django/core/serializers/python.py#L59

if you add following code in the else condition it works;

if field.name == '_hvad_query':
    return

spectras commented 5 years ago

Hello, the beta version is a release, I keep it beta precisely so people can test it and provide feedback, just like you did. I'll mark it as stable if it goes without issues for long enough.

By the way, if you could repost that exact message as a separate issue, it would be great!

bahattincinic commented 5 years ago

Also, I faced with a different bug.

When I try to add a new model, I got the following error while generating migrations.

python manage.py makemigrations
Traceback (most recent call last):
  File "manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/bahattincinic/Projects/personal/env/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/Users/bahattincinic/Projects/personal/env/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/bahattincinic/Projects/personal/env/lib/python3.7/site-packages/django/core/management/base.py", line 316, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/bahattincinic/Projects/personal/env/lib/python3.7/site-packages/django/core/management/base.py", line 353, in execute
    output = self.handle(*args, **options)
  File "/Users/bahattincinic/Projects/personal/env/lib/python3.7/site-packages/django/core/management/base.py", line 83, in wrapped
    res = handle_func(*args, **kwargs)
  File "/Users/bahattincinic/Projects/personal/env/lib/python3.7/site-packages/django/core/management/commands/makemigrations.py", line 170, in handle
    migration_name=self.migration_name,
  File "/Users/bahattincinic/Projects/personal/env/lib/python3.7/site-packages/django/db/migrations/autodetector.py", line 44, in changes
    changes = self._detect_changes(convert_apps, graph)
  File "/Users/bahattincinic/Projects/personal/env/lib/python3.7/site-packages/django/db/migrations/autodetector.py", line 160, in _detect_changes
    self.generate_renamed_models()
  File "/Users/bahattincinic/Projects/personal/env/lib/python3.7/site-packages/django/db/migrations/autodetector.py", line 462, in generate_renamed_models
    model_fields_def = self.only_relation_agnostic_fields(model_state.fields)
  File "/Users/bahattincinic/Projects/personal/env/lib/python3.7/site-packages/django/db/migrations/autodetector.py", line 100, in only_relation_agnostic_fields
    del deconstruction[2]['to']
KeyError: 'to'

Debugging Data:

> /Users/bahattincinic/Projects/personal/env/lib/python3.7/site-packages/django/db/migrations/autodetector.py(101)only_relation_agnostic_fields()
    100                 import ipdb; ipdb.set_trace()
--> 101                 del deconstruction[2]['to']
    102             fields_def.append(deconstruction)

ipdb> deconstruction
('hvad.fields.SingleTranslationObject', ['cms.Menu', 'cms.MenuTranslation'], {})

Django Version:

Django==2.1.4

chrisflink commented 5 years ago

I found that same bug when rebuilding all migrations from scratch. No solution but a workaround:

Remove language fields from your model(s)
Run python manage.py makemigrations
Add language fields to your models
Run python manage.py makemigrations again

This worked for me, hope it helps others too. I found no time for further debugging. Maybe good to know that in the models I use RichTextFields and Taggit tags, there might be a compatibility issue because I found this error on the taggit project: https://github.com/jazzband/django-taggit/issues/206

Hope this helps debugging and thanks for HVAD!

TheDeadOne commented 5 years ago

Python 3.6.3 Django 2.1.7 django-hvad 2.0.0 django-cms 3.6.0

Traceback:

File "D:\Projects\some-project\env3\lib\site-packages\django\core\handlers\exception.py" in inner
  34.             response = get_response(request)

File "D:\Projects\some-project\env3\lib\site-packages\django\core\handlers\base.py" in _get_response
  126.                 response = self.process_exception_by_middleware(e, request)

File "D:\Projects\some-project\env3\lib\site-packages\django\core\handlers\base.py" in _get_response
  124.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "D:\Projects\some-project\env3\lib\site-packages\django\contrib\admin\options.py" in wrapper
  604.                 return self.admin_site.admin_view(view)(*args, **kwargs)

File "D:\Projects\some-project\env3\lib\site-packages\django\utils\decorators.py" in _wrapped_view
  142.                     response = view_func(request, *args, **kwargs)

File "D:\Projects\some-project\env3\lib\site-packages\django\views\decorators\cache.py" in _wrapped_view_func
  44.         response = view_func(request, *args, **kwargs)

File "D:\Projects\some-project\env3\lib\site-packages\django\contrib\admin\sites.py" in inner
  223.             return view(request, *args, **kwargs)

File "D:\Projects\some-project\env3\lib\site-packages\django\utils\decorators.py" in _wrapper
  45.         return bound_method(*args, **kwargs)

File "D:\Projects\some-project\env3\lib\site-packages\django\utils\decorators.py" in _wrapped_view
  142.                     response = view_func(request, *args, **kwargs)

File "D:\Projects\some-project\env3\lib\site-packages\django\contrib\admin\options.py" in changelist_view
  1675.             cl = self.get_changelist_instance(request)

File "D:\Projects\some-project\env3\lib\site-packages\django\contrib\admin\options.py" in get_changelist_instance
  742.             sortable_by,

File "D:\Projects\some-project\env3\lib\site-packages\django\contrib\admin\views\main.py" in __init__
  45.         self.root_queryset = model_admin.get_queryset(request)

File "D:\Projects\some-project\env3\lib\site-packages\hvad\admin.py" in get_queryset
  347.             qs = qs.order_by(*ordering)

File "D:\Projects\some-project\env3\lib\site-packages\hvad\manager.py" in order_by
  647.         return super(TranslationQueryset, self).order_by(*fieldnames)

File "D:\Projects\some-project\env3\lib\site-packages\django\db\models\query.py" in order_by
  1024.         obj = self._chain()

File "D:\Projects\some-project\env3\lib\site-packages\django\db\models\query.py" in _chain
  1163.         obj = self._clone()

File "D:\Projects\some-project\env3\lib\site-packages\hvad\manager.py" in _clone
  201.         return super(TranslationQueryset, self)._clone(**kwargs)

Exception Type: TypeError at /admin/news/news/
Exception Value: _clone() got an unexpected keyword argument 'shared_model'

and

Traceback:

File "D:\Projects\some-project\env3\lib\site-packages\django\core\handlers\exception.py" in inner
  34.             response = get_response(request)

File "D:\Projects\some-project\env3\lib\site-packages\django\core\handlers\base.py" in _get_response
  126.                 response = self.process_exception_by_middleware(e, request)

File "D:\Projects\some-project\env3\lib\site-packages\django\core\handlers\base.py" in _get_response
  124.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "D:\Projects\some-project\env3\lib\site-packages\django\contrib\admin\options.py" in wrapper
  604.                 return self.admin_site.admin_view(view)(*args, **kwargs)

File "D:\Projects\some-project\env3\lib\site-packages\django\utils\decorators.py" in _wrapped_view
  142.                     response = view_func(request, *args, **kwargs)

File "D:\Projects\some-project\env3\lib\site-packages\django\views\decorators\cache.py" in _wrapped_view_func
  44.         response = view_func(request, *args, **kwargs)

File "D:\Projects\some-project\env3\lib\site-packages\django\contrib\admin\sites.py" in inner
  223.             return view(request, *args, **kwargs)

File "D:\Projects\some-project\env3\lib\site-packages\django\contrib\admin\options.py" in add_view
  1637.         return self.changeform_view(request, None, form_url, extra_context)

File "D:\Projects\some-project\env3\lib\site-packages\django\utils\decorators.py" in _wrapper
  45.         return bound_method(*args, **kwargs)

File "D:\Projects\some-project\env3\lib\site-packages\django\utils\decorators.py" in _wrapped_view
  142.                     response = view_func(request, *args, **kwargs)

File "D:\Projects\some-project\env3\lib\site-packages\django\contrib\admin\options.py" in changeform_view
  1525.             return self._changeform_view(request, object_id, form_url, extra_context)

File "D:\Projects\some-project\env3\lib\site-packages\django\contrib\admin\options.py" in _changeform_view
  1554.         ModelForm = self.get_form(request, obj, change=not add)

File "D:\Projects\some-project\env3\lib\site-packages\hvad\admin.py" in get_form
  172.         return translatable_modelform_factory(language, self.model, **defaults)

File "D:\Projects\some-project\env3\lib\site-packages\hvad\forms.py" in translatable_modelform_factory
  229.     klass = modelform_factory(model, form, *args, **kwargs)

Exception Type: TypeError at /admin/news/news/add/
Exception Value: modelform_factory() got an unexpected keyword argument 'change'

mohammedhammoud commented 5 years ago

Any progress on this one?

kcleong commented 5 years ago

Removing the change value from the defaults dictionary in hvad.admin.TranslatableAdmin.get_form fixes the modelform_factory() got an unexpected keyword argumentchange'` exception. See for example this commit: https://github.com/PythonUnited/django-hvad/commit/9ffd352342e0a911f97bef8dae7f95faffd051f0

I'm not sure what the change value does, so use this 'fix' at your own risk. Using this fix I got hvad working with Django 2.1.9.

In Django 2.1 the modelform_factory[1] method does not include the change argument, however in Django 2.0[2] I also do not see the change argument.

Where can I find the latest repo/branch for Django 2? The dev/hvad2 has been last updated 2 years ago. I've been using my own fork with fixes for Django 2.

DmytroLitvinov commented 5 years ago

Hello @kcleong . This is the latest branch with compatibility for Django>2

Could you please share link to your own fork?

kcleong commented 5 years ago

@DmytroLitvinov, thanks for the pointer to the Django 2.x branch. This is the fork I am using PythonUnited/django-hvad

silau2005 commented 5 years ago

I like django-hvad a lot, it works amazingly with RESTful API but the recent Django 2.0 upgrading in our project really brings some operation issues. I would like to share some tips here, for people with simple Django use case, you might considering a short term alternative django-parler, it was inspiring by hvad thus compitable with hvad as well, especially on ORM/model part.

For others use a lot of RESTful API and want to stay with hvad's awesome serializer, try to avoid using .language and get_translation_aware_manager. then you are fine. example here

this is a temporary fix till the official hvad 2.0, cheers

maltebeckmann commented 4 years ago

@silau2005 Your article saved me. What a beautiful short-term solution. 多谢你！真没想到django-hvad到现在还没更新过。

mroobert commented 4 years ago

@silau2005 the link is broken... and I would need so much to see how I can avoid using .language call... Can you point me to another link or something.. :D thx!

silau2005 commented 4 years ago

@silau2005 the link is broken... and I would need so much to see how I can avoid using .language call... Can you point me to another link or something.. :D thx!

@mroobert I just fixed the link, you might check it again. Wish it helps. Let me know your result. cheers

mroobert commented 4 years ago

@silau2005 thx <3, now the link is working

One more question... when you use this type of code to avoid the .language call :

def retrieve(self, request, pk=None):
        lang = self.request.query_params.get('lang', DEFAULT_LANGUAGE)
        queryset = LocalizedString.objects.filter(namespace__title=pk)
        queryTranslations = LocalizedString.translations.field.model.objects.filter(language_code=lang, master__in=queryset)
        serializer = LocalizedStringSerializer(queryTranslations, many=True)
        strings = {d['key']: d['value'] for d in serializer.data}
        return Response(strings)

and after that you pass the queryTranslations to the serializer:

serializer = LocalizedStringSerializer(queryTranslations, many=True) strings = {d['key']: d['value'] for d in serializer.data} return Response(strings)

This is triggering an error in serializer.data ===> /hvad/utils.py", line 70, in load_translation trans_model = instance._meta.translations_model AttributeError: \'Options\' object has no attribute \'translations_model'

It is this the right way of using the serializer if you're avoiding .language call.. or ?

The serializer:

class LocalizedStringSerializer(TranslatableModelSerializer):
    key = serializers.SerializerMethodField()
    value = serializers.SerializerMethodField()

    def get_key(self, obj):
        return obj.title

    def get_value(self, obj):
        return obj.value

    class Meta:
        model = LocalizedString
        fields = ('key', 'value')

The model:

class LocalizedString(TranslatableModel):
    title = models.CharField(max_length=255, unique=True)
    translations = TranslatedFields(
        value=models.TextField('description'),
    )
    # placeholder = PlaceholderField('localizable_string_placeholder')
    namespace = models.ForeignKey(StringsNamespace, related_name='strings', null=True, on_delete=models.CASCADE)
    is_translatable = models.BooleanField(default=True, verbose_name=_("Translatable"))

    panels = [
        MultiFieldPanel(
            [
                FieldPanel('title'),
                FieldPanel('namespace', widget=forms.Select),
                FieldPanel('is_translatable', help_text=_(
                    "Makes this string translatable. Adds a non translatable comment to the exported string if "
                    "unchecked"
                )),
            ],
            heading=_('Localized String')
        )
    ]

    def get_translated(self, language_code):
        translated = type(self).objects.language(language_code).filter(pk=self.pk)
        if translated.exists():
            return translated.first()
        return super(LocalizedString, self).translate(language_code)

silau2005 commented 4 years ago

This is triggering an error in serializer.data ===> /hvad/utils.py", line 70, in load_translation trans_model = instance._meta.translations_model AttributeError: 'Options' object has no attribute 'translations_model'

I will talk a little bit about this exception django-hvad adds translations_model (pointing to Master model by foreign key master) behind the scenes for every TranslatableModel

LocalizedString.translations.field.model.objects.filter(language_code=lang, master__in=queryset) is actually equivalent to translations_model.objects.filter(language_code=lang, master__in=queryset)

and the serializer you are using is LocalizedStringSerializer, while the model/queryTranslations you put into the serializer is from translations_model

so they don't match each other

my suggestion is to modify

queryset = LocalizedString.objects.filter(namespace__title=pk)
queryTranslations = LocalizedString.translations.field.model.objects.filter(language_code=lang, master__in=queryset)
#serializer = LocalizedStringSerializer(queryTranslations, many=True)
#strings = {d['key']: d['value'] for d in serializer.data}
#v1 = queryset.objects.values()
#v2 = queryTranslations.objects.values()
#fields = queryset.model._meta.fields
#fieldsTranslations = queryTranslations..model._meta.fields

you could construct the dictionary from queryset and queryTranslations getting dictionary by queryset.objects.values() change the prefix if needed and queryset.model._meta.fields might help to shorten the result

checking would be needed if the result should be a single dictionary instead of multiple dict from the querysey

wish the information helps, I don't have time to do deep investigation, good luck!

KristianOellegaard / django-hvad

Django 2.0 support #344