Autocomplete does not work with Django 3.2

[FrozenAlex]

VERSIONS: Grappelli 2.15.1, Django 3.2 STATICFILES: WARNINGS, Details: I see a lot of errors with text like this

Found another file with the destination path 'admin/js/autocomplete.js'. It will be ignored since only the first encountered file is collected. If this is not what you want, make sure every static file has a unique path.

JAVASCRIPTS: ERROR Failed to load resource: the server responded with a status of 403 (Forbidden) CUSTOMIZATIONS: NONE

I upgraded django from 2.2 to 3.2 and it broke the autocomplete on related fields it just always says 403 and gives the following error:

Forbidden (Permission denied): /admin/autocomplete/
Traceback (most recent call last):
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/utils/datastructures.py", line 76, in __getitem__
    list_ = super().__getitem__(key)
KeyError: 'app_label'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/contrib/admin/views/autocomplete.py", line 61, in process_request
    app_label = request.GET['app_label']
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/utils/datastructures.py", line 78, in __getitem__
    raise MultiValueDictKeyError(key)
django.utils.datastructures.MultiValueDictKeyError: 'app_label'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/contrib/admin/sites.py", line 250, in wrapper
    return self.admin_view(view, cacheable)(*args, **kwargs)
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/views/decorators/cache.py", line 44, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/contrib/admin/sites.py", line 232, in inner
    return view(request, *args, **kwargs)
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/contrib/admin/sites.py", line 417, in autocomplete_view
    return AutocompleteJsonView.as_view(admin_site=self)(request)
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/views/generic/base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/views/generic/base.py", line 98, in dispatch
    return handler(request, *args, **kwargs)
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/contrib/admin/views/autocomplete.py", line 20, in get
    self.term, self.model_admin, self.source_field, to_field_name = self.process_request(request)
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/contrib/admin/views/autocomplete.py", line 65, in process_request
    raise PermissionDenied from e
django.core.exceptions.PermissionDenied
[02/May/2021 20:11:33] "GET /admin/autocomplete/ HTTP/1.1" 403 135

This ticket seems to be related to the issue https://code.djangoproject.com/ticket/32659?cversion=2&cnum_hist=2

Downgrade to 3.1 fixes the issue.

[sehmaschine]

not sure I can help here. the warning looks "ok" (since we overwrite django files), you can ignore that. if the JS is not being loaded properly, there will be errors of course. I'm not sure why you get a 403, but it's definitely not related to grappelli.

[int-ua]

Django 3.2, Grappelli 2.15.1

It looks like a problem is the fact that there are two jQuery versions being loaded, 3.3.1 from Grappelli and 3.5.1 from Django 3.2.

https://code.djangoproject.com/ticket/32659

It's definitely a Grappelli problem at least partially unless Grappelli does not intend to support ModelAdmin.autocomplete_fields. And while on usual fields it's possible to replace with raw_id_fields, the https://github.com/farhan0581/django-admin-autocomplete-filter may never work without it. The simple solution may be updating jQuery in Grappelli, not sure if that will work.

Again, it does work if I remove grappelli from installed apps and do collectstatic and it stops working immediately after I add it back and perform collectstatic again.

[int-ua]

You get 403 because with Grappelli there are no app_label, model_name and field_name in GET parameters, only term.

[int-ua]

And they are not there because /static/admin/js/autocomplete.js is outdated. Please sync it with upstream.

Workaround for anyone else experiencing this: create an empty Django app with the upstream version of autocomplete.js in static/admin/js and put it before grappelli in the INSTALLED_APPS. Edit: I've also included override for emptying jquery-migrate and update for grappelli/jquery to 3.5.1. Let's see what will break.

Please comment if you discover more outdated JS files that have to be synced with upstream. I guess it's likely there are more of them.

[sehmaschine]

just to clarify. which "autocompletes" do you refer to? the django autocompletes or the grappelli autocompletes?

[int-ua]

Django autocompletes, the ones used in https://github.com/farhan0581/django-admin-autocomplete-filter Sorry, that was ambiguous, yes.

[sehmaschine]

thanks, I see. I'll check that.

[int-ua]

Thank you! Just noticed that admin/filter_listing.html doesn't support these Django autocompletes, shows only All option without the select element. Should I create a separate issue for that?

[sehmaschine]

not sure I can follow. filter with autocompletes? does django support that? if so, I haven´t been aware. but yes, please open a separate ticket ...

[int-ua]

Not by default, only with the aforementioned django-admin-autocomplete-filter. So it's more like support for that module rather than a bug. I'll try to do it with a custom change_list_filter_template first.

[Kiougar]

Any updates on this?

[sehmaschine]

We'll look into this as soon as we have some spare time. If you're in a hurry, a PR might help.

[Kiougar]

Any direction for creating a PR? Anything I should know?

[wangzhaohe]

Just use Django3.1

Django 3.2 still does not work.

[ahmedsafadii]

no fix 3.2.4

[ajakubo1]

Just to clarify as I encountered exactly the same problem (maybe that will help pinpointing a solution):

$ pip freeze | grep Django
Django==3.2.4
$ pip freeze | grep -i grappelli
django-grappelli==2.15.1
$ find ./lib/ | grep autocomplete.js
./lib/python3.8/site-packages/django/contrib/admin/static/admin/js/autocomplete.js
./lib/python3.8/site-packages/grappelli/static/admin/js/autocomplete.js

This is autocomplete.js code from grappelli:

(function($) {
    'use strict';
    var init = function($element, options) {
        var settings = $.extend({
            ajax: {
                data: function(params) {
                    return {
                        term: params.term,
                        page: params.page
                    };
                }
            }
        }, options);
        $element.select2(settings);
    };

    $.fn.djangoAdminSelect2 = function(options) {
        var settings = $.extend({}, options);
        $.each(this, function(i, element) {
            var $element = $(element);
            init($element, settings);
        });
        return this;
    };

    $(function() {
        // Initialize all autocomplete widgets except the one in the template
        // form used when a new formset is added.
        $('.admin-autocomplete').not('[name*=__prefix__]').djangoAdminSelect2();
    });

    $(document).on('formset:added', (function() {
        return function(event, $newFormset) {
            return $newFormset.find('.admin-autocomplete').djangoAdminSelect2();
        };
    })(this));
}(django.jQuery));

this is the code from django3:

'use strict';
{
    const $ = django.jQuery;
    const init = function($element, options) {
        const settings = $.extend({
            ajax: {
                data: function(params) {
                    return {
                        term: params.term,
                        page: params.page,
                        app_label: $element.data('app-label'),
                        model_name: $element.data('model-name'),
                        field_name: $element.data('field-name')
                    };
                }
            }
        }, options);
        $element.select2(settings);
    };

    $.fn.djangoAdminSelect2 = function(options) {
        const settings = $.extend({}, options);
        $.each(this, function(i, element) {
            const $element = $(element);
            init($element, settings);
        });
        return this;
    };

    $(function() {
        // Initialize all autocomplete widgets except the one in the template
        // form used when a new formset is added.
        $('.admin-autocomplete').not('[name*=__prefix__]').djangoAdminSelect2();
    });

    $(document).on('formset:added', (function() {
        return function(event, $newFormset) {
            return $newFormset.find('.admin-autocomplete').djangoAdminSelect2();
        };
    })(this));
}

I guess an easy fix would be to add (1):

                        app_label: $element.data('app-label'),
                        model_name: $element.data('model-name'),
                        field_name: $element.data('field-name')

to Grappelli lib, or (2) copy the code from django3. Or (3)... I think removing autocomplete.js from Grappelli will also work (tested it locally and the original Django file was loaded instead of Grappelli version when I did python manage.py collectstatic -c). I guess I can open a PR with one of the "fixes", just the question is - which one (1, 2 or 3)?

[maximdeclercq]

I can confirm that removing the file fixes it, however, the visuals are still bugged.

[sehmaschine]

@maximdeclercq what you're showing (visually) looks like a Grappelli autocomplete, not like a Django autocomplete field. Not sure if that's the result of removing the Django file, but it certainly does not look right. There are 2 different autocompletes: the Django autocomplete (which relies on the mentioned file) and the Grappelli autocomplete – and we do not style the first one.

[maximdeclercq]

This is the Django autocomplete. It is defined like this.

class MyAdmin(admin.ModelAdmin):
    ...
    autocomplete_fields = ("user",)

Removing the file shouldn't change anything I think, because the only real difference were the fields that were missing from the ajax request.

[sehmaschine]

I see. Then it´s interesting that it doesn't look like the Django autocomplete. There still has to be another issue with this.

[ajakubo1]

Maybe there's also a .css file for autocomplete that's being overridden?

[maximdeclercq]

The problem seems to be that these CSS variables are not defined. I'm looking into it.

[maximdeclercq]

Adding admin/css/base.css did not seem like the right choice since it conflicted a lot. I tweaked a bit with the grappelli CSS and managed to get this layout. This looks fine to me. If you are happy with it you can merge my PR.

[sehmaschine]

Thanks! I’ll take a look on Monday.

On Sat 10. Jul 2021 at 18:47, Maxim De Clercq @.***> wrote:

Adding admin/css/base.css did not seem like the right choice since it conflicted a lot. I tweaked a bit with the grappelli CSS and managed to get this layout. [image: preview] https://camo.githubusercontent.com/20198b1b4da22877cac644959c8625d271cf6edd904c5986e8de8ee39003a299/68747470733a2f2f692e696d6775722e636f6d2f63597455695a4b2e706e67 This looks fine to me. If you are happy with it you can merge my PR.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/sehmaschine/django-grappelli/issues/963#issuecomment-877667291, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABNIWH2SEJPJXZH7URMWX3TXB2TRANCNFSM437S7E4Q .

[sehmaschine]

@maximdeclercq Looks ok with ForeignKey, but not with M2M (that said, M2M looks kind of broken with the vanilla admin as well – but maybe that's just my view). I would say the PR is fine since it seems to work now. Styling these fields is almost impossible IMO.

[selected-pixel-jameson]

Django 3.2 is still not working. I had to revert to 3.1. I'm not using grapelli at all. Getting the exacts errors seen here.

[sehmaschine]

@selected-pixel-jameson what do you mean with "I'm not using grappelli at all"? because if that's the case, you're most probably wrong with this issue tracker. besides, the problem should be solved with current master (not the pypi version though).

[selected-pixel-jameson]

@sehmaschine I don't have Grappelli installed. I'm not using it. How do I grab the master using pip instead of the published version?

[sehmaschine]

@selected-pixel-jameson If you're not using grappelli ... why do you want to install it? And why do you comment on an issue which you obviously don't have (since you're not using grappelli)? sorry, but I don't get it.

[selected-pixel-jameson]

@sehmaschine I'm sorry. For some reason I thought this was the DAL repo. I apologize for the confusion.

[waketzheng]

This worked for me:

python manage.py collectstatic

Then activate the Disable cache in webbrowser develpor tool and refresh the page.

[sdsy888]

Disable cache

tried this, but no luck for me.

waiting for the devs to check where the real problem is.

[sehmaschine]

Can someone please explain what you mean with "not working"? I tried this several times now and I can´t see an issue (besides the look&feel which we currently not plan to fix with the Django autocomplete).

Here are 3 Screenshots with my setup ...

The first one shows the Django version (without Grappelli):

Now with Grappelli installed, but still using the Django autocompletes:

And now the Grappelli version with the Grappelli autocompletes:

[sehmaschine]

See also #980

[sehmaschine]

stable/2.15.x should fix all issues with django autocompletes. I've removed all Django javascripts from grappelli – besides the ones we need to overwrite (actions.js, datetimeshortcuts.js and relatedobjectlookup.js). please give that branch a try and let me know if there are any issues left (otherwise we'll do a new release).

[sehmaschine]

solved with the latest release (django autocompletes are also properly styled now).