search

yourlabs / django-autocomplete-light

A fresh approach to autocomplete implementations, specially for Django. Status: v4 alpha, v3 stable, v2 & v1 deprecated.
https://django-autocomplete-light.readthedocs.io
MIT License
1.8k stars 467 forks source link

3.5.1 new Media order is not safely compatible with ModelAdmin.autocomplete_fields #1143

Closed raratiru closed 4 years ago

raratiru commented 4 years ago

In the following setup, one inline uses autocomplete_fields the other inline uses DAL.

It works in 3.5.0. In order to work in 3.5.1. the Media meta-calss should also be defined in the inline that uses autocomplete_fields. 

class WithDALForm(forms.ModelForm):
    field_with_DAL = forms.ModelMultipleChoiceField(
        ...
        widget=autocomplete.ModelSelect2Multiple()
    )

    class Media:
        js = ("//code.jquery.com/jquery-3.4.1.min.js",)

class WithDALInline(admin.StackedInline):
    form = WithDALForm

class WithAutocompleteInline(admin.StackedInline)
    autocomplete_fields = ('field_1', 'field_2')

class Admin(admin.ModelAdmin):
    inlines = ('WithAutocompleteInline', 'WithDALIline')

What follows is the order of the scripts in each case:

3.5.1 with admin autocomplete (not working):

<script type="text/javascript" src="/static/admin/js/vendor/jquery/jquery.js"></script>
<script type="text/javascript" src="/static/autocomplete_light/jquery.init.js"></script>
<script type="text/javascript" src="//code.jquery.com/jquery-3.4.1.min.js"></script>
<script type="text/javascript" src="/static/admin/js/vendor/select2/select2.full.js"></script>
<script type="text/javascript" src="/static/vendor/select2/dist/js/select2.full.js"></script>
<script type="text/javascript" src="/static/admin/js/vendor/select2/i18n/en.js"></script>
<script type="text/javascript" src="/static/vendor/select2/dist/js/i18n/en.js"></script>
<script type="text/javascript" src="/static/admin/js/jquery.init.js"></script>
<script type="text/javascript" src="/static/autocomplete_light/autocomplete.init.js"></script>
<script type="text/javascript" src="/static/admin/js/core.js"></script>
<script type="text/javascript" src="/static/admin/js/inlines.js"></script>
<script type="text/javascript" src="/static/admin/js/autocomplete.js"></script>
<script type="text/javascript" src="/static/autocomplete_light/forward.js"></script>
<script type="text/javascript" src="/static/admin/js/admin/RelatedObjectLookups.js"></script>
<script type="text/javascript" src="/static/autocomplete_light/select2.js"></script>
<script type="text/javascript" src="/static/admin/js/collapse.js"></script>
<script type="text/javascript" src="/static/admin/js/actions.js"></script>
<script type="text/javascript" src="/static/autocomplete_light/jquery.post-setup.js"></script>
<script type="text/javascript" src="/static/admin/js/urlify.js"></script>
<script type="text/javascript" src="/static/admin/js/prepopulate.js"></script>
<script type="text/javascript" src="/static/admin/js/vendor/xregexp/xregexp.js"></script>

3.5.1 without admin autocomplete (working):

<script type="text/javascript" src="/static/admin/js/vendor/jquery/jquery.js"></script>
<script type="text/javascript" src="/static/autocomplete_light/jquery.init.js"></script>
<script type="text/javascript" src="//code.jquery.com/jquery-3.4.1.min.js"></script>
<script type="text/javascript" src="/static/admin/js/jquery.init.js"></script>
<script type="text/javascript" src="/static/vendor/select2/dist/js/select2.full.js"></script>
<script type="text/javascript" src="/static/admin/js/core.js"></script>
<script type="text/javascript" src="/static/admin/js/inlines.js"></script>
<script type="text/javascript" src="/static/vendor/select2/dist/js/i18n/en.js"></script>
<script type="text/javascript" src="/static/admin/js/admin/RelatedObjectLookups.js"></script>
<script type="text/javascript" src="/static/autocomplete_light/autocomplete.init.js"></script>
<script type="text/javascript" src="/static/admin/js/collapse.js"></script>
<script type="text/javascript" src="/static/admin/js/actions.js"></script>
<script type="text/javascript" src="/static/autocomplete_light/forward.js"></script>
<script type="text/javascript" src="/static/admin/js/urlify.js"></script>
<script type="text/javascript" src="/static/autocomplete_light/select2.js"></script>
<script type="text/javascript" src="/static/admin/js/prepopulate.js"></script>
<script type="text/javascript" src="/static/autocomplete_light/jquery.post-setup.js"></script>
<script type="text/javascript" src="/static/admin/js/vendor/xregexp/xregexp.js"></script>

3.5.0 with admin autocomplete (working):

<script type="text/javascript" src="/static/admin/js/vendor/jquery/jquery.js"></script>
<script type="text/javascript" src="//code.jquery.com/jquery-3.4.1.min.js"></script>
<script type="text/javascript" src="/static/admin/js/vendor/select2/select2.full.js"></script>
<script type="text/javascript" src="/static/autocomplete_light/jquery.init.js"></script>
<script type="text/javascript" src="/static/admin/js/vendor/select2/i18n/en.js"></script>
<script type="text/javascript" src="/static/vendor/select2/dist/js/select2.full.js"></script>
<script type="text/javascript" src="/static/admin/js/jquery.init.js"></script>
<script type="text/javascript" src="/static/vendor/select2/dist/js/i18n/en.js"></script>
<script type="text/javascript" src="/static/admin/js/core.js"></script>
<script type="text/javascript" src="/static/admin/js/inlines.js"></script>
<script type="text/javascript" src="/static/admin/js/autocomplete.js"></script>
<script type="text/javascript" src="/static/autocomplete_light/autocomplete.init.js"></script>
<script type="text/javascript" src="/static/admin/js/admin/RelatedObjectLookups.js"></script>
<script type="text/javascript" src="/static/autocomplete_light/forward.js"></script>
<script type="text/javascript" src="/static/admin/js/collapse.js"></script>
<script type="text/javascript" src="/static/admin/js/actions.js"></script>
<script type="text/javascript" src="/static/autocomplete_light/select2.js"></script>
<script type="text/javascript" src="/static/admin/js/urlify.js"></script>
<script type="text/javascript" src="/static/autocomplete_light/jquery.post-setup.js"></script>
<script type="text/javascript" src="/static/admin/js/prepopulate.js"></script>
<script type="text/javascript" src="/static/admin/js/vendor/xregexp/xregexp.js"></script>
jpic commented 4 years ago

Does it work with #1147 ?

jpic commented 4 years ago

Any idea where this is coming from ? 

<script type="text/javascript" src="//code.jquery.com/jquery-3.4.1.min.js"></script>
jpic commented 4 years ago

Also published in : django-autocomplete-light==3.6.0-dev1

Please try it out, it is expected to break fix in theory, but to fix things in practice, regarding jquery et al.

raratiru commented 4 years ago

I tried 3.6.0-dev1, the fix still works:

Insert jquery both:

The expected(?) scenario of inserting jquery only in the modelform that uses DAL is still not working.

(?) Since I may be missing this:

Is there a single source of truth regarding the introduction of jquery in the admin, for DAL to work in ModelAdmin and its inlines?

<script type="text/javascript" src="//code.jquery.com/jquery-3.4.1.min.js"></script> comes from class WithDALForm(forms.ModelForm).Media:

It is the class that defines a form field which uses DAL and its Media class brings jquery in the admin.

jpic commented 4 years ago

I thought that it would work as long as there's only one jquery loaded in the page, in which case you would not need WithDALForm anymore, does it work without WithDALForm ?

Outside the admin, I think that just loading jquery manually would work.

raratiru commented 4 years ago

WithDAL(ModelForm), provides field_with_DAL which is not a model field (otherwise, I could use ModelAdmin.autocomplete_fields).

https://github.com/yourlabs/django-autocomplete-light/blob/e639471eef0813239baba7ee2b812d9a3c63da3c/CHANGELOG#L7-L10

From DAL-3.5.0, I need to load jquery manually, and I did it in WithDAL(ModelForm).Media because it is also being used in its change form (not only as inline).

The above works in 3.5.0.

In 3.5.1, it stopped working when another inline of the same parent, uses autocomplete_fields.

It will work, however, if I also add jquery in Media metaclass of the inline that uses autocomplete_fields.

Outside the admin there is no reason not to work. It is inside that things get complicated with autocomplete_fields of another inline.

jpic commented 4 years ago

I'm deeply sorry but actually in 3.5.0, it was still loading jquery despite the announce, it was fixed by e46300d6085510ccd6cce3e0cd664fa7dcd5b974 which got in 3.5.1 release.

Ok so I think we have a documentation problem, but this is what I think should happen:

What we're trying to avoid: multiple jqueries loading in the same page, which creates inconsistencies such as : select2 loaded in one jquery, then another jquery is loaded and it doesn't have select2 anymore as such inlines won't work because they are created dynamically with the jquery that is loaded at this point in time.

Does that help ?

raratiru commented 4 years ago

Yes, picking up django.JQuery and avoiding multiple jqueries in the admin is great, thank you!

However, (it should have been a cache mishap?) currently with 3.6.0.dev1:

WithDALInline is dead. :-(

This is the relevant <script> part without external JQuery:

<script type="text/javascript" src="/admin/jsi18n/"></script>
<link href="/static/admin/css/vendor/select2/select2.css" type="text/css" media="screen" rel="stylesheet">
<link href="/static/vendor/select2/dist/css/select2.css" type="text/css" media="screen" rel="stylesheet">
<link href="/static/admin/css/autocomplete.css" type="text/css" media="screen" rel="stylesheet">
<link href="/static/autocomplete_light/select2.css" type="text/css" media="screen" rel="stylesheet">
<script type="text/javascript" src="/static/admin/js/vendor/jquery/jquery.js"></script>
<script type="text/javascript" src="/static/autocomplete_light/jquery.init.js"></script>
<script type="text/javascript" src="/static/admin/js/vendor/select2/select2.full.js"></script>
<script type="text/javascript" src="/static/vendor/select2/dist/js/select2.full.js"></script>
<script type="text/javascript" src="/static/admin/js/vendor/select2/i18n/en.js"></script>
<script type="text/javascript" src="/static/vendor/select2/dist/js/i18n/en.js"></script>
<script type="text/javascript" src="/static/admin/js/jquery.init.js"></script>
<script type="text/javascript" src="/static/autocomplete_light/autocomplete.init.js"></script>
<script type="text/javascript" src="/static/admin/js/core.js"></script>
<script type="text/javascript" src="/static/admin/js/inlines.js"></script>
<script type="text/javascript" src="/static/admin/js/autocomplete.js"></script>
<script type="text/javascript" src="/static/autocomplete_light/forward.js"></script>
<script type="text/javascript" src="/static/admin/js/admin/RelatedObjectLookups.js"></script>
<script type="text/javascript" src="/static/autocomplete_light/select2.js"></script>
<script type="text/javascript" src="/static/admin/js/actions.js"></script>
<script type="text/javascript" src="/static/admin/js/urlify.js"></script>
<script type="text/javascript" src="/static/admin/js/prepopulate.js"></script>
<script type="text/javascript" src="/static/admin/js/vendor/xregexp/xregexp.js"></script>
jpic commented 4 years ago

Right, I think the situation is still stuck here

rogeriomgatto commented 4 years ago

As a workaround, I got it working in the admin by adding the following Media class to my ModelAdmin class (django 2.2, dal 3.5.1). In my case, one admin worked (with a regular django autocomplete) and one didn't (without django autocomplete). I haven't had the time to find a minimal workaround.

    class Media:
        css = {
            'screen': [
                "vendor/select2/dist/css/select2.css",
                "admin/css/vendor/select2/select2.css",
                "admin/css/autocomplete.css",
                "autocomplete_light/select2.css",
            ]
        }
        js = [
            "admin/js/vendor/jquery/jquery.js",
            "autocomplete_light/jquery.init.js",
            "vendor/select2/dist/js/select2.full.js",
            "admin/js/vendor/select2/select2.full.js",
            "admin/js/jquery.init.js",
        ]
raratiru commented 4 years ago

... ne admin worked (with a regular django autocomplete) and one didn't (without django autocomplete)

This solution makes everything work in my case and it uses inhouse jquery!

However, it breaks everything in DAL-3.6.0-dev1.

mrts commented 4 years ago

As a workaround, I got it working in the admin by adding the following Media class to my ModelAdmin class (django 2.2, dal 3.5.1). In my case, one admin worked (with a regular django autocomplete) and one didn't (without django autocomplete). I haven't had the time to find a minimal workaround.

Thank you @rogeriomgatto ! This fixed jQuery loading in admin with Django 2.2.6 and DAL 3.5.1 for me as well.

jpic commented 4 years ago

We won't be affected by this with next AL release because we won't be using form media with the webcomponent.

In general, I'm leaving form.media, it's nice for little things for DAL it's been a disaster, using djwc for static instead, works much better and there's no conflict.

raratiru commented 4 years ago

@jpic OK, thank you!

It was for a while that I missed what happened in this issue due to work overload...

Cheers to the next release, 3.8.1 is on as I see, I can't wait to see the results when I get home.

I am closing this issue in favor of #1168 .

raratiru commented 3 years ago

@jpic As I am following your conversation with @orientalperil in #1168 , I entered the completely new world of front-end, SPAs, PWAs, web components (+ djwc ! ) and most of all, I re-tuned my thinking about javascript:

I realized that after 2015 this language has evolved into a really interesting OO programming language leaving behind the hacky bitter taste I thought it had. Wow!

I have just started learning Vue.js because it feels like it respects javascript and HTML as well (despite their attitude, svelte.js and svelteKit are also very promising), I got to meet snowpack and I feel excited.

Thank you for introducing me to this radically changing playground!

I decided to delve into Vue.js-3 using Vite which is created by Evan You along the lines of snowpack.

Anyway, these are just my first impressions I would like to share. Thank you again!

jpic commented 3 years ago

Nice video, thank you !

Have you used djsnowpack ? How did you implement?

raratiru commented 3 years ago

@jpic I am still learning javascript and understanding what are those tools trying to do. I have not yet reached the step of integrating them with Django.

However, I understand that decoupling the front-end from the back-end, is the future.

I read somewhere that javascript has matured enough to be able to benefit from the horse power of lower level languages. The idea of a static html file able to dynamically transform itself in an effective and efficient way is just awesome.

In that scope, djwc and djsnowpack pave the way for a better... offline tomorrow!

I am excited.

jpic commented 3 years ago

The reason I'm so much into Django is because it's the fastest way that I know from an idea to production, particularly when using an MVC pattern like Django admin which for me is absolutely necessary to get more value out of less code.

Decoupling between apps like Django does is great, but from what I've seen: separating frontend and backend into two distinct projects makes it twice as costly to make a change, at the cost of DRYness. Basically I want my backend to be able to generate a sane default frontend, and override the bits I need.

So with web components I feel like I can get the best of both worlds: generating HTML with custom tags on the backend and let the browser load the web components which basically replace jquery plugins but WAY better in many ways.

I believe the big question is: where do you want to have your router ? You need a backend router for sure, do you also want to maintain a frontend router ? It seems to me like twice the work that won't create any value for the users who actually just want something that works and don't really care.

Offline first is appealing, but this won't be magic and will have a cost in terms of development too, you need to deal with conflicts, you have to synchronize between the server DB and the DBs that are in all clients ... also server side rendering is nice for SEO and performance.

I'm getting very close to the "replace turbolinks+stimulusjs with sveltejs in crudlfa+" item in my todo list, I started this morning but then figured I had to clean the hackerspace ... I'm not planning on having a frontend router, but to bind clicks to a function that queries the server and updates the DOM, like turbolinks; same for the form: replace native POST with JS POST which is actually what crudlfa+ is already doing in its stimulusjs form_controller

Anyway, exciting stuff going on for sure !

Keep us up to date ;)

raratiru commented 3 years ago

@jpic Thank you for the insight, I really see the big picture now! Indeed having one source of truth like django provides a huge development boost. It became very clear to me how I can merge the benefits and why, wow! -D

Thank you also for the crucial point of keeping the dbs synced when it comes to the offline feature. Actually I am planning to build an app and the data entry will be performed on the street by a handful of people that could be -for example- located in a basement.

crudlfa+ -> I just met this guy, nice tool, let me explore it more!

Α few seconds ago I saw this very interesting video by Evan You, explaining clearly, important details that are not obvious in the relevant pages of Vue-3 documentation. I am not sure I understand everything but the layout presented there made me feel that things evolve in a really interesting and mind blowing way!

I will surely stay tuned, thank you again!

jpic commented 3 years ago

VueJS is fine but for making web components it will generate a bundle 10 times the size of a SvelteJS web component: https://webcomponents.dev/blog/all-the-ways-to-make-a-web-component/

Only Angular and React are even bigger, not that they are bad, they just pre-dates the web components standards which were heavily inspired from them.

Well, crudlfa+ like many other things are my personal toys, I'm using it in production on major sites but v0.9 is on the way and it already replaces MaterializeCSS (which I would NOT recommend) with Material Design Web Components, and also replaces some templates with Ryzom components.

Ryzom is a Python library to make components, I've started to replace templates with them in crudlfa+ and it's been nothing but pure joy, you can see this commit where we replace a complex Jinja2 template with Python components : https://yourlabs.io/oss/crudlfap/-/commit/7537d8d8f3f5e4e0004e076575d1056dc264cb04 Needless to say, the code is much more readable, and bits are much easier to reuse or override.

Ryzom also features data-binding and the meteor protocol, so if you have a django-channels server too, and you render say a list with Ryzom, then another user creates an item: it will automatically load the item in the list without reloading the browser. That's already working, but the developer @tmignot "I didn't know Django very well at the time, there's a layer of abstraction I wrote that I now know I can remove".

Another thing is that with Ryzom components, you're supposed to be able to code DOM events in Python, ie:

class MyComponent(ryzom.Component):
    tag = 'input'
    def onchange(self, dom):
        # do something ...

The conversion to efficient JavaScript is supposed to be done by Transcrypt which we've been sponsoring for more than a year now.

I'm hoping we can get to where https://facond.rtfd.io is, towards our goal of "generating a sane frontend by default from the backend code".

So yeah, pretty exciting times, and actually getting somewhere, all lights are green for crudlfa+ and ryzom to release v1 in 2021, wish us good luck ;)

raratiru commented 3 years ago

Good luck by all means!

The research I made those days enables me to understand that bringing web components into Python is a completely innovative change and I clearly see what makes Carlton Gibson (tough dude!) so excited about everything happening in #1168 ...

So yes, this is a solution!

Svelte has some syntactic idioms that deviate from javascript and I cannot easily spot them, yet. This is why I explored Vue.js hoping that the new tree-shaking compliance feature will help...

OK, it helps but there is a huge gap as I see in that article!

The web changes and Django community wins brave fights in the very front line. This is not only exciting but also soothing and relieving. Thank you for being here and sharing!

All these "details" were like a little "black spot" I avoided in my daily schedule. Thanks to COVID, I approached it and !OH! it is not a spot rather a whole new world... happening now!

jpic commented 3 years ago

All right, I'm trying to make the AutocompleteLight Web Component really ready, and have taken the following design decisions:

You can where it's at here:

https://yourlabs.io/oss/autocomplete-light/-/tree/master/new

However, given that I'm getting away with Python powered TDD, pure ECMAScript without dependencies, I'm thinking "I might as well write my Web Component in Python", yes, with Transcrypt ... Extremely lightweight pure Python frontend development might become a reality, I confirm that throwing away my JS test code in favor of pytest-splinter was an extremely pleasant experience

And trashing typescript was really nice too, going pure TDD so no time nor need for that

raratiru commented 3 years ago

@jpic Thank you * N! pytest-bdd with pytest-splinter will form an amazing tool to drive TDD.

I am just recompiling my head based on the configuration provided by this thorough presentation of Python 2009 yield.

I lived a "close to killer joke" extra-terrestrial experience regarding the power of Python when combined with a free as in Freedom mental setup.

Many things have changed since then, but yes, I will definitely be using Python as the basis of my projects and I would locally plug the most suited tools if a task requires so, I primarily regard this as an exercise to strengthen my imagination.

Transcrypt seems to be rocking, and your intuitive tests are awesome.