EmilStenstrom / django-components

Create simple reusable template components in Django.
MIT License
973 stars 62 forks source link

feat: Support for HTML attributes and `html_attrs` tag #491

Closed JuroOravec closed 3 weeks ago

JuroOravec commented 3 weeks ago

TODO:

EmilStenstrom commented 3 weeks ago

Happy to see you are working on this! 👏👏👏

I have a couple of big questions about this approach:

JuroOravec commented 3 weeks ago

@EmilStenstrom thanks for feedback! Yes, it's exactly these big questions that I wanted to discuss here.

For me, the steps I'm making/suggesting are with the end goal of having a similar developer experience (DevExp; ease of coding) for writing frontend code in Python/Django as I had with Vue/React. django-components is the closest thing I found in that regard. But it means that my goal may not be entirely aligned with your goal of pushing django-components to django.

IMO neither of the two is better, it's just different opinions, with different modes of operations, but IMO both roads could lead to stable maintenance of the django-components codebase.

Pursuing DevExp vs integration into Django codebase

My feeling from Django so far is that while it's great at being a "batteries-included" starting point, I feel like almost every aspect of it could be replaced with another library that "fixes" what Django lacks:

So from the above, it feels like Django is great at being the backbone that ties everything together, but spreads itself too thin, and then provides only basic support for each of the subcategories of web app/server development. Overall I don't feel like the Django landscape is going in the direction of merging successful projects into core Django. Instead, when I look at e.g. django-ninja, I get the feeling that while they maintain compatibility with Django, they are focusing on DevExp by porting popular features from FastAPI, rather than trying to maintain the codebase compatible with Django codebase (e.g. djang-ninja uses pydantic for validation).

So IMO if I could have a say, I'd rather for django-components to go in the similar direction as django-ninja. Still work within the Django framework, but provide the best possible experience for users.

I believe your point on pushing django-components upstream is coming from the point of concern for maintainability, and that's a very valid point (correct me if wrong). But in that case you have to educate me on this - have there been similar projects which have been integrated into code Django codebase like this? And has there already been any discussions on merging django-components into django? I haven't researched this so I'm unable to consider this side of the argument.

My concern is that I'm not entirely sure why Django would want to merge django-components, when they already have include, block and extends tags. Sure, django-components expands on this and does things a bit better. But by the same logic then I would expect also an effort from Django maintainers to integrate popular features from django-ninja. (Again, I don't know if there are or are not such efforts, as I simply haven't come across them).

Moreover, because of my feeling that Django spreads itself too thin, and because Django has much larger maintenance burden because of the framework's age and popularity, I feel like merging django-components into django effectively means an end to new features (maybe a few once a year). Which in itself is not necessarily a bad thing, but because I feel like there's still so much to do on django-components to improve dev experience, merging to django would hinder that.

Then, what could be a sustainable alternative to offloading django-components to django? Could be either minimizing support/development hours for django-components (declaring the project as feature-complete), or educating more people on the project so it can be community-driven, or getting sponsorship/donations to make maintenance sustainable. While first option is easily achievable, it would also mean an end to better user experience. Ofc, the latter two are easier said than done. However, IMO this project still has a plenty of room to grow and community members to attract, which could make the latter two options feasible. Personally, in this space I'm still missing a UI component library like what Vuetify does for Vue.

Also, important to point out, is that things don't have to be either-or. If it came down to actual merge of django-components into django, I would expect that there would still be plenty of API changes or deletions, as probably not all features would be accepted. So what would live in Django would not be 100% copy of this project. What means that this project can afford some deviances (altho, ofc still minimizing them if possible). Or, you can always take an older version with less features but better compatibility.


To understand where I'm coming from - I have deep experience with React/Vue/Vuetify, so I have a good sense for what I'm missing. I also for long wanted to work on open source project, and (for personal reasons) it's only now that I'm ticking things off of my bucket list. And, by chance, it seems landed on this project. I'm also currently at stage of life where I have very few family or other duties (still need to work part time tho). Hence the drive and the belief that this project could still be improved.

So with this in mind, yesterday I estimated that to get in many of the features that I miss (ofc, only if accepted), I could work on django-components for next ~ 3-4 months (assuming 1 feature per week). But as I said above, that would also mean going in the direction of e.g. django-ninja, prioritizing user experience over compatibility with Django codebase.

So I'd be interested to hear what you think about that.

JuroOravec commented 3 weeks ago

@EmilStenstrom on the topic of {% component attrs:x-data="..." %} vs {% component my_array:x-data="..." %}

Have a look at https://vuejs.org/guide/components/attrs#attribute-inheritance, this was where I was coming from. There, you can see that attributes are collected into $attrs. Component user can specify any extra attrs like x-data=... or @click="...", it's all collected into $attrs, and then it's the component author that decides where the attributes are rendered. So it's as if the component author chooses the prefix name.

Also to me it feels like it makes sense to allow only the special attrs: prefix, because it has special meaning ("These inputs will be rendered as HTML attributes (most likely) of the top-level HTML tag"). Because if I have something else like my_array:abc=def, then it doesn't have the meaning of HTML attributes anymore, and it's effectively only collecting inputs. So it would be as if instead of:

def get_context_data(self, abc):
    my_array = {"abc": abc}
    return {"my_array": my_array}

we do it for them:

def get_context_data(self, my_array):
    return {"my_array": my_array}

To me, this mental models feels like a leaky abstraction. If my_array:abc=def doesn't have a special meaning, then why use my_array:abc if I could instead use something else like my_array__abc=def or simply abc=def. Why would the component user need/want to know what variable is used inside of the component to group the inputs?

Also, if we think of my_array:some=value as simply grouping inputs, then it also raises questions like:

EmilStenstrom commented 3 weeks ago

@JuroOravec I agree with you that a great developer experience, definitely can go hand in hand with great maintainability. It requires that we are careful in taking on technical debt though.

About merging django-components into Django

You are right that it's not at all clear that this is possible. There are a couple of instances where this has happened though: South, that became django's migration system, Channels, which is now maintained in parallel with Django, and Crispy forms which was the blueprint for improvements in Django's form handling.

The core of why I think being part of Django is interesting is that Django needs this. It needs a better story of how to build complex interfaces with it. Currently people are not able to use big parts of the library and instead fall back to just using DRF or django-ninja. Django can be much more than an API builder, and I think this project might be the solution to that. I actually held a presentation about this almost 10 years ago: https://www.youtube.com/watch?v=Niq-HoraNPo (slightly dated, but the same core idea).

Does this mean that we have to be part of Django? Not necessarily, but I think the prospect of helping 100x more people "see the light" is a worthy goal. We could put django-components in a state where it could be included, and see if we could get it merged. If not, then fine, no harm done, and we could continue like it has for years.

What would it take to be included?

That what we build would match the other things that are already in Django. That we added as few new concepts as possible. That the code was maintainable and easy to understand. That we had a clear story around how you should use components, and were the scope of the project ended, and customization left off.

I think the above are all worthy goals, anyways, that would also make the project easier to explain and maintain over time.

What would happen after an eventual merge?

This library could be a "components-extended" package, including all the parts that didn't get merged into django. It would take the core parts of the library and move that into django core. After a couple of years we could remove those parts from our code. Would things significantly change for this project? I think not. Would django benefit? I think it definitely would!

I'm not at all saying we should stop developing the library, put it into maintenance mode, or anything like that.

EmilStenstrom commented 3 weeks ago

@JuroOravec I'm extremely happy that you are able to put some time into working on this project, and I think donating time like this should mean that you have an even greater say in where the project is going. And I share your vision of building a great developer experience for users. So I'll just share my image of what a great experience means, to see if we share a similar view.

How to build a great developer experience

I think the most important dev experience is how quickly you can onboard new users. I have done a couple of stints as a programming teacher, teaching beginners how Django works from scratch. I saw first hand how all the new concepts, sometimes introduced to make dev experience easier for advanced users, confused new users just because they had to learn something new. This is actually the reason why I proposed Django drop regexps from urls and switch to something easier, leading to the current path() function that is now the default: https://groups.google.com/g/django-developers/c/u6sQax3sjO4/m/l3nvZCuuAAAJ

Sometimes simplifying for new users colldes with adding new stuff for experienced users. I think this is very tricky, as maintainers of a library always are the most advanced users. That's why I really appreciate all the great stuff that you've done to simplify things, merging settings and possibly removing the middleware. Of course we should also build for more advanced users, but I feel very strongly about "first user experience", and will keep advocating for them! :)

My view of React

I have a little experience with React, but not enough that I have built anything substantial. I find it super confusing, with lots of magic going on, and way too many concepts crammed into a library that is supposed to be usable to beginners, while in reality being built for Facebook's needs. You even have to learn a new language, JSX, that has it's own kinks, to be productive. There's build steps and hydration, and fibers, and hooks, and, and and... All this in addition to understanding Django.

This just can't be the best way to build a website in 2024!

That's why I really appreciate HTMX and Alpine.js. They are trying to simplify things for everyone. And building on a "heavy" backend framework like Django, it makes sense the the frontend is light. I have talked about this before, but in the ideal world, I think being able to build interactive sites fully in python, not caring about the js layer at all, could be a really good goal to reach. I've talked before about Phoenix Liveview, which I think is a really elegant way to build heavy backend fullstack apps.

EmilStenstrom commented 3 weeks ago

@EmilStenstrom on the topic of {% component attrs:x-data="..." %} vs {% component my_array:x-data="..." %}

Have a look at https://vuejs.org/guide/components/attrs#attribute-inheritance, this was where I was coming from. There, you can see that attributes are collected into $attrs. Component user can specify any extra attrs like x-data=... or @click="...", it's all collected into $attrs, and then it's the component author that decides where the attributes are rendered. So it's as if the component author chooses the prefix name.

I definitely see your use-case here. As with all these features, I think it makes sense to back up one step from their implementation to the problem they are solving, and try to figure out what makes the most sense for this project. I don't think it's important to pick a solution that is similar to another project.

Also to me it feels like it makes sense to allow only the special attrs: prefix, because it has special meaning ("These inputs will be rendered as HTML attributes (most likely) of the top-level HTML tag").

I thought of the interface differently: We're building a way to send an array of parameters to the component, instead of just key=value pairs that are supported today. I know I talked about a HtmlDict, which I agree doesn't work with the mental model I talk about above. Maybe the merge_attrs component tag could handle all the html specific stuff?

This would mean that {% component variable:key1=value1 variable:key2=value2 %} would be a way to send the dict variable={key1: value1, key2: value2} to the get_context_data method, so that it could do whatever it wanted with it. Maybe it passes it to the template, maybe it does something else, doesn't matter. The elegant part to me, is that the name you pick, in this case "variable", will be present as a get_context_data parameter, so you can easily track where stuff is ending up. A concept like Vue's "fallthrough" saves some characters, but adds another concept to the mix that you have to learn. I think it goes against Python's "explicit over implicit" ethos...

I'm open to changing my mind here, just trying to explain what I'm thinking.

If my_array:abc=def doesn't have a special meaning, then why use my_array:abc if I could instead use something else like my_array__abc=def or simply abc=def. Why would the component user need/want to know what variable is used inside of the component to group the inputs?

my_array__abc=def would be an OK api for me too. The point of my_array is that you'll find the same name in the get_context_data method, so you can follow the flow of data easily.

Also, if we think of my_array:some=value as simply grouping inputs, then it also raises questions like:

* Can a component user apply the prefix multiple times to create a nested input? E.g. `my_data:my_array:abc=def`

I have a very hard time to see who would find that useful, so I think we can just specify that this is something we support for just one level of arrays.

JuroOravec commented 3 weeks ago

@EmilStenstrom Btw, just want to say, whether these concept may get into Django codebase or not, I think it's still extremely useful to have that sort of "selective pressure" from your side (or as you said, advocating for "first user experience"), really appreciate it, and hope my message didn't come off as too strong.

And thanks for the examples of projects that became part of Django.

Agree with you on wanting people to see the light, altho I'm maybe on the advanced end of the spectrum - but I'm still annoying that with Vue+Vuetify I can build a whole webpage in a day, whereas with Django (+components), I spend a day just creating an autocomplete field. And then I see someone on the Django forums who shrugs at the idea of using JS because "it's a mess", and I'm just perplexed at their ignorance. (ofc, there may be a bias in Django community to be more "anti-other-programming-languages", because those who can/want use other languages for web apps may find better tooling elsewhere and not come back). Anyway...

Agree that having a clear story is definitely useful to have either way!

And also like the idea that this could potentially shift into "components-extended". Overall, it is encouraging, hearing from you that you too believe that two ideas (keep improving the project vs being open to pushing some concepts to upstream) don't have to be exclusive!

Personally, achieving "everything in Python" is not a goal for me, because I have experience with the whole stack, so I don't feel the need. But still happy to help if that becomes one of the goals. I just won't be the one to spearhead that, as I don't have the end goal vision like I have for the templating part of django-components.

On React / first user experience

I keep mentioning Vue AND React so there's higher chance people can relate with their experience. But truth be told, my inspiration is 99% Vue. To Vue, React feels low-level / unpolished, requiring a lot of micromanagement, and being more chaotic because it has less prescribed structure*. In other words, I think Vue has better developer experience.

* But also here there's more nuance - here I'm talking about variables / component state. When it comes purely to templating, they have different approaches, but not necessarily better/worse.

Sidenote: That's one thing that I find ironic, that in JS ecosystem, there is (or at least was) a sentiment that "React has less magic than Vue and is therefore more straightforward to understand". But then Vue has more guardrails, and it requires more mental power to maintain React apps than it does for Vue. So as always, there's no silver bullets, no absolutes, and there's always trade-offs.

Also for django-components, Vue is much more relevant, simply because the template is separated from the code. In React, you have JSX where you directly inline components as variables. In Vue, you render a component by "referencing" the component via it's registered name (Which is the same how django-components does it).

So basically for me, I want to bring the ease of use of Vue+Vuetify to Django via components + Alpine/HTMX.

JuroOravec commented 3 weeks ago

@EmilStenstrom

This would mean that {% component variable:key1=value1 variable:key2=value2 %} would be a way to send the dict variable={key1: value1, key2: value2} to the get_context_data method

Ok, we can also go in this direction. So that means that attrs: would NOT be a special prefix. In that case, I'm thinking of renaming the merge_attrs tag to html_attrs, so it's more declarative, as it's about taking any dict, and formatting it as a string of HTML attributes.

So in terms of documenting it, I'm thinking:

EmilStenstrom commented 3 weeks ago

@EmilStenstrom Btw, just want to say, whether these concept may get into Django codebase or not, I think it's still extremely useful to have that sort of "selective pressure" from your side (or as you said, advocating for "first user experience"), really appreciate it, and hope my message didn't come off as too strong.

Absolutely not! In open source the rarest resource is time, so all arguments should be weighted by the time you are willing to spend. See everything I say as opinions, and I hope I didn't come off as too strong either.

And thanks for the examples of projects that became part of Django.

Agree with you on wanting people to see the light, altho I'm maybe on the advanced end of the spectrum - but I'm still annoying that with Vue+Vuetify I can build a whole webpage in a day, whereas with Django (+components), I spend a day just creating an autocomplete field.

This is actually a really good use-case. Maybe a good subgoal could be to make creating an autocomplete field really straightforward? Something similar to https://dhc.iwanalabs.com/cascading_selects/ perhaps?

And then I see someone on the Django forums who shrugs at the idea of using JS because "it's a mess", and I'm just perplexed at their ignorance. (ofc, there may be a bias in Django community to be more "anti-other-programming-languages", because those who can/want use other languages for web apps may find better tooling elsewhere and not come back). Anyway...

I think it's partly because of ignorance, as you say, but also partly because of the mental load of being in two development stacks at once. Knowing all the ins and outs, and that's not even including all the additional libraries you need to learn, and keep in your head. For advanced users like us, it's easy to forget the huge set of knowledge it takes to understand say Django + requirements.txt + Node + React + package.json. SOOO many concepts!

Agree that having a clear story is definitely useful to have either way!

And also like the idea that this could potentially shift into "components-extended". Overall, it is encouraging, hearing from you that you too believe that two ideas (keep improving the project vs being open to pushing some concepts to upstream) don't have to be exclusive!

Personally, achieving "everything in Python" is not a goal for me, because I have experience with the whole stack, so I don't feel the need. But still happy to help if that becomes one of the goals. I just won't be the one to spearhead that, as I don't have the end goal vision like I have for the templating part of django-components.

Fully understand. Back to the open source ethos: it's "free" work, so everyone needs to find motivation for their work elsewhere, not doing what other people tell you to. That's work! :)

On React / first user experience

I keep mentioning Vue AND React so there's higher chance people can relate with their experience. But truth be told, my inspiration is 99% Vue. To Vue, React feels low-level / unpolished, requiring a lot of micromanagement, and being more chaotic because it has less prescribed structure*. In other words, I think Vue has better developer experience.

I should look into Vue again. Read up on it many years ago, but didn't get to building anything.

So basically for me, I want to bring the ease of use of Vue+Vuetify to Django via components + Alpine/HTMX.

That's a really good goal I think. We want to make Alpine/HTML + Django a much better developer experience than Django + DRF/django-ninja + Vue/React (the most common development stack right now).

EmilStenstrom commented 3 weeks ago

@EmilStenstrom

This would mean that {% component variable:key1=value1 variable:key2=value2 %} would be a way to send the dict variable={key1: value1, key2: value2} to the get_context_data method

Ok, we can also go in this direction. So that means that attrs: would NOT be a special prefix. In that case, I'm thinking of renaming the merge_attrs tag to html_attrs, so it's more declarative, as it's about taking any dict, and formatting it as a string of HTML attributes.

Makes sense to have html_attrs be the name instead.

So in terms of documenting it, I'm thinking:

* We should show the example with `{% component variable:key1=value1 variable:key2=value2 %}`

Not sure. The most common use-case will be setting HTML attributes, so maybe it still makes sense to have those as examples? I think the alpine params that you used shows the reason behind the feature well. Do we need an example when two different attributes are used at the same time? {% component attrs:class="card" attrs:id="avatar" wrapper_attrs:class="wrapper_card" wrapper_attrs:id="wrapper_avatar" %}?

* For `html_attrs`, I suggest we show an example where we call the variable `attrs:`, e.g. `{% component attrs:key1=value1 attrs:key2=value2 %}`, so it's at least the "conventional" variable name for HTML attributes.

Yeah, or we could do a tailwind example: attrs:class="m-0" attrs:class="flex-wrap-reverse"? You decide.

* We should explain the caveat that `variable:` cannot be nested.

Sure!

JuroOravec commented 3 weeks ago

@EmilStenstrom I've refactored this so that the construct prefix:key=val can be used for any props. I also changed it from single colon : to two colons ::, so it becomes prefix::key=val

The reason for using :: is that I realized that I don't know if there are any frameworks or libraries that make use of colon in HTML attribute name. But if there is, then we wouldn't be able to distinguish it from our prefix. So it would be safer to use ::. Coincidentally, I feel like this makes it also easier to read, and it's similar to scoped variables in some programming languages, so hopefully it'd be more intuitive at a glance.

I kept the ability of the html_attrs tag to "append" values, so we can join class attribute that is coming from component user with class defined by component author. But instead of key+=val, I reused the prefix::key=val construct. So user's can set add::class="extra-class" to indicate that the value "extra-class" should be appended to whatever is set to attribute class.

I've also updated README, trying to document this from user's perspective, introducing one concept at a time and finishing it with a full example.

And since we don't treat class as a special attribute (like the code from django-web-components originally did), I was able to shave off about half of the code + tests copied from django-web-components.

JuroOravec commented 3 weeks ago

@EmilStenstrom

This is actually a really good use-case. Maybe a good subgoal could be to make creating an autocomplete field really straightforward? Something similar to dhc.iwanalabs.com/cascading_selects perhaps?

Yeah, that's a good goal. Creating pre-made components will be a whole new can of worms, but it would be IMO very valuable. However, I already feel like there might be a desire for 2 different libraries, one for AlpineJS and one for HTMX (unless we figure out how to merge them). And there's also the question of supporting Jinja2 vs Django templates. So definitely a lot of things to consider before we get to making components.

For advanced users like us, it's easy to forget the huge set of knowledge it takes to understand say Django + requirements.txt + Node + React + package.json. SOOO many concepts!

Very true!

EmilStenstrom commented 3 weeks ago

@EmilStenstrom I've refactored this so that the construct prefix:key=val can be used for any props. I also changed it from single colon : to two colons ::, so it becomes prefix::key=val

The reason for using :: is that I realized that I don't know if there are any frameworks or libraries that make use of colon in HTML attribute name. But if there is, then we wouldn't be able to distinguish it from our prefix. So it would be safer to use ::.

: is a character reserved for XML namespaces, so it can't be used for other things in HTML. https://stackoverflow.com/questions/16021123/is-colon-valid-in-attribute-names-for-html5 - so it would be safe to use for this.

Coincidentally, I feel like this makes it also easier to read, and it's similar to scoped variables in some programming languages, so hopefully it'd be more intuitive at a glance.

Double colon feels a bit "foreign" I think, but so does the colon, so you decide :)

I kept the ability of the html_attrs tag to "append" values, so we can join class attribute that is coming from component user with class defined by component author. But instead of key+=val, I reused the prefix::key=val construct. So user's can set add::class="extra-class" to indicate that the value "extra-class" should be appended to whatever is set to attribute class.

It feels strange to have reserved prefixes like that. Could we instead just say that if you use the same variable that means you add an attribute? {% merge_attrs attrs attrs::class="some-class" %}?

I've also updated README, trying to document this from user's perspective, introducing one concept at a time and finishing it with a full example.

Awesome! I see a couple of mentions of merge_attrs in the docs, I think they should be html_attrs.

And since we don't treat class as a special attribute (like the code from django-web-components originally did), I was able to shave off about half of the code + tests copied from django-web-components.

Awesome! I think this is shaping up to be a really nice feature. Thanks for being patient with all my side-thoughts :)

JuroOravec commented 3 weeks ago

: is a character reserved for XML namespaces, so it can't be used for other things in HTML. stackoverflow.com/questions/16021123/is-colon-valid-in-attribute-names-for-html5 - so it would be safe to use for this.

Ok, that's great, that means we can use just single colon!

It feels strange to have reserved prefixes like that. Could we instead just say that if you use the same variable that means you add an attribute? {% merge_attrs attrs attrs::class="some-class" %}?

I don't like using a same variable attrs::, because then we're breaking expectations on how prefix:: should work, as seen in component tag. Sure enough, the suggested add:: also doesn't work 100% the same as in component tag (we can pass add::class multiple times to html_attrs, but we cannot do that in component), but IMO then it at least feels like the input to html_attrs tag would be {% html_attrs attrs add key1=val1 key2=val2 ... %}, where both attrs and add are dicts.

Since it's safe to use single colon, then one more options comes to mind - we could simply use :class="abc" instead of add:class="abc". Similarly how in django-web-components, they use colon-prefixed :let. Altho then it might be harder to distinguish between the "appended" value and "default" values, e.g. {% html_attrs attrs key1=val1 :key1=val1 %},

JuroOravec commented 3 weeks ago

One more idea on this:

It feels strange to have reserved prefixes like that. Could we instead just say that if you use the same variable that means you add an attribute? {% merge_attrs attrs attrs::class="some-class" %}?

In previous comment, I suggested interface like:

{% html_attrs attrs add key1=val1 key2=val2 ... %}

But actually, we could avoid the problem with trying to distinguish between "defaults" and "appends" by switching it up:

{% html_attrs attrs defaults key1=val1 key2=val2 ... %}

So then we could say that all extra kwargs key1=val1 key2=val2, etc, they would be appended.

And the attrs defaults would be parsed the same way we parse components, so component author could do:

{% html_attrs
    attrs                       <- attrs from component user
    defaults:class="some-class" <- defaults from component author
    defaults:data-id="123"
    class="extra-class"         <- appended values
    class=class_var
%}
JuroOravec commented 3 weeks ago

Ok, I like this last approach, with attrs and defaults, where we're treating them almost the same as component kwargs (The only difference is that we allow them to be unset).

So the idea is that:

Examples:

Assuming that:

class_from_var="from-var"
attrs = {
    "class": "from-attrs",
    "type": "submit",
}
defaults = {
    "class": "from-defaults",
    "role": "button",
}
JuroOravec commented 3 weeks ago

Updated the code, tests, and README to the API described in my last 2 comments.

@EmilStenstrom @dylanjcastillo Would you give this one a review?

JuroOravec commented 3 weeks ago

Thanks @EmilStenstrom, I'm also happy with how it turned out! And thanks again for all the feedback, we wouldn't have arrived at this without it 😄

And thanks for the review, I've updated the docs.