Cerca CSS compiler project

[Thomasorus]

Hey folks!

Following @cblgh wish and @sansfontieres idea, let's talk about our own custom CSS compiler to write in GO!

A CSS compiler? Isn't that a big... BIG?!

Yeah a little bit, so let's reduce the scope and evacuate a few things first. The goal isn't to reproduce what the NPM ecosystem proposes, aka compiling from superset of CSS (SCSS, SASS, etc) and compatibility features like autoprefixing, fallbacks, stuff like that. We want solely a small generator, importer and cleaner.

Generator

Like I explained in #17, my goal is to have design tokens. Designs tokens are single piece of design, like a color, a size, a font-size, a border, an underline... Design tokens can then be combined into classes.

Let's say for example we have a scale of sizes. Those sizes are incremented by 0.25 and we start at 0.75. I'll use the CSS notation but that could be JSON or anything else.

:root {
--1: 0.75;
--2: 1;
--3: 1.25;
}

We can use this scale and CSS logical properties to create utility classes:

.padding-1 {
    padding: 0.75rem;
}
.padding-2 {
    padding: 1rem;
}
.padding-3 {
    padding: 1.25rem;
}

.font-size-1 {
    font-size: 0.75rem;
}
.font-size-2 {
    font-size: 1rem;
}
.font-size-3 {
    font-size: 1.25rem;
}

And many more!

Why use design tokens and generate those classes? For two reasons:

1. Design tokens create an harmony of sizes and colors based on scales make the design more cohesive, pleasant, and avoid writing hard random values.
2. Utility classes based on design tokens allow us to style single elements without create a dedicated class like .box-card

With these, we can do a first pass on all templates, adding padding, margins, font-sizes, colors, font-families, etc... It's also way more easy to understand for people not too familiar with CSS. Want bigger font-sizes? Change the scale! Want another color for your links? Just change it!

So we generate those classes and put them into a file. But that doesn't mean it will work for everything, far from it!

Importer

We still have some classes to write by hand. Some of them can use the design tokens. For example the flow classes, that are used to detect if two elements are adjacent and add margin between them, are often a good way of having a good vertical flow in a page and cannot be generated solely by design tokens.

.flow-1 *+* {
    margin-top: 0.75rem;
}
.flow-2 *+* {
    margin-top: 1rem;
}
.flow-3 *+* {
    margin-top: 1.25rem;
}

If you did not know, using + avoid writing :first-child or :last-child to remove borders applied to all elements.

To organize those classes into logical elements, we could use partials of CSS. There's a convention that we could reuse from SCSS, which is to prefix all imported files with _. For example: _flow.css.

Writing in the index.css a list of files we want to import already works in CSS... by requesting all files one after each other. Ideally, we want everything into a single file that we can compress using gzip or brotli (the server can do it).

So the goal would be to create a list from the index file and find in subfolders each _*.css files and concatenate them. In the end we would have everything inside a second file, both our generated classes and our manually crafted classes.

And that's an issue: the file is going to be way too large.

Cleaner

Let's say we want to generate a lot of utility classes with our design tokens. We want padding, padding-left, padding-top, padding-right, padding-bottom, padding. The same for margin. Then we wan gap and grid-gap. We want background-color and color for our colors. We're gonna have a ton of classes we could be using, but might not use in the end.

That's why we need to clean our file. The best way to do it would be:

1. Scan classes declarations in our template files
2. Using this dictionary of names, remove all the unused ones inside our compiled css file.

In the end, only what we end up using would be there, while still keeping the possibility to use new utility classes. Note that we could clean the declarations before we import all files, but doing it an the end also allows us to remove unused manually crafted classes, like the .flow classes.

Conclusion

I don't know GO at all so I can't really say exactly how to proceed since I don't know what the language is capable of. If you have suggestions about how we should declare design tokens, if importing files is easy or not, if the cleaning process is doable or not, please share!

I'll leave you with a few links:

• Gorko, a token class generator written in SASS: https://github.com/hankchizljaw/gorko#getting-started
• PurgeCSS, an unused CSS remover: https://purgecss.com/
• Cube CSS, the methodology that consist of styling with your own utilities, them doing specific classes: cube.fyi/

[cblgh]

thank you sooo much for this clear and well-written proposal @Thomasorus! looking forward to conducting a bit of an experiment on this with ya :> what follows are some quick follow-on ideas based on the above :~

for generating and specifying the design tokens, i think it would be fun to play with some conventions and try to see how far we can get with only using css (like you are doing in your example):

e.g.

/* some file, like settings.css or whatever */
:root {
--1: 0.75;
--2: 1;
--3: 1.25;
}

.setup {
    padding: 0;     /* we want to scaffold out tokens for padding */
    font-size: 0;   /* same for font-size */
}

i'm sure we'll run in to some hurdles eventually, but if we could avoid having to use a separate language/syntax to specify css, then i think that would be really fun :> we could even look into having some support for the basics of the 100r themes concept by using their naming conventions for colours and colour use.

when it comes to importing, how about the following convention: css is written in whatever named files you want inside a single folder, and the compiler outputs a single concatenated file generated.css (or something more fun, perhaps literally-everything.css).

if we're using the cleaner pass, hierarchies of imports or whatever don't really matter—but i guess they could matter for css precedence rules, huh? so maybe we revise that: by convention, we have some file—let's say index.css—that lists the order of precedence using filenames or imports. if that file is not found, then we just go nuts and concatenate all the files we can find in alphabetical order, and dump the output into generated.css

when cerca's developing flag is set to true, let's include the entire generated file always! this ideally prevents people playing around from being confused why the css isn't changing, despite them changing it—i've been there with weird frameworks before!!

instead of having one file and then cleaning it, i think we could do the same thing but in the opposite way. we have one file that contains all of the generated tokens and then we, separately, have the file that will be served as part of http responses. the latter file is essentially populated by moving over the in-use design tokens from the generated file, based on what has been found by scanning the templates!

(and tbh i don't really care about the compression bit, but if we only use one file then maybe we/someone else can take that on towards the end ^^)

---

before we proceed, i'll probably need a smol concrete example to start with for fleshing out a structure and more honed idea of what shape things and procedures will take. unless anyone else has any thoughts or arguments against this i'm happy to explore the area and see what comes out of it :3

[Thomasorus]

Hey I'm back with a bit of feedback!

i'm sure we'll run in to some hurdles eventually, but if we could avoid having to use a separate language/syntax to specify css, then i think that would be really fun

Here's an idea for you. It's not correct CSS but the syntax is similar:

• Variables are usually declared in :root in CSS, we could use something similar with a specific name for our groups of tokens.
• Specific rules like mediaqueries are declared using the @ in CSS, we could use a similar style to declare the classes we want to generate.

Example:

:colors {
    --main: black;
    --second: red;
}

:scale {
    --01: 0.75;
    --0: 1;
    --1: 1.25;
}

@classes {
    .pad { 
            padding: rem !scale;
        }
        .text {
            color: !colors;
        }
}

In this example:

• -- declares a token like a classic CSS variable.
• The .classname inside @classes are the utility classes we want for our project, following this syntax: CSS property: optional css unit !tokenGroup.
• We concatenate our variable name with our token name with a simple - between them.

In this example, we would generate:

.pad-01 {
    padding: 0.75rem;
}
.pad-0 {
    padding: 1rem;
}
.pad-1 {
    padding: 1.25rem;
}
.text-main {
    color: black;
}
.text-second {
    color: red;
}

but i guess they could matter for css precedence rules, huh? so maybe we revise that: by convention, we have some file—let's say index.css—that lists the order of precedence using filenames or imports.

Yeeeees you guessed right! The goal is to go hand to hand with the cascade and avoiding reflows. To do this we need first to declare our CSS in a specific order. If you never saw it, it's basically following the inverted triangle CSS methodology.

The goal of this order is to avoid specificity:

In our case, we probably want something like this:

• Reset file to specify a few rules we don't want to write ourselves.
• Base rules like HTML elements for links, lists, titles, paragraphs, etc that are not design tokens.
• Layout classes with flexbox and responsive, as well as the fallbacks for old browsers, or flow classes.
• Automatically generated classes.
• Block classes that require hyper specific things, for example the logo with an SVG but with a background image as a fallback for old browsers.

Cerca is a small project so it might feel we're overdoing it, and I kinda agree. But it's a good and easy to understand way of styling it. It allows you to make specific or general updates without effort.

• For example if the community agrees that there is not enough padding on the main tag. You can just go in the template and replace pad-0 by pad-1 and you are done.
• But if you decide that there is not enough padding everywhere, you can just up the values of the scale, recompile, and everything will be updated without you touching the HTML.

when cerca's developing flag is set to true, let's include the entire generated file always! this ideally prevents people playing around from being confused why the css isn't changing, despite them changing it—i've been there with weird frameworks before!!

Yup for development that's a good idea since we don't care about the size at this moment.

instead of having one file and then cleaning it, i think we could do the same thing but in the opposite way. we have one file that contains all of the generated tokens and then we, separately, have the file that will be served as part of http responses. the latter file is essentially populated by moving over the in-use design tokens from the generated file, based on what has been found by scanning the templates!

Yup that works too! There is another way of doing this, which is JIT (just in time) but that means making speedy code that scans first and generate classes on demand, which is probably way more work and not really needed for such a small project.

before we proceed, i'll probably need a smol concrete example to start with for fleshing out a structure and more honed idea of what shape things and procedures will take. unless anyone else has any thoughts or arguments against this i'm happy to explore the area and see what comes out of it :3

I hope the examples will help you! I tried doing a small parser using regex in GO, but the arrays (or slices?) are confusing to me, as well as the syntax. If we were using JS I would have probably done the compiler as a fun project, but right now I'm not into the mood to learn a new language, so I'll be counting on you. 👍

[cblgh]

thanks again for the easily-digestible response @Thomasorus 🖤

I tried doing a small parser using regex in GO, but the arrays (or slices?) are confusing to me, as well as the syntax

ooohh nooo i never meant for you to do that ahhh. let me get onto it this week :>

Here's an idea for you. It's not correct CSS but the syntax is similar:

Variables are usually declared in :root in CSS, we could use something similar with a specific name for our groups of tokens. Specific rules like mediaqueries are declared using the @ in CSS, we could use a similar style to declare the classes we want to generate.

i like it, let's simplify!

an iteration on the idea:

:colors {
    --main: black;
    --second: red;
}

:scale {
    --01: 0.75;
    --0: 1;
    --1: 1.25;
}

.pad {
    padding: var(--scale)rem;
}

.text {
    color: var(--colors);
}

/* now let's redeclare :scale to generate the margin classes `m-<scale>` */
:scale {
    --1: 1rem;
    --2: 2rem;
    --3: 3rem;
}
.m {
    margin: var(--scale);
}

[Thomasorus]

I really like this part:

:colors {
    --main: black;
    --second: red;
}

:scale {
    --01: 0.75;
    --0: 1;
    --1: 1.25;
}

.pad {
    padding: var(--scale)rem;
}

.text {
    color: var(--colors);
}

But not much that one:

/* now let's redeclare :scale to generate the margin classes `m-<scale>` */
:scale {
    --1: 1rem;
    --2: 2rem;
    --3: 3rem;
}
.m {
    margin: var(--scale);
}

For two reasons that are separate:

• If you use a scale for spacings (gap, margins, padding, etc) you should use the same for all of them, because even if they are a different property, their end result is spacing, and spacing should be the same for all.
• If you want to use a second scale for something else (for example font sizes), you should declare it first instead of rewriting an existing one, for clarity.

You idea is indeed very "cascady" in the spirit of CSS, but I think we should leave tokens out of this so it's easier to scan, modify, and without surprises.

[cblgh]

But not much that one:

i'm fine with scrapping that then :~

this'll be a fun little experiment!

[Thomasorus]

Let's goooo! What is cool with this approach, I think, is that once it's done people can actually fork the forum project, create their own theme, then ask for a PR so all members can use it. :3

[cblgh]

@Thomasorus aight! i've gotten the first naive version of the password reset out, i've got my bread for the week baked. let's see if we can make some progress on some experiments in this arena this week! :>:>

[Thomasorus]

@cblgh I'm testing the compiler and here are some suggestions:

1. We should have the ability to declare where the generated classes are going to be placed inside the compiled file. Maybe by using the @import syntax? We could have a hardwired import name @import "generated";?
2. I have a strange issue with my code editor (sublime). When declaring the unit type padding: var(--scale)rem; is saved to padding: var(--scale) rem; and thus breaks the generator. I think it's because most css declarations are either closed by ; or have a space for another declaration. Can we make it possible to have a space between the var(--scale) and rem declaration? 😅
3. Right now the dev experience for making styling is very poor. You have to relaunch the main go server if you change the html and use the cercass command line to recompile if you change the css. Can we have a watcher or something that execute those commands?

Otherwise, everything seems to be working!

[cblgh]

@Thomasorus 2 and 3 are pretty clear to me, but can you expand a bit on what you want with 1 and in what situation? are you basically asking about a way of putting the generated block of design tokens somewhere in the final css?

e.g. do you always want it to be last? or does it differ?

[cblgh]

@Thomasorus just fixed #2 :)

regarding #3, do you know of entr? it's a really cool tool i learned about when reading julia evan's blog: https://jvns.ca/blog/2020/06/28/entr/

e.g. standing in the cmd/cercacss folder, i can use it like follows to achieve the behaviour you're after:

git ls-files ../../html | entr ./cercacss --html ../../html

[Thomasorus]

you basically asking about a way of putting the generated block of design tokens somewhere in the final css?

yes!

e.g. do you always want it to be last? or does it differ?

Yes but not only if possible! Being able to put it anywhere would probably be the best. If not possible/no time, just put the generated code at the end.

do you know of entr?

Yup I know about it but haven't used it in a while. I'll try it and update the docs in the CSS branch. 👍

[cblgh]

alright, i put the generated css at the bottom of the block for now and if it becomes essential we'll tackle that hurdle when we get there :)

good idea about the docs, thanks!