w3suli commented 4 years ago

A list of things to keep in sync:

See the css_consistency branch.

[x] vendor specific prefixes (attributes) https://github.com/highlightjs/highlight.js/pull/2425
[x] vendor specific prefixes (at-rules) https://github.com/highlightjs/highlight.js/pull/2425
[x] tag selectors https://github.com/highlightjs/highlight.js/pull/2425 (just adds tests)
[x] class selectors
[x] ID selectors
[x] pseudo selectors : https://github.com/highlightjs/highlight.js/pull/2425
[x] pseudo selectors :: https://github.com/highlightjs/highlight.js/pull/2425
[x] top level psuedo-selector :lang(de)
[x] selection attribute, i.e. a[href*="example" I] (See: #2243)
[ ] function highlighting, ie (grayscale(0.5))
[ ] at-rules @import, etc.
[x] media at-rules highlighting ("@media", "not", "only" and "and")
[x] selection of @media attributes ("(orientation: landscape)" or "(min-width: 600px)")
[ ] variables
[ ] attributes
[ ] What are we missing?
[x] use uniform TAG list
[x] use uniform PSUEDO-SELECTOR list
[x] use uniform ATTRIBUTE list
[x] !important

Sass bug list:

[x] attribute prefix -I like it but different from css and less - unfortunately bad (-webkit-animation-name:)
[x] for the selection attribute, the string is not highlighted (a[href*="example" I])
[ ] lack of function highlighting (grayscale(0.5))
[ ] at-rules with highlighting difference (@keyframes "from" "to")
[x] lack of selection of @media attributes ("(orientation: landscape)" or "(min-width: 600px)")
[x] other differences: ":lang(de)"

Less bug list:

[x] for the selection attribute, the string is not highlighted (a[href*="example" I])
[x] lack of pseudo selectors highlighting (q::after)
[ ] lack of function highlighting (grayscale(0.5))
[x] lack of selection of @media attributes ("not", "only" and "and")
[x] other differences: ":lang(de)"

Stylus bug list:

[x] attribute prefix -I like it but different from css and less - unfortunately bad (-webkit-animation-name:)
[x] lack of attribute selectors highlighting (a[href*="example" I])
[x] lack of pseudo selectors highlighting (q::after)
[ ] lack of function highlighting (grayscale(0.5))
[ ] lack of at-rules highlighting ("@font-face" "@keyframes", "from", "to")
[x] lack of media at-rules highlighting ("@media", "not", "only" and "and")
[x] lack of selection of @media attributes ("(orientation: landscape)" or "(min-width: 600px)")
[x] other differences: ":lang(de)"

There is a lot of difference. If I have enough time, I'll make examples one by one. I welcome any help. Thanks in advance for all your help!

joshgoebel commented 4 years ago

Although really you should tear the look ahead regex apart, understand them... and they should eat the spaces, then only look-ahead for the content itself... that means the very first "starts" wouldn't need to deal with leading whitespace, etc.

w3suli commented 4 years ago

Thank you very much! I apologize! I was completely misunderstood. That makes sense. I'll keep doing it. I would like to have a unified look for css, scss, less, sass and stylus. As soon as there is a properly developed version, I will launch a Pull request.

joshgoebel commented 4 years ago

I would like to have a unified look for css, scss, less, sass and stylus

I did think that was the idea. :-) Looking forward to seeing something. Also be aware you can have a "WIP" pull request and just put that in the title and continue to push your new work there... that gives us a chance to see your stuff as you're building it out.

w3suli commented 4 years ago

@yyyc514 The css functions use the built_in class. Preprocessors can be used to create arbitrary functions. Do they also get the built_in class? If you want to distinguish between functions, you must add the built-in ones to a list. In this case, the function class can be used for other functions.

Be it different or not?

joshgoebel commented 4 years ago

I'm not sure I have an opinion. :-)

joshgoebel commented 4 years ago

This will probably end up being a potentially breaking change if we depend on CSS or a common core (which we've never done before)... so this would be good for 10.0.

I'm thinking the next release might be the first in 10.0 series... so that's a 6-7 week clock starting now. :-) Just a heads up.

Would be nice if we can get all the breaking things in one big release then go back to more quiet path for a while.

w3suli commented 4 years ago

Yes, I think it will be necessary for a few more weeks. There are even minor problems and issues to resolve. There is no problem with basic css (scss, less, sass, stylus) rendering, that's fine. The functions and other additional features of css preprocessors still give some work.

Regarding the list of CSS properties, I would suggest: I think it would be a good idea to include items in both the experimental and recommended stages. For example, prism already highlights css --* as a variable.

:root {
  --first-color: #488cff;
  --second-color: #ffff8c;
}

#firstParagraph {
  background-color: var(--first-color);
  color: var(--second-color);
}

#secondParagraph {
  background-color: var(--second-color);
  color: var(--first-color);
}

#container {
  --first-color: #48ff32;
}

#thirdParagraph {
  background-color: var(--first-color);
  color: var(--second-color);
}

https://developer.mozilla.org/en-US/docs/Web/CSS/--*

I've collected a list of internal functions: annotation() attr() blur() brightness() calc() character-variant() circle() clamp() conic-gradient() contrast() counters() cross-fade() cubic-bezier() drop-shadow() element() ellipse() env() fit-content() format() grayscale() hsl() hsla() hue-rotate() image() image-set() inset() invert() leader() linear-gradient() local() matrix() matrix3d() max() min() minmax() opacity() ornaments() paint() perspective() polygon() radial-gradient() rect() repeat() repeating-linear-gradient() repeating-radial-gradient() rgb() rgba() rotate() rotate3d() rotateX() rotateY() rotateZ() saturate() scale() scale3d() scaleX() scaleY() scaleZ() selector() sepia() skew() skewX() skewY() steps() styleset() stylistic() swash() symbols() target-counter() target-counters() target-text() translate() translate3d() translateX() translateY() translateZ() url() var()

If I differentiate the built-in functions, I need to list them. hljs.CSS_NUMBER_MODE is missing the units "fr", "Q", and "x". The "grad" and "rad" can be merged into "g?rad".

joshgoebel commented 4 years ago

The "grad" and "rad" can be merged into "g?rad".

Not worth it, just list them both.

That's not that long of a list... I think the real decision is whether there is value in differentiating built-in functions vs user functions. I think probably?

joshgoebel commented 4 years ago

I think it would be a good idea to include items in both the experimental and recommended stages.

No objection.

w3suli commented 4 years ago

Current state:

This is used by css: calc(100% - 80px)

This is used by stylus: sub(a, b)

The two representations are clearly different. It is true that the functions used in the property values are not highlighted by stylus at all.

If we do not differentiate them, then we have to decide what the uniform highlighting structure of the function should be.

Each solution has its advantages and disadvantages.

The benefit of differentiation is that it shows you where to look for the function. A non-built-in function in the style files, while a built-in function in the style language description.

joshgoebel commented 4 years ago

I'm not really sure what I'm looking at. It would probably help if you used the same underlying code for both examples and then showed what it looks like in the different languages.

I looks like you're using entirely different snippets so that makes it really hard to see anything (at least for me).

w3suli commented 4 years ago

The two solutions are very different. If the functions are not differentiated then a third solution might be good. What is currently used by (stylus) is only the non-built-in functions highlighted. The parenthesis and all parameters within it. What css uses is highlighted before the parenthesis word as built-in. Brackets and other characters are not highlighted. Only numbers and strings are highlighted within parentheses.

If you need a unified solution, you might want to keep the css complemented by highlighting numbers and strings, highlighting each letter or word as a parameter.

Then, the question is whether the built_in or function class is used to highlight the word before the parenthesis?

joshgoebel commented 4 years ago

I feel like I already answered this. I saw lets track both functions and built-ins as different things and see how that works out.

joshgoebel commented 4 years ago

Any luck? :)

joshgoebel commented 4 years ago

FYI: Now that we have modules we also have the ability to pull common things out into a lib/css module and then share them at compile-time rather than run-time. Currently all the languages are still individually compiled but as soon as we have common modules that start to cross-language I'll update the build process to build one huge bundle for included languages which will cut down on duplication, etc.

I'm not 100% sure that's what we want here, but just mentioning it as a possibility. It's definitely preferable to copy-and-paste for simple things.

joshgoebel commented 4 years ago

Where is the source code example for your very first post? And how did you get the 4 side by side? Do you have a HTML template you're using for the previews? It'd be helpful if you attached those things so others could also use them.

I think we need to go back and split this into bite size pieces so that each one can turn into a small PR that is easy to review and get merged. If fear we've gone and made this a huge project and hence it's stalled out. You could do it by language but I think it'd be easier to do it by feature...

So you could start with say vendor prefixes, get those working the same everywhere, etc... then pseudo-selectors, etc... these could be separate PRs or separate commits inside a larger PR... If you made a complete list of CSS/CSS processor features how long would it be? 10-20 items?

And adding markup tests for each as you went... that way once you had things looking the same you could refactor the languages easily knowing we had solid tests to catch any issues.

joshgoebel commented 4 years ago

I created a checklist (above in the first message of this thread) and an example: https://github.com/highlightjs/highlight.js/pull/2425

First, add tests. Then make the grammars consistent. Optimization and sharing code can come last - so I didn't even focus on that. Obviously we should share vendor prefixes globally, etc.

I have a small script to make keeping the tests in sync easier:

#!/bin/bash
cp test/markup/css/css_consistency.txt test/markup/less
cp test/markup/css/css_consistency.expect.txt test/markup/less
cp test/markup/css/css_consistency.txt test/markup/scss
cp test/markup/css/css_consistency.expect.txt test/markup/scss
cp test/markup/css/css_consistency.txt test/markup/stylus
cp test/markup/css/css_consistency.expect.txt test/markup/stylus

If you want to work on this you can make PRs against the css_consistency branch rather than master so we keep all this work isolated for now.

joshgoebel commented 4 years ago

@taufik-nurrohman This is a big chunk of work if you wanted something large to cut your teeth on. :-)

w3suli commented 4 years ago

I will have time again by June. I will continue to work on the topic. Sorry I didn't have time for the new year :(.

There is a need on which I would like to ask @yyyc514 opinion, as well as the opinion of others. In the case of css, would it be possible to highlight css correctly even without selectors?

When you leave the selector in an article, because only the property and the value are important: border : 2px solid green;

joshgoebel commented 4 years ago

Maybe, maybe not.

I feel like we're wasting time answering questions like this without specific context. I made a long list (see initial message) of individual items to work thru... so lets focus on that list - knocking things off one at a time - and then when one of the items on it leads to us asking this kind of question, I say discuss it at that point - with the full context on hand, a PR to look at, etc.

I've also been trying to make the minimal invasive changes to do this so far... and work on building up a comprehensive test suite across all the CSS-like grammars... then once we have very high test coverage we can circle back at how we might make any of the individual grammars simpler.

w3suli commented 4 years ago

@yyyc514 Thank you for your support!

When I cut in, it seemed simpler at first. However, during the work, it turned out that the basic css highlighter also has shortcomings.

I don’t want to waste time, but a lot of little little things came up.

Maybe I lost a little bit of the details. Such as unicode selectors, which are also highlighted in the Visual studio code.

For example, that's why I created the following solution:

CSS_SPECIAL = '!|"|\'|`|#|\\$|%|&|\\(|\\)|\\*|\\+|,|-|\\.|\\/|:|;|<|=|>|\\?|@|\\[|\\\\|\\]|\\^|{|\\||}|~';
CSS_SPECIALS = CSS_SPECIAL.replace(/\|/gi, '');

...

className: 'selector-id', begin: '\\#(-?((\\\\('+CSS_SPECIAL+'|[0-9a-f]{1,6}\\s?))|(?![-\\d'+CSS_SPECIALS+'\\s]))|--)((\\\\('+CSS_SPECIAL+'|[0-9a-f]{1,6}\\s?))|[^'+CSS_SPECIALS+'\\s])*'

There are lots and lots of little things, yet. I started working on these, but they stopped. I will have time to continue soon.

._

_

._7

_7

._a

_a

.--

--

.-a

-a

.--3

--3

.class

id

.Class

Id

.clAss

iD

.CLASS

ID

.cláss

íd

.cläss

iđ

.člasŝ

ίδ

.класс

идентичность

.クラス

身元

.מעמד

זהות

.صنف

هوية

.कक्षा

पहचान

.c-lass

i_d

.c-lass

i_d

.-class

-id

._class

_id

.►♫♪™

►♫♪™

.cla\$s

\:d

.cl\@ss

i\&

.\31 #\31 .\31 0cl\31 ass #\31 0id\31abcd gass .\31 fclass #\31 fid #\31fid .\31gclass #\31gid .\31\@ #\31\@ ..

.

.#

\

.\@

\@

.!

!

.\$

\$

.\%

\%

.\&

\&

.(

(

.)

)

.*

*

.+

+

.\,

\,

.-

-

.\/

\/

.\:

\:

.\;

\;

.\<

\<

.\=

\=

.>

>

.\?

\?

.[

[

.\

\

.]

]

.\^

\^

.{

{

.|

|

.}

}

.\~

\~

.\"

\"

.\'

\'

.`

`

.cs!#\$\%\&()*+\,-.\/\:\;\<\=>\?\@[\]\^{|}\~\"\'`

id!#\$\%\&()*+\,-.\/\:\;\<\=>\?\@[\]\^{|}\~\"\'`

/ NOT Classes/ .- .-4 .5 .. .@ .!#$%&()*+,-./:;<=>?@[]^{|}~"'``'"

!#$%&()*+,-./:;<=>?@[]^{|}~"'``'"

toastal commented 4 years ago

Title: ~CSS, SASS(SCSS), LESS, STYLUS highlighting differences~ CSS, Sass(SCSS), Less, Stylus highlighting differences

joshgoebel commented 4 years ago

When I cut in, it seemed simpler at first. However, during the work, it turned out that the basic css highlighter also has shortcomings.

They each have their own shortcomings. Bringing them all into alignment is going to be a lot of slow, laborious effort I think. I started it, but I think someone else may have to finish it. I'm not interested in patches that ONLY fix one while ignoring the others.

Such as unicode selectors, which are also highlighted in the Visual studio code.

Picking a single item like that (Unicode selectors) and making a PR is a great idea... add a test, sync it across the CSS languages (I believe I have a script and that most of them support "plain jane CSS" in addition to their more complex form) and then make changes to guarantee the same behavior.

This work (overall) should be done in lots of tiny PRs or else it will be impossible to review. That's why I have a detailed list of CSS features in checklist form at the top of this item... hopefully each item would be a small PR that could be easily reviewed.

joshgoebel commented 3 years ago

What issues did you notice with :lang(de)? Currently I see consistency across all 4 (could be I already fixed):

<span class="hljs-selector-pseudo">:lang</span>(en) {}

joshgoebel commented 3 years ago

  src: url("/fonts/OpenSans-Regular-webfont.woff2") format("woff2"),
       url("/fonts/OpenSans-Regular-webfont.woff") format("woff");

What are url and format here? They aren't technically functions are they?

joshgoebel commented 3 years ago

lack of at-rules highlighting ("@font-face" "@Keyframes", "from", "to")

Could you clarify if you remember? I'm sure keyframes needs work, but what was the issue with font-face (other than the function aren't highlighted, see question above). Right now I have this: (Stylus)

joshgoebel commented 3 years ago

One could question whether we highlight the explicit values like swap, but first I'm just trying to get basic consistency.

highlightjs / highlight.js

CSS, SASS(SCSS), LESS, STYLUS highlighting differences. #2269

_

_7

_a

--

-a

--3

id

Id

iD

ID

íd

iđ

ίδ

идентичность

身元

זהות

هوية

पहचान

i_d

i_d

-id

_id

►♫♪™

\:d

i\&

.

\

\@

!

\$

\%

\&

(

)

*

+

\,

-

\/

\:

\;

\<

\=

>

\?

[

\

]

\^

{

|

}

\~

\"

\'

`

id!#\$\%\&()*+\,-.\/\:\;\<\=>\?\@[\]\^{|}\~\"\'`

!#$%&()*+,-./:;<=>?@[]^{|}~"'``'"