Proposal: use and recommend .solidity file extension (for syntax highlighting on github)

[veox]

EDIT: The .gitattributes approach mentioned below now works with linguist. Add to .gitattributes:

*.sol linguist-language=Solidity
*.or-any-other-extension-you-like linguist-language=Solidity

Original issue description follows.

---

The main reason there's still no Solidity syntax highlighting on github is that the .sol extension is too generic.

(There have been several attempts made so far, which I've listed in PR https://github.com/github/linguist/pull/3560. That PR itself is likely to get scrapped.)

Therefore, linguist can't classify .sol files as "Solidity" based on extension alone. It must run a heuristic to determine a language based on the file's contents.

However, due to linguist's current design, heuristics have no way of indicating "no match" (documented in issue https://github.com/github/linguist/issues/3589). That, coupled with the same fact that .sol is too generic, will mean that a lot of unrelated files will still be mis-classified, which in turn means the heuristic approach is a no-go.

So, there are two options I see now:

.gitattributes file

Don't rely on github's automated classification, and ask everyone to manually hint the language via .gitattributes (as in PR https://github.com/github/linguist/pull/3559).

Both new and old projects will have to do this if they want highlighing.

This will likely be made available regardless (probably by @frangio, as discussed in PR https://github.com/github/linguist/pull/3559 in PR https://github.com/github/linguist/pull/3755).

.solidity extension

Use a file extension that unambiguosly describes the contents.

Old projects would have to mass-rename files, as well as mass-replace import statements, which will change their Swarm hashes AFAIU.

[veox]

AFAIU gists would have manual selection for "Solidity" as soon as some form of an actual classifier is merged into linguist.

Gists no longer have a "Language" drop-down selection. They likely now completely rely on the file extension.

[axic]

Any chance of fixing linguist to indicate "no match"?

[veox]

Not sure. It seems to be low on their priority list, and in either case might require changing the approach to classification in linguist. Since it's used and maintained by github, with a humongous set of files that might get erroniously re-classified... the process is necessarily slow.

Essentially, what I'm proposing is for Solidity to step out of the "edge case territory" that is the .sol extension.

---

@Alhadis also (jokingly?..) proposed .sold in one of the above PRs, which has seen very minimal use on github so far.

This might be a compromise if a short extension is still desired, although it has the same downsides as .solidity.

EDIT: Since the purpose of an extension seems to be aiding machine classification, I'd still recommend .solidity, since, well, that describes a Solidity file best.

[Alhadis]

Half-jokingly. My best ideas tend to come out while talking shit. Seriously.

.solid also looks ripe for the taking, albeit with more in-the-wild results (717 so far, which is still trivial).

[veox]

Would like to note that some people are already using .solidity (24 results ATM). Not sure why.

[Alhadis]

They're visionaries. God bless their .sols.

EDIT: I am absolutely killin' it tonight.

[federicobond]

If we are considering alternatives, .sldt sounds good too and has minimal usage in GitHub.

[axic]

In my opinion I'd only consider .solidity (preferred if there must be a new name) or .solid, but I think this is way more down to the users and not to the compiler as it is agnostic to filenames.

[Alhadis]

Then make it a pushy, angry atheist to filenames, because indecisiveness or indifference towards standardised file-extensions is the sort of thing that leads to our original problem of thousands of moderately-used .sol extensions, and me screaming in fatigue-encrypted Latin wondering where the hell to begin organising files based on non-existent formats. :(

TL:DR; Rename language .sparta and nobody's gonna fuck with you.

(Hey, it's my third day without sleep, whoooooo!0

[Alhadis]

I think what I'm trying to say is this: it would help to decide on a definitive extension, and having a technical rationale behind that adds incentive for the community to follow. Otherwise, if it's just a community recommendation, well... there're dicks like me who openly go out of their way to push stuff like that. :D

coughTabs-in-Ruby-and-Python*

[axic]

I am sure this has been raised several times, but is looking for pragma solidity [^;]+; not good enough to find a Solidity file? 😉

[Alhadis]

It would be, if only Linguist's classifier worked that way... to TL;DR it for you: having that course of logic would require every other .sol file that doesn't match that pattern to fall under a particular language type. So there's currently no real option for a not-a-language or inconclusive result. :(

[axic]

This is aside the discussion whether .sol is a good name or not:

I am still puzzled a bit why we are trying to change the extension of a language to workaround a bug in a piece of software used by a commercial entity. Is it nigh impossible to implement the not-a-language feature in linguist?

[Alhadis]

It's a technical issue that happens to be beyond our current ability to fix: it involves digging around and under the base tier of logic that happens when Linguist's classifier runs - the last stage is the Bayesian classifier, which returns a list of sample/language pairs rated with varying levels of confidence. @pchaigno assures me there's no way of working outside that thread of logic, but I'm not so sure...

Let's be realistic though: the morass of varied search results would imply many other users have the .sol extension used for their own purposes... wouldn't you rather Solidity be represented by a distinctive, easy-to-recognise extension? ;)

[VoR0220]

Craziest proposal so far: Solidity compiles filename extension solidarity...since my friends always misnames the language to that anyway...might be worthwhile to atleast add legitimacy to their mistake. 🥉

[garrett92895]

Are there any reasons why both .sol and .solidity can't be accepted? People could choose the .solidity extension moving forward if they want the benefits of automatic github syntax highlighting and completely unambiguous file extensions, and anyone else could just create a .gitattribute file (if they even want syntax highlighting) without the need to refactor their codebase.

[bronoman]

suggest to vote for it: http://poll.lab.io/solidity-ext

[Alhadis]

Choose both, just make sure you give the .sol extension the 🖕. 😀

I voted for .sold because it has a chime to it and a subtle wit makes it memorable.

[oktapodia]

Any update about this one?

[veox]

IMO ATM there's no actionable for the repo maintainers here.

The Solidity developer community historically uses .sol; there are costs to switching (in time and broken pipelines at least), and no immediate benefits.

Voting won't help with removing the costs, but it does indicate already that quite a few are reluctant to do it in either case.

The .gitattributes approach is therefore more likely. The relevant linguist PR is https://github.com/github/linguist/pull/3755.

[axic]

@veox if that PR is merged, what is the appropriate line to be placed into .gitattributes?

It would be nice to include that in the Solidity documentation once it is merged.

[veox]

According to their README:

*.sol linguist-language=Solidity

The PR in question doesn't pass CI currently, so is distant from being merged.

[oktapodia]

Hey guys, I fixed the CI, can I have a review, please?

[oktapodia]

@axic @federicobond @VoR0220, one of you can confirm that the .solidity extension is good for now in the linguist package?

We could use the *.sol linguist-language=Solidity line in the .gitattributes file to override it (I will make a pull request on solidity when the linguist PR will be merged) and having the highlighting syntax which will be awesome!

[veox]

The PR by @oktapodia has just been merged.

Solidity syntax highlighting should become available on github (with the above-mentioned .gitattributes approach, for files of any extension) with the next linguist release or so.

If anyone still sees value in Solidity files being recognised automatically, this issue should IMO stay open. Otherwise OK to close.

[oktapodia]

It has been released and deployed 4 days ago, we can now enjoy and use it! https://github.com/github/linguist/releases/tag/v6.0.0

[codingthat]

What about for gists? (I don't see any manual selection method available anywhere--maybe I'm not looking in the right place?)

[veox]

@codingthat Gists used to have a language selection drop-down, but not anymore. I don't know when the change happened.

It seems to completely rely on the file extension now.

[codingthat]

@veox thanks for confirming. So...I have a .sol gist that is not highlighting, should it be? Or would a further change be required somewhere?

Alternatively, is there a way to embed syntax-highlighted files from a repo as if they were a gist? :)

[roberto-butti]

For files on github repo (not gist) I solved creating .gitattributes file and adding this line: *.sol linguist-language=Solidity I wrote a little post about this: https://medium.com/@robertodev/solidity-and-github-42a3ca375618

[codingthat]

@roberto-butti That's true, that's what's already suggested above in this thread. Hoping someone can comment on how to do this for gists, since they're embeddable, unlike repo files, as far as I know.

[NachoPal]

Yeah! It would be nice to be able to highlight solidity sintaxis for Gists as well. Apparently it's not yet possible. I didn't find any solution.

[iCodeSometime]

@codingthat, @NachoPal according to the pull request, the language was added under the .solidity extension. Changing your gist to that extension should give you highlighting. In a repo, you can keep using the .sol extension and add the extension to your .gitattributes.

edit: Actually that's not working for me. no idea.

edit2: Apparently they removed the extension right before merging. Only gitattributes will work I guess

[codingthat]

An example gist (with .solidity) in case this ever gets implemented: https://gist.github.com/codingthat/4286f86fe946a49829b3897499defc1e

[k06a]

@codingthat still not implemented

[cameel]

Looks like the issue has been finally solved for .sol extension in #4936.

https://twitter.com/NomicLabs/status/1303010441954762754

Solidity_lang is now a first-class citizen on Github!

Using EF funding we sponsored a maintainer of GitHub’s language detection library to do the refactoring to enable automatic detection of Solidity. No more .gitattributes to get highlighting!

[codingthat]

Thanks for the heads up, @cameel ... looks great in repos! Still doesn't work in gists like https://gist.github.com/codingthat/4286f86fe946a49829b3897499defc1e though, unless there's something extra that needs doing there? (The main thing I was looking for was the ability to embed the code, with highlighting, in an external site.)

[cameel]

@codingthat I think it only works for .sol extension.

But .solidity was just a workaround for .sol being too ambiguous to be properly supported, right? Now github's detection can deal with it. This issue was opened in 2017 so the question is: how widespread is .solidity in practice. Should we keep pushing for wider support for it or is it obsolete now?

[chriseth]

This issue is certainly obsolete. Changing the extension now is probably not possible since the community has adopted .sol a long time ago.

[Alhadis]

This issue is certainly obsolete.

☝️ This. Also, to clarify a point of confusion — language stats are cached on GitHub, so repositories that haven't been updated recently will still show stats computed when the last change was pushed. Making a small edit to a file is enough to purge the cache of stale data and recompute usage statistics.

*drops the mic* 🎤