Proposal: Attributes everywhere

[stephentoub]

Background

Today, attributes have a few restrictions relevant to this conversation:

• They can only be used in places allowed by their AttributeUsage, which is itself limited to where attributes can be represented in IL. This means they can’t be used in the body of methods.
• They must derive from Attribute.
• They are always output to the MSIL, unless annotated as Conditional with an appropriate compilation constant, but that Conditional applies to the type definition, and thus any instance of that attribute abides by that same conditionality.

  Proposal

• Whereas today we have the […] syntax for attributes, I propose we introduce an additional [[…]] syntax.
• […] remains exactly as it is today, whereas [[…]] represents attribution that is source-only and will never be emitted to IL. Thus you can write:

[SomeAttribute(1, 2)]
[[SomeAttribute(3,4)]]
class C { }

and the first attribute will end up in metadata whereas the latter won’t.

• The […] continues to be limited by AttributeUsage, whereas [[…]] doesn’t. [[…]] can be used anywhere in source where we can make it syntactically unambiguous (which should be most, if not all, locations). The purpose of this is to have it represented as a special kind of syntax node in a syntax tree available to analyzers, likely as another form of trivia.
• [[…]] would also not be limited to types derived from Attribute. Arbitrary expressions could be used in the …, with those expressions showing up as children of the special syntax node in the syntax tree. For example, you could do things like:

public class NoCaptureAttribute : Attribute { }
public static class Allocs { public static void AllowCapture(params object[] args) { } }
…
[[NoCapture]] i => i * this.Value; // error from analyzer… lambda captures when it’s not allowed to
[[Allocs.AllowCapture(this)]] i => i * this.Value; // analyzer ok… lambda allowed to capture this

or:

throw new Exception( [[Issue(123)]]“message to be localized” ); // analyzer dispays info about issue from github

or:

Console.WriteLine($"The value is {[[BoxOk]]this._result}."); // analyzer warns about boxing unless it’s annotated

• Since the attribute would never be serialized to metadata, libraries could depend on “attributes” but not need to ship those attributes, making some customers more willing or able to take a dependency on an analyzer library in their source. And for cases where that’s not even possible, this could be done without special types at all, e.g.

Console.WriteLine($"The value is {[["boxok"]]this._result}.");

[bbarry]

So really these are just a fancy form of a comment that gets some type checking. It looks like they should be permitted anywhere /**/ is. Should it be a form of trivia on the syntax node?

[stephentoub]

So really these are just a fancy form of a comment that gets some type checking.

The whole contents would be transformed into a syntax tree and get the same level of checking as any other syntax tree.

Should it be a form of trivia on the syntax node?

That's what I was suggesting: "The purpose of this is to have it represented as a special kind of syntax node in a syntax tree available to analyzers, likely as another form of trivia."

[bbarry]

I meant a new kind of trivia. If it derives from SyntaxTrivia and it contains CSharpSyntaxNode (or vb?) representing an expression, does that mean refactoring code needs to look inside trivia instead of passing it along for the ride?

If you have trivia referencing this for example and a refactoring that converts a method to a static method, does it fail because of this in the attribute?

[stephentoub]

I meant a new kind of trivia.

Ah, yeah, that's what I had in mind.

does that mean refactoring code needs to look inside trivia instead of passing it along for the ride?

Yes, it would. Similar to how refactoring code today needs to look inside XML comment trivia when doing renames.

[AdamSpeight2008]

@stephentoub How would this look / work in VB? Where < > is used for attributes and not [ ]?

[stephentoub]

How would this look / work in VB?

Same idea. Instead of [[...]], maybe it could be <!...!> or or something else that's similar to VB's <...> syntax for attributes and that could be unambiguous (I expect <<...>> would be problematic due to shift operators).

[AdamSpeight2008]

@stephentoub Also we have take into account the xml literals. .<! !> would have to be check to see if it the start of a xml comment literal <!-- content -->

[bbarry]

<[[...]]> perhaps, similar to CDATA sections in xml?

[mattwar]

VB already has this feature, its the tick-bracket token.

Try it...

'[ attribute ]'

It already works!

[stephentoub]

@mattwar, cute :wink:

In any event, I appreciate the enthusiasm around choosing syntax, but I'm not particularly concerned with the exact choice of tokens, in either language. I'm confident that if we like the general idea and decided to implement it that we could find reasonable characters to use.

[khellang]

I bet @KathleenDollard has something to say about this proposal :smile:

[bondsbw]

I like the idea, but if it's not an attribute let's find some other name and preferably a quite different syntax from attributes.

What about ambiguities? How do we know that the "attribute" in the following cases applies to the lambda instead of to parameter i?

[[NoCapture]] i => i * this.Value;
[[Allocs.AllowCapture(this)]] i => i * this.Value;

[alrz]

@bondsbw ([[Attribute]] int i) => i * this.Value;?

[bondsbw]

@alrz That is still questionable, what if [[Attribute]] needs to apply to specifically to the type int instead of the parameter declaration int i?

[alrz]

@bondsbw ([[type: Attribute]] int i) => i * this.Value;? (see "Attribute Targets")

[paulomorgado]

@stephentoub,

Why can't it be a type of usage of the attribute? Or another property, like Persistence?

I guess you're trying to keep this a language-only feature without any dependencies from CLR changes.

[stephentoub]

Why can't it be a type of usage of the attribute? Or another property, like Persistence?

The intent is that you can use these in places where you can't even represent attributes in IL, and as a result you can do things with them that you can't do with attributes.

I guess you're trying to keep this a language-only feature without any dependencies from CLR changes

Yes.

but if it's not an attribute let's find some other name and preferably a quite different syntax from attributes

I personally very much like the [[...]] syntax, but I'm of course open to other suggestions... my main goal is the functionality, and if it's exposed in a different way, that's fine. As for name, again, that was just a moniker I threw on it as I thought it helped to introduce the concept. If it's confusing, it can of course be called something else, for example "expression comments".

[alrz]

"expression comments"

Smooth.

[KathleenDollard]

To state clearly some reasons between the lines for this...

• We are currently quite restricted in what we can put into attributes. The most important additions would be generics and lambdas. But this is also an opportunity for further creativity - including (just a little) JSON (or XML) (only inside "parameters" for the "attribute-things") which would allow very clean syntax for some interesting data.
• The purpose of code is to be as expressive as possible to the humans. Expressing that a notation is intended for the compiler pipeline as opposed to runtime is important in itself. This is particularly interesting with compiler coloration. I had (for a while) my hacked version with purple comments. It was truly remarkable how this concise statement helped illustrate code

Also: The known spaces (again between the lines in the description above:

• Analyzers - both to guide (method length rules) and to ignore with more granularity
• Editor support - Allowing key hints and external documentation links
• Code Generation and expansion - (yes of course)
• (potentially) optimizer hints
• Probably loads of other things smart people will think of

I am most interested in applicability to Analyzers and Code Expansion/generation. The code expansion space is remarkably broad, including things like declaring validation lambdas in place and letting your templates/expanders determine how to code that up. And I think it will make it easier to uptake both Semantic Logging/EventSource and the new Diagnostic Source.

Kathleen

[ghord]

One possible elegant usage would be hints to JIT compiler

[[Unlikely]] //branch prediction hint, also [[Likely]]
if(x < 0)
    throw new ArgumentOutOfRangeException(nameof(x));
else
{
   //following code
}

var p = [[NoInline]] Foo(x); //inlining hint, also [[Inline]]

[[Hot]] //spend more time here on code optimization - this function will run a lot
void Foo()
{
   //complicated code to follow
}

[GSPP]

@ghord True. Unfortunately, the JIT team does not seem to agree that it's optimizer will always lack the information to make certain choices. It took 4 .NET versions to get the AggressiveInlining hint into the JIT. This is one of the most basic and obviously useful hints there is.

Another important hint would be to specify that code is hot. This would raise the inliner limit, cause loops to be unrolled and trigger all other optimizations that are not usually done for code size concerns.

The JIT can't statically know what code is hot and what code is cold. Therefore, hints are helpful.

[GeirGrusom]

@ghord I think the intention is that this data is lost after compilation, so it couldn't work for JIT hinting.

[iam3yal]

Just an idea but C# uses the at (@) symbol to allow us to name things that are otherwise reserved maybe we can use this idea and use the exclamation mark (!) to hint the compiler that this isn't going to emit anything? or it absolutely needs to be verbose? :)

What I mean is something like this:

[SomeAttribute(1, 2), 
!SomeAttribute(3,4)]
class C { }

Or/and this:

[SomeAttribute(1, 2)]
[!SomeAttribute(3,4)]
class C { }

The idea is that they blend in to the code like normal attributes.

[choikwa]

While I'm in agreement that this addresses many shortcomings such as performance issues with JIT by providing "hints" (but I still fear that "hints" are workarounds for improper characteristics of the JIT), I hope that it doesn't make C# become Attribute-Driven Language where code is littered with Attributes. I disagree using Attributes where actual coding can do the job since some attributes may ever live as "hints" which is non-deterministic.

[briandrennan]

If I understood your comment about [[Issue(123)]] correctly @stephentoub, this would essentially mean I could connect an analyzer to something like my work item/issue tracking system? So, for instance, there could be a GitHub issue or Visual Studio Team Services analyzer capable of inspecting the trivia from Roslyn, and then link it to other systems? I know this is not "the point" of the feature, but I thought the idea was intriguing. Also, do you see such references as being perpetual? What sort of lifetime do you envision for the attributes?

[alrz]

The whole contents would be transformed into a syntax tree and get the same level of checking as any other syntax tree.

So why relate this to "attributes"? It could be a structured comment, e.g. markdown syntax works just fine,

// `SomeAttribute`
class X

Aside: I think there was also a proposal to support markdown for doc comments.

Alternatively, some comments could have meaning (heh).

{
  // no capture
  object F() => null;
}

This is already used in, for example, R# to disable code fixes etc.

I think the real problem here is associating comments to syntax nodes and a better API around it. Perhaps we could use triple-slash or /** */ to denote meaningful comments (that are associated to a syntax node).

{
  /// no capture
  object F() => null;
}

AFAIK we already have this for doc comments but the API is not optimized for these use cases.

[iam3yal]

@alrz The only issue with comments is it doesn't matter what style you would use to annotate the code it's one that might be used today by some people and this might have some implications in particular IDE experience:

1. People would probably want these special kind of comments to be distinguished from regular comments let's call them annotations and they would probably want them to get coloured differently so if people are already using /// and /** */ or any other style and their intention is not really to annotate their code but to comment block of codes, write notes or whatever they would be surprised that their comments are coloured differently.

2. A tool like R# may use these annotations to treat the code differently and offer all kinds of enhancements but then these people may have issues because again their comments isn't even close to be an annotation it's just a style so now tools will have to take into account that some annotations aren't really annotations but comments and this may lead to some false positives.

R# provides an alternative way to annotate the code in the form of comments only because there's no other proper way to do it and unfortunately this is a hack not a solution.

The only thing that really bothers me is the syntax it looks pretty verbose whereas comments seems terser.

So how about having a new character for annotations something like #?

Annotation as Attribute:

# [SomeAttribute]
class X

Annotation as text:

{
  # no capture
  object F() => null;
}

The benefit of this is that a tool like R# can then offer specialized intellisense for available annotations when typing #.

[alrz]

People would probably want these special kind of comments to be distinguished from regular comments let's call them annotations and they would probably want them to get coloured differently so if people are already using /// and /* / or any other style and their intention is not really to annotate their code but to comment block of codes, write notes or whatever they would be surprised that their comments are coloured differently.

/// and /** */ have already a different meaning. They denote doc comments for the subsequent declaration symbol. However, it doesn't currently work on locals, etc.

A tool like R# may use these annotations to treat the code differently and offer all kinds of enhancements but then these people may have issues because again their comments isn't even close to be an annotation it's just a style so now tools will have to take into account that some annotations aren't really annotations but comments and this may lead to some false positives.

It's up the tool to treat a particular comment differently. You can't stop that. But the main issue that this proposal is addressing is that it associates these annotations to a particular syntax node. You may use // anywhere for any purpose. But when you use /// it is a structured trivia bound to that syntax node. My suggestion doesn't change that. I'm saying that we can extend it to other syntax nodes and provide a better API to fetch associated structured trivia for any particular syntax node.

So how about having a new character for annotations something like #?

My point is that we have already a similar mechanism to do this but it just should be extended to work anywhere else and an API optimized for these use cases i.e. for analyzers to consume.

So I'd call this "structured trivia everywhere" rather than attributes everywhere or the like. :)

[alrz]

Don't get me wrong, I'm not against source-only attributes, i.e. attributes that do not emitted to the IL. I like the idea of [[Attr]] or #[Attr] where Attr is actually a type (not necessarily inherited from Attribute). It would be actually useful to have it "everywhere". But when you say [["boxok"]] it no longer makes sense for me. I think that is just a comment with special meaning. so /// box ok would also just work. Type-safety of these comments is another concern. That's where I suggested backtick syntax where you want your comment to be verified, i.e. /// `Type`.

[iam3yal]

@alrz

/// and /* / have already a different meaning. They denote doc comments for the subsequent declaration symbol. However, it doesn't currently work on locals, etc.

Comments should be used to comment out code whereas xml comments are used for documentation purposes, I think it would be inappropriate to add yet another kind of comments that are considered as annotations or reusing doc comments to denote annotations at this time.

It's up the tool to treat a particular comment differently. You can't stop that.

Vendors are abusing or exploiting comments today to enhance productivity because there's no other way to do it.

I'd expect a productivity tool to give me an intellisense for xml comments when I'm using /// and a different kind of intellisense for annotations.

Today Visual Studio colour xml comment with grey this is the opposite of what some people might expect from annotations and they may want to choose a different colour for it.

One more thing, Visual Studio automatically adds the xml summary element when you type ///.

Sorry for nagging about colours and intellisense but I think that it is a vital part to this proposal.

But the main issue that this proposal is addressing is that it associates these annotations to a particular syntax node. You may use // everywhere for any purpose. But when you use /// it is a structured trivia bound to that syntax node. My suggestion doesn't change that. I'm saying that we can extend it to other syntax nodes and provide a better API to fetch associated structured trivia for any particular syntax node.

I think that annotations mean one thing and comments mean another thing therefor we need to treat them differently and the syntax for them should be different.

If it was C# 1.0 then I'd agree that it was cool if they formalized annotations as part of comments the way they did with xml comments but it would be wrong to introduce annotations as comments today because you might end up with cases where comments are interpreted as annotations from a tooling perspective but not from the standpoint of developers.

My point is that we have already a similar mechanism to do this but it just should be extended to work anywhere else and an API optimized for these use cases i.e. for analyzers to consume.

Yes but like I've pointed out above I don't think that reusing the syntax for comments is the right approach to take here.

So I'd call this "structured trivia everywhere" rather than attributes everywhere or the like. :)

Indeed, I like it. 👍

Don't get me wrong, I'm not against source-only attributes, i.e. attributes that do not emitted to the IL. I like the idea of [[Attr]] or #[Attr] where Attr is actually a type (not necessarily inherited from Attribute). But when you say [["boxok"]] it no longer makes sense for me. I think that is just a comment with special meaning. so /// box ok would also just work. Type-safety of these comments is another concern. That's where I suggested backtick syntax where you want your comment to be verified, i.e. /// Type.

I agree, I think that they need to stand out in the code like attributes and yet be as plain as comments.

p.s. Completely forgot that # is used as preprocessor directive so dunno maybe [#...].

Just as an example for how R# language injection might look like with this:

[#language=html]
string = @"<html></html>";

Alternatively:

string x = [#language=html] @"<html></html>";

[bondsbw]

There could be use cases for a start and end tag. Maybe something like this?

[#Issue(123)]
// multiple methods which are related to this issue
[#/Issue]

[alrz]

@eyalsk #[] would not be ambiguous at all. I'd prefer it over [[]] as it is less verbose IMO. The former would look like Rust attributes (In Rust #[] applies to the next item whereas #![] applies to the item enclosing it), while the latter is from C++. Anyways, my real concern is that, what is this language=html, I mean, syntactically? OP suggests that it is a valid expression or an attribute-like type construction (which doesn't apply to your example either way). At any rate, I think it is a bad idea to allow a general expression there, it would be confusing and convoluted (even if it might make life easier to work with analyzers so you don't need to declare a type every time you want to specify a different behavior, however, I think that's against C#'s goal as a strongly-typed language). I think it should just be an attribute-like construction, then it would make sense to call it "attributes everywhere". I agree with you regarding doc comments, I think I get confused about the proposal at that point that it suggests the [["boxok"]] syntax. If it has a meaning, it should have its own type otherwise you are relying on magic strings.

[iam3yal]

@alrz Yeah I generalized because you and other mentioned comments so I said okay, let's have anything in there and let the tool decide what it is but yeah either way #[SomeAttribute] or [#SomeAttribute] look so much better than [[SomeAttribute]].

[bondsbw]

@alrz Agreed, VB and C# should utilize their respective expression/attribute syntax, and the trivia be constructed such that semantically-equivalent tags look identical to an analyzer.

[bondsbw]

Assuming these tags have such a well-defined language-based structure, perhaps they should also include analyzer-provided constraints (e.g. C#-only, can only be applied within a statement list, the parameter can only be a property name, etc.). This would imply that the end user cannot just make up ad hoc tags without writing the associated analyzer.

[KathleenDollard]

I really like the notion of "structured trivia everywhere" as a way to think of the alternative to "attributes everywhere." It's a better description than "comments."

I think it allows a better format for a relatively strict syntax. I believe there is a set of simple rules that allow syntax checking, colorization and IntelliSense, strong typing against the code and safe use of lambdas enforced by a simple (read that as fast) "structured trivia compiler" It would be most interesting if this could also provide a better experience for XML comments as a more-or-less-free tag-along features

[bondsbw]

And more strict means more room to open up the syntax in a later version. Better that, than making it wide open today and wanting to restrict it later (breaking BC).

[jnm2]

@gafter Something to move to csharplang?

[agocke]

This issue has been around for a while, but I wanted to remark that we would have a compelling use case for trimming. Right now you can annotate System.Type variables with [DynamicallyAccessedMembers] to indicate that reflection will be used against the type instance. This falls over for local variables, which can't have attributes applied to them. We propagate this information via flow analysis in the trimming tools, but sometimes this runs into problems where the output IL is very different from the source code, e.g. in nested functions and async methods.