Operater optimization

[shigma]

Some of the changes are proposals and others are features.

1. [Adjustment] Add context declarations into expressions for supporting following condition:

   entranceFunction = Module[{internalFunc},
   internalFunc[foo_] := bar;
   Return[doSomethingWith[internalFunc]];
   ]

2. [Chore] Delete unnecessary regex grouping and quotation marks.
3. [Feature] Support numbers with base, scientific notation, precision or accuracy. See here.

   11^^11.11`11*^11
   (* ^^^^^^^^^^^^^^^^^ constant.numeric *)

4. [Adjustment] Display built_in_options with variable.parameter. I know this is not that accurate, but much better than variable.function. Any ideas for this scope are welcomed.
5. [Feature] Support shorthand syntax for Out.

   %%%
   (*^^^ storage.type.Out *)
   %12
   (*^^^ storage.type.Out *)

6. [Feature] Support operator for Power.
7. [Adjustment] Display & with variable.function instead of keyword.operator. I think this would be more appropriate.
8. [Break] Remove the rule below. I don't know what does a prefix have to do with "constant".

   match: ((?:{{symbol}}`)*(\${{symbol}}))(?=\s*:?=)
   scope: entity.name.constant.wolfram

9. [Feature] Support shorthand syntax for Infix.

   a ~ f ~ b ~ g ~ c
   (*    ^ variable.function *)
   (*            ^ variable.function *)

10. [Feature] Support operator for Definition and FullDefinition.
11. [Adjustment] Better display with specific functions, for example:

    Sum[x, {x, 1, 100}]
    (*        ^ variable.parameter *)
    Map[func, list]
    (*    ^^^^ variable.function *)
    TakeWhile[list, func]
    (*                ^^^^ variable.function *)

    These functions are currently listed as variables. With further development of branch new-build-system, we can easily generate them within lines of code.

[shigma]

Updates

1. Optimize number pattern, support scope names with base, precision, accuracy and exponent.

   11`
   (*^^ constant.numeric *)
   (*  ^ constant.numeric.precision *)
   11.`11
   (*^^^ constant.numeric *)
   (*   ^^^ constant.numeric.precision *)
   .11``
   (*^^^ constant.numeric *)
   (*   ^^ constant.numeric.accuracy *)
   11.11``
   (*^^^^^ constant.numeric *)
   (*     ^^ constant.numeric.accuracy *)
   11^^1a
   (*^^^^ constant.numeric.base *)
   (*    ^^ constant.numeric *)
   11.11*^-11
   (*^^^^^ constant.numeric.wolfram *)
   (*     ^^^^^ constant.numeric.exponent *)

2. Optimize parameter pattern, support scope names with blank, head and default (which has not been supported in the past).

   _.
   (*^^ variable.parameter.default *)
   
   var_head
   (*^^^^^^^^ meta.parameter *)
   (*^^^ variable.parameter *)
   (*   ^ variable.parameter.blank *)
   (*    ^^^^ variable.parameter.head *)
   
   var__head : foo
   (*          ^ meta.pattern keyword.operator.Optional *)
   
   var___head ? EvenQ
   (*           ^ meta.pattern keyword.operator.PatternTest *)
   
   var: patt ? EvenQ : foo
   (*^^^ variable.parameter *)
   (*   ^ keyword.operator.Pattern *)
   (*     ^^^^^^^^^^^^ meta.pattern *)
   (*     ^^^^ variable.other*)
   (*          ^ keyword.operator.PatternTest *)
   (*            ^^^^^ variable.function *)
   (*                  ^ keyword.operator.Optional *)
   (*                    ^^^ variable.other *)

3. Remove function classifications, which will be discussed in another PR.
4. Add DynamicModule into scoping_functions.
5. Merge pattern-short into variables for better semantics.
6. Fix a pattern capture mistake.
7. Add some comments.

[chere005]

Re 7 I would keep & an operator.. There's some good features in this PR, but a lot to go through all at once. Smaller PRs might make it easier to get things merged since we can't approve it until all changes are OK. I see good work on supporting : as Optional, I haven't had a chance to check this out and try it yet but does it work for recognizing foo : (_Integer | _String) as pattern? I'll try to look at this more carefully in the next couple weeks.

[chere005]

I did install it to start looking at things actually... first issue:

If[rowStart > 1,
    rowList = Replace[rowList, x_Integer :> x - rowStart + 1, {1}];
];

[chere005]

Another issue:

Module[{data, fileMode, dims, enc, dataStrm, ignoreEmptyLines, summary},

    (* Load LibraryLink functions *)
    If[!loadAdapter[],
        Return@CreateSVToolsException["AdapterNotLoaded"];
    ];

    enc                 = ValidateOption[True, SVGetDimensions, CharacterEncoding, Alternatives @@ Join[System`$CharacterEncodings, {None, Automatic, "UTF8ISOLatin1"}], opts];
    ignoreEmptyLines    = ValidateOption[True, SVGetDimensions, "IgnoreEmptyLines", True | False, opts];
    td                  = ValidateOption[True, SVGetDimensions, "TextDelimiters", t_String /; 0 <= StringLength[t] <= 1, opts];
    summary             = ValidateOption[True, SVGetDimensions, "Summary", _?BooleanQ, opts];
]

[chere005]

Re 7 I would keep & an operator.. There's some good features in this PR, but a lot to go through all at once. Smaller PRs might make it easier to get things merged since we can't approve it until all changes are OK. I see good work on supporting : as Optional, I haven't had a chance to check this out and try it yet but does it work for recognizing foo : (_Integer | _String) as pattern? I'll try to look at this more carefully in the next couple weeks.

I tested the : as pattern and it works

[chere005]

This is causing MAJOR performance detriments on my machine. I couldn't even open a file with 10k+ lines of code, but when I switched to the plugin on master it loaded instantly.