wincent / docvim

Documentation generator for Vim plug-ins
MIT License
25 stars 0 forks source link

Add support for extracting documentation from Lua files #42

Open wincent opened 4 years ago

wincent commented 4 years ago

Seeing as I am porting a bit of stuff from Vimscript to Lua lately... I don't really want to have to maintain .vim files just to hold documentation comments, although I will if I have to.

Related: #43

wincent commented 3 years ago

Just leaving this here in case it proves useful:

Also:

wincent commented 3 years ago

Also, from "The Complete Syntax of Lua":

chunk ::= {stat [`;´]} [laststat [`;´]]

block ::= chunk

stat ::=  varlist `=´ explist | 
         functioncall | 
         do block end | 
         while exp do block end | 
         repeat block until exp | 
         if exp then block {elseif exp then block} [else block] end | 
         for Name `=´ exp `,´ exp [`,´ exp] do block end | 
         for namelist in explist do block end | 
         function funcname funcbody | 
         local function Name funcbody | 
         local namelist [`=´ explist] 

laststat ::= return [explist] | break

funcname ::= Name {`.´ Name} [`:´ Name]

varlist ::= var {`,´ var}

var ::=  Name | prefixexp `[´ exp `]´ | prefixexp `.´ Name 

namelist ::= Name {`,´ Name}

explist ::= {exp `,´} exp

exp ::=  nil | false | true | Number | String | `...´ | function | 
         prefixexp | tableconstructor | exp binop exp | unop exp 

prefixexp ::= var | functioncall | `(´ exp `)´

functioncall ::=  prefixexp args | prefixexp `:´ Name args 

args ::=  `(´ [explist] `)´ | tableconstructor | String 

function ::= function funcbody

funcbody ::= `(´ [parlist] `)´ block end

parlist ::= namelist [`,´ `...´] | `...´

tableconstructor ::= `{´ [fieldlist] `}´

fieldlist ::= field {fieldsep field} [fieldsep]

field ::= `[´ exp `]´ `=´ exp | Name `=´ exp | exp

fieldsep ::= `,´ | `;´

binop ::= `+´ | `-´ | `*´ | `/´ | `^´ | `%´ | `..´ | 
         `<´ | `<=´ | `>´ | `>=´ | `==´ | `~=´ | 
         and | or

unop ::= `-´ | not | `#´

plus:

Names (also called identifiers) in Lua can be any string of letters,
digits, and underscores, not beginning with a digit. This coincides with
the definition of names in most languages. (The definition of letter
depends on the current locale: any character considered alphabetic by
the current locale can be used in an identifier.) Identifiers are used
to name variables and table fields.

The following keywords are reserved and cannot be used as names:

     and       break     do        else      elseif
     end       false     for       function  if
     in        local     nil       not       or
     repeat    return    then      true      until     while

Lua is a case-sensitive language: and is a reserved word, but And and
AND are two different, valid names. As a convention, names starting
with an underscore followed by uppercase letters (such as _VERSION) are
reserved for internal global variables used by Lua.

The following strings denote other tokens:

     +     -     *     /     %     ^     #
     ==    ~=    <=    >=    <     >     =
     (     )     {     }     [     ]
     ;     :     ,     .     ..    ...

Literal strings can be delimited by matching single or double quotes,
and can contain the following C-like escape sequences: '\a' (bell),
'\b' (backspace), '\f' (form feed), '\n' (newline), '\r' (carriage
return), '\t' (horizontal tab), '\v' (vertical tab), '\\' (backslash),
'\"' (quotation mark [double quote]), and '\'' (apostrophe [single
quote]). Moreover, a backslash followed by a real newline results in a
newline in the string. A character in a string can also be specified
by its numerical value using the escape sequence \ddd, where ddd is a
sequence of up to three decimal digits. (Note that if a numerical escape
is to be followed by a digit, it must be expressed using exactly three
digits.) Strings in Lua can contain any 8-bit value, including embedded
zeros, which can be specified as '\0'.

Literal strings can also be defined using a long format enclosed by long
brackets. We define an opening long bracket of level n as an opening
square bracket followed by n equal signs followed by another opening
square bracket. So, an opening long bracket of level 0 is written as
[[, an opening long bracket of level 1 is written as [=[, and so on. A
closing long bracket is defined similarly; for instance, a closing long
bracket of level 4 is written as ]====]. A long string starts with an
opening long bracket of any level and ends at the first closing long
bracket of the same level. Literals in this bracketed form can run for
several lines, do not interpret any escape sequences, and ignore long
brackets of any other level. They can contain anything except a closing
bracket of the proper level.

For convenience, when the opening long bracket is immediately followed
by a newline, the newline is not included in the string. As an example,
in a system using ASCII (in which 'a' is coded as 97, newline is coded
as 10, and '1' is coded as 49), the five literal strings below denote
the same string:

     a = 'alo\n123"'
     a = "alo\n123\""
     a = '\97lo\10\04923"'
     a = [[alo
     123"]]
     a = [==[
     alo
     123"]==]

A numerical constant can be written with an optional decimal part and
an optional decimal exponent. Lua also accepts integer hexadecimal
constants, by prefixing them with 0x. Examples of valid numerical
constants are

     3   3.0   3.1416   314.16e-2   0.31416E1   0xff   0x56

A comment starts with a double hyphen (--) anywhere outside a
string. If the text immediately after -- is not an opening long
bracket, the comment is a short comment, which runs until the end
of the line. Otherwise, it is a long comment, which runs until the
corresponding closing long bracket. Long comments are frequently used to
disable code temporarily.
wincent commented 3 years ago

For comparison, here is a slightly more comprehensive example (from the 5.3 manual; the one above is from the 5.1 manual) — but note, Neovim uses LuaJIT, which means 5.1.

chunk ::= block

block ::= {stat} [retstat]

stat ::=  ‘;’ | 
         varlist ‘=’ explist | 
         functioncall | 
         label | 
         break | 
         goto Name | 
         do block end | 
         while exp do block end | 
         repeat block until exp | 
         if exp then block {elseif exp then block} [else block] end | 
         for Name ‘=’ exp ‘,’ exp [‘,’ exp] do block end | 
         for namelist in explist do block end | 
         function funcname funcbody | 
         local function Name funcbody | 
         local namelist [‘=’ explist] 

retstat ::= return [explist] [‘;’]

label ::= ‘::’ Name ‘::’

funcname ::= Name {‘.’ Name} [‘:’ Name]

varlist ::= var {‘,’ var}

var ::=  Name | prefixexp ‘[’ exp ‘]’ | prefixexp ‘.’ Name 

namelist ::= Name {‘,’ Name}

explist ::= exp {‘,’ exp}

exp ::=  nil | false | true | Numeral | LiteralString | ‘...’ | functiondef | 
         prefixexp | tableconstructor | exp binop exp | unop exp 

prefixexp ::= var | functioncall | ‘(’ exp ‘)’

functioncall ::=  prefixexp args | prefixexp ‘:’ Name args 

args ::=  ‘(’ [explist] ‘)’ | tableconstructor | LiteralString 

functiondef ::= function funcbody

funcbody ::= ‘(’ [parlist] ‘)’ block end

parlist ::= namelist [‘,’ ‘...’] | ‘...’

tableconstructor ::= ‘{’ [fieldlist] ‘}’

fieldlist ::= field {fieldsep field} [fieldsep]

field ::= ‘[’ exp ‘]’ ‘=’ exp | Name ‘=’ exp | exp

fieldsep ::= ‘,’ | ‘;’

binop ::=  ‘+’ | ‘-’ | ‘*’ | ‘/’ | ‘//’ | ‘^’ | ‘%’ | 
         ‘&’ | ‘~’ | ‘|’ | ‘>>’ | ‘<<’ | ‘..’ | 
         ‘<’ | ‘<=’ | ‘>’ | ‘>=’ | ‘==’ | ‘~=’ | 
         and | or

unop ::= ‘-’ | not | ‘#’ | ‘~’

And:

It ignores spaces (including new lines) and comments between lexical
elements (tokens), except as delimiters between names and keywords.

Names (also called identifiers) in Lua can be any string of letters,
digits, and underscores, not beginning with a digit and not being a
reserved word. Identifiers are used to name variables, table fields, and
labels.

The following keywords are reserved and cannot be used as names:

     and       break     do        else      elseif    end
     false     for       function  goto      if        in
     local     nil       not       or        repeat    return
     then      true      until     while

The following strings denote other tokens:

     +     -     *     /     %     ^     #
     &     ~     |     <<    >>    //
     ==    ~=    <=    >=    <     >     =
     (     )     {     }     [     ]     ::
     ;     :     ,     .     ..    ...

A short literal string can be delimited by matching single or double
quotes, and can contain the following C-like escape sequences: '\a'
(bell), '\b' (backspace), '\f' (form feed), '\n' (newline), '\r'
(carriage return), '\t' (horizontal tab), '\v' (vertical tab), '\\'
(backslash), '\"' (quotation mark [double quote]), and '\'' (apostrophe
[single quote]). A backslash followed by a line break results in a
newline in the string. The escape sequence '\z' skips the following span
of white-space characters, including line breaks; it is particularly
useful to break and indent a long literal string into multiple lines
without adding the newlines and spaces into the string contents. A short
literal string cannot contain unescaped line breaks nor escapes not
forming a valid escape sequence.

We can specify any byte in a short literal string by its numeric value
(including embedded zeros). This can be done with the escape sequence
\xXX, where XX is a sequence of exactly two hexadecimal digits, or with
the escape sequence \ddd, where ddd is a sequence of up to three decimal
digits. (Note that if a decimal escape sequence is to be followed by a
digit, it must be expressed using exactly three digits.)

The UTF-8 encoding of a Unicode character can be inserted in a literal
string with the escape sequence \u{XXX} (note the mandatory enclosing
brackets), where XXX is a sequence of one or more hexadecimal digits
representing the character code point.

Literal strings can also be defined using a long format enclosed by long
brackets. We define an opening long bracket of level n as an opening
square bracket followed by n equal signs followed by another opening
square bracket. So, an opening long bracket of level 0 is written as
[[, an opening long bracket of level 1 is written as [=[, and so on. A
closing long bracket is defined similarly; for instance, a closing long
bracket of level 4 is written as ]====]. A long literal starts with an
opening long bracket of any level and ends at the first closing long
bracket of the same level. It can contain any text except a closing
bracket of the same level. Literals in this bracketed form can run
for several lines, do not interpret any escape sequences, and ignore
long brackets of any other level. Any kind of end-of-line sequence
(carriage return, newline, carriage return followed by newline, or
newline followed by carriage return) is converted to a simple newline.

For convenience, when the opening long bracket is immediately followed
by a newline, the newline is not included in the string. As an example,
in a system using ASCII (in which 'a' is coded as 97, newline is coded
as 10, and '1' is coded as 49), the five literal strings below denote
the same string:

     a = 'alo\n123"'
     a = "alo\n123\""
     a = '\97lo\10\04923"'
     a = [[alo
     123"]]
     a = [==[
     alo
     123"]==]

Any byte in a literal string not explicitly affected by the previous
rules represents itself. However, Lua opens files for parsing in text
mode, and the system file functions may have problems with some control
characters. So, it is safer to represent non-text data as a quoted
literal with explicit escape sequences for the non-text characters.

A numeric constant (or numeral) can be written with an optional
fractional part and an optional decimal exponent, marked by a letter 'e'
or 'E'. Lua also accepts hexadecimal constants, which start with 0x or
0X. Hexadecimal constants also accept an optional fractional part plus
an optional binary exponent, marked by a letter 'p' or 'P'. A numeric
constant with a radix point or an exponent denotes a float; otherwise,
if its value fits in an integer, it denotes an integer. Examples of
valid integer constants are

     3   345   0xff   0xBEBADA

Examples of valid float constants are

     3.0     3.1416     314.16e-2     0.31416E1     34e1
     0x0.1E  0xA23p-4   0X1.921FB54442D18P+1

A comment starts with a double hyphen (--) anywhere outside a
string. If the text immediately after -- is not an opening long
bracket, the comment is a short comment, which runs until the end
of the line. Otherwise, it is a long comment, which runs until the
corresponding closing long bracket. Long comments are frequently used to
disable code temporarily.