Add Modula-2 and Modula-3 lexer

[1997tn]

I asked on Lexilla https://github.com/ScintillaOrg/lexilla/issues/125 but it will take very long for them to actually make the lexer. I know TextAdept supports custom lexers in Lua but I have no knowledge about Lua, please add the custom lexers for Modula-2 and Modula-3. Thank you.

[orbitalquark]

Thanks for the idea. I have no knowledge of Modula-2 and Modula-3, so this will take a while for me to do when I find the time.

[1997tn]

Oh, I don't know there is already a lexer for Modula. Sorry everyone :(

https://github.com/ScintillaOrg/lexilla/blob/master/lexers/LexModula.cxx

[1997tn]

There is some problems with it, though: https://github.com/ScintillaOrg/lexilla/issues/128

[1997tn]

@orbitalquark Hi. Scintilla already has lexer for Modula (both Modula-2 and Modula-3), but why I have no syntax highlighting when opening a Modula-3 .i3 or .m3 source file?

[orbitalquark]

Textadept uses Scintillua, which uses Lua based lexers, not Scintilla ones. Scintillua currently not have a Modula lexer.

[paaguti]

@1997tn: Modula sort of kicked in a dormant neuron. If memory serves me well, it was somehow related to Pascal, right? Maybe you could start from the Pascal lexer and evolve that to a Modula-2 lexer.

Just my .2 cents, /PA

[paaguti]

Ten minutes worth of work. Put this in .textadept/lexers/modula.lua and continue from there

-- Copyright 2006-2023 Mitchell. See LICENSE.
-- Pascal LPeg lexer.

local lexer = require('lexer')
local token, word_match = lexer.token, lexer.word_match
local P, S = lpeg.P, lpeg.S

local lex = lexer.new('modula')

-- Whitespace.
lex:add_rule('whitespace', token(lexer.WHITESPACE, lexer.space^1))

-- Keywords.
lex:add_rule('keyword', token(lexer.KEYWORD, word_match({
  'and', 'array', 'as', 'at', 'asm', 'begin', 'case', 'class', 'const', 'constructor', 'destructor',
  'dispinterface', 'div', 'do', 'downto', 'else', 'end', 'except', 'exports', 'file', 'final',
  'finalization', 'finally', 'for', 'from', 'function', 'goto', 'if', 'implementation', 'import', 'in', 'inherited',
  'initialization', 'inline', 'interface', 'is', 'label', 'mod', 'module', 'not', 'object', 'of', 'on', 'or',
  'out', 'packed', 'procedure', 'program', 'property', 'raise', 'record', 'repeat',
  'resourcestring', 'set', 'sealed', 'shl', 'shr', 'static', 'string', 'then', 'threadvar', 'to',
  'try', 'type', 'unit', 'unsafe', 'until', 'uses', 'var', 'while', 'with', 'xor', 'absolute',
  'abstract', 'assembler', 'automated', 'cdecl', 'contains', 'default', 'deprecated', 'dispid',
  'dynamic', 'export', 'external', 'far', 'forward', 'implements', 'index', 'library', 'local',
  'message', 'name', 'namespaces', 'near', 'nodefault', 'overload', 'override', 'package', 'pascal',
  'platform', 'private', 'protected', 'public', 'published', 'read', 'readonly', 'register',
  'reintroduce', 'requires', 'resident', 'safecall', 'stdcall', 'stored', 'varargs', 'virtual',
  -- 'write', 'writeln',
  'writeonly', --
  'false', 'nil', 'self', 'true'
}, true)))

-- Functions.
lex:add_rule('function', token(lexer.FUNCTION, word_match({
  'chr', 'ord', 'succ', 'pred', 'abs', 'round', 'trunc', 'sqr', 'sqrt', 'arctan', 'cos', 'sin',
  'exp', 'ln', 'odd', 'eof', 'eoln',
  'float', 'max', 'min'
}, true)))

-- Types.
lex:add_rule('type', token(lexer.TYPE, word_match({
  'shortint', 'byte', 'char', 'smallint', 'integer', 'word', 'longint', 'cardinal', 'boolean',
  'bytebool', 'wordbool', 'longbool', 'real', 'single', 'double', 'extended', 'comp', 'currency',
  'pointer'
}, true)))

-- Strings.
--lex:add_rule('string', token(lexer.STRING, S('uUrR')^-1 * lexer.range("'", true, false)))
local modula_string = lexer.range("\"", true, false)
local modula_char = S("'")*P(1)*S("'")
lex:add_rule('string', token(lexer.STRING, modula_string + modula_char))

-- Identifiers.
lex:add_rule('identifier', token(lexer.IDENTIFIER, lexer.word))

-- Comments.
--local line_comment = lexer.to_eol('//', true)
--local bblock_comment = lexer.range('{', '}')
local pblock_comment = lexer.range('(*', '*)')
--lex:add_rule('comment', token(lexer.COMMENT, line_comment + bblock_comment + pblock_comment))
lex:add_rule('comment', token(lexer.COMMENT, pblock_comment))

-- Numbers.
lex:add_rule('number', token(lexer.NUMBER, lexer.number * S('LlDdFf')^-1))

-- Operators.
lex:add_rule('operator', token(lexer.OPERATOR, S('.,;^@:=<>+-/*()[]')))

lexer.property['scintillua.comment'] = '//'

return lex

Of course, you know more Modula than what I remember, so you might clean it up and expand it

Ahh, and associate this with the appropriate file extensions ;-) /PA

[1997tn]

I could manage the keywords but I don't understand the regular expressions. Your memory didn't serve you well. On Modula, it doesn't use single quotes for strings, single quotes is only used for chars, strings used double quotes. Modula only supports one kind of comment, ( and ), e.g: (* Foo *). It's used for both single line or multiline comment.

http://modula3.github.io/cm3/reference/complete/html/2_6_5Text_character.html

http://modula3.github.io/cm3/reference/complete/html/2_8_4Comments.html

[1997tn]

Here is a Modula-2 lang report I found on the internet:

https://www.modula2.org/downloads//wirth-modula2/Wirth_Modula2.pdf

I have not used Modula-2 for a long time, now I use only Modula-3.

Modula-3 is a much simpler language:

http://modula3.github.io/cm3/reference/complete/html/Modula_3_Language_definitio.html

Note: Modula-3's numeric literals syntax is different from Pascal:

http://modula3.github.io/cm3/reference/complete/html/2_6_4Numeric_literals.html

[paaguti]

I could manage the keywords but I don't understand the regular expressions. Your memory didn't serve you well.

My last contact with Modula-2 was on a CP/M-80 system... Guess when that was :-)

On Modula, it doesn't use single quotes for strings, single quotes is only used for chars, strings used double quotes.

Then https://www.modula2.org is wrong ;-)

Modula only supports one kind of comment, ( and ), e.g: (* Foo *). It's used for both single line or multiline comment.

Yeap, that's why the are comment types are commented(lines that start with a -- in Lua)

http://modula3.github.io/cm3/reference/complete/html/2_6_5Text_character.html http://modula3.github.io/cm3/reference/complete/html/2_8_4Comments.html

Fine, give me a couple of days and I'll assemblesomething workable ;-)

[paaguti]

Here is a Modula-2 lang report I found on the internet:

https://www.modula2.org/downloads//wirth-modula2/Wirth_Modula2.pdf

I have not used Modula-2 for a long time, now I use only Modula-3.

Modula-3 is a much simpler language:

http://modula3.github.io/cm3/reference/complete/html/Modula_3_Language_definitio.html

Note: Modula-3's numeric literals syntax is different from Pascal:

http://modula3.github.io/cm3/reference/complete/html/2_6_4Numeric_literals.html

Thanks for the references.

[1997tn]

Modula-2 source file extension: .def, .mod Modula-3 source file extension: .i3, .m3, .ig, .mg

[paaguti]

Still alpha, but take a loot at https://github.com/textadept-modula-lexers. Comments welcome

[1997tn]

Still alpha, but take a loot at https://github.com/textadept-modula-lexers. Comments welcome

Thank you. Btw, the correct link is https://github.com/paaguti/textadept-modula-lexers

[paaguti]

Yup, thx... @Mitchell: can you add them to Scintillua, or would you prefer a PR?

[orbitalquark]

I will add them when I get the chance. Thanks!

[1997tn]

I will add them when I get the chance. Thanks!

No. Don't add them. He didn't test to see if his lexers worked or not. His lexers were unable to be loaded into TextAdept because of his typos (very trivial, just missing of brackets, otherwise I'm unable to fix it as I have no knowledge about Lua). After I fixed the typos, the lexers loaded but not work right. His rules for hex numbers and real numbers in E notation on Modula-2 lexer doesn't work (they are not correctly highlighted). The Modula-3 lexer is the Modula-2 lexer with the keywords updated for Modula-3. The numeric literals syntax is still of Modula-2. As he doesn't answer bug reports so I will report the bugs here.

He always insists on me having to send him the PR. It's not always that simple. If I know what to do I should do it already. I don't understand Lua and especially LPEG. I'm a Modula-3 user but I don't understand EBNF grammar, so there could be some language corner cases I'm not yet know. e.g: a keyword could be both a data type and a function, who know? Sometimes I really need someone to ask and discuss if I should do it this way or that way or I understand and do it right or not. I need someone to help me to do things correctly, not that I'm lazy I want you to do all of this for me. Please make it clear.

p/s: if he wants PR, here is his PR: https://github.com/paaguti/textadept-modula-lexers/pull/4. Only correct the typos, other issues remain.

[orbitalquark]

By criticizing the work of someone trying to help, you are not doing yourself any favors and are unlikely to find others willing to help you further. Please be more civil.

[1997tn]

By criticizing the work of someone trying to help, you are not doing yourself any favors and are unlikely to find others willing to help you further. Please be more civil.

What do you mean by criticizing? I just tell the fact. If the fact is unfit in your ears then it's OK. I'm more than appreciated of his help and his kindness and he will be my hero if he managed to help me further by answering my bug reports. The fact is I don't know what to do. He wanted me to make PR? OK. But first tell me what to do. I don't want him to do everything for me, but I need pointers. But not, he decided to stop. His work is half-assed. If you find it's fit to include such a half-assed work into your text editor (which I assumed is just another half-assed work), you are OK.

From the beginning, you are just talk. You have given no aid to me, even a pointer about how to do thing. He stepped up to help me. I'm very surprised and thankful to him. He also told me to do bug reports and PR as he welcomed it. But when I actually do bug reports, what I received is just me should fork and make PR. If I know what and how to do it, I should already do it without the help of any of you. I have read the document of LPEG and the fact is I have no idea why his code for Modula-2 hex and real in E notation failed to work. I need your aid, I need your pointers, I need to discuss with you to solve the problem. So far, you didn't give me anything other than criticizing me. You are a hypocrite!