Dependencies are sometimes not processed in the correct order

[JLPLabs]

Context:

Writing a parser in Teal, using Cyan to build it.

• token.tl -- defines tokens and map strings to/from integers
• parser.tl -- scans source code and emits tokens

Versions:

> tl --version
0.15.1
> cyan version
      Info Cyan version: 0.3.0
       ... Teal version: 0.15.1
       ...  Lua version: Lua 5.4

I am not redeclaring globals. Teal agrees (and often Cyan agrees).

Teal's view:

> tl check src/parser.tl 
========================================
Type checked src/parser.tl
0 errors detected -- you can use:

   tl run src/parser.tl

       to run src/parser.tl as a program

   tl gen src/parser.tl

       to generate parser.lua
> tl check src/token.tl 
========================================
Type checked src/token.tl
0 errors detected -- you can use:

   tl run src/token.tl

       to run src/token.tl as a program

   tl gen src/token.tl

       to generate token.lua

Cyan's view:

ebc> cyan build
      Info Type checked src/token.tl
      Info Type checked src/parser.tl
      Info Wrote build/token.lua
      Info Wrote build/parser.lua

Sometimes Cyan Disagrees.

Say I modify the token.tl file to add a new token (but NOT change any types). Sometimes Cyan complains as shown:

> cyan build
      Info Type checked src/parser.tl
     Error 2 type errors in src/token.tl
       ... src/token.tl 37:8
       ...    37 | global TOK <total>: {Tokstr:TokenType} = {
       ...       |        ^^^
       ...       | cannot reassign to <total> global: TOK
       ... 
       ... src/token.tl 61:8
       ...    61 | global TOKSTR: {TokenType:Tokstr} = {
       ...       |        ^^^^^^
       ...       | cannot redeclare global with a different type: previous type of TOKSTR is {TokenType : Tokstr}

One way to "fix" the error is to change the type name from "TOKSTR" to "TOK_STR", save the file, build it (the error is gone), go back in and rename back to "TOKSTR", build it (the error is still gone).

Issue is related to case-sensitive differences?

I have an enum named 'Tokstr'. I have a map named 'TOKSTR'.

> grep -in tokstr src/token.tl 
8:local type Tokstr = enum
37:global TOK <total>: {Tokstr:TokenType} = {
61:global TOKSTR: {TokenType:Tokstr} = {

I can post the token file if that is useful.

[hishamhm]

@JLPLabs thank you for the report!

@euclidianAce since the behavior seems consistent on the Teal side, could you check first whether it's a bug in Cyan, or in the Teal API? Thank you in advance!

[euclidianAce]

hmm, if i had to guess this would probably be cyan misusing the teal api. Since its types aren't exposed I have to do a lot of casting to custom types and am probably misreporting something?

@JLPLabs the files (or a minimal reproduction) would be nice to have for testing

[JLPLabs]

Here you go. Please let me know if you need more information.

tlconfig.lua

return {
   build_dir   = "build",
   source_dir  = "src",
   include_dir = {"src"},
}

src/token.tl

-- token.tl
-- Defines tokens used by parser and compiler.
-- Establishes globals:
--   * type TokenType
--   * TOK.x -- maps TOK.x to integer 
--   * record type Token

local type Tokstr = enum
  -- single character tokens
  "LEFT_PAREN" "RIGHT_PAREN"             -- ()
  "LEFT_BRACKET" "RIGHT_BRACKET"         -- []
  "LEFT_BRACE"   "RIGHT_BRACE"           -- {}

  "MINUS" "PLUS" "SLASH" "STAR" "UNDER" "COMMA"

  -- one or two character tokens
  "BANG"    "BANG_EQUAL"
  "EQUAL"   "EQUAL_EQUAL"
  "GREATER" "GREATER_EQUAL"
  "LESS"    "LESS_EQUAL"

  -- literals
  "IDENTIFIER" "STRING" "NUMBER"

  -- reserved
  "AND"   "BOX"   "FALSE" "I"
  "OR"    "OBS"   "P"     "PRINT"
  "RAND"  "RANDI" "RANDN" "RANDSEED"
  "SCENE" "TRUE"

  -- support
  "ERROR" "EOF"
end

global type TokenType = integer

global TOK <total>: {Tokstr:TokenType} = {
  ["LEFT_PAREN"]   =  4, ["RIGHT_PAREN"]   =  5,
  ["LEFT_BRACKET"] =  6, ["RIGHT_BRACKET"] =  7,
  ["LEFT_BRACE"]   =  8, ["RIGHT_BRACE"]   =  9,

  ["MINUS"]        = 10, ["PLUS"]          = 11,
  ["SLASH"]        = 12, ["STAR"]          = 13,
  ["UNDER"]        = 14, ["COMMA"]         = 15,

  ["BANG"]         = 20, ["BANG_EQUAL"]    = 21,
  ["EQUAL"]        = 22, ["EQUAL_EQUAL"]   = 23,
  ["GREATER"]      = 24, ["GREATER_EQUAL"] = 25,
  ["LESS"]         = 26, ["LESS_EQUAL"]    = 27,

  ["IDENTIFIER"]   = 30, ["STRING"]        = 31, ["NUMBER"] = 32,

  ["AND"]   =  40, ["BOX"]   =  45, ["FALSE"] = 50, ["I"]        = 55,
  ["OR"]    =  60, ["OBS"]   =  65, ["P"]     = 70, ["PRINT"]    = 75,
  ["RAND"]  =  80, ["RANDI"] =  85, ["RANDN"] = 90, ["RANDSEED"] = 95,
  ["SCENE"] = 100, ["TRUE"]  = 105,

  ["ERROR"] = 220, ["EOF"]   = 221
}

global TOKSTR: {TokenType:Tokstr} = {
  [4]  = "LEFT_PAREN",     [5]  = "RIGHT_PAREN",
  [6]  = "LEFT_BRACKET",   [7]  = "RIGHT_BRACKET",
  [8]  = "LEFT_BRACE",     [9]  = "RIGHT_BRACE",

  [10] = "MINUS",         [11] = "PLUS",
  [12] = "SLASH",         [13] = "STAR",          
  [14] = "UNDER",         [15] = "COMMA",

  [20] = "BANG",          [21] = "BANG_EQUAL",
  [22] = "EQUAL",         [23] = "EQUAL_EQUAL",
  [24] = "GREATER",       [25] = "GREATER_EQUAL",
  [26] = "LESS",          [27] = "LESS_EQUAL",

  [30] = "IDENTIFIER",    [31] = "STRING",   [32] = "NUMBER",

  [40]  = "AND",   [45]  = "BOX",   [50] = "FALSE", [55] = "I",
  [60]  = "OR",    [65]  = "OBS",   [70] = "P",     [75] = "PRINT",
  [80]  = "RAND",  [85]  = "RANDI", [90] = "RANDN", [95] = "RANDSEED",
  [100] = "SCENE", [105] = "TRUE",

  [220] = "ERROR", [221] = "EOF",
}

global type Token = record
  ttype:  TokenType
  start:  integer
  length: integer
  line:   integer
  msg:    string
end

src/parser.tl (the small portion that refers to TOKSTR)

-- parser.tl
-- Given source code, returns tokens.

require "common"
require "token"

...

    local str = TOKSTR[token.ttype]
    if str ~= nil then
      printf("%13s '%s'\n", str, p.src:sub(i,j))
    else

[euclidianAce]

Okay, i've pinned down the issue. There is an intermittent bug where cyan for some reason doesn't process dependencies in the correct order, therefore parser.tl may get checked first (which pulls in token.tl to get checked), then token.tl gets checked again, but the global was already added to the environment, so teal sees it as a redeclaration.

Will try to narrow the root cause of this down at a later time, thanks for the report and example!