search

Facepunch / garrysmod-issues

Garry's Mod issue tracker
145 stars 56 forks source link

Valid Lua parses on Lua start, but fails on Lua refresh #4388

Closed TiberiumFusion closed 4 years ago

TiberiumFusion commented 4 years ago

Introduction

While creating an addon, I've been blocked by what could be a defect in Gmod's Lua parser. Depending on the content of a particular Lua file of mine, Gmod will only parse it on initial server load/client lua start. Aka, the same file does NOT parse when it is reloaded from a Lua refresh. The exact same file, the exact same bytes, just suddenly fail to parse in a way that does not agree with manual syntax review and Falco's GLua Lint utility.

This is a bizarre issue that is difficult to explain in summary. The Lua file experiencing the problem is large-ish (about 800 lines), and the problem goes away when I start removing comments and/or chunks of 100% valid code. Just about any large section (>50 lines) can be removed to make the issue go away. It seems very random, like the code that I remove doesn't matter. It seems like the fact that the file is >800 lines is related to what's causing parsing to fail. So, I have to share the full file in order to demonstrate the issue.

The steps to reproduce will best show what is happening.

Steps to reproduce

Addon Setup

There are a couple of pathways to demonstrate this glitch. All of them start with the same setup, as shown below. A) Addon structure /garrysmod/addons/glitchdemo/ |----> lua/ |-------- autorun/ |------------ init.lua |-------- demo/ |------------ basecontrol.lua B) Contents of init.lua are always the same. https://pastebin.com/Ma1F8rzS

Here are some variants of the Lua file that experiences the glitchy parsing. Verison 1 is the basis that all other files derive their changes from. Version 1: https://pastebin.com/Dc0gsPMV Version 2: https://pastebin.com/9s7uZzEz Version 3: https://pastebin.com/duQhTT0W Version 4: https://pastebin.com/HYqn7j3w Version 5: https://pastebin.com/cwX0HCuu Version 6: https://pastebin.com/p6qsJXqA Version 7: https://pastebin.com/asZ65h0s

Changes from Version 1 -> Version 2: -> The large comment block from lines 63-211 is deleted.

Changes from Version 1 -> Version 3: -> The method from lines 450-457 is deleted.

Changes from Version 1 -> Version 4: -> The method from lines 778-794 is deleted.

Changes from Version 1 -> Version 5: -> The two methods from lines 242-254 are deleted.

Changes from Version 1 -> Version 6: -> The variable from line 19 is removed.

Changes from Version 1 -> Version 7: -> The fields defined by lines 19-60 are removed.

Here are a few demos using Version 1 and Version 2.

This is a good place to start, because the only change from Version 1 to Version 2 is the removal of a large comment block, which should have zero impact on the legality of the syntax. Please see the Overview section after these demos for more information on the other Versions.

Demo #.1:

  1. Before starting server, set the contents of basecontrol.lua to Version 1
  2. Start server
  3. Connect client
  4. No parsing errors in client or server console
  5. Overwrite basecontrol.lua with the exact same contents (Version 1 -> Version 1) to trigger a Lua refresh
  6. The following parsing error occurs [ERROR] addons/glitchdemo/lua/demo/basecontrol.lua:793: 'end' expected (to close 'function' at line 778) near '<eof>'
  7. Repeat as many times as desired. Each Lua refresh prints the same parse error.

Demo #.2:

  1. Before starting server, set the contents of basecontrol.lua to Version 2
  2. Start server
  3. Connect client
  4. No parsing errors in client or server console
  5. Overwrite basecontrol.lua with the exact same contents (Version 2 -> Version 2) to trigger a Lua refresh
  6. No parsing errors in client or server console
  7. Repeat as many times as desired. No parsing errors are printed.

Demo #.3:

  1. Before starting server, set the contents of basecontrol.lua to Version 1
  2. Start server
  3. Connect client
  4. No parsing errors in client or server console
  5. Overwrite basecontrol.lua with the exact same contents (Version 1 -> Version 1) to trigger a Lua refresh
  6. The following parsing error occurs [ERROR] addons/glitchdemo/lua/demo/basecontrol.lua:793: 'end' expected (to close 'function' at line 778) near '<eof>'
  7. You can repeat Step 5 anywhere from 0 to infinity times and it will not change the outcome of the next several steps.
  8. Overwrite basecontrol.lua with Version 2 (change from Version 1 -> Version 2) to trigger a Lua refresh
  9. No parsing errors in client or server console.
  10. Repeat as many times as desired. Each Lua refresh with Version 2 does not print parsing errors.

Demo #.4:

  1. Before starting server, set the contents of basecontrol.lua to Version 2
  2. Start server
  3. Connect client
  4. No parsing errors in client or server console
  5. Overwrite basecontrol.lua with the exact same contents (Version 2 -> Version 2) to trigger a Lua refresh
  6. No parsing errors in client or server console. You can repeat Step 5 anywhere from 0 to infinity times and it will not change the outcome of the next several steps.
  7. Overwrite basecontrol.lua with Version 1 (change from Version 2 -> Version 1) to trigger a Lua refresh
  8. The following parsing error occurs [ERROR] addons/glitchdemo/lua/demo/basecontrol.lua:793: 'end' expected (to close 'function' at line 778) near '<eof>'
  9. Repeat as many times as desired. Each Lua refresh with Version 1 spits the same parse error.

Other versions

You can repeat the above demos with the other file Versions I provided. The results are very consistent. See the Overview section below for the changes and behavior of each Version.

Overview

-> Version 1 works on initial boot, but fails on Lua refresh ----> Version 1 gives this error: [ERROR] addons/glitchdemo/lua/demo/basecontrol.lua:793: 'end' expected (to close 'function' at line 778) near '<eof>' -> Version 2 works both on initial boot and Lua refresh -> Version 3 works on initial boot, but fails on Lua refresh ----> Version 3 gives this error: [ERROR] addons/glitchdemo/lua/demo/basecontrol.lua:798: '=' expected near '<eof>' -> Version 4 works both on initial boot and Lua refresh -> Version 5 works both on initial boot and Lua refresh -> Version 6 works on initial boot, but fails on Lua refresh ----> Version 6 gives this error: [ERROR] addons/glitchdemo/lua/demo/basecontrol.lua:793: 'end' expected (to close 'function' at line 777) near 'en' -> Version 7 works both on initial boot and Lua refresh

All seven Versions are successfully parsed and considered valid Lua by Falco's GLua Lint utility. While this tool isn't always 100% accurate (I've seen it get something wrong before), it is at least a good cross-reference.

Environment

This glitch also occurs in x64 branch version 2020.02.18.

Summary

While this looks like a bug in Gmod's lua parser, perhaps something dealing with condition/loop nesting, this issue may actually be related to an FTP upload size bug, as suggested by @robotboy655

robotboy655 commented 4 years ago

Are you using some sort of FTP client to upload files to your server by chance?

If I remember correctly there is an issue with very large files, FTP and autorefresh (although I can't find a bug report for it) and you will notice all files that have the problem exceed 32K in size.

TiberiumFusion commented 4 years ago

@robotboy655 Yes, I am using SFTP protocol ver 3 to upload files to my server.

If this is an issue with live uploading >32KB files... ...will it be fixed? ...is there a workaround?

A note regarding this bug should be included on the new wiki site, yes? I can't find anything regarding this. Although, the new site has quite a few issues and regressions, so maybe there was a note, but it got lost? This page seems like the right place to put a warning.

thegrb93 commented 4 years ago

use the -disableluarefresh flag. I do this anyway because it prevents addons from breaking when I update them while the server is running.

TiberiumFusion commented 4 years ago

@thegrb93 As I stated in my OP, "While creating an addon..." I am developing an addon. Lua refresh saves 10-20 project hours otherwise spent waiting 1 minute to reboot the server between every file change.

For a production server, disabling Lua refresh is the (usually) correct practice. For developing an addon, especially a rather complex one, losing Lua refresh incurs a massive penalty on manhour efficiency.

thegrb93 commented 4 years ago

lua_openscript?

TiberiumFusion commented 4 years ago

@thegrb93 I've heard of lua_openscript_cl for executing clientside lua, but I haven't seen any official material on lua_openscript. But... I can guess that this is the workaround to the >32KB bug? How would I effectively use this command, though? The only material I could find is on the old maurits.tv site, which does not specify the path structure you have to provide to the command.

To test live changes in my addon (with lua refresh enabled), I re-save a single autorun file (after uploading all modified files) which is responsible for including and initializing everything. I'm working with about 50 files at the moment, and might change as many as 10 before invoking the next lua refresh to test behavior. Would I be able to run lua_openscript on this autorun file and expect everything else to Just Work™? Or do I have to use lua_openscript on every single modified file to get the server to realize it needs to reload them? If the latter, then it's hardly faster than just restarting the server.

Kefta commented 4 years ago

https://github.com/Facepunch/garrysmod-issues/issues/2996 Bo98 provided a solution in this issue.

TiberiumFusion commented 4 years ago

@Kefta Very interesting.

Bo98's idea was posted on December 30, 2016. In the 3 years since, has any progress been made on his seemingly simple solution?

robotboy655 commented 4 years ago

So it is essentially duplicate of #2996