Automatically adapt to the project environment

[sumneko]

As an embedded language, Lua is often used in various environments, which have their own API environment and Lua version. I plan to let the extension automatically adapt to these environments. For example, when I find that the project you are running is likely to be an OpenResty project, I will ask for one-click adaptation, including: Enable API definition file of OpenResty (thanks to https://github.com/CppCXY for providing, see https://github.com/sumneko/lua-language-server/discussions/389#discussioncomment-380323) Set Lua.runtime.version to LuaJIT

For this, I need to collect some commonly used environments, please help me to complete the following functions:

1. How to detect these environments (characteristics of the environment, such as specific file names in the workspace)
2. What settings need to be modified (Lua running version, disable some diagnostics, enable non-standard symbol support, etc.)
3. Environment API definition file (this workload is relatively large, you can use PR to supplement: https://github.com/sumneko/lua-language-server/tree/master/meta/3rd)

Any ideas and suggestions can be put forward, thank you!

EDIT: I have basically completed this feature, you can contribute or update third-party libraries here: https://github.com/sumneko/lua-language-server/tree/master/meta/3rd

[justarandomgeek]

How would you feel about having some mechanism for other vscode extensions to notify you "hey, this is a $thing workspace, i've got api definitions here (inside other extension, or from some other public location, not sure which would work better) for you and our runtime is X"? This would allow my Factorio debugger to provide that context to you (i already recognize the workspace for some of my own functions, and i've got somebody working on emmylua docs) and keep the workload for maintaining those more on us instead of on you.

Alternatively, you could allow extension to provide some criteria for you to recognize by (like factorio's info.json, having factoriomod debug launch config, etc) along with a the needed context (runtime, api docs, plugin.lua, etc) to use when it matches

[CelticMinstrel]

Is there any information on how to enable an API definition file?

[sumneko]

Is there any information on how to enable an API definition file?

See setting Lua.workspace.library

[CelticMinstrel]

So that's a list of "meta" files to load?

[sumneko]

So that's a list of "meta" files to load?

Yes

[CelticMinstrel]

The environment I work in does not include the built-in io library. Is there a way to specify that the io and file meta files should not be loaded?

[Ketho]

1. For World of Warcraft we use .TOC files and Lua 5.1 without for example the os and io libraries.
2. Maybe disable "undefined-global" because there are thousands of globals in the environment, but nonlazy users should try to mark them as defined global in "Lua.diagnostics.globals" to be honest.
3. Are the Environment API definition files loaded the same way as adding a path to Lua.workspace.library?

[sumneko]

3. Are the Environment API definition files loaded the same way as adding a path to Lua.workspace.library?

yes

[sewbacca]

I often use Love2D. How are you going to account for environment changes? In the case of Love2D, there is an automatic API generation, written by the author of EmmyLua. However for other environments it is not that easy, like ComputerCraft for example. What I did to get autocompletion anyway, was, to generate docs from this file and correct some mistakes manually.

EDIT:

In my case, i would be perfectly happy for installing locally my environments and could switch them via a combo box in the status bar, so i could switch from ComputerCraft to Love2D to PureLua on the fly. Currently, when i setup a project, i have to add the path variable to Lua.workspace.library

[cryi]

Hi, we use adjusted lua 5.4 with 1 possible environments. Lua 5.4 and eli. Eli adds additional APIs. I would like option to set custom environment in the project settings. Also it is common for us to work on multiple projects in parallel (single workspace). Is it possible to set somewhere that dir a and dir b are independent projects and that lua-language-server should not interfere types from each other?

[sumneko]

Hi, we use adjusted lua 5.4 with 1 possible environments. Lua 5.4 and eli. Eli adds additional APIs. I would like option to set custom environment in the project settings. Also it is common for us to work on multiple projects in parallel (single workspace). Is it possible to set somewhere that dir a and dir b are independent projects and that lua-language-server should not interfere types from each other?

If your are using VSCode, you could try multi workspace. Otherwise, it is not supported for now.

[cryi]

Thank you for quick response. That is unfortunate. How complex would it be to add support? Maybe we can optionally treat folder as workspace?

Yes I use VS Code. But multi workspace is overkill. We have multiple micro projects which acts as packages/plugins and it is expected that they will grow in numbers. So I would rather prefer way to say treat this as separate project/workspace.

[cryi]

Actually never mind. I got confused by global variables. This shouldn't be a deal breaker as long as projects share same globals. But if it is something what can be integrated into language server without too much effort. I would take a look. We would be able to switch environment by changing library paths on per project basis in that case. Would be nice to have.

[justarandomgeek]

So, is there any mechanism for another extension to provide the definitions for a runtime environment? I'd rather not have to actually clobber the users's library settings directly, and having to PR the definitions from my extension's emmylua generator (which takes in the game's doc format) into your extension seems counter productive from a maintenance standpoint. I'd really love a way to just call an API on your extension and just register things for only the current session directly from my extension!

[sumneko]

So, is there any mechanism for another extension to provide the definitions for a runtime environment? I'd rather not have to actually clobber the users's library settings directly,

When I ask the user about the application environment, I can provide an option to only modify the settings in the current session memory and not write the settings to the file. But the disadvantage is that once the user modifies the settings file, the settings in the memory will be overwritten.

and having to PR the definitions from my extension's emmylua generator (which takes in the game's doc format) into your extension seems counter productive from a maintenance standpoint. I'd really love a way to just call an API on your extension and just register things for only the current session directly from my extension!

I also don't like to include the definition files of third-party libraries directly in my project, because it will undoubtedly increase the size of this extension (some definition files are very large). A better way may be that my project only contains configuration files. When the user decides to use the configuration, I download these definition files from a certain place through the method of configuration definition, but this may be more complicated.

[sumneko]

I have basically completed this feature, you can contribute or update third-party libraries here: https://github.com/sumneko/lua-language-server/tree/master/meta/3rd

[lhemkendreis]

Hi, I really like the idea behind this feature, but I couldn't get it to work yet. :-(

When I use the keyword thisIsAnExampleWord.ifItExistsInFile.thenTryLoadThisLibrary to trigger the preconfigured work environment 'example' then I get the following message (see image). But when I click on "apply and modify settings" nothing happens except that the workspace configuration file gets a new entry "Lua.workspace.checkThirdParty": false.

None of the settings seem to be applied and no library loaded. :'-(

I also tried the preconfigured work environment 'jass' (keyword jass.common) but with the same (non-)result.

What am I doing wrong?

Thank you.

[sumneko]

@lhemkendreis I just test it and seems works well. What's your extension version? Is there any error in output?

[serg3295]

I am testing the function of automatically including libraries. I have two different libraries for which I have written two different config.lua. Then I copied the library files, config and plugin to ~/.vscode/extensions/sumneko.lua-2.2.3/server/meta/3rd/ Everything works well, the libraries are included depending on the contents of the files in the workspace.

I have a question. I tried using ---@meta at the beginning of my library files, and also tried without @meta. I did not understand what the presence or absence of this annotation affects. Could you explain its purpose in more detail?

[sumneko]

I am testing the function of automatically including libraries. I have two different libraries for which I have written two different config.lua. Then I copied the library files, config and plugin to ~/.vscode/extensions/sumneko.lua-2.2.3/server/meta/3rd/ Everything works well, the libraries are included depending on the contents of the files in the workspace.

I have a question. I tried using ---@meta at the beginning of my library files, and also tried without @meta. I did not understand what the presence or absence of this annotation affects. Could you explain its purpose in more detail?

I just updated the wiki, see https://github.com/sumneko/lua-language-server/wiki/EmmyLua-Annotations#meta

[serg3295]

I just updated the wiki, see https://github.com/sumneko/lua-language-server/wiki/EmmyLua-Annotations#meta

Thank you.

[lhemkendreis]

@sumneko

@lhemkendreis I just test it and seems works well. What's your extension version? Is there any error in output?

Thanks for the quick reply and especially for the GIF! That behavior was exactly what I expected, but just didn't happen for me. My extension version is sumneko.lua-2.2.3 and I just reinstalled it just because of this issue.

I did a new test and suddenly it did work! Then I did some further testing. The behavior seems to be as follows.

• When I don't save the workspace and just work in a folder, then it works as expected. The settings are applied to the local .vscode/settings.json file. I.e. exactly like you showed in your reply.

• When I do save the workspace in a workspace configuration file (suffix .code-workspace) then it does not work as expected. In this case it only modifies the setting in the workspace configuration file - which would be fine, except that it only applies the setting "Lua.workspace.checkThirdParty": false and nothing else :-(

Conclusion: It does work but not with saved workspaces. Is that intended?

[serg3295]

As discussed above, including user libraries directly in your extension can be inconvenient for a number of reasons. Perhaps, it might make sense to add global setting "Lua.usermeta.path" which specify path to directory usermeta. This directory will contains subdirectories with files: config.lua, plugin.lua and / library.

In this case your extension will check content of usermeta directory and process it just like/ 3rd / meta

[sumneko]

As discussed above, including user libraries directly in your extension can be inconvenient for a number of reasons. Perhaps, it might make sense to add global setting "Lua.usermeta.path" which specify path to directory usermeta. This directory will contains subdirectories with files: config.lua, plugin.lua and / library.

In this case your extension will check content of usermeta directory and process it just like/ 3rd / meta

The original intention of this function is to provide a one-key configuration method. If a person has the ability to write usermeta by himself, he can completely modify the configuration instead of relying on the one-click configuration.

Can you explain the application scenario in detail?

[serg3295]

Here is my usecase. I have two libraries esp8266 and esp32. In a project uses  either esp32 or esp8266 . I  cannot set path to both libraries  in User-> lua.workspace.library , because ones contains many functions with the same signatures. Therefore, I have to set path to the library in every workspace  separatly. When I  start new project, I copy folder .vscode with all settings to new project folder. When I open my old script  or  script from Internet, it has not contains lua.workspace.library setting, but contains keywords for automatically defining environment.

Currently, I have copied my libraries to the 3d/meta folder and when opening the file, the necessary library is automatically included. Unfortunately, after updating the extension version, my libraries will be deleted. The solution that I proposed allows to set the path to the libraries only once in the global parameter.  This is just an option for convinience. In case only file config.lua is located in 3rd/meta directory, then we can use a predefined path to the user's libraries. Perhaps there is another solution to this case and my suggestion does not make sense.

[serg3295]

I tried new feature "Lua.workspace.userThirdParty". It works fine. Thanks a lot! Yet another trick with new feature -- When starting new project you can first type in short keyword (it defined in config.lua) e.g ---esp32 into new file and environment will set up automatically.

[CelticMinstrel]

Is there any way to specify a table with a specific structure? For example, a JSON-compatible table - all keys must be strings, and values must be number, boolean, string, array, or a nested JSON-compatible table.

Looking through the documentation both here and on EmmyLua, I can't see anything relevant; but I figure it can't hurt to ask in case I missed something.

If it's not possible, then I'll have to annotate it as "table". I could make an alias but it seems that won't be shown in the tooltip so there's not much point.

Also: Is there a way to add a key-value pair with the config.lua? I want to add something to "Lua.runtime.special". And is there a way to specify that the io library (among other things) doesn't exist in my environment?

[sumneko]

Is there any way to specify a table with a specific structure? For example, a JSON-compatible table - all keys must be strings, and values must be number, boolean, string, array, or a nested JSON-compatible table.

You may use ---@alias json table<string, string|boolean|number|json> . Since there is currently no type checking, there is no benefit even if it is defined.

Also: Is there a way to add a key-value pair with the config.lua? I want to add something to "Lua.runtime.special". And is there a way to specify that the io library (among other things) doesn't exist in my environment?

Use Lua.runtime.builtin

[CelticMinstrel]

Use Lua.runtime.builtin

Almost perfect… my environment also disables "loadfile" though, which seems to be bundled up into "basic" there. But I guess it's close enough. Also, is it possible to add these in a config.lua?

[sumneko]

Also, is it possible to add these in a config.lua?

Yes, see https://github.com/sumneko/lua-language-server/blob/bc6dd9b20925d21e6def32170854b27f69ca50d9/meta/3rd/example/config.lua

[CelticMinstrel]

I saw that example file, but it doesn't show how to add entries to a "dictionary" in the settings - only a "scalar" (the runtiime) or an "array" (the globals). Would you mind adding an example of how to add something to a dictionary setting such as "Lua.runtime.special" or "Lua.runtime.builtin"?

[sumneko]

my environment also disables "loadfile" though, which seems to be bundled up into "basic" there.

There is currently no good way. I now have a setting that if a function has multiple definitions, as long as one is not marked as deprecated, then you can use this function. Maybe I can reverse the setting, as long as a definition is marked as deprecated, then you can not use this function, maybe this can meet your needs.

[sumneko]

I saw that example file, but it doesn't show how to add entries to a "dictionary" in the settings - only a "scalar" (the runtiime) or an "array" (the globals). Would you mind adding an example of how to add something to a dictionary setting such as "Lua.runtime.special" or "Lua.runtime.builtin"?

You asked me down, I will add this feature now.

[CelticMinstrel]

Maybe I can reverse the setting, as long as a definition is marked as deprecated, then you can not use this function, maybe this can meet your needs.

Sure. I don't really mind if this isn't exactly perfect. I even decided to give up on specifying the table structure and just declare it as a "class" with no members.

[CelticMinstrel]

After writing luadoc annotations for the entirety of the API I work with, I've noticed a couple of things that seem to be missing in the luadoc syntax, but is this the correct place to open a GitHub issue for it or is there an EmmyLua repository somewhere that would be a better place to open such an issue? There's a lot of results searching for EmmyLua on GitHub so I'm not sure which one is correct, if any.

[sumneko]

After writing luadoc annotations for the entirety of the API I work with, I've noticed a couple of things that seem to be missing in the luadoc syntax, but is this the correct place to open a GitHub issue for it or is there an EmmyLua repository somewhere that would be a better place to open such an issue? There's a lot of results searching for EmmyLua on GitHub so I'm not sure which one is correct, if any.

You may just open issue here. The current LuaDoc is not exactly the same as EmmyLua, and I don't mind adding more proprietary functions.

[oowl]

In my project, I don't have resty/resdis*.lua file, so i can not auto detect openresty. Can you consider to change this way?

[sumneko]

In my project, I don't have resty/resdis*.lua file, so i can not auto detect openresty. Can you consider to change this way?

You could open a PR about it https://github.com/sumneko/lua-language-server/blob/master/meta/3rd/OpenResty/config.lua