EdgeTX / edgetx

EdgeTX is the cutting edge open source firmware for your R/C radio
https://edgetx.org
GNU General Public License v2.0
1.57k stars 333 forks source link

LUAC is not read by edge tx #5230

Open i3dm opened 3 months ago

i3dm commented 3 months ago

Is there an existing issue for this problem?

What part of EdgeTX is the focus of this bug?

Transmitter firmware

Current Behavior

my LUA runs fine and creates a LUAC. if i remove the LUA and leave only the LUAC, the widget does not appear in the select widget menu at all. seems LUAC is not read by TX fw.

Expected Behavior

LUAC should run just like LUA.

Steps To Reproduce

  1. grab my widget: https://github.com/i3dm/Zavionix/tree/main/OpenTX-EdgeTX%20Widgets
  2. load to edgetx radio or sim.
  3. run the widget, LUAC is created.
  4. remove LUA file. restart tx.
  5. widget is no longer seen and can not be selected.

tested and reproducible on the simulator.

Version

2.10.1

Transmitter

RadioMaster TX16S / TX16SMK2, Other (Please specify below)

Operating System (OS)

No response

OS Version

No response

Anything else?

tested and reproducible on the simulator.

pfeerick commented 3 months ago

I'm not entirely convinced this is a bug but is "by design". Luac was directly runnable for standalone Lua tools due to memory constraints (primarily b&w radios). This isn't the case for radios that support widgets. So was never made possible, to strongly encourage you to ship the source for the widget, not just the compiled bytecode.

On Sun, 30 June 2024, 6:26 am i3dm, @.***> wrote:

Is there an existing issue for this problem?

  • I have searched the existing issues

What part of EdgeTX is the focus of this bug?

Transmitter firmware Current Behavior

my LUA runs fine and creates a LUAC. if i remove the LUA and leave only the LUAC, the widget does not appear in the select widget menu at all. seems LUAC is not read by TX fw. Expected Behavior

LUAC should run just like LUA. Steps To Reproduce

  1. grab my widget: https://github.com/i3dm/Zavionix/tree/main/OpenTX-EdgeTX%20Widgets
  2. load to edgetx radio or sim.
  3. run the widget, LUAC is created.
  4. remove LUA file. restart tx.
  5. widget is no longer seen and can not be selected.

tested and reproducible on the simulator. Version

2.10.1 Transmitter

RadioMaster TX16S / TX16SMK2, Other (Please specify below) Operating System (OS)

No response OS Version

No response Anything else?

tested and reproducible on the simulator.

— Reply to this email directly, view it on GitHub https://github.com/EdgeTX/edgetx/issues/5230, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJ66KI6747TNDGA225632TZJ4J7LAVCNFSM6AAAAABKDOZFBCVHI2DSMVQWIX3LMV43ASLTON2WKOZSGM4DCOJUHA4TKMQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

i3dm commented 3 months ago

This was implemented way back from OTX days. Any reason why it shouldn't be implemented in ETX?

Any dev can choose whether to post his Lua or Luac.

pfeerick commented 3 months ago

True, but that is against the spirit of open source. And any Lua dev should know that a launcher stub should be able to work around this without any trouble if they really insist on not releasing the source.

JimB40 commented 3 months ago

@pfeerick not relevant argument.

Byte code (luac) has smaller footprint and it's faster. If script is "one liner" it does not matter, if it is lua application like iNav Telemetry it does matter. In fact to run properly on small memory BW radios it is distributed compiled and .luac extension for bytecode files is exchanged to .lua to overcome this bug.

Then there was always source code folder with readable luac files to meet what you call "open source spirit".

Luadoc says that if you use loadScript() with 'Tx' argument, it should check for both .lua and .luac and choose .luac if existing. This is true only if both types are on SD alongide. If you remove .lua EdgeTX throws error still looking for .lua

This is definetely bug not conected to "spirit" and should be fixed. :)

i3dm commented 3 months ago

I agree.

pfeerick commented 3 months ago

Byte code (luac) has smaller footprint and it's faster. If script is "one liner" it does not matter, if it is lua application like iNav Telemetry it does matter. In fact to run properly on small memory BW radios it is distributed compiled and .luac extension for bytecode files is exchanged to .lua to overcome this bug.

You are just repeating what I said above. Did you only read the last reply? Also not relevant since this is about color lcd, not B&W.

Then there was always source code folder with readable luac files to meet what you call "open source spirit".

This is a logical fallacy - how exactly is the firmware supposed to magically know you distributed the source separately (if at all)?

Luadoc says that if you use loadScript() with 'Tx' argument, it should check for both .lua and .luac and choose .luac if existing. This is true only if both types are on SD alongside. If you remove .lua EdgeTX throws error still looking for .lua

Did you have to do something special to make it work with TBS Agent Lite widget mode then? As that appears to be doing exactly what you say isn't working there... using a .lua stub that then uses loadScript to load a .luac, and uses the Tx parameters to do so?

If it does not actually work somehow in accordance with the docs, then yes, that is a bug, which should be raised separately.

btw, it's been two weeks since I pinged you about https://github.com/EdgeTX/edgetx/pull/5181 Still waiting. ;)

JimB40 commented 2 months ago

Then there was always source code folder with readable luac files to meet what you call "open source spirit".

This is a logical fallacy - how exactly is the firmware supposed to magically know you distributed the source separately (if at all)?

It is logical fallacy indeed @pfeerick Why firmware should know anything about source code? Is there a folder with source code of EdgeTX firmware on radio so firmware knows it is open source? Nope. Can I sent compiled EdgeTX (firmware.bin) to someone to be used on his radio not breaching ETX open-source licence? Yes I can. Can I download compiled EdgeTX to my radio via Buddy without source code? Yes I can.

Luadoc says that if you use loadScript() with 'Tx' argument, it should check for both .lua and .luac and choose .luac if existing. This is true only if both types are on SD alongside. If you remove .lua EdgeTX throws error still looking for .lua

Did you have to do something special to make it work with TBS Agent Lite widget mode then? As that appears to be doing exactly what you say isn't working there... using a .lua stub that then uses loadScript to load a .luac, and uses the Tx parameters to do so?

You bet I did. :) 20% more speed and stripped commments in LUA compiled verscion are substancial not to hit 'CPU Limit' or 'C-boundary' error.

To recap

  1. luac (compiled version) has some substancial advantages and shoud be treated equally to lua (text version) in terms of run-time possibilty
  2. Fact that OS is open-source doesn't mean (in legal tems) that applicatations working on this OS must be open-source too. BTW I'm using few Linux tools that are distrubuted without source code and they do not breach Linux open-source licence.

@i3dm Workaround I use for widgets is that widget main.lua file is "one-liner" with: loadScript("/MY_ACTUAL_WIDGET_FILES/my_widget", "Tx") ("bx")

  1. Widget's main.lua will be compiled every time radio is stared or model is reloaded "Tx" will force loadScript to look for either .lua or .luac (with caveat that if lua is missing luac won't load)
  2. This script passes as argument of further loadScript() loading mode "tx" - use .lua and do not compile "bx" - use .luac and do not compie

You can also specify full name for loading ommiting loadScript mode ie. loadScript("/MY_ACTUAL_WIDGET_FILES/my_widget.luac") but I find more convinient to write code without extensions and let loadScript mode decide what to load.

btw, it's been two weeks since I pinged you about #5181 Still waiting. ;)

Sorry missed this ping, comenting now.

i3dm commented 2 months ago

How is that a workaround? If Lua is still needed to run?

JimB40 commented 2 months ago

.lua is needed only as /WIDGETS/MY_WIDGET/main.lua rest of loaded scrtipts are .lua or *.luac whatever you decide in your LUA code.

i3dm commented 2 months ago

Not sure I understand how that works. You add another Lua with one line to allow read Luac?

Can you share an example?

JimB40 commented 2 months ago
  1. loader

    -------------------------------------------------------------------------------
    --  /WIDGETS/MY_WIDGET/main.lua
    -------------------------------------------------------------------------------
    return loadScript("/WIDGETS/MY_WIDGET/code.luac") ()
  2. actual widget code that is copiled

    
    -------------------------------------------------------------------------------
    --  /WIDGETS/MY_WIDGET/code.luac
    -------------------------------------------------------------------------------
    local name = "WidgetName"

-- Create a table with default options -- Options can be changed by the user from the Widget Settings menu -- Notice that each line is a table inside { } local options = { { "Source", SOURCE, 1 }, -- BOOL is actually not a boolean, but toggles between 0 and 1 { "Boolean", BOOL, 1 }, { "Value", VALUE, 1, 0, 10}, { "Color", COLOR, ORANGE }, { "Text", STRING, "Max8chrs" } }

local function create(zone, options) -- Runs one time when the widget instance is registered -- Store zone and options in the widget table for later use local widget = { zone = zone, options = options } -- Add local variables to the widget table, -- unless you want to share with other instances! widget.someVariable = 3 -- Return widget table to EdgeTX return widget end

local function update(widget, options) -- Runs if options are changed from the Widget Settings menu widget.options = options end

local function background(widget) -- Runs periodically only when widget instance is not visible end

local function refresh(widget, event, touchState) -- Runs periodically only when widget instance is visible -- If full screen, then event is 0 or event value, otherwise nil end

return { name = name, options = options, create = create, update = update, refresh = refresh, background = background }

i3dm commented 2 months ago
  1. loader
-------------------------------------------------------------------------------
--  /WIDGETS/MY_WIDGET/main.lua
-------------------------------------------------------------------------------
return loadScript("/WIDGETS/MY_WIDGET/code.luac") ()
  1. actual widget code that is copiled
-------------------------------------------------------------------------------
--  /WIDGETS/MY_WIDGET/code.luac
-------------------------------------------------------------------------------
local name = "WidgetName"

-- Create a table with default options
-- Options can be changed by the user from the Widget Settings menu
-- Notice that each line is a table inside { }
local options = {
  { "Source", SOURCE, 1 },
  -- BOOL is actually not a boolean, but toggles between 0 and 1
  { "Boolean", BOOL, 1 },
  { "Value", VALUE, 1, 0, 10},
  { "Color", COLOR, ORANGE },
  { "Text", STRING, "Max8chrs" }
}

local function create(zone, options)
  -- Runs one time when the widget instance is registered
  -- Store zone and options in the widget table for later use
  local widget = {
    zone = zone,
    options = options
  }
  -- Add local variables to the widget table,
  -- unless you want to share with other instances!
  widget.someVariable = 3
  -- Return widget table to EdgeTX
  return widget
end

local function update(widget, options)
  -- Runs if options are changed from the Widget Settings menu
  widget.options = options
end

local function background(widget)
  -- Runs periodically only when widget instance is not visible
end

local function refresh(widget, event, touchState)
  -- Runs periodically only when widget instance is visible
  -- If full screen, then event is 0 or event value, otherwise nil
end

return {
  name = name,
  options = options,
  create = create,
  update = update,
  refresh = refresh,
  background = background
}

just tried this and main.lua is not being read by Edgetx, i can not find it in the widgets list. any idea why?

JimB40 commented 2 months ago
  1. If you can't find widget on widget list it's 99% due to an error during widget initializing. Look at companion debug what is causing error.

  2. It may be that code.luac can't be found as you need to compile it first.

i3dm commented 2 months ago
  1. If you can't find widget on widget list it's 99% due to an error during widget initializing. Look at companion debug what is causing error.
  2. It may be that code.luac can't be found as you need to compile it first.
  1. the luac i used was compiled by edgetx, but i thought the lua should still appear as an option in the menu even if the luac is somewhat defective?
  2. heres some of the debugger output:

130ms: f_stat(C:/Users/USER/Documents/EdgeTX/WIDGETS/Test/main.luac) = OK 130ms: f_stat(C:/Users/USER/Documents/EdgeTX/WIDGETS/Test/main.lua) = OK 130ms: luaLoadScriptFileToState(/WIDGETS/Test/main.luac, bt): loading /WIDGETS/Test/main.luac 130ms: f_open(C:/Users/USER/Documents/EdgeTX/W968a90) (FIL:0000001a355fe338) 130ms: f_stat(C:/Users/USER/Documents/EdgeTX/WIDGETS/Test/main.luac) = OK 130ms: f_stat(C:/Users/USER/Documents/EdgeTX/WIDGETS/Test/main.lua) = OK 130ms: luaLoadScriptFileToState(/WIDGETS/Test/main.luac, bt): loading /WIDGETS/Test/main.luac

this occurs many times and then...

570ms: f_stat(C:/Users/USER/Documents/EdgeTX/WIDGETS/Test/main.luac) = OK 570ms: f_stat(C:/Users/USER/Documents/EdgeTX/WIDGETS/Test/main.lua) = OK 570ms: luaLoadFile(/WIDGETS/Test/main.lua): Error parsing script: CPU limit

JimB40 commented 2 months ago

Hmm not sure what is causing your error. Example with code.luac works fine (tested on ETX 2.10)

Screenshot 2024-07-23 at 10 34 30

screenshot_tx16s_24-07-23_10-34-35

i3dm commented 2 months ago

Hmm not sure what is causing your error. Example with code.luac works fine (tested on ETX 2.10)

Screenshot 2024-07-23 at 10 34 30

screenshot_tx16s_24-07-23_10-34-35

  1. can you upload your files so i could test them?
  2. why do you have both code lua and main lua? i only used main lua and main luac. this is unclear.
  3. how do you compile your luac files?
  4. i tried to replicate these 2 files from the above text and still no go. but anyway your LUAC is non compiled?
JimB40 commented 2 months ago
  1. Here you are Archive.zip

  2. Once finished I leave only main.lua (main.luac will be generated anyway with widget initialization) and code.luac

  3. For one file project like in this example I change in main.lua return loadScript("/WIDGETS/MY_WIDGET/code.luac") () to return loadScript("/WIDGETS/MY_WIDGET/code.lua") ()

that will force code.lua to be compiled thenb revert main.lua to return loadScript("/WIDGETS/MY_WIDGET/code.luac") ()

then point 2

for bigger projects with many folders and lua files I have separate Lua Compiler thar traverse through project folder tree compile every .lua found into .luac then deletes *.lua

JimB40 commented 2 months ago

I know it's cumbersome for Lua dev but that's because implementation if loadScript API function is not considering *.luac as valid Lua chunk. To clarify https://luadoc.edgetx.org/part_iii_-_opentx_lua_api_reference/general-functions-less-than-greater-than-luadoc-begin-general/loadscript

  1. if you use loadScript('code') ()
    • it shorthand for loadScript('code','bt') ()
    • EdgeTX will use either binary or text (code.lua or code.lua), whichever is newer (binary preferred when timestamps are equal).
    • If only code.lua is present EdgeTx will force compilation and use code.luac according to rule above.
    • If only code.luac is present ... well :) that's when things go diffrent to what is dectibed in documentation.

That is why I'm not allowing loadScript to automate this proces but I'm compiling files to *.luac then I'm using loadScript('code', 'bx') That will:

BTW last parenthesis mean that you want to execute what's inside code.lua as loaded chunk is treated as function

JimB40 commented 2 months ago

last but not least EdgeTX API loadScript will load any file

That means below code is valid as long as loaded files are valid Lua chunks (text or compiled)

loadScript('code')
loadScript('code.lua')
loadScript('code.luac')
loadScript('code.png')
loadScript('code.blabla')

In project implementation you can use somehing like

local path = '/WIDGETS/MY_WIDGET/'
local extension = '.luac'
local function myLoad(fileName)
  return loadScript(path..filenNme..extension, 'x') ()
end
local chunk = myLoad('code')

Function will only look for files with *.luac extension and prevent compilation.

i3dm commented 2 months ago

this might not be a viable solution for my issue. my issue is that i want to publicly post only a compiled luac, and can add a small lua to run it thats what it takes, but not ask the user to run x file, and then replace with Y file and so on.

could this even be done?

JimB40 commented 2 months ago

You haven't understood.

Choose solution which best suits your project's needs. Core EdgeTX assumprton is flexibility so you can reach your goals in many ways.

i3dm commented 2 months ago

OK. how do i compile code.lua (which will have the content of my existing mian.lua widget file) ?

JimB40 commented 2 months ago

For me best solution is to have another script (compiler) that will just compile files I need to be compiled with loadScript('fileName','c') command

i3dm commented 2 months ago

for some reason i cant seem to cause my lua to compile while your code lua does compile. can you try to run the same for TD-RDT and upload the example? i.e compile it and run it from another lua.

https://github.com/i3dm/Zavionix/tree/main/OpenTX-EdgeTX%20Widgets

JimB40 commented 2 months ago

Starting holiday soon so my calendar is packed. I may have some free slot to have look on holiday but do not promise. BTW you got all info to figure it out.

i3dm commented 2 months ago

@JimB40 thank you for your help i managed to get it to work. i will leave the issue open as i still think EdgeTx should support LUAC by nature.