Lape debug mode

[Vizit0r]

As i see: No debug mode implemented for lape. ATM i found only way to take values of global vars, and thats all. No params&result values, no methods vars, no step into&over, etc. Compiler provide all needed info, but with values from interpreter big problems.

If all above is correct, please advise, do you have a plans for this feature in your roadmap? If i mess something - please advice, where to dig in.

[ollydev]

I think the basic idea is to pass StackInfo to the interpreter then use VarToString(AVar: Pointer): lpString; on the current VarStack.

[Vizit0r]

dive deeper and deeper there.

One more question: when script is running, if its possible to exec script method from Delphi (in same script area, means shared vars etc) ? I found the way how to exec it in lpffiwrappers.pas, but not sure about same script area, seems like its separated.

P.S. Main script at this time will invoke imported method, who is checking queue and calling script methods, if needed. So, no problem with data interference between them.

[ollydev]

It works fine using ffi. https://i.imgur.com/5D8QK9n.png

So LapeExportWrapper. or Natify in the script and pass the pointers back to FPC.

[Vizit0r]

1) ffi was developed and designed for FPC only, not so easy to transfer it to Delphi. 2)i see only Win dlls, nothing for other platforms, Without ffi no way?

[ollydev]

I think FFI should work on Delphi? There might be some minor compile errors as it's never been compiled using Delphi (I think) Lape loads FFI from the system on macOS/Linux. So apt-get install libffi-dev or brew install libffi.

You can call script methods using RunCode by using InitialJump: RunCode(Compiler.Emitter.Code, Compiler.Emitter.CodeLen, [{var stack}], PCodePos(Compiler['SomeScriptMethod'].Ptr)^);

But variables, I am not sure on. I've only used above to call just to call the most basic procedure SomeScriptMethod; method.

[Vizit0r]

will try it later, thanks.

Now i try to find the way to obtain list of vars for current stack area, still nothing. Some vars i can get, but thats not a result.

P.S. Regarding FFI - i change all places with error quickly(a lot of them!), but it doesnt work, need to pay more attention there.

[Vizit0r]

Question with script methods calling solved fast and easy, thanks @ollydev again,

For my purpose enough simple types in params, without retur values.

Now only question with debug left, but i'm stacked again. Have no idea, what to do. Too much of temp vars, all translated to pointers, and no some common origin var to refer on. Have some crazy ideas, like make magic methods for every var and param, and call them outside -but its really crazy.

example on script method calling:

script:

var bbb : Integer;
procedure SomeScriptMethod(Xorig, Yorig : Word; Dir : Byte; X, Y : Word; St : String);
begin
  AddToSystemJournal('Xorig='+IntToStr(Xorig));
  AddToSystemJournal('Yorig='+IntToStr(Yorig));
  AddToSystemJournal('Dir='+IntToStr(Dir));
  AddToSystemJournal('X='+IntToStr(X));
  AddToSystemJournal('Y='+IntToStr(Y));
  AddToSystemJournal('St='+St);
  bbb := 100;
end;
Begin
  bbb := 800;//ttest(199);
  AddToSystemJournal('b='+IntToStr(bbb));
  Wait(1);
  AddToSystemJournal('b='+IntToStr(bbb));
End.

and Delphi:

    Compiler : TLapeCompiler;
    VarStack: lptypes.TByteArray;
    Xorig, Yorig : Word; Dir : Byte; X, Y : Word; St : String;
    StP : PChar;
    TotalParamsLen : Integer;
begin
  Compiler := TBaseLapeScriptThread(CurrScriptThread).Compiler;
  Xorig := 500; Yorig := 600; Dir := 91; X := 700; Y := 800; St := 'Delphi Test';
  TotalParamsLen := SizeOf(Xorig) + SizeOf(Yorig) + SizeOf(Dir) +SizeOf(X) + SizeOf(Y) + SizeOf(PChar);
  SetLength(VarStack, TotalParamsLen);
  k := 0;
  Move(Xorig,VarStack[0],SizeOf(Xorig));
  Inc(k,SizeOf(Xorig));
  Move(Yorig,VarStack[k],SizeOf(Yorig));
  Inc(k,SizeOf(Yorig));
  Move(Dir,VarStack[k],SizeOf(Dir));
  Inc(k,SizeOf(Dir));
  Move(X,VarStack[k],SizeOf(X));
  Inc(k,SizeOf(X));
  Move(Y,VarStack[k],SizeOf(Y));
  Inc(k,SizeOf(Y));
  StP := PChar(St);
  Move(StP,VarStack[k],SizeOf(PChar));

  RunCode(Compiler.Emitter.Code, Compiler.Emitter.CodeLen, VarStack, PCodePos(Compiler['SomeScriptMethod'].Ptr)^);

returns

22:26:18:740 : Compiling
22:26:18:798 : Compiled succesfully
22:26:18:799 : b=800
22:26:18:799 : Xorig=500
22:26:18:799 : Yorig=600
22:26:18:799 : Dir=91
22:26:18:799 : X=700
22:26:18:799 : Y=800
22:26:18:799 : St=Delphi Test
22:26:18:799 : b=100
22:26:18:799 : Succesfully executed
22:26:18:799 : Script Test.sc stopped successfuly

[ollydev]

Hmmm? You just want a easier way to generate the var stack? If only needing base types, this comes to mind.

function getVar(BaseType, Value: String): TByteArray;
var
  Variable: TLapeGlobalVar; // will need to manage (free) me
begin
  Variable := Compiler.getBaseType(BaseType).NewGlobalVarStr(Value);
  SetLength(Result, Variable.Size);
  Move(PPointer(Variable.Ptr)^, Result[0], Variable.Size);
end; 

RunCode(Compiler.Emitter.Code, Compiler.Emitter.CodeLen, getVar('AnsiString', 'Hello world') + getVar('Int32', '123456'), PCodePos(Compiler['SomeScriptMethod'].Ptr)^);

Script:

procedure SomeScriptMethod(Str: AnsiString; I: Int32);
begin
  Writeln('Str: ', Str);
  WriteLn('I: ', I);
end;

begin
end;

Output:

Str: Hello world
I: 123456

You could also verify you've passed the correct varstack by comparing what you generate to TLapeType_Method(Compiler['SomeScriptMethod'].VarType).Params; and so on...

Going to bed now, Good luck!

[Vizit0r]

its just for test purposes, in real code i will obtain VariantArray and transfer it to code I think to do some method like this, but generic.

P.S. Constant param size also for tests only, first i check it by TLapeType_Method(TBaseLapeScriptThread(CurrScriptThread).Compiler['SomeScriptMethod'].VarType).Params.

Have a good night)

[ollydev]

If you didn't know the script can accept a variant array very easily as it's just a pointer. Just pass the pointer to a variant array and lape (should) handle it fine. :)

var test: TVariantArray; 
begin
SetLength(test, 3);
test[0] := 123;
test[1] := 'hello';
test[2] := 123.4;

SetLength(bytes, SizeOf(Pointer));
Move(Pointer(Test), bytes[0], SizeOf(Pointer));

RunCode(Compiler.Emitter.Code, Compiler.Emitter.CodeLen, bytes, PCodePos(Compiler['SomeScriptMethod'].Ptr)^); 

Might be easier?

Or this is what you want. (No error checking)

  function AddVar(VarType: TLapeType; Value: String): TByteArray;
  var
    Variable: TLapeGlobalVar; // will need to manage (free) me
  begin
    Variable := VarType.NewGlobalVarStr(Value);
    SetLength(Result, Variable.Size);
    Move(PPointer(Variable.Ptr)^, Result[0], Variable.Size);
  end;

  procedure Call(Method: String; Params: array of Variant);
  var
    m: TLapeType_Method;
    i: Int32;
    Vars: TByteArray;
  begin
    m := TLapeType_Method(Compiler[Method].VarType);
    for i := 0 to m.Params.Count - 1 do
      Vars += AddVar(m.Params[i].VarType, VarToStr(Params[i]));

    RunCode(Compiler.Emitter.Code, Compiler.Emitter.CodeLen, Vars, PCodePos(Compiler[Method].Ptr)^);
  end;

Okay, Actually bed time...

[Vizit0r]

Might be easier?

really not. Thank you.

[Vizit0r]

Does exists some way to add method, who will look like a var in script? I want to import class properties. Now i have Compiler.addGlobalFunc('function DiscordMessenger.Token: string;overload;', @_GetMesNameWrap); Compiler.addGlobalMethod('procedure DiscordMessenger.Token(Value : String);overload;', @_SetMesNameWrap, nil); in script DiscordMessenger.Token('124zzzz'); AddToSystemJournal(DiscordMessenger.Token); works fine.

But i need to do it in classic way, DiscordMessenger.Token := '124zzzz'; otherwise compatibility with old user scripts will be broken, that is not what i want.

But i have error: Can't assign "UnicodeString" to "Overloaded Method [2]" its expected, but i still cant find a solution.

[ollydev]

Properties have not been implemented into Lape.

[Vizit0r]

@ollydev Implemented, lcoAutoProperties in Compiler Options solve this question.

So, now left only the question about local vars&params value obtaining, ang get call stack as string list. Will explore it more.

BTW, i'm add to parser support of Pascal unit files (uses XXX;) for compatibility with old PS scripts, not sure anyone else need this.

[Vizit0r]

one more question regardings compilation to bytecode. After successfull compilation i save bytecode to file. Next time i read it, pass to Interpreter, it runs few first instruction, than dying on lpePointer_ASGN_Pointer, ptr = nil

LapeCompiler.Options := LapeCompiler.Options - [lcoAlwaysInitialize];

It there some way to run it without sources compilation?

[ollydev]

No work on bytecode loading/saving has ever been done. First thought you will have to relink all imported methods, global variables and generate all constant values. Likely other stuff too.

I don't think you've done since lcoAlwaysInitialize just defaults the variable with zeros. Does not do any memory allocation, so has nothing to with a nil pointer.

[Vizit0r]

i try with lcoAlwaysInitialize, and without. no matter. In PS for correct running compiled scripts for PSExec you have special methods for settings pointer addresses for imported vars. For Lape, ofc if no this setup done before running - all imported global vars pointing to dev>null. But atm i have no idea where and how this have to be done.

[ollydev]

You could save PS compiled code? Never knew that. We used to use that in Simba. And yeah I said there has been no work done on loading/saving bytecode, nor have I attempted so I don't really know "where and how this has to be done"

[Vizit0r]

If you need, i can send you a code.

well understood. Thats not a critical option, will pack and ecrypt whole file (with inserted includes) and compile it in silent mode every time. Bad way, but way :)

Still try to understand tree logic, to find a ptrs of local vars&params, but still failed...

P.S. Took a code part with callstack getting by EnterMethod&LeaveMethod from Simba, its really good, thanks.

[JohnPeel]

Not sure anymore how Lape does it, but to support bytecode saving/loading it would need to compile the script to an intermediate bytecode and have an interpreter for that bytecode.

That bytecode could be saved with a symbol table for external functions, then when loading you replace the table with the external implementations.

I doubt Lape supports this with it's current implementation.

[Vizit0r]

yes, it doesnt support.

BTW, quesion with local vars\params values solved (based on priciples of Enter\Leave Method by olly).

P.S. calling script methods from Delphi as described before - works, but cause memory corruption sooner or later - some common script String vars, which in use, suddenly disposed. Cant locate the exact place, its floating.

Now try to use ffi...

[Vizit0r]

@ollydev i need help. my skills not enough to understood the reason of bug, lost 6 days, but

still not found root reason of error.

I have a small script, cuted from huge script(more than 15k lines), to reproduce error. Error coming in line 489 (preparation function params), but real reason is bit earlier. In my own log i write execution of all opcodes(with addresses) + deep logging in method lpeUString_ASGN_UString + detection of writing value $2106C, its "constant" (until script changing).

So, in my log most interesting lines 9090 (error detected), and 9067-9068 (setting non string-value). Pointer to non-string value assigned, and when on error line it try to check string by ptr and its refcount - Access Violation raised.

For both operations StackPos = 48, its seems like error, but i'm not sure.

The biggest problem, that 100% error repeating available in my program only, seems like due to specific Lape imported methods and types. I try to execute script in Simba - all is ok. Look it please, if you have free time...

Attached 4 files: 1) script with bug 2)disassembled log 3) my additional log 4) small log with AV info.

LapeOps_612.log DISASM.log Exec_Error.log script_BUGGG.txt

[ollydev]

@Vizit0r There is some kind low level bug related to records (as far as I can tell)

Here is a (bad) bug report I made a little bit ago. https://github.com/nielsAD/lape/issues/147 It seems to be your issue also.

When I constref your script so no record copies are made, it succesfully executes. function GetFatigueThreshold(constref city:TCity; constref Info:TGumpInfo2 ): Integer;

Today I found another bug with record compare only on 32 bit, in this example at least.

https://i.imgur.com/bkcYm4S.png

I will try to fix these bugs in the coming days with niels.

Edit: bug was fixed in 5589b76dcbc72a111fddfcb6324c81fa56a31b31

[Vizit0r]

good that you already knows about this deep bug, hope you will fix it. "ConstRef" or "var" works as temporary solution, thanks.

BTW, the subject is already solved, file with cuted code pieces attached.

Not sure it covers ALL types, but on my tests its works fine. Not test it for Variants at all. Also not tested for all params "type" like "var", "constref", etc.

Big thanks to @ollydev for showing the right way.

Maybe will be useful for someone.

DELPHI lape local methods debug.txt