high memory usage on loop

[TorchedSammy]

i'm not sure how to title this or to properly report this.

n = 0; while n < 10000000000 do n = n + 1 end running this sample of code in hilbish (which in reality just uses golua) and I get this:

for some reason, this doesn't happen in golua-repl

[arnodel]

What is the global table in Hilbish? If its __index metamethod is redefined (as a function) then the runtime will try to call it when it retrieves the value of n, as n is short for _G["n"]. If you do that 100 million times, youi will create 10 billion terminations, which will make the go GC busy. But that's an interesting point, in the past I added object pools for continuations and register sets because the Go runtime was spending a lot of time garbage collecting those. Perhaps a similar approach could be taken with terminations.

EDIT: to check my theory is right, you can declare n as local. This should make it fast again.

EDIT 2: 100 million -> 10 billion

On Tue, 19 Dec 2023 at 01:49, sammyette @.***> wrote:

i'm not sure how to title this or to properly report this.

n = 0; while n < 10000000000 do n = n + 1 end running this sample of code in hilbish (which in reality just uses golua) and I get this:

https://camo.githubusercontent.com/7fff5053af0d54616241fde0257251807d7962f7679201dc803347ca4d9c500d/68747470733a2f2f736166652e736179612e6d6f652f6761524646493279644370432e706e67

https://camo.githubusercontent.com/5af21c1e34e375f5210debdcb0181af4bfdf3fc1d60fdd87a28b72cd0db3e4ca/68747470733a2f2f736166652e736179612e6d6f652f37346e4d447572466d686c4a2e706e67

https://camo.githubusercontent.com/7897fd75c3ad3d15d1b1c7b11c7bc45db96164f40870e60fa87d27c749e249e9/68747470733a2f2f736166652e736179612e6d6f652f5a3377546a4c724148647a682e706e67

for some reason, this doesn't happen in golua-repl

— Reply to this email directly, view it on GitHub https://github.com/arnodel/golua/issues/101, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMJKOLKM2KU3P2XNRV4L5TYKDXCNAVCNFSM6AAAAABA2NIW3KVHI2DSMVQWIX3LMV43ASLTON2WKOZSGA2DONZXGI2DKOA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[TorchedSammy]

What is the global table in Hilbish? If its __index metamethod is redefined (as a function) then the runtime will try to call it

I set it in Lua to define global variables as environment variables. It can be seen here: https://github.com/Rosettea/Hilbish/blob/master/nature/init.lua#L35-L63

EDIT: to check my theory is right, you can declare n as local. This should make it fast again

yes, no change in memory usage at all.

[arnodel]

What is the global table in Hilbish? If its __index metamethod is redefined (as a function) then the runtime will try to call it

I set it in Lua to define global variables as environment variables. It can be seen here: https://github.com/Rosettea/Hilbish/blob/master/nature/init.lua#L35-L63

Ah, so does it mean every time n is executed, if the global table doesn't have. key called n, the program checks if there is an environment variable called n and returns it if that is the case? Also do you define __newindex as well?

[TorchedSammy]

Yes, it reads and sets environment variables as Lua variables. To and back:

[arnodel]

Ok, so your code snippet basically calls os.Getenv("n") and os.Setenv("n", ...) 10 billion times. That may account for a significant portion of the execution time - also I imagine the number is converted to and from a string each time (this is all speculative, it would need to be checked). As for the memory leak, I am not sure there is one - it may just be that 10 billion instances of Termination are allocated and so it keeps the GC busy, but all memory is probably reclaimed. If the terminations were not GCed, there would be a leak of the order of 10 billion * (size of a termination >= 48) bytes, which would be hundreds of GB.

I think a termination pool might mitigate that, as I mentioned previously. It's something I can try implementing - it might be straightforward enough as it could be modelled on continuation pool implementations. I don't know if I would have time to do that soon though.

Another approach would be to convince the Go compiler that the termination created in the runtime.Index function can be allocated on the stack, but that's probably impossible.