Better way of making loops.

[ghost]

I added these keywords: 'incf' , 'decf' and 'dotimes' Usage of 'incf' & 'decf' :

(print (incf 100 50)) ; exactly like += in C
(print (decf 100 5)) ; exactly like -= in C

Why the need for these two new keywords? Better performance to keyword 'while' which now I renamed it to 'loop'. Usage of 'dotimes':

(= i 0)
(dotimes i 100 ; exactly like for (int start = 0; start < limit; start++) 
  (print "I'm going to be printed 100 times!"))

The reason behind 'dotimes' is also performance. 'dotimes' is a lot faster than 'loop'. I'm thinking for a new keyword which will go from limit to start instead of start to limit. I'd like to hear your opinion on that matter. I'd encourage you to use the "time" command and see for yourself the difference in performance before and after this commit. Thanks!

'while' -> 'loop' usage: Before:

(= i 0)
(while (< i 10) 
;some code 
(= i (+ i 1) ; slower than the new version
)

Now:

(= i 0)
(loop (< i 10)
;some code
(incf i 1) ;faster!
)

P.S: Can you please explain to me a bit more about this piece of code:

/* Truncate stack so we don't accumulate protected values */
S->gc_stack_idx = orig_stack_idx;

What does it actually do? Tbh I haven't looked in detail over the GC yet but if you have some docs or anything where I can read more about this I'd be very grateful.

[rxi]

Unfortunately the goal of the project is to be concise and simple — although these changes would improve performance for these operations they're fairly specific cases and hundreds of other similar functions could be added to a similar effect — for example, a native "map" function. Such additions come with the cost of added code size and program size, as well as complexity.

A better course of action would be to implement native functions as needed in the project which is embedding the language, and if a faster scripting language is required to use an embeddable language which has a greater focus on performance.

The implementations of incf and decf are also problematic in that they mutate the number value instead of creating a new number value, and would result in odd behaviour, for example, assigning x to y then performing incf on x would change y's value.

Regarding the gc code you mentioned, the implementation uses a simple mark and sweep garbage collector — when it runs it searches for values, finds those that are not referenced by anything in the program and frees them. Garbage collection can potentially trigger any time a new value is created which becomes a problem for intermediate values which aren't referenced by anything in the program but are still being used — for example, those passed as function arguments, those returned by eval or a list of values that is still being built — thus an internal stack of values is used which are protected from garbage collection. These include return values and newly created values.

In the line you linked the return values from the while loop's eval call are each pushed to the stack. As these values won't be used and are safe to discard we reset the stack manually. If this wasn't done the stack would keep accumulating these return values until the loop exited and gc_stack_idx was reset when exiting the frame.

[ghost]

I fully understand your point of view, thanks for explaining everything so well.

[ghost]

Not sure if this is the right place to post a question but I don't know any better place. Why does the function push_value_to_stack have a check for stack capacity? Isn't stack space in relation to CHUNK_LEN ? And since CHUNK_LEN is a hard coded value which won't grow ever why does the stack have to grow bigger than CHUNK_LEN ? Wouldn't it better to alloc S->gc_stack directly to 1024? Also, why does MAX_STACK exist in the first place? Isn't it enough to have CUNK_LEN?

[rxi]

The memory used by values is stored in chunks, each chunk is value struct array of length CHUNK_LEN and a pointer to the next chunk. If we try to create a value and there are no free values in the freelist a new chunk is created and all the values in its array are pushed to the freelist.

The GC stack is a stack separate from the chunks, this holds references to values which we don't want to be garbage collected, I explain this in more detail in the previous comment.

The MAX_STACK value relates to the call stack rather than the GC stack — it is used as a hard limit of how many nested calls there can be before an error occurs. This is to prevent the underlying C stack from overflowing which would result in a segfault.

In regard to where to ask questions: The usual approach is to create a new issue and ask it there.

[ghost]

Thanks, now everything really makes sense!

P.S: The whole garbage collection system is very smart. I really appreciate all your work from github and twitter!