Input passed to a function is int. But we need to use that integer in a floating operation.

[adi-lb-phoenix]

my c-lisp code

(c-lisp
    (define ((sqrtf float ) (ss float)))

    (define ((rnsorm void) (o (ptr float)) (x (ptr float)) (weight (ptr float)) (size int))
        (declare ss float)
        (declare n float)
        (declare j int)
        (declare e float)
        (set e 2.71828)
        (set ss 0.0)
        (for ((set j 0) (lt j size) (set j (add j 1)))
            (set ss (fadd ss (fmul (load (ptradd x j))  (load (ptradd x j)) ))) 
        )

        (set ss (fdiv ss size))
        (set ss (fadd ss (fsub e 5.0) ))
        (set ss (fdiv 1.0 (call sqrtf ss )))

        (for ((set j 0) (lt j size) (set j (add j 1)))
            (store (ptradd o j) (fmul (load (ptradd weight j)) (fmul ss (load(ptradd x j)))))
        )

    )

)

C code :

void rmsnorm(float* o, float* x, float* weight, int size) {
    // calculate sum of squares
    float ss = 0.0f;
    for (int j = 0; j < size; j++) {
        ss += x[j] * x[j];
    }
    ss /= size;
    ss += 1e-5f;
    ss = 1.0f / sqrtf(ss);
    // normalize and scale
    for (int j = 0; j < size; j++) {
        o[j] = weight[j] * (ss * x[j]);
    }
}

In the C function definition we have int size defined. But in the one of the instruction that integer value is used to compute floating point value ss /= size;. ss is defined to be float. The issue here is how can a function in c-lisp receive a int value and use that value in a floating point computation.

[adi-lb-phoenix]

My approach to solve this : Just like we did (define ((fprint float) (n float))). We should be able to define (define ((float float) (n int))) . Since fprint is a call to the c function. We must be able to implement something similar here

[adi-lb-phoenix]

So I have added the statements .

(define ((float float) (n int)))

(declare size_float float) 
(set size_float (call float size))

We will use floating point value in places floating point operations are done and integer value in places integer operations are done. The variables representing int and float formats are size and size_float respectively. (define ((float float) (n int))) is a function call to C where int value will be converted into float.

[adi-lb-phoenix]

Command executed : guile ../../utils/sexp-json.scm < llama2.c/rnsnorm/rns.sexp | python ../../c-lisp.py | python ../../brilisp.py | python ../../llvm.py status : no error. Successfully compiled.

command executed : guile ../../utils/sexp-json.scm < llama2.c/rnsnorm/rns.sexp | python ../../c-lisp.py | python ../../brilisp.py | python ../../llvm.py | bash ../brilisp/run.sh {args}

Output :

/usr/bin/ld: /gnu/store/ln6hxqjvz6m9gdd9s97pivlqck7hzs99-glibc-2.35/lib/Scrt1.o: in function `_start':
(.text+0x17): undefined reference to `main'
/usr/bin/ld: /tmp/tmp-1f644f.o: in function `rnsorm':
tmp.cGr2cmLmuI.ll:(.text+0x16): undefined reference to `float'
/usr/bin/ld: tmp.cGr2cmLmuI.ll:(.text+0xda): undefined reference to `sqrtf'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
../brilisp/run.sh: line 9: /tmp/tmp.kvsrEhQyWh.out: No such file or directory
rm: cannot remove '/tmp/tmp.kvsrEhQyWh.out': No such file or directory

[GlowingScrewdriver]

I think it would be better to implement casting as an operator, rather than as a function. We can use LLVM's casting operators. Syntax for casting could be something like:

(declare i-val int)
(set i-val 5) ; i-val is integer 5

(declare f-val float)
(set f-val (int->float i-val)) ; f-val is i-val converted to float

[adi-lb-phoenix]

I tried this. In the runtime.c file, included the below code :

float int_to_float(int x){
    float y ;
    y = x ;
    return y ;
}

float sqrtf(float x){
    return sqrtf(x);
}

[adi-lb-phoenix]

Command executed :

guile ../../utils/sexp-json.scm < llama2.c/rnsnorm/rns.sexp | python ../../c-lisp.py | python ../../brilisp.py | python ../../llvm.py | bash ../brilisp/run.sh {args}

C- lisp code .

(c-lisp
    (define ((sqrtf float ) (ss float)))

    (define ((int_to_float float) (n int)))

    (define ((rnsorm void) (o (ptr float)) (x (ptr float)) (weight (ptr float)) (size int))
        (declare ss float)
        (declare n float)
        (declare j int)
        (declare e float)
        (declare size_float float)
        (set e 2.71828)
        (set ss 0.0)
        (set size_float (call int_to_float size))

        (for ((set j 0) (lt j size) (set j (add j 1)))
            (set ss (fadd ss (fmul (load (ptradd x j))  (load (ptradd x j)) ))) 
        )
        (set ss (fdiv ss size_float))
        (set ss (fadd ss (fsub e 5.0) ))
        (set ss (fdiv 1.0 (call sqrtf ss)))

        (for ((set j 0) (lt j size) (set j (add j 1)))
            (store (ptradd o j) (fmul (load (ptradd weight j)) (fmul ss (load(ptradd x j)))))
        )

    )

)

Output :

/usr/bin/ld: /gnu/store/ln6hxqjvz6m9gdd9s97pivlqck7hzs99-glibc-2.35/lib/Scrt1.o: in function `_start':
(.text+0x17): undefined reference to `main'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
../brilisp/run.sh: line 9: /tmp/tmp.vGEByLrCxB.out: No such file or directory
rm: cannot remove '/tmp/tmp.vGEByLrCxB.out': No such file or directory

[GlowingScrewdriver]

You don't have a main function.

(.text+0x17): undefined reference to `main'

[GlowingScrewdriver]

Also, your definition of sqrtf will just go into a recursion loop... It will just keep calling itself.

But your int_to_float function is a good workaround for the time being.

[adi-lb-phoenix]

@GlowingScrewdriver

Command executed :

guile ../../utils/sexp-json.scm < llama2.c/rnsnorm/rns.sexp | python ../../c-lisp.py

Output :

Error in statement: ['set', 'f-val', ['int->float', 'size']]
Traceback (most recent call last):
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../c-lisp.py", line 473, in <module>
    json.dump(brilisp_code_generator.c_lisp(json.loads(sys.stdin.read())), sys.stdout)
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../c-lisp.py", line 59, in c_lisp
    return ["brilisp"] + [self.gen_function(fn) for fn in prog[1:]]
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../c-lisp.py", line 59, in <listcomp>
    return ["brilisp"] + [self.gen_function(fn) for fn in prog[1:]]
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../c-lisp.py", line 125, in gen_function
    *self.gen_compound_stmt(
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../c-lisp.py", line 303, in gen_compound_stmt
    instr_list += self.gen_stmt(s)
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../c-lisp.py", line 163, in gen_stmt
    raise e
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../c-lisp.py", line 158, in gen_stmt
    return self.gen_expr(stmt)
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../c-lisp.py", line 452, in gen_expr
    return self.gen_set_expr(expr, res_sym)
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../c-lisp.py", line 317, in gen_set_expr
    instr_list = self.gen_expr(expr[2], res_sym=res_sym)
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../c-lisp.py", line 468, in gen_expr
    raise CodegenError(f"Bad expression: {expr}")
__main__.CodegenError: Bad expression: ['int->float', 'size']

My -lisp code :

(c-lisp
    (define ((sqrtf float ) (ss float)))

    (define ((int_to_float float) (n int)))

    (define ((rnsorm void) (o (ptr float)) (x (ptr float)) (weight (ptr float)) (size int))
        (declare ss float)
        (declare n float)
        (declare j int)
        (declare e float)
        (declare size_float float)
        (set e 2.71828)
        (set ss 0.0)
        (set size_float (call int_to_float size))
        (declare f-val float)
        (set f-val (int->float size))  

        (for ((set j 0) (lt j size) (set j (add j 1)))
            (set ss (fadd ss (fmul (load (ptradd x j))  (load (ptradd x j)) ))) 
        )
        (set ss (fdiv ss size_float))
        (set ss (fadd ss (fsub e 5.0) ))
        (set ss (fdiv 1.0 (call sqrtf ss)))

        (for ((set j 0) (lt j size) (set j (add j 1)))
            (store (ptradd o j) (fmul (load (ptradd weight j)) (fmul ss (load(ptradd x j)))))
        )

    )

)

Statements added :

(declare f-val float)
(set f-val (int->float size)) 

size is a int variable defined in the parameter list of the function definition .

[adi-lb-phoenix]

You don't have a main function.

(.text+0x17): undefined reference to `main'

wanted to compile first without errors in the function

Also, your definition of sqrtf will just go into a recursion loop... It will just keep calling itself.

oh yes got it

[adi-lb-phoenix]

float sqrt_f(float x){ float y = sqrtf(x); return y ; }

[GlowingScrewdriver]

main.CodegenError: Bad expression: ['int->float', 'size']

Like I said, casting isn't implemented yet. When we do, we can adopt syntax like int->float. For the time being, defining int_to_float in runtime.c like you did earlier is a good idea.

[GlowingScrewdriver]

float sqrt_f(float x){ float y = sqrtf(x); return y ; }

This should work, yes

[adi-lb-phoenix]

My runtime.c contains

#include < math.h>
float int_to_float(int x){
    float y ;
    y = x ;
    return y ;
}

float sqrt_f(float x){
    float y = sqrt(x);
    return y ;
}

These are the contents of run.sh

#!/bin/bash
cd "$(dirname "$0")"

tmp_in=$(mktemp --suffix '.ll')
tmp_out=$(mktemp --suffix '.out')

cp /dev/stdin $tmp_in
clang $tmp_in runtime.c -o $tmp_out -Wno-override-module -O2 -lm
$tmp_out $@

rm $tmp_in $tmp_out

[adi-lb-phoenix]

Command executed :

guile ../../utils/sexp-json.scm < llama2.c/rnsnorm/rns.sexp | python ../../c-lisp.py | python ../../brilisp.py | python ../../llvm.py 

Output Received :

Exception: Unknown variable: j; in instruction Munch({'op': 'id', 'type': 'int', 'dest': 'tmp_clisp.inp_0-zfhebfqz', 'args': ['j']}):
Traceback (most recent call last):
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../llvm.py", line 359, in <module>
    main()
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../llvm.py", line 354, in main
    code_gen.generate(bril_prog)
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../llvm.py", line 44, in generate
    self.gen_function(fn)
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../llvm.py", line 345, in gen_function
    self.gen_instructions(fn_instrs)
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../llvm.py", line 261, in gen_instructions
    raise e
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../llvm.py", line 250, in gen_instructions
    gen_id(instr)
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../llvm.py", line 221, in gen_id
    self.gen_symbol_store(instr.dest, self.gen_symbol_load(instr.args[0]))
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../llvm.py", line 305, in gen_symbol_load
    raise CodegenError(f"Unknown variable: {name}")
__main__.CodegenError: Unknown variable: j

my c-lisp code :

(c-lisp
    (define ((sqrt_f float ) (ss float)))

    (define ((fprint float) (n  float) ))

    (define ((int_to_float float) (n int)))

    (define ((rnsorm void) (o (ptr float)) (x (ptr float)) (weight (ptr float)) (size int))
        (call fprint 1.0)
        (ret)
        (declare ss float)
        (declare n float)
        (declare j int)
        (declare e float)
        (declare size_float float)
        (set e 2.71828)
        (set ss 0.0)
        (set size_float (call int_to_float size)) 
        (set j 0)

        (for ((set j 0) (lt j size) (set j (add j 1)))
            (set ss (fadd ss (fmul (load (ptradd x j))  (load (ptradd x j)) ))) 
        )
        (set ss (fdiv ss size_float))
        (set ss (fadd ss (fsub e 5.0) ))
        (set ss (fdiv 1.0 (call sqrt_f ss)))

        (for ((set j 0) (lt j size) (set j (add j 1)))
            (store (ptradd o j) (fmul (load (ptradd weight j)) (fmul ss (load(ptradd x j)))))
        )

    )
)

It says unknown variable 'j' But I have defined (declare j int) and (set j 0).

[adi-lb-phoenix]

Command executed :

guile ../../utils/sexp-json.scm < llama2.c/rnsnorm/rns.sexp | python ../../c-lisp.py | python ../../brilisp.py | python ../../llvm.py | bash ../brilisp/run.sh {args}

(c-lisp
    (define ((sqrt_f float ) (ss float)))

    (define ((fprint float) (n  float) ))

    (define ((int_to_float float) (n int)))

    (define ((rnsorm void) (o (ptr float)) (x (ptr float)) (weight (ptr float)) (size int))
        (call fprint 1.0)
        (declare ss float)
        (declare n float)
        (declare j int)
        (declare e float)
        (declare size_float float)
        (set e 2.71828)
        (set ss 0.0)
        (set size_float (call int_to_float size)) 
        (set j 0)

        (for ((set j 0) (lt j size) (set j (add j 1)))
            (set ss (fadd ss (fmul (load (ptradd x j))  (load (ptradd x j)) ))) 
        )
        (set ss (fdiv ss size_float))
        (set ss (fadd ss (fsub e 5.0)))
        (set ss (fdiv 1.0 (call sqrt_f ss)))

        (for ((set j 0) (lt j size) (set j (add j 1)))
            (store (ptradd o j) (fmul (load (ptradd weight j)) (fmul ss (load(ptradd x j)))))
        )
        (ret)

    )
    (define ((main void))
    (ret)
    )
)

The above code compiles.

[adi-lb-phoenix]

Command executed :

guile ../../utils/sexp-json.scm < llama2.c/rnsnorm/rns.sexp | python ../../c-lisp.py | python ../../brilisp.py | python ../../llvm.py 

Output Received :

Exception: Unknown variable: j; in instruction Munch({'op': 'id', 'type': 'int', 'dest': 'tmp_clisp.inp_0-zfhebfqz', 'args': ['j']}):
Traceback (most recent call last):
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../llvm.py", line 359, in <module>
    main()
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../llvm.py", line 354, in main
    code_gen.generate(bril_prog)
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../llvm.py", line 44, in generate
    self.gen_function(fn)
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../llvm.py", line 345, in gen_function
    self.gen_instructions(fn_instrs)
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../llvm.py", line 261, in gen_instructions
    raise e
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../llvm.py", line 250, in gen_instructions
    gen_id(instr)
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../llvm.py", line 221, in gen_id
    self.gen_symbol_store(instr.dest, self.gen_symbol_load(instr.args[0]))
  File "/home/aadithya.bhat/c_lisp/bril/bril-txt/llama.lisp/src/backend/tests/c-lisp/../../llvm.py", line 305, in gen_symbol_load
    raise CodegenError(f"Unknown variable: {name}")
__main__.CodegenError: Unknown variable: j

my c-lisp code :

(c-lisp
    (define ((sqrt_f float ) (ss float)))

    (define ((fprint float) (n  float) ))

    (define ((int_to_float float) (n int)))

    (define ((rnsorm void) (o (ptr float)) (x (ptr float)) (weight (ptr float)) (size int))
        (call fprint 1.0)
        (ret)
        (declare ss float)
        (declare n float)
        (declare j int)
        (declare e float)
        (declare size_float float)
        (set e 2.71828)
        (set ss 0.0)
        (set size_float (call int_to_float size)) 
        (set j 0)

        (for ((set j 0) (lt j size) (set j (add j 1)))
            (set ss (fadd ss (fmul (load (ptradd x j))  (load (ptradd x j)) ))) 
        )
        (set ss (fdiv ss size_float))
        (set ss (fadd ss (fsub e 5.0) ))
        (set ss (fdiv 1.0 (call sqrt_f ss)))

        (for ((set j 0) (lt j size) (set j (add j 1)))
            (store (ptradd o j) (fmul (load (ptradd weight j)) (fmul ss (load(ptradd x j)))))
        )

    )
)

It says unknown variable 'j' But I have defined (declare j int) and (set j 0).

I removed the (ret) instructions from rnsnorm function in the third line.

(call fprint 1.0)
        (ret)

[GlowingScrewdriver]

The issue was the ret at the top of the function. Brilisp removed a bunch of unreachable instructions, after the jump instruction emitted by ret. However, since an access of j appeared under some other label, it didn't get removed. This is an interesting issue.