Update to 24.2 WIP

[anthony-chaudhary]

Related to https://github.com/tairov/llama2.mojo/issues/88

Work in progress, still getting these errors. I'm new to Mojo so if anyone can assist:

/root/llama2.mojo/llama2.mojo:506:20: error: 'String' value has no attribute 'bitcast'
    buf.data = data.bitcast[DType.uint8]()
               ~~~~^~~~~~~~
/root/llama2.mojo/llama2.mojo:517:29: error: no matching method in call to 'load'
        a.load[width=_nelts](j, a.load[width=_nelts](j) + b.load[width=_nelts](j))
        ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: method argument #1 cannot be converted from 'SIMD[f32, _nelts]' to 'Int'
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 1 positional argument, got 2
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:543:39: error: no matching method in call to 'load'
        o.offset(j).load[width=_nelts](0, val)
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 1 positional argument, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:580:29: error: no matching method in call to 'load'
        x.load[width=_nelts](start + ii, val)
        ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: method argument #1 cannot be converted from 'SIMD[f32, _nelts]' to 'Int'
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 1 positional argument, got 2
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:589:29: error: no matching method in call to 'load'
        x.load[width=_nelts](start + ii, x.load[width=_nelts](start + ii) / ssum)
        ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: method argument #1 cannot be converted from 'SIMD[f32, _nelts]' to 'Int'
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 1 positional argument, got 2
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:821:48: error: no matching method in call to 'load'
                    state.xb.load[width=_nelts](xb_offset + i, xbi)
                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: method argument #1 cannot be converted from 'SIMD[f32, _nelts]' to 'Int'
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 1 positional argument, got 2
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:850:40: error: no matching method in call to 'load'
            state.hb.load[width=_nelts](i, hbi * state.hb2.load[width=_nelts](i))
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: method argument #1 cannot be converted from 'SIMD[f32, _nelts]' to 'Int'
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 1 positional argument, got 2
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:881:32: error: invalid call to 'rand': missing 1 required positional argument: 'size'
    var r = rand[DType.float32](1)
            ~~~~~~~~~~~~~~~~~~~^~~
/root/llama2.mojo/llama2.mojo:1:1: note: function declared here
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:50:37: warning: 'SIMD[T, _width]' value is unused
        self.data.load[width=_width](newVal)
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~
        _ = 
/root/llama2.mojo/llama2.mojo:111:38: error: no matching method in call to 'load'
        return self._data.load[nelts](idx)
               ~~~~~~~~~~~~~~~~~~~~~~^~~~~
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 1 positional argument, got 2
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: callee parameter #2 has 'Intable' type, but value has type 'Int'
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:119:38: error: no matching method in call to 'load'
        return self.load[width=nelts](indices[0] * self._shape[1] + indices[1])
               ~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/root/llama2.mojo/llama2.mojo:110:8: note: candidate not viable: unknown keyword parameter: 'width'
    fn load[nelts: Int](self, idx: Int) -> SIMD[DType.float32, nelts]:
       ^
/root/llama2.mojo/llama2.mojo:113:8: note: candidate not viable: unknown keyword parameter: 'width'
    fn load[nelts: Int](self, *indices: Int) -> SIMD[DType.float32, nelts]:
       ^
/root/llama2.mojo/llama2.mojo:121:8: note: candidate not viable: unknown keyword parameter: 'width'
    fn load[
       ^
/root/llama2.mojo/llama2.mojo:129:8: note: candidate not viable: missing 1 required positional argument: 'val'
    fn load[nelts: Int](self, idx: Int, val: SIMD[DType.float32, nelts]):
       ^
/root/llama2.mojo/llama2.mojo:130:38: error: no matching method in call to 'load'
        return self._data.load[nelts](idx, val)
               ~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 1 positional argument, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:133:34: error: no matching method in call to 'load'
        return self.load[width=1](idx, val)
               ~~~~~~~~~~~~~~~~~~^~~~~~~~~~
/root/llama2.mojo/llama2.mojo:110:8: note: candidate not viable: expected at most 2 positional arguments, got 3
    fn load[nelts: Int](self, idx: Int) -> SIMD[DType.float32, nelts]:
       ^
/root/llama2.mojo/llama2.mojo:113:8: note: candidate not viable: unknown keyword parameter: 'width'
    fn load[nelts: Int](self, *indices: Int) -> SIMD[DType.float32, nelts]:
       ^
/root/llama2.mojo/llama2.mojo:121:8: note: candidate not viable: expected at most 2 positional arguments, got 3
    fn load[
       ^
/root/llama2.mojo/llama2.mojo:129:8: note: candidate not viable: unknown keyword parameter: 'width'
    fn load[nelts: Int](self, idx: Int, val: SIMD[DType.float32, nelts]):
       ^
/root/llama2.mojo/llama2.mojo:382:40: error: 'List[SIMD[si8, 1]]' value has no attribute 'bitcast'
        var int32_ptr = config_data_raw.bitcast[DType.int32]()
                        ~~~~~~~~~~~~~~~^~~~~~~~
/root/llama2.mojo/llama2.mojo:469:27: error: 'List[SIMD[si8, 1]]' value has no attribute 'bitcast'
            var data = tmp.bitcast[DType.float32]()
                       ~~~^~~~~~~~
/root/.modular/pkg/packages.modular.com_max/bin/mojo: error: failed to parse the provided Mojo

[mikowals]

I pulled this and tried against Mojo 24.2 but I think maybe you are working against the nightly branch? So these changes are likely to be needed in 24.4 assuming we get a release later this month that makes these new Mojo APIs public.

update: This looks like my own mistake not shifting the VS Code extension to 24.2. Your changes here are indeed for 24.2 so ignore this comment.

[anthony-chaudhary]

Hi Michael, thanks for all the updates 😎, I'll take a closer look and check version tonight! 😊

On Mon, Apr 15, 2024, 7:58 PM Michael Kowalski @.***> wrote:

I pulled this and tried against Mojo 24.2 but I think maybe you are working against the nightly branch? So these changes are likely to be needed in 24.4 assuming we get a release later this month that makes these new Mojo APIs public.

— Reply to this email directly, view it on GitHub https://github.com/tairov/llama2.mojo/pull/89#issuecomment-2058142139, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEJ6DJAHEC5CLJLKT2XO3YDY5SHT7AVCNFSM6AAAAABGIKGSHOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJYGE2DEMJTHE . You are receiving this because you authored the thread.Message ID: @.***>

[anthony-chaudhary]

Hello!

Okay so I think I fixed most of the items mentioned. Sorry about simd_store mix up I read the change log a little too fast and missed that.

I fixed a new error https://github.com/anthony-sarkis/llama2.mojo/commit/017081e329c540770ee718f584f6cf36f5669e15

The only two sets of remaining errors seem to be

1. Attempting to add a None check to fix subscript error:

   /home/anthony/llama2.mojo/llama2.mojo:886:13: error: 'None' is not subscriptable, it does not implement the `__getitem__`/`__setitem__` or `__refitem__` methods
       if r[0] < cdf:

   https://github.com/anthony-sarkis/llama2.mojo/commit/394d31f8ad4056f78718943cffe99fd546b6a20a But it still yields:

   /home/anthony/llama2.mojo/llama2.mojo:886:12: error: 'None' does not implement the '__bool__' method
       if r:

   Not surer if I'm missing something obvious but it seems like this isn't implemented in mojo the same as python?

2. The bitcast set of errors (which seems to actually be attribute errors, since it seems to be there either way with or without steal pointer) https://github.com/tairov/llama2.mojo/pull/89#discussion_r1568266979

anthony@Anthony:~/llama2.mojo$ mojo llama2.mojo stories15M.bin -s 100 -n 256 -t 0.5 -i "Mojo is a language"
/home/anthony/llama2.mojo/llama2.mojo:506:20: error: 'String' value has no attribute 'bitcast'
    buf.data = data.bitcast[DType.uint8]()
               ~~~~^~~~~~~~
/home/anthony/llama2.mojo/llama2.mojo:886:12: error: 'None' does not implement the '__bool__' method
        if r:
           ^
/home/anthony/llama2.mojo/llama2.mojo:382:40: error: 'List[SIMD[si8, 1]]' value has no attribute '_steal_ptr'
        var int32_ptr = config_data_raw._steal_ptr().bitcast[DType.int32]()
                        ~~~~~~~~~~~~~~~^~~~~~~~~~~
/home/anthony/llama2.mojo/llama2.mojo:469:27: error: 'List[SIMD[si8, 1]]' value has no attribute 'bitcast'
            var data = tmp.bitcast[DType.float32]()
                       ~~~^~~~~~~~
mojo: error: failed to parse the provided Mojo

Edit: Sounds like version thing isn't a concern but just in case this is the output of mojo -v mojo 24.2.1 (58157dc0)

[mikowals]

This error:

/home/anthony/llama2.mojo/llama2.mojo:886:13: error: 'None' is not subscriptable, it does not implement the `__getitem__`/`__setitem__` or `__refitem__` methods
        if r[0] < cdf:

Is because rand that creates a random Tensor moved from the random module to the tensor module. So rand needs to be added to the list of imports from tensor at the top of the file. And the from random line can be removed.

[tairov]

Hi @anthony-sarkis , I added a few fixes. Now it llama2 is compiling fine

[anthony-chaudhary]

Awesome, thanks everyone! 😊

On Wed, Apr 17, 2024, 1:44 AM Aydyn Tairov @.***> wrote:

Merged #89 https://github.com/tairov/llama2.mojo/pull/89 into master.

— Reply to this email directly, view it on GitHub https://github.com/tairov/llama2.mojo/pull/89#event-12504050837, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEJ6DJBD56NAAFTDUW25ANTY5YZAHAVCNFSM6AAAAABGIKGSHOVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJSGUYDIMBVGA4DGNY . You are receiving this because you were mentioned.Message ID: @.***>