Google Colab Notebook Error

[ericborgos]

FYI, when I run your demo notebook at https://github.com/harrisonvanderbyl/rwkvstic/blob/master/notebooks/chatbot.ipynb in Colab I get this error:

init RWKVOPS, from super

[?] Max quantization target size (in GB):: 100

---

error Traceback (most recent call last)

in 2 from rwkvstic.load import RWKV 3 ----> 4 model = RWKV( 5 "https://huggingface.co/Hazzzardous/RWKV-8Bit/resolve/main/RWKV-4-Pile-7B-Instruct.pqth" 6 )

9 frames

/usr/local/lib/python3.8/dist-packages/rwkvstic/load.py in RWKV(path, mode, tokenizer, *args, **kwargs) 50 return rwkvRs.initRwkvRsFile(path, tokenizer) 51 elif path.endswith(".pqth"): ---> 52 return prequantized.loadPreQuantized(path, tokenizer) 53 elif path.endswith(".jax.npy"): 54 return preJax.loadPreJax(path, tokenizer)

/usr/local/lib/python3.8/dist-packages/rwkvstic/interOpLoaders/prequantized.py in loadPreQuantized(path, tokenizer) 22 n_layers = int(ww[0]) 23 ---> 24 ops = RWKVCudaQuantOps( 25 preQuantized=True, embed=len(weights["blocks.0.ln2.weight"]), layers=(n_layers+1), chunksize=32, target=100, useLogFix="logfix" in path, useGPU=torch.cuda.is_available(), runtimedtype=torch.bfloat16) 26 if "logfix" in path:

/usr/local/lib/python3.8/dist-packages/rwkvstic/agnostic/backends/torch.py in init(self, layers, embed, runtimedtype, dtype, useGPU, chunksize, preQuantized, maxQuantTarget, target, *args, **kwargs) 195 default="100")) if target is None and dev == "cuda" else target 196 --> 197 maxQuantTarget = float(inquirer.text("Max quantization target size (in GB):", 198 default="100")) if maxQuantTarget is None and dev == "cuda" else maxQuantTarget 199

/usr/local/lib/python3.8/dist-packages/inquirer/shortcuts.py in text(message, autocomplete, render, kwargs) 6 render = render or ConsoleRender() 7 question = questions.Text(name="", message=message, autocomplete=autocomplete, kwargs) ----> 8 return render.render(question) 9 10

/usr/local/lib/python3.8/dist-packages/inquirer/render/console/init.py in render(self, question, answers) 36 37 try: ---> 38 return self._event_loop(render) 39 finally: 40 print("")

/usr/local/lib/python3.8/dist-packages/inquirer/render/console/init.py in _event_loop(self, render) 49 self._print_options(render) 50 ---> 51 self._process_input(render) 52 self._force_initial_column() 53 except errors.EndOfInput as e:

/usr/local/lib/python3.8/dist-packages/inquirer/render/console/init.py in _process_input(self, render) 93 def _process_input(self, render): 94 try: ---> 95 ev = self._event_gen.next() 96 if isinstance(ev, events.KeyPressed): 97 render.process_input(ev.value)

/usr/local/lib/python3.8/dist-packages/inquirer/events.py in next(self) 20 21 def next(self): ---> 22 return KeyPressed(self._key_gen())

/usr/local/lib/python3.8/dist-packages/readchar/_posix_read.py in readkey() 32 read and returned as noted in _posix_key.py.""" 33 ---> 34 c1 = readchar() 35 36 if c1 in config.INTERRUPT_KEYS:

/usr/local/lib/python3.8/dist-packages/readchar/_posix_read.py in readchar() 16 17 fd = sys.stdin.fileno() ---> 18 old_settings = termios.tcgetattr(fd) 19 term = termios.tcgetattr(fd) 20 try:

error: (25, 'Inappropriate ioctl for device')

[dbddv01]

Same here. Trying to figure out where it could come from i stumble on another one. Don't know if it's linked so :

[harrisonvanderbyl]

ahh, I will update the colab

[harrisonvanderbyl]

Please try with current (0.2.3) and let me know if there is any more issues

[ericborgos]

It gave the same error again (I shut the notebook down and ran it from scratch). Do I need to change the notebook to specify your new version number? It currently just uses: !pip install inquirer transformers sty rwkvstic

[harrisonvanderbyl]

Yes,you may need to completely restart the colab

[ericborgos]

It works now.

[dbddv01]

Thx ! works perfect.