NolanoOrg / cformers

SoTA Transformers with C-backend for fast inference on your CPU.
MIT License
311 stars 29 forks source link

created a pip package #37

Open kamalojasv181 opened 1 year ago

tejasvaidhyadev commented 1 year ago

It seems I am still facing an issue with the execution of a subprocess in Python.
Using pip install cformers

  File "<stdin>", line 1, in <module>
  File "/Users/tejasvaidhya/miniconda3/lib/python3.9/site-packages/cformers/interface.py", line 218, in generate
    process = Popen(command, stdout=PIPE, stderr=PIPE)
  File "/Users/tejasvaidhya/miniconda3/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/Users/tejasvaidhya/miniconda3/lib/python3.9/subprocess.py", line 1821, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: '/Users/tejasvaidhya/miniconda3/lib/python3.9/site-packages/cformers/cpp/main'
>>> x = ai.generate("<|prompter|>What's the Earth total population<|endoftext|><|assistant|>", num_tokens_to_generate=100)z

Also getting similar error on windows

mallorbc commented 1 year ago

I was just about to create an issue to potentially help with this. Glad to see work on this is ongoing.

I will try this PR soon.

@tejasvaidhyadev whatever issue you are facing is dealing with the binary file that is used for running gmml models

kamalojasv181 commented 1 year ago

Hey @tejasvaidhyadev I cannot replicate this. I have tried to load models on my own machine (Ubuntu 20.04) and google colab and I am successfully able to load models. I do not have access to a windows machine. Perhaps could you state the exact command that you used to run the models? Thanks

kamalojasv181 commented 1 year ago

@tejasvaidhyadev I understand the issue. The Linux binary does not work in windows. Can you make and send the binary over to me? Thanks

tejasvaidhyadev commented 1 year ago

sure, I will send binary for you. Give me sometime