su77ungr / CASALIOY

♾️ toolkit for air-gapped LLMs on consumer-grade hardware
Apache License 2.0
230 stars 31 forks source link

Miscellaneous #93

Open williamspindox opened 1 year ago

williamspindox commented 1 year ago

Feature request

Hi, some questions:

1) is it possible to integrate your solution on AWS SageMaker? How? 2) did you try your solution on Windows? After following your steps, I get: ModuleNotFoundError: No module named 'casalioy'. Please enhance README for Windows users too. 3) I can run your docker image (please add -p 8501:8501 in the README for container run in order to access to the gui), but how can I update image after every new release without lost model and source_documents folders? Using a volume? Can you enhance README?

Thanks

Motivation

Improve your solution

Your contribution

-

su77ungr commented 1 year ago

Gonna take on this issue tomorrow. Did you test the release v0.1.0 on windows? Edit: not the issue

1. is it possible to integrate your solution on AWS SageMaker? How?

No familiar with the AWS ecosystem. I know about SageMaker Labs that colab clone but that's it. If it's akin to a docker implemenation feel free to pitch me your approach of implementing it.

2. did you try your solution on Windows? After following your steps, I get: ModuleNotFoundError: No module named 'casalioy'. Please enhance README for Windows users too.

On remote and inactive rn. Gonna check this tomorrow.

3. I can run your docker image (please add -p 8501:8501 in the README for container run in order to access to the gui), but how can I update image after every new release without lost model and source_documents folders? Using a volume? Can you enhance README?

This will be added Edit: done

williamspindox commented 1 year ago

Yes, v0.1.0

su77ungr commented 1 year ago

Docker for Windows is not supported atm.

Is the (built from source) windows issue resolved? It might be a badly configured virtual env. Since I could recreate this error when running ingest.py after deactivating my venv. So I'll update the ReadMe for this case.

williamspindox commented 1 year ago

Hi, I can run your solution on Docker for Windows (the steps on your README are not complete, but it works), why do you say it's not supported?

I was able to run on Windows shell too (built from source) simply run "poetry shell" before call python commands.

About AWS SageMaker, yes it's like colab with jupyter notebook, I think you should deepen this integration, because different companies are migrating on cloud to get more cpu/gpu/storage performance. Regards,

su77ungr commented 1 year ago

There were several issues with docker on Windows see #83. Also I did not test it for myself since I want Ubuntu to be running stable. Glad they have been resolved on your side.

Could you elaborate on the additional steps used by you to get this running on Windows, thanks.