search

pytorch / torchchat

Run PyTorch LLMs locally on servers, desktop and mobile
BSD 3-Clause "New" or "Revised" License
3.4k stars 225 forks source link

Add Intel XPU device support to generate and serve #1361

Open jenniew opened 2 weeks ago

jenniew commented 2 weeks ago

Add XPU device support exclude distributed mode, workflow, documentation.

pytorch-bot[bot] commented 2 weeks ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/1361

Note: Links to docs will display an error until the docs builds have been completed.

:heavy_exclamation_mark: 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

:white_check_mark: No Failures

As of commit 4d16351421bcb0240fc4463e3620bf2e47581501 with merge base 2cf1a17a74799cc797a525eafa27da575d568193 (image): :green_heart: Looks good so far! There are no failures yet. :green_heart:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot commented 2 weeks ago

Hi @jenniew!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

mikekgfb commented 2 weeks ago

Is there a way to run at least a few simple tests on an xpu to avoid inadvertent breakage?

jenniew commented 2 weeks ago

Is there a way to run at least a few simple tests on an xpu to avoid inadvertent breakage?

For generate, run a simple test by python3 torchchat.py generate llama3.1 --prompt "write me a story about a boy and his bear --device xpu For serve, run a simple test by python3 torchchat.py server llama3.1 --device xpu

Jack-Khuu commented 2 weeks ago

Welcome to torchchat and thanks for adding @jenniew!!

Super stoked to see that it didn't require much lift to get XPU set up. Added a few questions on the versioning difference. I'll try to get some folks to test and see if we can get some recurring jobs set up.

What device did you test on btw?

Tagging a few folk who I'm trying to help taking a larger role

Jack-Khuu commented 2 weeks ago

@jenniew Also do you mind filling out the CLA? It'll allow you to contribute to Meta repos

jenniew commented 2 weeks ago

@jenniew Also do you mind filling out the CLA? It'll allow you to contribute to Meta repos

Yes, I just signed the CLA

jenniew commented 2 weeks ago

Welcome to torchchat and thanks for adding @jenniew!!

Super stoked to see that it didn't require much lift to get XPU set up. Added a few questions on the versioning difference. I'll try to get some folks to test and see if we can get some recurring jobs set up.

What device did you test on btw?

Tagging a few folk who I'm trying to help taking a larger role

I tested on Intel Data Center GPU Max 1100.

Jack-Khuu commented 1 week ago

Just an update: