A Docker-powered service for PDF document layout analysis. This service provides a powerful and flexible PDF analysis service. The service allows for the segmentation and classification of different parts of PDF pages, identifying the elements such as texts, titles, pictures, tables and so on.
output = client.chat.completions.create(
model="meta-llama/Meta-Llama-3-8B-Instruct",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Count to 10"},
],
stream=True,
max_tokens=1024,
)
for chunk in output:
print(chunk.choices[0].delta.content)
Why switching to InferenceClient if you already use OpenAI then? Because it's better integrated with HF services, such as the Serverless Inference API and Dedicated Endpoints. Check out the more detailed answer in this HF Post.
For more details about OpenAI compatibility, check out this guide's section.
True OpenAI drop-in replacement by InferenceClient by @Wauplin in #2384
Promote chat_completion in inference guide by @Wauplin in #2366
(other) InferenceClient improvements
Some new parameters have been added to the InferenceClient, following the latest changes in our Inference API:
prompt_name, truncate and normalize in feature_extraction
model_id and response_format, in chat_completion
adapter_id in text_generation
hypothesis_template and multi_labels in zero_shot_classification
Of course, all of those changes are also available in the AsyncInferenceClient async equivalent 🤗
Support truncate and normalize in InferenceClient by @Wauplin in #2270
Add prompt_name to feature-extraction + update types by @Wauplin in #2363
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Bumps huggingface-hub from 0.23.4 to 0.24.0.
Release notes
Sourced from huggingface-hub's releases.
... (truncated)
Commits
44b6e0f
Release: v0.24.083b5d7f
Merge branch 'main' into v0.24-release1f9dbf8
fix: Handle single return value. (#2396)b5e0d76
Update _async_client.py7fe286b
Merge branch 'main' into tv0.24-release9c98a4b
Do not mention gitalyUid in expand parameter (#2395)d8e92aa
Release: v0.24.0.rc036396f1
[InferenceClient] Add support foradapter_id
(text-generation) and `respons...6ddaf44
Fix list_accepted_access_requests if grant user manually (#2392)e370fa6
Prevent empty commits if files did not change (#2389)Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase
.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show