Open AbhiPawar5 opened 1 month ago
Good catch!
I've added GPU support for nsfw text here: https://github.com/guardrails-ai/nsfw_text
For the profanity check, did you mean https://github.com/guardrails-ai/profanity_free ?
I believe this validator runs off of sci-kit on the backend, so I am not sure if it provides GPU support natively.
Thanks! I see the log saying pipeline is loaded and device set as CPU. I believe its still a Hugging Face pipeline
On Thu, 11 Jul, 2024, 10:09 pm Wyatt Lansford, @.***> wrote:
Good catch!
I've added GPU support for nsfw text here: https://github.com/guardrails-ai/nsfw_text
For the profanity check, did you mean https://github.com/guardrails-ai/profanity_free ?
I believe this validator runs off of sci-kit on the backend, so I am not sure if it provides GPU support natively.
— Reply to this email directly, view it on GitHub https://github.com/guardrails-ai/guardrails/issues/907#issuecomment-2223403445, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK3AUDEMIXLLG23RHOPLUSLZL2YLPAVCNFSM6AAAAABKWGDUJOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRTGQYDGNBUGU . You are receiving this because you authored the thread.Message ID: @.***>
Can you provide the logs? From what I can tell it uses,
https://github.com/vzhou842/profanity-check
Which is just an SVM on the backend. There shouldn't be any huggingface calls made for it
here is the code: https://github.com/guardrails-ai/profanity_free/blob/main/validator/main.py
Description For Profanity and NSFW detection, the code uses a transformer pipeline under the hood. However, the device argument should be passed to use the best hardware accelerator.
Why is this needed This is a general feature to speedup the pipeline inference significantly
Implementation details Adding
device
with proper integer should solve the problem. More details here - https://huggingface.co/transformers/v3.0.2/main_classes/pipelines.html#transformers.PipelineEnd result Across multiple pipelines and use cases.