Spark NLP getting BART models for Zero Shot Classifications

JohnSnowLabs / spark-nlp

State of the Art Natural Language Processing

https://sparknlp.org/

Apache License 2.0

3.81k stars 708 forks source link

Spark NLP getting BART models for Zero Shot Classifications #7162

Closed ericbugin closed 10 months ago

ericbugin commented 2 years ago

Hello, Was wondering if John Snow Labs had its eye on adding BART models from Huggingface for Zero Shot Classifications?

Here a model example from Huggingface https://huggingface.co/facebook/bart-large-mnli

Thanks

maziyarpanahi commented 2 years ago

I reopen this since it’s related to the discussion of how to import BART models.

dodo-robot commented 2 years ago

I'm also interested in importing a MBART model to make summaries, is it difficult to implement??

ericbugin commented 2 years ago

You do not need to wait for SparkNLP to implement it, you can download the HF BART model and use the spark "broadcast" function to push it to all your nodes, this will allow inference on a spark dataframe. you'll have to make certain all of the necessary python libraries (i.e. transformers, pytorch) are installed on all of your nodes as well

github-actions[bot] commented 1 year ago

This issue is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 5 days

maziyarpanahi commented 1 year ago

Reopening this feature request, it is still a valid request

github-actions[bot] commented 1 year ago

This issue is stale because it has been open 180 days with no activity. Remove stale label or comment or this will be closed in 5 days

maziyarpanahi commented 1 year ago

I am re-opening this as we already released BartTransformer for summarization, translation, and text-generation.
We have also released BertForZeroShotClassification annotator
I think the next best thing would be to also implement BartForZeroShotClassification as it seems to have more models out there

github-actions[bot] commented 11 months ago

This issue is stale because it has been open 180 days with no activity. Remove stale label or comment or this will be closed in 5 days

khanhvy31 commented 5 months ago

@maziyarpanahi "I am re-opening this as we already released BartTransformer for summarization, translation, and text-generation." Hi, could you direct me to the notebook where you transfer Bart to spark nlp please?