Open head-iie-vnr opened 5 months ago
& Generate key
They are many models (27308 as on today) available. They are clearly classified into groups. Research work: 80% of your research work would be already done by others so you can focus on the remaining 20% and complete the research.
Explanation of the webpage
Certainly! Here are the steps to create an API key on Hugging Face and use a model for a simple task.
Sign Up or Log In:
Access API Keys:
Create an API Key:
Let's say you want to use a pre-trained model for text classification. Here's how you can do it using the Hugging Face Transformers library in Python.
First, make sure you have the transformers
library installed. You can install it using pip:
pip install transformers
You may also need the requests
library for making HTTP requests if you don't have it installed:
pip install requests
Here's an example of how to use the Hugging Face API to perform a simple text classification task:
import requests
# Your Hugging Face API key
api_key = "YOUR_HUGGING_FACE_API_KEY"
# The model you want to use
model = "distilbert-base-uncased-finetuned-sst-2-english"
# The text you want to classify
text = "I love using Hugging Face models!"
# Set up the headers and data for the request
headers = {
"Authorization": f"Bearer {api_key}"
}
data = {
"inputs": text
}
# Make the request to the Hugging Face API
response = requests.post(
f"https://api-inference.huggingface.co/models/{model}",
headers=headers,
json=data
)
# Parse the response
result = response.json()
print(result)
"YOUR_HUGGING_FACE_API_KEY"
with the API key you generated."distilbert-base-uncased-finetuned-sst-2-english"
model, which is a text classification model fine-tuned on the SST-2 dataset.This example demonstrates how to use a pre-trained text classification model, but you can easily adapt it for other tasks like text generation, translation, or summarization by changing the model and input accordingly.
transformers
Library LocallyAlternatively, you can use the transformers
library to load and run models locally without making API requests:
from transformers import pipeline
# Load the pipeline for sentiment-analysis
classifier = pipeline("sentiment-analysis")
# Classify the text
result = classifier("I love using Hugging Face models!")
print(result)
This method does not require an API key and runs the model locally, which can be faster and avoids API request limits.
These steps should help you get started with using Hugging Face models for your tasks. If you have any specific task in mind or run into any issues, feel free to ask for further guidance!
python3 1_sentiment.py
text = "I hate using Hugging Face models!"
[[{'label': 'NEGATIVE', 'score': 0.9932259917259216}, {'label': 'POSITIVE', 'score': 0.00677394587546587}]]
text = "I like using Hugging Face models!"
[[{'label': 'POSITIVE', 'score': 0.9786928296089172}, {'label': 'NEGATIVE', 'score': 0.0213072020560503}]]
text = "I love so much using Hugging Face models!"
[[{'label': 'POSITIVE', 'score': 0.9998047947883606}, {'label': 'NEGATIVE', 'score': 0.00019523841910995543}]]
Sentiment Analysis:
distilbert-base-uncased-finetuned-sst-2-english
Named Entity Recognition (NER):
dbmdz/bert-large-cased-finetuned-conll03-english
Text Summarization:
facebook/bart-large-cnn
Text Generation:
gpt-2
Question Answering:
deepset/roberta-base-squad2
By changing code a little, we can download the code to local system. The models are stored in
.cache/huggingface/transformers
Then the models are run from local system
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ python3 2_text_process.py
Choose a text analysis task:
1. Sentiment Analysis
2. Named Entity Recognition
3. Text Summarization
4. Text Generation
5. Question Answering
Enter the number of your choice: 1
Enter text for sentiment analysis: I love taj Mahal.
[[{'label': 'POSITIVE', 'score': 0.9998650550842285}, {'label': 'NEGATIVE', 'score': 0.00013494079757947475}]]
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ python3 2_text_process.py
Choose a text analysis task:
1. Sentiment Analysis
2. Named Entity Recognition
3. Text Summarization
4. Text Generation
5. Question Answering
Enter the number of your choice: 2
Enter text for named entity recognition: Taj Mahal
{'error': 'Model dbmdz/bert-large-cased-finetuned-conll03-english is currently loading', 'estimated_time': 53.3741569519043}
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ python3 2_text_process.py
Choose a text analysis task:
1. Sentiment Analysis
2. Named Entity Recognition
3. Text Summarization
4. Text Generation
5. Question Answering
Enter the number of your choice: 2
Enter text for named entity recognition: Taj Mahal
[{'entity_group': 'MISC', 'score': 0.6656236052513123, 'word': 'Taj Mahal', 'start': 0, 'end': 9}]
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ python3 2_text_process.py
Choose a text analysis task:
1. Sentiment Analysis
2. Named Entity Recognition
3. Text Summarization
4. Text Generation
5. Question Answering
Enter the number of your choice: 2
Enter text for named entity recognition: Hyderabad
[{'entity_group': 'LOC', 'score': 0.9817549586296082, 'word': 'Hyderabad', 'start': 0, 'end': 9}]
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ python3 2_text_process.py
Choose a text analysis task:
1. Sentiment Analysis
2. Named Entity Recognition
3. Text Summarization
4. Text Generation
5. Question Answering
Enter the number of your choice: 2
Enter text for named entity recognition: Gandhi
[{'entity_group': 'PER', 'score': 0.9895268678665161, 'word': 'Gandhi', 'start': 0, 'end': 6}]
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ python3 3
python3: can't open file '/home/kp/kp-gen-ai/exp3-huggf/3': [Errno 2] No such file or directory
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ python3 2_text_process.py
Choose a text analysis task:
1. Sentiment Analysis
2. Named Entity Recognition
3. Text Summarization
4. Text Generation
5. Question Answering
Enter the number of your choice: 3
Enter text for summarization: Tommy was a boy with an insatiable love for sweets. From dawn till dusk, his pockets bulged with candies, chocolates, and cookies. His chubby cheeks and round belly spoke of his indulgence. Despite knowing he was already overweight, Tommy couldn’t resist the allure of sugar. One day, while savoring a giant lollipop, he caught his reflection and sighed. Determined to change, Tommy decided to balance his love for sweets with healthier choices and exercise. With every step, he found joy in his progress, proving that change, though challenging, was possible. His journey had just begun, sweet and steady.
[{'summary_text': 'Tommy was a boy with an insatiable love for sweets. His pockets bulged with candies, chocolates, and cookies. Determined to change, Tommy decided to balance sweets with healthier choices and exercise. With every step, he found joy in his progress, proving that change, though challenging, was possible.'}]
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ python3 2_text_process.py
Choose a text analysis task:
1. Sentiment Analysis
2. Named Entity Recognition
3. Text Summarization
4. Text Generation
5. Question Answering
Enter the number of your choice: 4
Enter text for text generation: Education system in Rural India.
{'error': 'Model gpt-2 does not exist'}
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ python3 2_text_process.py
Choose a text analysis task:
1. Sentiment Analysis
2. Named Entity Recognition
3. Text Summarization
4. Text Generation
5. Question Answering
Enter the number of your choice: 5
Enter context for question answering: Tommy was a boy with an insatiable love for sweets. From dawn till dusk, his pockets bulged with candies, chocolates, and cookies. His chubby cheeks and round belly spoke of his indulgence. Despite knowing he was already overweight, Tommy couldn’t resist the allure of sugar. One day, while savoring a giant lollipop, he caught his reflection and sighed. Determined to change, Tommy decided to balance his love for sweets with healthier choices and exercise. With every step, he found joy in his progress, proving that change, though challenging, was possible. His journey had just begun, sweet and steady.
Enter question: What is the boy name?
{'error': 'Model deepset/roberta-base-squad2 is currently loading', 'estimated_time': 20.0}
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ python3 2_text_process.py
Choose a text analysis task:
1. Sentiment Analysis
2. Named Entity Recognition
3. Text Summarization
4. Text Generation
5. Question Answering
Enter the number of your choice: 5
Enter context for question answering: Enter context for question answering: Tommy was a boy with an insatiable love for sweets. From dawn till dusk, his pockets bulged with candies, chocolates, and cookies. His chubby cheeks and round belly spoke of his indulgence. Despite knowing he was already overweight, Tommy couldn’t resist the allure of sugar. One day, while savoring a giant lollipop, he caught his reflection and sighed. Determined to change, Tommy decided to balance his love for sweets with healthier choices and exercise. With every step, he found joy in his progress, proving that change, though challenging, was possible. His journey had just begun, sweet and steady.
Enter question: What is the boy name
{'score': 0.8775681853294373, 'start': 38, 'end': 43, 'answer': 'Tommy'}
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ python3 2_text_process.py
Choose a text analysis task:
1. Sentiment Analysis
2. Named Entity Recognition
3. Text Summarization
4. Text Generation
5. Question Answering
Enter the number of your choice: 5
Enter context for question answering: Enter context for question answering: Tommy was a boy with an insatiable love for sweets. From dawn till dusk, his pockets bulged with candies, chocolates, and cookies. His chubby cheeks and round belly spoke of his indulgence. Despite knowing he was already overweight, Tommy couldn’t resist the allure of sugar. One day, while savoring a giant lollipop, he caught his reflection and sighed. Determined to change, Tommy decided to balance his love for sweets with healthier choices and exercise. With every step, he found joy in his progress, proving that change, though challenging, was possible. His journey had just begun, sweet and steady.
Enter question: Explain about Tommy
{'score': 0.09513478726148605, 'start': 38, 'end': 88, 'answer': 'Tommy was a boy with an insatiable love for sweets'}
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ python3 2_text_process.py
Choose a text analysis task:
1. Sentiment Analysis
2. Named Entity Recognition
3. Text Summarization
4. Text Generation
5. Question Answering
Enter the number of your choice: 3
Enter text for summarization: asd
sdf
sdfsdf
sdf
sdf
sdf
d[{'summary_text': 'Asd. asd asd. is an online magazine for young people. The magazine is published by iReporter.com. For more information on the magazine, visit: http://www.iReport.com/Asd-asd-is-an-Online-Magazine-for-Young-People.'}]
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ sdf
Command 'sdf' not found, but can be installed with:
sudo apt install sdf
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ sdfsdf
sdfsdf: command not found
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ sdf
Command 'sdf' not found, but can be installed with:
sudo apt install sdf
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ sdf
Command 'sdf' not found, but can be installed with:
sudo apt install sdf
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ sdf
Command 'sdf' not found, but can be installed with:
sudo apt install sdf
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ python3 2_text_process.py
Choose a text analysis task:
1. Sentiment Analysis
2. Named Entity Recognition
3. Text Summarization
4. Text Generation
5. Question Answering
Enter the number of your choice: 4
Enter text for text generation: Hyderabad Biryani
[{'generated_text': 'Hyderabad Biryani, who was also a member of the BJP\'s national executive committee, said the party was "very happy" with the decision.\n\n"We are very happy with the decision. We are very happy with the decision of the BJP. We are very happy with the decision of the BJP. We are very happy with the decision of the BJP," he said.\n\nThe BJP\'s national executive committee, which is headed by former chief minister Manmohan Singh, has been in the process of'}]
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ python3 2_text_process.py
Choose a text analysis task:
1. Sentiment Analysis
2. Named Entity Recognition
3. Text Summarization
4. Text Generation
5. Question Answering
Enter the number of your choice: 4
Enter text for text generation: Explain about how to go from Hyderabad to London.
[{'generated_text': 'Explain about how to go from Hyderabad to London. \xa0This is a very simple and easy way to get to London. \xa0You can also get to London by going to the airport. \xa0You can also get to London by going to the airport by going to the airport by going to the airport by going to the airport by going to the airport by going to the airport by going to the airport by going to the airport by going to the airport by going to the airport by going to the airport by going to the airport by g
```oing to the'}]
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ python 3_local_text_gen.py Choose a text analysis task:
grouped_entities
is deprecated and will be removed in version v5.0.0, defaulted to aggregation_strategy="AggregationStrategy.SIMPLE"
instead.
warnings.warn(
Enter text for named entity recognition: Football
[]
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ python 3_local_text_gen.py
Choose a text analysis task:grouped_entities
is deprecated and will be removed in version v5.0.0, defaulted to aggregation_strategy="AggregationStrategy.SIMPLE"
instead.
warnings.warn(
Enter text for named entity recognition: Hyderabad
[{'entity_group': 'LOC', 'score': 0.98175496, 'word': 'Hyderabad', 'start': 0, 'end': 9}]
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ python 3_local_text_gen.py
Choose a text analysis task:max_length
is provided a specific value, please use truncation=True
to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to truncation
.
Setting pad_token_id
to eos_token_id
:50256 for open-end generation.
[{'generated_text': 'Hyderabad Biryani, D.H.G. (1995). Sex differences in sexual orientation and gender identity in children. Journal of Pediatric Psychology, 34(6), 879-864. Pubmed Abstract | Pubmed Full Text'}]
(GenAI) (base) kp@kuru:~/kp-gen-ai/exp3-huggf$ python 3_local_text_gen.py
Choose a text analysis task: