This Python script creates a password-protected ZIP file using the pyzipper library. It allows you to specify the files to include in the ZIP and set a password for encryption. The resulting ZIP file requires the provided password to access its contents, providing an additional layer of security.
Project OverviewProject Introduction
Imagine you are an AI engineer. Your team is exploring innovative ways to enhance customer experience and streamline information delivery. You've been entrusted by your boss to develop a proof of concept (POC) for a domain expert model. This model will be trained on a dataset of domain-specific knowledge (finance, medical, or IT domain knowledge). The model can then be used to create chat applications, internal knowledge applications, or text content generation for company collateral.
Project Objective
Your task in this project is to train (fine-tune) a large language model. This model should become a domain expert, capable of generating informative, accurate, and contextually relevant text responses. Think of it as creating a knowledgeable consultant for the company!
The Challenge
Selecting the Dataset: Choose an appropriate, copyright-free unstructured text dataset relevant to a domain. You'll choose from finance, medical, or IT datasets. This dataset will be the training ground for your model to learn domain-specific language and concepts.
Overview of Project Tasks
Fine-tuning the Language Model:
Utilize Amazon Sagemaker and other AWS tools to fine-tune the Meta Llama 2 7B foundation model. This model has been trained for text-generation tasks. The goal is to adapt this model to your selected domain, enhancing its ability to understand and generate domain-specific text.
Deliverables
Trained Model: A fine-tuned language model proficient in your chosen domain.
Report and Presentation: Documentation of your process, challenges, and solution.
What You Will Learn
Advanced skills in machine learning and natural language processing.
Hands-on experience with AWS tools like Amazon Sagemaker.
Insights into domain-specific language model training.
Practical application of AI in solving real-world business challenges.
On the following pages, you'll find detailed instructions on project steps, environment configuration, and resources, and directions for completing and filling out deliverables.
You'll submit your project and it will be assessed by a project reviewer against this rubric(opens in a new tab). Take a moment to read through this rubric(opens in a new tab) to familiarize yourself with the key deliverables for the project and what the reviewers will be looking for in your submission.
Using Python and the Sagemaker API:
Deploy the Meta Llama 2 7B foundation model on the AWS platform
Test and evaluate the model for its responses to domain knowledge and text-generation tasks
Fine-tune the model on your chosen dataset
Deploy the fine-tuned model
Test the fine-tuned model on domain-specific knowledge and text generation tasks relevant to your dataset.
Document and submit!
Project Overview Project Introduction Imagine you are an AI engineer. Your team is exploring innovative ways to enhance customer experience and streamline information delivery. You've been entrusted by your boss to develop a proof of concept (POC) for a domain expert model. This model will be trained on a dataset of domain-specific knowledge (finance, medical, or IT domain knowledge). The model can then be used to create chat applications, internal knowledge applications, or text content generation for company collateral.
Project Objective
Your task in this project is to train (fine-tune) a large language model. This model should become a domain expert, capable of generating informative, accurate, and contextually relevant text responses. Think of it as creating a knowledgeable consultant for the company!
The Challenge
Selecting the Dataset: Choose an appropriate, copyright-free unstructured text dataset relevant to a domain. You'll choose from finance, medical, or IT datasets. This dataset will be the training ground for your model to learn domain-specific language and concepts.
Overview of Project Tasks
Fine-tuning the Language Model: Utilize Amazon Sagemaker and other AWS tools to fine-tune the Meta Llama 2 7B foundation model. This model has been trained for text-generation tasks. The goal is to adapt this model to your selected domain, enhancing its ability to understand and generate domain-specific text. Deliverables
Trained Model: A fine-tuned language model proficient in your chosen domain. Report and Presentation: Documentation of your process, challenges, and solution. What You Will Learn
Advanced skills in machine learning and natural language processing. Hands-on experience with AWS tools like Amazon Sagemaker. Insights into domain-specific language model training. Practical application of AI in solving real-world business challenges. On the following pages, you'll find detailed instructions on project steps, environment configuration, and resources, and directions for completing and filling out deliverables.
You'll submit your project and it will be assessed by a project reviewer against this rubric(opens in a new tab). Take a moment to read through this rubric(opens in a new tab) to familiarize yourself with the key deliverables for the project and what the reviewers will be looking for in your submission.
Using Python and the Sagemaker API: Deploy the Meta Llama 2 7B foundation model on the AWS platform Test and evaluate the model for its responses to domain knowledge and text-generation tasks Fine-tune the model on your chosen dataset Deploy the fine-tuned model Test the fine-tuned model on domain-specific knowledge and text generation tasks relevant to your dataset. Document and submit!