Notebook to demo the synthetic data generation process with Gemini 1.5

deep-diver / llamaduo

This project showcases an LLMOps pipeline that fine-tunes a small-size LLM model to prepare for the outage of the service LLM.

https://huggingface.co/papers/2408.13467

Apache License 2.0

286 stars 29 forks source link

Notebook to demo the synthetic data generation process with Gemini 1.5 #10

Closed sayakpaul closed 6 months ago

sayakpaul commented 6 months ago

The notebook isn't complete yet. I guess I am hitting ratelimits.

But my idea is to take hints from the original dataset periodically (which is controllable by the user) to achieve a bit of diversity in the responses. The goal of the notebook is to demonstrate the following things:

Generate response with Gemini 1.5 similar to the original samples
Format the samples such that they match original structure of the dataset
Pack the final representation and have it as a datasets compatible structure

review-notebook-app[bot] commented 6 months ago

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

sayakpaul commented 6 months ago

@deep-diver now, I am facing read timeouts. I have created a little utility in the notebook that makes concurrent calls to the Gemini API with the prompts crafted from the original dataset. But it's leading read timeouts.

Would love to have your opinion.

deep-diver commented 6 months ago

How about trying out with Gemini 1.0? 1.5 Pro usually takes a long time.

If wanting to use 1.5 pro, we may need to make calls in a naive approach like using requests

https://g.co/gemini/share/24c4912aed2c

sayakpaul commented 6 months ago

Hmm that means we need concurrency as well as proper handling of the rate-limits (keeping the API limits in mind). Sorry, but I won't have the bandwidth to go this deep for this.

sayakpaul commented 6 months ago

@deep-diver I have added a bunch of things that are working. I have left comments to make it easy for you to review things. But let me know in case questions. I would appreciate if the reviewing was done with enough time and not hurried.

The PR now supports concurrency along with rate-limit handling (basic support).

Generated dataset: https://huggingface.co/datasets/sayakpaul/generated-gemini-responses.

Sample Gemini (1.0 ultra) responses for you to try and validate things: sample_json_responses.zip

deep-diver commented 6 months ago

other than the asyncio stuff, I think it is good to go.

sayakpaul commented 6 months ago

@deep-diver pushed the changes. Thanks for catching the bug.