It will show the error like below:
05/28/2024 10:16:38 - WARNING - llamafactory.data.preprocess - Dropped invalid example: [{'role': 'user', 'content': 'Having a DisplayPort with you at your house will always keep you entertained if you love gaming and live to stream. The cables fit perfectly in a monitor, projector, laptop or tablet. There are many products in the market you can choose from, but you have to do proper research to avoid purchasing a wrong product.\nThis is the perfect cable for your laptop or desktop that is equipped with a display port. You can also use it to enjoy your movies where you can also expand your laptop for better view or when you are presenting at school. It provides you with quality and high-definition audio and videos from your laptop.\nThis cable is gold plated, and this helps to resist corrosion or resist any damage and enhance durability. This kind of plating also ensures a perfect signal transmission. This display port connector provides a secure connection where it has a release button that is always lowered before unplugging. The display port is also portable where you can carry it anywhere with you.\nThis mini display port cable can give you quality transmission of videos and audios. You can connect it to your computer or tablet to get HD videos. Has gold plated connectors, copper conductors, and foil-and-braid shielding where all these provide enough energy and convenient connection.\nThis cable measures a maximum of 10 feet and gives an easy time when connecting it to your laptop. The cable has a sleek design that gives you an easy time to carry and store the cable. It is secure and reliable where it will provide you with a good connection when you need it.\nIf you are looking for a perfect cable for video streaming and gaming, this is the perfect fit. It has a construction design that prevents the cable from cracking or any damage. It contains gold-plated connectors and multiple shielding where this prevents any damage and enhance the durability of the cable.\nThe cable provides you with free non-interfered data transmission from any time and position. Provides a secure and reliable connection where you only plug and play without struggling. You can directly connect the display port that is equipped to your desktop for high definition videos and clear audios.\nThis is a convenient cable that connects a display port cable directly to a projector or HD monitor with display port unit. This cable is portable and mobile where you can use it anywhere. It features gold-plated connectors that with copper connectors and braid shielding where they all combine to provide powerful cable performance reliable connections.\nIt transmits high definition videos and clear music from your computer to a monitor for gaming or video streaming. It provides a secure connection where it has a button that is always lowered before unplugging. This cable is 10 foot long, and this gives you an easy time when using it from any position.\nThis cable has gold-plated connectors that resist corrosion and any form of damage. The gold connectors also enhance reliable and increased connectivity. It features inner braided foil that reduces interference and increases video and audio quality. The 6 feet cable connects a display port enabled computer to an HD monitor or projector with DP port unit for audio and video streaming.\nThe cable gives you the best and quality video resolution you need. The DP has a latch that provides a secure and reliable connection with the port where the latch must be pressed down before unplugging the connector. This is perfect for extended display or mirrored displays.\nThe cable supports resolutions of up to 3840 by 2160 and high-definition videos. This display port provides you with a secure and reliable connection any time you need it. It has pure copper conductors and triple metal shielding that enhance a perfect performance.\nThis cable transmits both high definition video from your computer or tablet to an HD display. It has gold-plated and copper conductors that protect the cable from damage and improve its performance.\nHas foil and braid shielding that helps in reducing electromagnetic interference.\nIt has an ergonomic design where it has secured grips for easy plugging and unplugging. The cable measures 6 foot and supports a high video resolution. This cable provides you with a high-quality connection no matter where you are.\nThis is a perfect choice when you need quality and clear videos and pictures. The cable provides you with a high video resolution. The cable is 10 foot long to provide you with enough and secure connection from any position.\nThe cable is gold-plated for preventing corrosion and providing a secure connection. Has a release button that is always lowered before unplugging. Has a slim design where you can fold it easily for easy carrying.\nThis cable has latching gold-plated connectors for providing a secure and reliable connection. The gold plating also helps in resisting damage, and this enhances the durability of the cable. Features braided copper conductors for providing a reliable signal from source to display design with foil design for an uninterrupted connection.\nThe display port provides you with a secure connection where it has a release button that must be lowered before unplugging. It transmits HD audio, and video from you’re your computer to a monitor for video streaming and gaming. You can also connect and configure your monitor for an extended monitor display.\nThis cable is easily connected to a display port that is equipped with a desktop to an HD monitor or projector for easy streaming and gaming. The cable transmits high definition video and audio from your computer to a monitor where it also convenient for gaming and live streaming. It supports it provides you with a high video resolution.\nIt can work with various display port modes for providing you with deep color depths and sharp images. It is gold plated for increasing providing quality connection where the plating also prevents damage. The cable has an available price that will favor your pocket.\nFlexibility: Before buying any cable, you ask yourself the questions like what type of install is the cable going into? Is everything fixed? You need to find a cable that is flexible enough such that it is mobile to be used everywhere and in different scenarios. A flexible will always give you the freedom of using the cable everywhere provided the cable has a slim design for easy carrying and storage.\nDurability: Buying a durable display port cable means you will keep yourself out of the shops buying new products time by time. Consider buying a cable that is plated with strong material that prevents corrosion or any form of damage. There are other cables that are made from strong materials that are strong enough to enhance the durability of the cable. You have to do better research on the product you want to buy because there are many products in the market and buying a durable cable can be a tiring job.\nCost: Consider buying a port that is very well priced and a cable that will suit your pocket. Mostly, the high-quality cable is costly than the low-quality cables. You can still buy the low priced cables where they perform their function efficiently. Buying an expensive or low costing product only depends on the strength of your pocket. Avoid buying a product that will affect your budget, but if you are financially stable, you can spend on the expensive cables.\nDesign: A good cable should always have a perfect design that will make its use simple. It should have a slim design for easy carrying and storage. A foldable cable is always portable, and this makes it best to use in any position. A cable that is molded with good design will always provide you with the best results you want.\nQuality: Consider buying a cable that is of high quality and a cable that will suit your needs very well. If you purchase a low-quality cable, you will be forced to return it to the shops or buy a new one, and this will be costly. To avoid all these problems, do proper research on the product you want to buy.\nChoosing the best Display port on the market can give you stress because the products are numerous in the market and you have to do proper research before buying any cable. You have to consider many things to avoid buying a product that will not suit your functions. Consider the products in the guide above because they are of high quality and well researched to suit your functions.'}]
LLaMA-Factory/src/llamafactory/data/loader.py", line 176, in get_dataset
raise RuntimeError("Cannot find valid samples, check data/README.md for the data format.")
RuntimeError: Cannot find valid samples, check data/README.md for the data format.
hiyouga
commented
4 months ago
Reminder
Reproduction
Here is my yaml file.
model
model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct
method
stage: sft do_train: true finetuning_type: full use_galore: true galore_layerwise: true galore_target: mlp,self_attn galore_rank: 128 galore_scale: 2.0
dataset
dataset: c4_demo template: llama3 cutoff_len: 1024 max_samples: 1000 overwrite_cache: true preprocessing_num_workers: 16
output
output_dir: saves/llama3-8b/full/sft logging_steps: 10 save_steps: 500 plot_loss: true overwrite_output_dir: true
train
per_device_train_batch_size: 1 gradient_accumulation_steps: 1 learning_rate: 0.0001 num_train_epochs: 3.0 lr_scheduler_type: cosine warmup_steps: 0.1 pure_bf16: true
eval
val_size: 0.1 per_device_eval_batch_size: 1 evaluation_strategy: steps eval_steps: 500
Expected behavior
I want to do continuous pre-training in GaLora with c4-demo.
System Info
transformersversion: 4.41.0Others
It will show the error like below:
05/28/2024 10:16:38 - WARNING - llamafactory.data.preprocess - Dropped invalid example: [{'role': 'user', 'content': 'Having a DisplayPort with you at your house will always keep you entertained if you love gaming and live to stream. The cables fit perfectly in a monitor, projector, laptop or tablet. There are many products in the market you can choose from, but you have to do proper research to avoid purchasing a wrong product.\nThis is the perfect cable for your laptop or desktop that is equipped with a display port. You can also use it to enjoy your movies where you can also expand your laptop for better view or when you are presenting at school. It provides you with quality and high-definition audio and videos from your laptop.\nThis cable is gold plated, and this helps to resist corrosion or resist any damage and enhance durability. This kind of plating also ensures a perfect signal transmission. This display port connector provides a secure connection where it has a release button that is always lowered before unplugging. The display port is also portable where you can carry it anywhere with you.\nThis mini display port cable can give you quality transmission of videos and audios. You can connect it to your computer or tablet to get HD videos. Has gold plated connectors, copper conductors, and foil-and-braid shielding where all these provide enough energy and convenient connection.\nThis cable measures a maximum of 10 feet and gives an easy time when connecting it to your laptop. The cable has a sleek design that gives you an easy time to carry and store the cable. It is secure and reliable where it will provide you with a good connection when you need it.\nIf you are looking for a perfect cable for video streaming and gaming, this is the perfect fit. It has a construction design that prevents the cable from cracking or any damage. It contains gold-plated connectors and multiple shielding where this prevents any damage and enhance the durability of the cable.\nThe cable provides you with free non-interfered data transmission from any time and position. Provides a secure and reliable connection where you only plug and play without struggling. You can directly connect the display port that is equipped to your desktop for high definition videos and clear audios.\nThis is a convenient cable that connects a display port cable directly to a projector or HD monitor with display port unit. This cable is portable and mobile where you can use it anywhere. It features gold-plated connectors that with copper connectors and braid shielding where they all combine to provide powerful cable performance reliable connections.\nIt transmits high definition videos and clear music from your computer to a monitor for gaming or video streaming. It provides a secure connection where it has a button that is always lowered before unplugging. This cable is 10 foot long, and this gives you an easy time when using it from any position.\nThis cable has gold-plated connectors that resist corrosion and any form of damage. The gold connectors also enhance reliable and increased connectivity. It features inner braided foil that reduces interference and increases video and audio quality. The 6 feet cable connects a display port enabled computer to an HD monitor or projector with DP port unit for audio and video streaming.\nThe cable gives you the best and quality video resolution you need. The DP has a latch that provides a secure and reliable connection with the port where the latch must be pressed down before unplugging the connector. This is perfect for extended display or mirrored displays.\nThe cable supports resolutions of up to 3840 by 2160 and high-definition videos. This display port provides you with a secure and reliable connection any time you need it. It has pure copper conductors and triple metal shielding that enhance a perfect performance.\nThis cable transmits both high definition video from your computer or tablet to an HD display. It has gold-plated and copper conductors that protect the cable from damage and improve its performance.\nHas foil and braid shielding that helps in reducing electromagnetic interference.\nIt has an ergonomic design where it has secured grips for easy plugging and unplugging. The cable measures 6 foot and supports a high video resolution. This cable provides you with a high-quality connection no matter where you are.\nThis is a perfect choice when you need quality and clear videos and pictures. The cable provides you with a high video resolution. The cable is 10 foot long to provide you with enough and secure connection from any position.\nThe cable is gold-plated for preventing corrosion and providing a secure connection. Has a release button that is always lowered before unplugging. Has a slim design where you can fold it easily for easy carrying.\nThis cable has latching gold-plated connectors for providing a secure and reliable connection. The gold plating also helps in resisting damage, and this enhances the durability of the cable. Features braided copper conductors for providing a reliable signal from source to display design with foil design for an uninterrupted connection.\nThe display port provides you with a secure connection where it has a release button that must be lowered before unplugging. It transmits HD audio, and video from you’re your computer to a monitor for video streaming and gaming. You can also connect and configure your monitor for an extended monitor display.\nThis cable is easily connected to a display port that is equipped with a desktop to an HD monitor or projector for easy streaming and gaming. The cable transmits high definition video and audio from your computer to a monitor where it also convenient for gaming and live streaming. It supports it provides you with a high video resolution.\nIt can work with various display port modes for providing you with deep color depths and sharp images. It is gold plated for increasing providing quality connection where the plating also prevents damage. The cable has an available price that will favor your pocket.\nFlexibility: Before buying any cable, you ask yourself the questions like what type of install is the cable going into? Is everything fixed? You need to find a cable that is flexible enough such that it is mobile to be used everywhere and in different scenarios. A flexible will always give you the freedom of using the cable everywhere provided the cable has a slim design for easy carrying and storage.\nDurability: Buying a durable display port cable means you will keep yourself out of the shops buying new products time by time. Consider buying a cable that is plated with strong material that prevents corrosion or any form of damage. There are other cables that are made from strong materials that are strong enough to enhance the durability of the cable. You have to do better research on the product you want to buy because there are many products in the market and buying a durable cable can be a tiring job.\nCost: Consider buying a port that is very well priced and a cable that will suit your pocket. Mostly, the high-quality cable is costly than the low-quality cables. You can still buy the low priced cables where they perform their function efficiently. Buying an expensive or low costing product only depends on the strength of your pocket. Avoid buying a product that will affect your budget, but if you are financially stable, you can spend on the expensive cables.\nDesign: A good cable should always have a perfect design that will make its use simple. It should have a slim design for easy carrying and storage. A foldable cable is always portable, and this makes it best to use in any position. A cable that is molded with good design will always provide you with the best results you want.\nQuality: Consider buying a cable that is of high quality and a cable that will suit your needs very well. If you purchase a low-quality cable, you will be forced to return it to the shops or buy a new one, and this will be costly. To avoid all these problems, do proper research on the product you want to buy.\nChoosing the best Display port on the market can give you stress because the products are numerous in the market and you have to do proper research before buying any cable. You have to consider many things to avoid buying a product that will not suit your functions. Consider the products in the guide above because they are of high quality and well researched to suit your functions.'}]LLaMA-Factory/src/llamafactory/data/loader.py", line 176, in get_dataset raise RuntimeError("Cannot find valid samples, check
data/README.mdfor the data format.") RuntimeError: Cannot find valid samples, checkdata/README.mdfor the data format.