h2oai / h2ogpt

Private chat with local GPT with document, images, video, etc. 100% private, Apache 2.0. Supports oLLaMa, Mixtral, llama.cpp, and more. Demo: https://gpt.h2o.ai/ https://gpt-docs.h2o.ai/
http://h2o.ai
Apache License 2.0
11.3k stars 1.24k forks source link

With some models and it i ask about a text files that i submitted i get a bug #186

Closed lelapin123 closed 1 year ago

lelapin123 commented 1 year ago

I receive a bug with some models but not all. I get that error below with the [Model: junelee/wizard-vicuna-13b], when i ask about a text. I can ask questions to the model, but when i try to interrogate it about a text file, i receive this bug:

Traceback (most recent call last): File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/gradio/routes.py", line 414, in run_predict output = await app.get_blocks().process_api( File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/gradio/blocks.py", line 1323, in process_api result = await self.call_function( File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/gradio/blocks.py", line 1067, in call_function prediction = await utils.async_iteration(iterator) File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/gradio/utils.py", line 339, in async_iteration return await iterator.anext() File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/gradio/utils.py", line 332, in anext return await anyio.to_thread.run_sync( File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run result = context.run(func, args) File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/gradio/utils.py", line 315, in run_sync_iterator_async return next(iterator) File "/home/lelapain123/work/h2oa/h2ogpt/gradio_runner.py", line 932, in bot for output_fun in fun1(tuple(args_list)): File "/home/lelapain123/work/h2oa/h2ogpt/generate.py", line 922, in evaluate for r in run_qa_db(query=query, File "/home/lelapain123/work/h2oa/h2ogpt/gpt_langchain.py", line 917, in _run_qa_db docs, chain, scores, use_context = get_similarity_chain(sim_kwargs) File "/home/lelapain123/work/h2oa/h2ogpt/gpt_langchain.py", line 1053, in get_similarity_chain docs_with_score = db.similarity_search_with_score(query, k=k_db, filter_kwargs)[:k] File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 229, in similarity_search_with_score results = self.__query_collection( File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/utils.py", line 52, in wrapper return func(*args, **kwargs) File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 121, in __query_collection return self._collection.query( File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/chromadb/api/models/Collection.py", line 227, in query return self._client._query( File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/chromadb/api/local.py", line 437, in _query uuids, distances = self._db.get_nearest_neighbors( File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/chromadb/db/clickhouse.py", line 585, in get_nearest_neighbors uuids, distances = index.get_nearest_neighbors(embeddings, n_results, ids) File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/chromadb/db/index/hnswlib.py", line 256, in get_nearest_neighbors labels = {self._id_to_label[hexid(id)] for id in ids} File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/chromadb/db/index/hnswlib.py", line 256, in labels = {self._id_to_label[hexid(id)] for id in ids} KeyError: '9028eb17b0cc44b79fcd6bda35271710'

Later, if i open the model again and chat with it, i receive a bug even if i dont ask anything about a document. The chat exception window, write: "cannot pickle '_thread.lock' object" I have to delete the content of "db_dir_UserData/" to make it work again. I dont get a similar bug with the model: h2oai/h2ogpt-oasst1-512-12b

lelapin123 commented 1 year ago

btw, i am not sure it is related, but if i dont use the option "--load_8bit=True", what happens ?

pseudotensor commented 1 year ago

Thanks, trying it. I'll run:

python generate.py --base_model=junelee/wizard-vicuna-13b --load_8bit=True --langchain_mode=UserData

What did you run?

On my main desktop I only have a 3090Ti with 24GB, and for 13b I'll have to use 8-bit to fit into GPU memory. For any 12B or 13B model on 24GB GPU, one would have to use 8-bit mode. One might be able to barely get it to fit upload initial load, but it would immediately fail with any use (what I found).

The cannot pickle '_thread.lock' object error sounds like what I saw with new transformers: https://github.com/huggingface/transformers/issues/23785 but that was fixed: https://github.com/h2oai/h2ogpt/blob/b1110490315bd425e8382e35a510615917e3ad4c/h2oai_pipeline.py#L75-L77 by avoiding the deepcopy in our pipeline. Had to copy the entire _forward they have to avoid that.

Can you give full stack trace for that pickle error?

lelapin123 commented 1 year ago

python generate.py --base_model=junelee/wizard-vicuna-13b --load_8bit=True --langchain_mode=UserData --user_path=/home/lelapin/work/h2oa/h2ogpt/dadata --share=False

(i modified my username)

3090 normal here

lelapin123 commented 1 year ago

Here is the whole error (i had some files in the database, the error appears directly)

===================================BUG REPORT=================================== Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues

bin /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: /home/lelapn123/anaconda3/envs/h2oa did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths... warn(msg) /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('@/tmp/.ICE-unix/1961,unix/lelapn123-B450-I-AORUS-PRO-WIFI'), PosixPath('local/lelapn123-B450-I-AORUS-PRO-WIFI')} warn(msg) /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/etc/xdg/xdg-ubuntu')} warn(msg) /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/share/gconf/ubuntu.mandatory.path')} warn(msg) /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('0'), PosixPath('1')} warn(msg) /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/org/gnome/Terminal/screen/2684e21c_ede4_4a9d_b1f3_9ab1ce87b39b')} warn(msg) /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/share/gconf/ubuntu.default.path')} warn(msg) /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('path=/run/user/1000/bus,guid=8f75ba35b6f469324bd1a77c64724bc8'), PosixPath('unix')} warn(msg) CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths... CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so CUDA SETUP: Highest compute capability among GPUs detected: 8.6 CUDA SETUP: Detected CUDA version 121 CUDA SETUP: Loading binary /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so... Using Model junelee/wizard-vicuna-13b Prep: persist_directory=db_dir_UserData exists, using DO Loading db: UserData Using embedded DuckDB with persistence: data will be stored in: db_dir_UserData DONE Loading db: UserData Removing old MyData: /tmp/db_dir_MyData_2f7f8e8b-60e2-41d1-b445-fe5a656039ab device_map: {'': 0} The argument trust_remote_code is to be used with Auto classes. It has no effect here and is ignored. Loading checkpoint shards: 100%|████████████| 6/6 [00:41<00:00, 6.93s/it] device_map: {'': 0} Running on local URL: http://0.0.0.0:7860

To create a public link, set share=True in launch(). Started GUI The model 'OptimizedModule' is not supported for text-generation. Supported models are ['BartForCausalLM', 'BertLMHeadModel', 'BertGenerationDecoder', 'BigBirdForCausalLM', 'BigBirdPegasusForCausalLM', 'BioGptForCausalLM', 'BlenderbotForCausalLM', 'BlenderbotSmallForCausalLM', 'BloomForCausalLM', 'CamembertForCausalLM', 'CodeGenForCausalLM', 'CpmAntForCausalLM', 'CTRLLMHeadModel', 'Data2VecTextForCausalLM', 'ElectraForCausalLM', 'ErnieForCausalLM', 'GitForCausalLM', 'GPT2LMHeadModel', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'GPTNeoForCausalLM', 'GPTNeoXForCausalLM', 'GPTNeoXJapaneseForCausalLM', 'GPTJForCausalLM', 'LlamaForCausalLM', 'MarianForCausalLM', 'MBartForCausalLM', 'MegaForCausalLM', 'MegatronBertForCausalLM', 'MvpForCausalLM', 'OpenLlamaForCausalLM', 'OpenAIGPTLMHeadModel', 'OPTForCausalLM', 'PegasusForCausalLM', 'PLBartForCausalLM', 'ProphetNetForCausalLM', 'QDQBertLMHeadModel', 'ReformerModelWithLMHead', 'RemBertForCausalLM', 'RobertaForCausalLM', 'RobertaPreLayerNormForCausalLM', 'RoCBertForCausalLM', 'RoFormerForCausalLM', 'RwkvForCausalLM', 'Speech2Text2ForCausalLM', 'TransfoXLLMHeadModel', 'TrOCRForCausalLM', 'XGLMForCausalLM', 'XLMWithLMHeadModel', 'XLMProphetNetForCausalLM', 'XLMRobertaForCausalLM', 'XLMRobertaXLForCausalLM', 'XLNetLMHeadModel', 'XmodForCausalLM']. Chroma collection UserData contains fewer than 1000 elements. Chroma collection UserData contains fewer than 999 elements. Chroma collection UserData contains fewer than 998 elements. Chroma collection UserData contains fewer than 997 elements. Chroma collection UserData contains fewer than 996 elements. Chroma collection UserData contains fewer than 995 elements. Chroma collection UserData contains fewer than 994 elements. Chroma collection UserData contains fewer than 993 elements. Chroma collection UserData contains fewer than 992 elements. Chroma collection UserData contains fewer than 991 elements. Chroma collection UserData contains fewer than 990 elements. Chroma collection UserData contains fewer than 989 elements. Chroma collection UserData contains fewer than 988 elements. Chroma collection UserData contains fewer than 987 elements. Chroma collection UserData contains fewer than 986 elements. Chroma collection UserData contains fewer than 985 elements. Chroma collection UserData contains fewer than 984 elements. Chroma collection UserData contains fewer than 983 elements. Chroma collection UserData contains fewer than 982 elements. Chroma collection UserData contains fewer than 981 elements. Chroma collection UserData contains fewer than 980 elements. Chroma collection UserData contains fewer than 979 elements. Chroma collection UserData contains fewer than 978 elements. Chroma collection UserData contains fewer than 977 elements. Chroma collection UserData contains fewer than 976 elements. Chroma collection UserData contains fewer than 975 elements. Chroma collection UserData contains fewer than 974 elements. Chroma collection UserData contains fewer than 973 elements. Chroma collection UserData contains fewer than 972 elements. Chroma collection UserData contains fewer than 971 elements. Chroma collection UserData contains fewer than 970 elements. Chroma collection UserData contains fewer than 969 elements. Chroma collection UserData contains fewer than 968 elements. Chroma collection UserData contains fewer than 967 elements. Chroma collection UserData contains fewer than 966 elements. Chroma collection UserData contains fewer than 965 elements. Chroma collection UserData contains fewer than 964 elements. Chroma collection UserData contains fewer than 963 elements. Chroma collection UserData contains fewer than 962 elements. Chroma collection UserData contains fewer than 961 elements. Chroma collection UserData contains fewer than 960 elements. Chroma collection UserData contains fewer than 959 elements. Chroma collection UserData contains fewer than 958 elements. Chroma collection UserData contains fewer than 957 elements. Chroma collection UserData contains fewer than 956 elements. Chroma collection UserData contains fewer than 955 elements. Chroma collection UserData contains fewer than 954 elements. Chroma collection UserData contains fewer than 953 elements. Chroma collection UserData contains fewer than 952 elements. Chroma collection UserData contains fewer than 951 elements. Chroma collection UserData contains fewer than 950 elements. Chroma collection UserData contains fewer than 949 elements. Chroma collection UserData contains fewer than 948 elements. Chroma collection UserData contains fewer than 947 elements. Chroma collection UserData contains fewer than 946 elements. Chroma collection UserData contains fewer than 945 elements. Chroma collection UserData contains fewer than 944 elements. Chroma collection UserData contains fewer than 943 elements. Chroma collection UserData contains fewer than 942 elements. Chroma collection UserData contains fewer than 941 elements. Chroma collection UserData contains fewer than 940 elements. Chroma collection UserData contains fewer than 939 elements. Chroma collection UserData contains fewer than 938 elements. Chroma collection UserData contains fewer than 937 elements. Chroma collection UserData contains fewer than 936 elements. Chroma collection UserData contains fewer than 935 elements. Chroma collection UserData contains fewer than 934 elements. Chroma collection UserData contains fewer than 933 elements. Chroma collection UserData contains fewer than 932 elements. Chroma collection UserData contains fewer than 931 elements. Chroma collection UserData contains fewer than 930 elements. Chroma collection UserData contains fewer than 929 elements. Chroma collection UserData contains fewer than 928 elements. Chroma collection UserData contains fewer than 927 elements. Chroma collection UserData contains fewer than 926 elements. Chroma collection UserData contains fewer than 925 elements. Chroma collection UserData contains fewer than 924 elements. Chroma collection UserData contains fewer than 923 elements. Chroma collection UserData contains fewer than 922 elements. Chroma collection UserData contains fewer than 921 elements. Chroma collection UserData contains fewer than 920 elements. Chroma collection UserData contains fewer than 919 elements. Chroma collection UserData contains fewer than 918 elements. Chroma collection UserData contains fewer than 917 elements. Chroma collection UserData contains fewer than 916 elements. Chroma collection UserData contains fewer than 915 elements. Chroma collection UserData contains fewer than 914 elements. Chroma collection UserData contains fewer than 913 elements. Chroma collection UserData contains fewer than 912 elements. Chroma collection UserData contains fewer than 911 elements. Chroma collection UserData contains fewer than 910 elements. Chroma collection UserData contains fewer than 909 elements. Chroma collection UserData contains fewer than 908 elements. Chroma collection UserData contains fewer than 907 elements. Chroma collection UserData contains fewer than 906 elements. Chroma collection UserData contains fewer than 905 elements. Chroma collection UserData contains fewer than 904 elements. Chroma collection UserData contains fewer than 903 elements. Chroma collection UserData contains fewer than 902 elements. Chroma collection UserData contains fewer than 901 elements. Chroma collection UserData contains fewer than 900 elements. Chroma collection UserData contains fewer than 899 elements. Chroma collection UserData contains fewer than 898 elements. Chroma collection UserData contains fewer than 897 elements. Chroma collection UserData contains fewer than 896 elements. Chroma collection UserData contains fewer than 895 elements. Chroma collection UserData contains fewer than 894 elements. Chroma collection UserData contains fewer than 893 elements. Chroma collection UserData contains fewer than 892 elements. Chroma collection UserData contains fewer than 891 elements. Chroma collection UserData contains fewer than 890 elements. Chroma collection UserData contains fewer than 889 elements. Chroma collection UserData contains fewer than 888 elements. Chroma collection UserData contains fewer than 887 elements. Chroma collection UserData contains fewer than 886 elements. Chroma collection UserData contains fewer than 885 elements. Chroma collection UserData contains fewer than 884 elements. Chroma collection UserData contains fewer than 883 elements. Chroma collection UserData contains fewer than 882 elements. Chroma collection UserData contains fewer than 881 elements. Chroma collection UserData contains fewer than 880 elements. Chroma collection UserData contains fewer than 879 elements. Chroma collection UserData contains fewer than 878 elements. Chroma collection UserData contains fewer than 877 elements. Chroma collection UserData contains fewer than 876 elements. Chroma collection UserData contains fewer than 875 elements. Chroma collection UserData contains fewer than 874 elements. Chroma collection UserData contains fewer than 873 elements. Chroma collection UserData contains fewer than 872 elements. Chroma collection UserData contains fewer than 871 elements. Chroma collection UserData contains fewer than 870 elements. Chroma collection UserData contains fewer than 869 elements. Chroma collection UserData contains fewer than 868 elements. Chroma collection UserData contains fewer than 867 elements. Chroma collection UserData contains fewer than 866 elements. Chroma collection UserData contains fewer than 865 elements. Chroma collection UserData contains fewer than 864 elements. Chroma collection UserData contains fewer than 863 elements. Chroma collection UserData contains fewer than 862 elements. Chroma collection UserData contains fewer than 861 elements. Chroma collection UserData contains fewer than 860 elements. Chroma collection UserData contains fewer than 859 elements. Chroma collection UserData contains fewer than 858 elements. Chroma collection UserData contains fewer than 857 elements. Chroma collection UserData contains fewer than 856 elements. Chroma collection UserData contains fewer than 855 elements. Chroma collection UserData contains fewer than 854 elements. Chroma collection UserData contains fewer than 853 elements. Chroma collection UserData contains fewer than 852 elements. Chroma collection UserData contains fewer than 851 elements. Chroma collection UserData contains fewer than 850 elements. Chroma collection UserData contains fewer than 849 elements. Chroma collection UserData contains fewer than 848 elements. Chroma collection UserData contains fewer than 847 elements. Chroma collection UserData contains fewer than 846 elements. Chroma collection UserData contains fewer than 845 elements. Chroma collection UserData contains fewer than 844 elements. Chroma collection UserData contains fewer than 843 elements. Chroma collection UserData contains fewer than 842 elements. Chroma collection UserData contains fewer than 841 elements. Chroma collection UserData contains fewer than 840 elements. Chroma collection UserData contains fewer than 839 elements. Chroma collection UserData contains fewer than 838 elements. Chroma collection UserData contains fewer than 837 elements. Chroma collection UserData contains fewer than 836 elements. Chroma collection UserData contains fewer than 835 elements. Chroma collection UserData contains fewer than 834 elements. Chroma collection UserData contains fewer than 833 elements. Chroma collection UserData contains fewer than 832 elements. Chroma collection UserData contains fewer than 831 elements. Chroma collection UserData contains fewer than 830 elements. Chroma collection UserData contains fewer than 829 elements. Chroma collection UserData contains fewer than 828 elements. Chroma collection UserData contains fewer than 827 elements. Chroma collection UserData contains fewer than 826 elements. Chroma collection UserData contains fewer than 825 elements. Chroma collection UserData contains fewer than 824 elements. Chroma collection UserData contains fewer than 823 elements. Chroma collection UserData contains fewer than 822 elements. Chroma collection UserData contains fewer than 821 elements. Chroma collection UserData contains fewer than 820 elements. Chroma collection UserData contains fewer than 819 elements. Chroma collection UserData contains fewer than 818 elements. Chroma collection UserData contains fewer than 817 elements. Chroma collection UserData contains fewer than 816 elements. Chroma collection UserData contains fewer than 815 elements. Chroma collection UserData contains fewer than 814 elements. Chroma collection UserData contains fewer than 813 elements. Chroma collection UserData contains fewer than 812 elements. Chroma collection UserData contains fewer than 811 elements. Chroma collection UserData contains fewer than 810 elements. Chroma collection UserData contains fewer than 809 elements. Chroma collection UserData contains fewer than 808 elements. Chroma collection UserData contains fewer than 807 elements. Chroma collection UserData contains fewer than 806 elements. Chroma collection UserData contains fewer than 805 elements. Chroma collection UserData contains fewer than 804 elements. Chroma collection UserData contains fewer than 803 elements. Chroma collection UserData contains fewer than 802 elements. Chroma collection UserData contains fewer than 801 elements. Chroma collection UserData contains fewer than 800 elements. Chroma collection UserData contains fewer than 799 elements. Chroma collection UserData contains fewer than 798 elements. Chroma collection UserData contains fewer than 797 elements. Chroma collection UserData contains fewer than 796 elements. Chroma collection UserData contains fewer than 795 elements. Chroma collection UserData contains fewer than 794 elements. Chroma collection UserData contains fewer than 793 elements. Chroma collection UserData contains fewer than 792 elements. Chroma collection UserData contains fewer than 791 elements. Chroma collection UserData contains fewer than 790 elements. Chroma collection UserData contains fewer than 789 elements. Chroma collection UserData contains fewer than 788 elements. Chroma collection UserData contains fewer than 787 elements. Chroma collection UserData contains fewer than 786 elements. Chroma collection UserData contains fewer than 785 elements. Chroma collection UserData contains fewer than 784 elements. Chroma collection UserData contains fewer than 783 elements. Chroma collection UserData contains fewer than 782 elements. Chroma collection UserData contains fewer than 781 elements. Chroma collection UserData contains fewer than 780 elements. Chroma collection UserData contains fewer than 779 elements. Chroma collection UserData contains fewer than 778 elements. Chroma collection UserData contains fewer than 777 elements. Chroma collection UserData contains fewer than 776 elements. Chroma collection UserData contains fewer than 775 elements. Chroma collection UserData contains fewer than 774 elements. Chroma collection UserData contains fewer than 773 elements. Chroma collection UserData contains fewer than 772 elements. Chroma collection UserData contains fewer than 771 elements. Chroma collection UserData contains fewer than 770 elements. Chroma collection UserData contains fewer than 769 elements. Chroma collection UserData contains fewer than 768 elements. Chroma collection UserData contains fewer than 767 elements. Chroma collection UserData contains fewer than 766 elements. Chroma collection UserData contains fewer than 765 elements. Chroma collection UserData contains fewer than 764 elements. Chroma collection UserData contains fewer than 763 elements. Chroma collection UserData contains fewer than 762 elements. Chroma collection UserData contains fewer than 761 elements. Chroma collection UserData contains fewer than 760 elements. Chroma collection UserData contains fewer than 759 elements. Chroma collection UserData contains fewer than 758 elements. Chroma collection UserData contains fewer than 757 elements. Chroma collection UserData contains fewer than 756 elements. Chroma collection UserData contains fewer than 755 elements. Chroma collection UserData contains fewer than 754 elements. Chroma collection UserData contains fewer than 753 elements. Chroma collection UserData contains fewer than 752 elements. Chroma collection UserData contains fewer than 751 elements. Chroma collection UserData contains fewer than 750 elements. Chroma collection UserData contains fewer than 749 elements. Chroma collection UserData contains fewer than 748 elements. Chroma collection UserData contains fewer than 747 elements. Chroma collection UserData contains fewer than 746 elements. Chroma collection UserData contains fewer than 745 elements. Chroma collection UserData contains fewer than 744 elements. Chroma collection UserData contains fewer than 743 elements. Chroma collection UserData contains fewer than 742 elements. Chroma collection UserData contains fewer than 741 elements. Chroma collection UserData contains fewer than 740 elements. Chroma collection UserData contains fewer than 739 elements. Chroma collection UserData contains fewer than 738 elements. Chroma collection UserData contains fewer than 737 elements. Chroma collection UserData contains fewer than 736 elements. Chroma collection UserData contains fewer than 735 elements. Chroma collection UserData contains fewer than 734 elements. Chroma collection UserData contains fewer than 733 elements. Chroma collection UserData contains fewer than 732 elements. Chroma collection UserData contains fewer than 731 elements. Chroma collection UserData contains fewer than 730 elements. Chroma collection UserData contains fewer than 729 elements. Chroma collection UserData contains fewer than 728 elements. Chroma collection UserData contains fewer than 727 elements. Chroma collection UserData contains fewer than 726 elements. Chroma collection UserData contains fewer than 725 elements. Chroma collection UserData contains fewer than 724 elements. Chroma collection UserData contains fewer than 723 elements. Chroma collection UserData contains fewer than 722 elements. Chroma collection UserData contains fewer than 721 elements. Chroma collection UserData contains fewer than 720 elements. Chroma collection UserData contains fewer than 719 elements. Chroma collection UserData contains fewer than 718 elements. Chroma collection UserData contains fewer than 717 elements. Chroma collection UserData contains fewer than 716 elements. Chroma collection UserData contains fewer than 715 elements. Chroma collection UserData contains fewer than 714 elements. Chroma collection UserData contains fewer than 713 elements. Chroma collection UserData contains fewer than 712 elements. Chroma collection UserData contains fewer than 711 elements. Chroma collection UserData contains fewer than 710 elements. Chroma collection UserData contains fewer than 709 elements. Chroma collection UserData contains fewer than 708 elements. Chroma collection UserData contains fewer than 707 elements. Chroma collection UserData contains fewer than 706 elements. Chroma collection UserData contains fewer than 705 elements. Chroma collection UserData contains fewer than 704 elements. Chroma collection UserData contains fewer than 703 elements. Chroma collection UserData contains fewer than 702 elements. Chroma collection UserData contains fewer than 701 elements. Chroma collection UserData contains fewer than 700 elements. Chroma collection UserData contains fewer than 699 elements. Chroma collection UserData contains fewer than 698 elements. Chroma collection UserData contains fewer than 697 elements. Chroma collection UserData contains fewer than 696 elements. Chroma collection UserData contains fewer than 695 elements. Chroma collection UserData contains fewer than 694 elements. Chroma collection UserData contains fewer than 693 elements. Chroma collection UserData contains fewer than 692 elements. Chroma collection UserData contains fewer than 691 elements. Chroma collection UserData contains fewer than 690 elements. Chroma collection UserData contains fewer than 689 elements. Chroma collection UserData contains fewer than 688 elements. Chroma collection UserData contains fewer than 687 elements. Chroma collection UserData contains fewer than 686 elements. Chroma collection UserData contains fewer than 685 elements. Chroma collection UserData contains fewer than 684 elements. Chroma collection UserData contains fewer than 683 elements. Chroma collection UserData contains fewer than 682 elements. Chroma collection UserData contains fewer than 681 elements. Chroma collection UserData contains fewer than 680 elements. Chroma collection UserData contains fewer than 679 elements. Chroma collection UserData contains fewer than 678 elements. Chroma collection UserData contains fewer than 677 elements. Chroma collection UserData contains fewer than 676 elements. Chroma collection UserData contains fewer than 675 elements. Chroma collection UserData contains fewer than 674 elements. Chroma collection UserData contains fewer than 673 elements. Chroma collection UserData contains fewer than 672 elements. Chroma collection UserData contains fewer than 671 elements. Chroma collection UserData contains fewer than 670 elements. Chroma collection UserData contains fewer than 669 elements. Chroma collection UserData contains fewer than 668 elements. Chroma collection UserData contains fewer than 667 elements. Chroma collection UserData contains fewer than 666 elements. Chroma collection UserData contains fewer than 665 elements. Chroma collection UserData contains fewer than 664 elements. Chroma collection UserData contains fewer than 663 elements. Chroma collection UserData contains fewer than 662 elements. Chroma collection UserData contains fewer than 661 elements. Chroma collection UserData contains fewer than 660 elements. Chroma collection UserData contains fewer than 659 elements. Chroma collection UserData contains fewer than 658 elements. Chroma collection UserData contains fewer than 657 elements. Chroma collection UserData contains fewer than 656 elements. Chroma collection UserData contains fewer than 655 elements. Chroma collection UserData contains fewer than 654 elements. Chroma collection UserData contains fewer than 653 elements. Chroma collection UserData contains fewer than 652 elements. Chroma collection UserData contains fewer than 651 elements. Chroma collection UserData contains fewer than 650 elements. Chroma collection UserData contains fewer than 649 elements. Chroma collection UserData contains fewer than 648 elements. Chroma collection UserData contains fewer than 647 elements. Chroma collection UserData contains fewer than 646 elements. Chroma collection UserData contains fewer than 645 elements. Chroma collection UserData contains fewer than 644 elements. Chroma collection UserData contains fewer than 643 elements. Chroma collection UserData contains fewer than 642 elements. Chroma collection UserData contains fewer than 641 elements. Chroma collection UserData contains fewer than 640 elements. Chroma collection UserData contains fewer than 639 elements. Chroma collection UserData contains fewer than 638 elements. Chroma collection UserData contains fewer than 637 elements. Chroma collection UserData contains fewer than 636 elements. Chroma collection UserData contains fewer than 635 elements. Chroma collection UserData contains fewer than 634 elements. Chroma collection UserData contains fewer than 633 elements. Chroma collection UserData contains fewer than 632 elements. Chroma collection UserData contains fewer than 631 elements. Chroma collection UserData contains fewer than 630 elements. Chroma collection UserData contains fewer than 629 elements. Chroma collection UserData contains fewer than 628 elements. Chroma collection UserData contains fewer than 627 elements. Chroma collection UserData contains fewer than 626 elements. Chroma collection UserData contains fewer than 625 elements. Chroma collection UserData contains fewer than 624 elements. Chroma collection UserData contains fewer than 623 elements. Chroma collection UserData contains fewer than 622 elements. Chroma collection UserData contains fewer than 621 elements. Chroma collection UserData contains fewer than 620 elements. Chroma collection UserData contains fewer than 619 elements. Chroma collection UserData contains fewer than 618 elements. Chroma collection UserData contains fewer than 617 elements. Chroma collection UserData contains fewer than 616 elements. Chroma collection UserData contains fewer than 615 elements. Chroma collection UserData contains fewer than 614 elements. Chroma collection UserData contains fewer than 613 elements. Chroma collection UserData contains fewer than 612 elements. Chroma collection UserData contains fewer than 611 elements. Chroma collection UserData contains fewer than 610 elements. Chroma collection UserData contains fewer than 609 elements. Chroma collection UserData contains fewer than 608 elements. Chroma collection UserData contains fewer than 607 elements. Chroma collection UserData contains fewer than 606 elements. Chroma collection UserData contains fewer than 605 elements. Chroma collection UserData contains fewer than 604 elements. Chroma collection UserData contains fewer than 603 elements. Chroma collection UserData contains fewer than 602 elements. Chroma collection UserData contains fewer than 601 elements. Chroma collection UserData contains fewer than 600 elements. Chroma collection UserData contains fewer than 599 elements. Chroma collection UserData contains fewer than 598 elements. Chroma collection UserData contains fewer than 597 elements. Chroma collection UserData contains fewer than 596 elements. Chroma collection UserData contains fewer than 595 elements. Chroma collection UserData contains fewer than 594 elements. Chroma collection UserData contains fewer than 593 elements. Chroma collection UserData contains fewer than 592 elements. Chroma collection UserData contains fewer than 591 elements. Chroma collection UserData contains fewer than 590 elements. Chroma collection UserData contains fewer than 589 elements. Chroma collection UserData contains fewer than 588 elements. Chroma collection UserData contains fewer than 587 elements. Chroma collection UserData contains fewer than 586 elements. Chroma collection UserData contains fewer than 585 elements. Chroma collection UserData contains fewer than 584 elements. Chroma collection UserData contains fewer than 583 elements. Chroma collection UserData contains fewer than 582 elements. Chroma collection UserData contains fewer than 581 elements. Chroma collection UserData contains fewer than 580 elements. Chroma collection UserData contains fewer than 579 elements. Chroma collection UserData contains fewer than 578 elements. Chroma collection UserData contains fewer than 577 elements. Chroma collection UserData contains fewer than 576 elements. Chroma collection UserData contains fewer than 575 elements. Chroma collection UserData contains fewer than 574 elements. Chroma collection UserData contains fewer than 573 elements. Chroma collection UserData contains fewer than 572 elements. Chroma collection UserData contains fewer than 571 elements. Chroma collection UserData contains fewer than 570 elements. Chroma collection UserData contains fewer than 569 elements. Chroma collection UserData contains fewer than 568 elements. Chroma collection UserData contains fewer than 567 elements. Chroma collection UserData contains fewer than 566 elements. Chroma collection UserData contains fewer than 565 elements. Chroma collection UserData contains fewer than 564 elements. Chroma collection UserData contains fewer than 563 elements. Chroma collection UserData contains fewer than 562 elements. Chroma collection UserData contains fewer than 561 elements. Chroma collection UserData contains fewer than 560 elements. Chroma collection UserData contains fewer than 559 elements. Chroma collection UserData contains fewer than 558 elements. Chroma collection UserData contains fewer than 557 elements. Chroma collection UserData contains fewer than 556 elements. Chroma collection UserData contains fewer than 555 elements. Chroma collection UserData contains fewer than 554 elements. Chroma collection UserData contains fewer than 553 elements. Chroma collection UserData contains fewer than 552 elements. Chroma collection UserData contains fewer than 551 elements. Chroma collection UserData contains fewer than 550 elements. Chroma collection UserData contains fewer than 549 elements. Chroma collection UserData contains fewer than 548 elements. Chroma collection UserData contains fewer than 547 elements. Chroma collection UserData contains fewer than 546 elements. Chroma collection UserData contains fewer than 545 elements. Chroma collection UserData contains fewer than 544 elements. Chroma collection UserData contains fewer than 543 elements. Chroma collection UserData contains fewer than 542 elements. Chroma collection UserData contains fewer than 541 elements. Chroma collection UserData contains fewer than 540 elements. Chroma collection UserData contains fewer than 539 elements. Chroma collection UserData contains fewer than 538 elements. Chroma collection UserData contains fewer than 537 elements. Chroma collection UserData contains fewer than 536 elements. Chroma collection UserData contains fewer than 535 elements. Chroma collection UserData contains fewer than 534 elements. Chroma collection UserData contains fewer than 533 elements. Chroma collection UserData contains fewer than 532 elements. Chroma collection UserData contains fewer than 531 elements. Chroma collection UserData contains fewer than 530 elements. Chroma collection UserData contains fewer than 529 elements. Chroma collection UserData contains fewer than 528 elements. Chroma collection UserData contains fewer than 527 elements. Chroma collection UserData contains fewer than 526 elements. Chroma collection UserData contains fewer than 525 elements. Chroma collection UserData contains fewer than 524 elements. Chroma collection UserData contains fewer than 523 elements. Chroma collection UserData contains fewer than 522 elements. Chroma collection UserData contains fewer than 521 elements. Chroma collection UserData contains fewer than 520 elements. Chroma collection UserData contains fewer than 519 elements. Chroma collection UserData contains fewer than 518 elements. Chroma collection UserData contains fewer than 517 elements. Chroma collection UserData contains fewer than 516 elements. Chroma collection UserData contains fewer than 515 elements. Chroma collection UserData contains fewer than 514 elements. Chroma collection UserData contains fewer than 513 elements. Chroma collection UserData contains fewer than 512 elements. Chroma collection UserData contains fewer than 511 elements. Chroma collection UserData contains fewer than 510 elements. Chroma collection UserData contains fewer than 509 elements. Chroma collection UserData contains fewer than 508 elements. Chroma collection UserData contains fewer than 507 elements. Chroma collection UserData contains fewer than 506 elements. Chroma collection UserData contains fewer than 505 elements. Chroma collection UserData contains fewer than 504 elements. Chroma collection UserData contains fewer than 503 elements. Chroma collection UserData contains fewer than 502 elements. Chroma collection UserData contains fewer than 501 elements. Chroma collection UserData contains fewer than 500 elements. Chroma collection UserData contains fewer than 499 elements. Chroma collection UserData contains fewer than 498 elements. Chroma collection UserData contains fewer than 497 elements. Chroma collection UserData contains fewer than 496 elements. Chroma collection UserData contains fewer than 495 elements. Chroma collection UserData contains fewer than 494 elements. Chroma collection UserData contains fewer than 493 elements. Chroma collection UserData contains fewer than 492 elements. Chroma collection UserData contains fewer than 491 elements. Chroma collection UserData contains fewer than 490 elements. Chroma collection UserData contains fewer than 489 elements. Chroma collection UserData contains fewer than 488 elements. Chroma collection UserData contains fewer than 487 elements. Chroma collection UserData contains fewer than 486 elements. Chroma collection UserData contains fewer than 485 elements. Chroma collection UserData contains fewer than 484 elements. Chroma collection UserData contains fewer than 483 elements. Chroma collection UserData contains fewer than 482 elements. Chroma collection UserData contains fewer than 481 elements. Chroma collection UserData contains fewer than 480 elements. Chroma collection UserData contains fewer than 479 elements. Chroma collection UserData contains fewer than 478 elements. Chroma collection UserData contains fewer than 477 elements. Chroma collection UserData contains fewer than 476 elements. Chroma collection UserData contains fewer than 475 elements. Chroma collection UserData contains fewer than 474 elements. Chroma collection UserData contains fewer than 473 elements. Chroma collection UserData contains fewer than 472 elements. Chroma collection UserData contains fewer than 471 elements. Chroma collection UserData contains fewer than 470 elements. Chroma collection UserData contains fewer than 469 elements. Chroma collection UserData contains fewer than 468 elements. Chroma collection UserData contains fewer than 467 elements. Chroma collection UserData contains fewer than 466 elements. Chroma collection UserData contains fewer than 465 elements. Chroma collection UserData contains fewer than 464 elements. Chroma collection UserData contains fewer than 463 elements. Chroma collection UserData contains fewer than 462 elements. Chroma collection UserData contains fewer than 461 elements. Chroma collection UserData contains fewer than 460 elements. Chroma collection UserData contains fewer than 459 elements. Chroma collection UserData contains fewer than 458 elements. Chroma collection UserData contains fewer than 457 elements. Chroma collection UserData contains fewer than 456 elements. Chroma collection UserData contains fewer than 455 elements. Chroma collection UserData contains fewer than 454 elements. Chroma collection UserData contains fewer than 453 elements. Chroma collection UserData contains fewer than 452 elements. Chroma collection UserData contains fewer than 451 elements. Chroma collection UserData contains fewer than 450 elements. Chroma collection UserData contains fewer than 449 elements. Chroma collection UserData contains fewer than 448 elements. Chroma collection UserData contains fewer than 447 elements. Chroma collection UserData contains fewer than 446 elements. Chroma collection UserData contains fewer than 445 elements. Chroma collection UserData contains fewer than 444 elements. Chroma collection UserData contains fewer than 443 elements. Chroma collection UserData contains fewer than 442 elements. Chroma collection UserData contains fewer than 441 elements. Chroma collection UserData contains fewer than 440 elements. Chroma collection UserData contains fewer than 439 elements. Chroma collection UserData contains fewer than 438 elements. Chroma collection UserData contains fewer than 437 elements. Chroma collection UserData contains fewer than 436 elements. Chroma collection UserData contains fewer than 435 elements. Chroma collection UserData contains fewer than 434 elements. Chroma collection UserData contains fewer than 433 elements. Chroma collection UserData contains fewer than 432 elements. Chroma collection UserData contains fewer than 431 elements. Chroma collection UserData contains fewer than 430 elements. Chroma collection UserData contains fewer than 429 elements. Chroma collection UserData contains fewer than 428 elements. Chroma collection UserData contains fewer than 427 elements. Chroma collection UserData contains fewer than 426 elements. Chroma collection UserData contains fewer than 425 elements. Chroma collection UserData contains fewer than 424 elements. Chroma collection UserData contains fewer than 423 elements. Chroma collection UserData contains fewer than 422 elements. Chroma collection UserData contains fewer than 421 elements. Chroma collection UserData contains fewer than 420 elements. Chroma collection UserData contains fewer than 419 elements. Chroma collection UserData contains fewer than 418 elements. Chroma collection UserData contains fewer than 417 elements. Chroma collection UserData contains fewer than 416 elements. Chroma collection UserData contains fewer than 415 elements. Chroma collection UserData contains fewer than 414 elements. Chroma collection UserData contains fewer than 413 elements. Chroma collection UserData contains fewer than 412 elements. Chroma collection UserData contains fewer than 411 elements. Chroma collection UserData contains fewer than 410 elements. Chroma collection UserData contains fewer than 409 elements. Chroma collection UserData contains fewer than 408 elements. Chroma collection UserData contains fewer than 407 elements. Chroma collection UserData contains fewer than 406 elements. Chroma collection UserData contains fewer than 405 elements. Chroma collection UserData contains fewer than 404 elements. Chroma collection UserData contains fewer than 403 elements. Chroma collection UserData contains fewer than 402 elements. Chroma collection UserData contains fewer than 401 elements. Chroma collection UserData contains fewer than 400 elements. Chroma collection UserData contains fewer than 399 elements. Chroma collection UserData contains fewer than 398 elements. Chroma collection UserData contains fewer than 397 elements. Chroma collection UserData contains fewer than 396 elements. Chroma collection UserData contains fewer than 395 elements. Chroma collection UserData contains fewer than 394 elements. Chroma collection UserData contains fewer than 393 elements. Chroma collection UserData contains fewer than 392 elements. Chroma collection UserData contains fewer than 391 elements. Chroma collection UserData contains fewer than 390 elements. Chroma collection UserData contains fewer than 389 elements. Chroma collection UserData contains fewer than 388 elements. Chroma collection UserData contains fewer than 387 elements. Chroma collection UserData contains fewer than 386 elements. Chroma collection UserData contains fewer than 385 elements. Chroma collection UserData contains fewer than 384 elements. Chroma collection UserData contains fewer than 383 elements. Chroma collection UserData contains fewer than 382 elements. Chroma collection UserData contains fewer than 381 elements. Chroma collection UserData contains fewer than 380 elements. Chroma collection UserData contains fewer than 379 elements. Chroma collection UserData contains fewer than 378 elements. Chroma collection UserData contains fewer than 377 elements. Chroma collection UserData contains fewer than 376 elements. Chroma collection UserData contains fewer than 375 elements. Chroma collection UserData contains fewer than 374 elements. Chroma collection UserData contains fewer than 373 elements. Chroma collection UserData contains fewer than 372 elements. Chroma collection UserData contains fewer than 371 elements. Chroma collection UserData contains fewer than 370 elements. Chroma collection UserData contains fewer than 369 elements. Chroma collection UserData contains fewer than 368 elements. Chroma collection UserData contains fewer than 367 elements. Chroma collection UserData contains fewer than 366 elements. Chroma collection UserData contains fewer than 365 elements. Chroma collection UserData contains fewer than 364 elements. Chroma collection UserData contains fewer than 363 elements. Chroma collection UserData contains fewer than 362 elements. Chroma collection UserData contains fewer than 361 elements. Chroma collection UserData contains fewer than 360 elements. Chroma collection UserData contains fewer than 359 elements. Chroma collection UserData contains fewer than 358 elements. Chroma collection UserData contains fewer than 357 elements. Chroma collection UserData contains fewer than 356 elements. Chroma collection UserData contains fewer than 355 elements. Chroma collection UserData contains fewer than 354 elements. Chroma collection UserData contains fewer than 353 elements. Chroma collection UserData contains fewer than 352 elements. Chroma collection UserData contains fewer than 351 elements. Chroma collection UserData contains fewer than 350 elements. Chroma collection UserData contains fewer than 349 elements. Chroma collection UserData contains fewer than 348 elements. Chroma collection UserData contains fewer than 347 elements. Chroma collection UserData contains fewer than 346 elements. Chroma collection UserData contains fewer than 345 elements. Chroma collection UserData contains fewer than 344 elements. Chroma collection UserData contains fewer than 343 elements. Chroma collection UserData contains fewer than 342 elements. Chroma collection UserData contains fewer than 341 elements. Chroma collection UserData contains fewer than 340 elements. Chroma collection UserData contains fewer than 339 elements. Chroma collection UserData contains fewer than 338 elements. Chroma collection UserData contains fewer than 337 elements. Chroma collection UserData contains fewer than 336 elements. Chroma collection UserData contains fewer than 335 elements. Chroma collection UserData contains fewer than 334 elements. Chroma collection UserData contains fewer than 333 elements. Chroma collection UserData contains fewer than 332 elements. Chroma collection UserData contains fewer than 331 elements. Chroma collection UserData contains fewer than 330 elements. Chroma collection UserData contains fewer than 329 elements. Chroma collection UserData contains fewer than 328 elements. Chroma collection UserData contains fewer than 327 elements. Chroma collection UserData contains fewer than 326 elements. Chroma collection UserData contains fewer than 325 elements. Chroma collection UserData contains fewer than 324 elements. Chroma collection UserData contains fewer than 323 elements. Chroma collection UserData contains fewer than 322 elements. Chroma collection UserData contains fewer than 321 elements. Chroma collection UserData contains fewer than 320 elements. Chroma collection UserData contains fewer than 319 elements. Chroma collection UserData contains fewer than 318 elements. Chroma collection UserData contains fewer than 317 elements. Chroma collection UserData contains fewer than 316 elements. Chroma collection UserData contains fewer than 315 elements. Chroma collection UserData contains fewer than 314 elements. Chroma collection UserData contains fewer than 313 elements. Chroma collection UserData contains fewer than 312 elements. Chroma collection UserData contains fewer than 311 elements. Chroma collection UserData contains fewer than 310 elements. Chroma collection UserData contains fewer than 309 elements. Chroma collection UserData contains fewer than 308 elements. Chroma collection UserData contains fewer than 307 elements. Chroma collection UserData contains fewer than 306 elements. Chroma collection UserData contains fewer than 305 elements. Chroma collection UserData contains fewer than 304 elements. Chroma collection UserData contains fewer than 303 elements. Chroma collection UserData contains fewer than 302 elements. Chroma collection UserData contains fewer than 301 elements. Chroma collection UserData contains fewer than 300 elements. Chroma collection UserData contains fewer than 299 elements. Chroma collection UserData contains fewer than 298 elements. Chroma collection UserData contains fewer than 297 elements. Chroma collection UserData contains fewer than 296 elements. Chroma collection UserData contains fewer than 295 elements. Chroma collection UserData contains fewer than 294 elements. Chroma collection UserData contains fewer than 293 elements. Chroma collection UserData contains fewer than 292 elements. Chroma collection UserData contains fewer than 291 elements. Chroma collection UserData contains fewer than 290 elements. Chroma collection UserData contains fewer than 289 elements. Chroma collection UserData contains fewer than 288 elements. Chroma collection UserData contains fewer than 287 elements. Chroma collection UserData contains fewer than 286 elements. Chroma collection UserData contains fewer than 285 elements. Chroma collection UserData contains fewer than 284 elements. Chroma collection UserData contains fewer than 283 elements. Chroma collection UserData contains fewer than 282 elements. Chroma collection UserData contains fewer than 281 elements. Chroma collection UserData contains fewer than 280 elements. Chroma collection UserData contains fewer than 279 elements. Chroma collection UserData contains fewer than 278 elements. Chroma collection UserData contains fewer than 277 elements. Chroma collection UserData contains fewer than 276 elements. Chroma collection UserData contains fewer than 275 elements. Chroma collection UserData contains fewer than 274 elements. Chroma collection UserData contains fewer than 273 elements. Chroma collection UserData contains fewer than 272 elements. Chroma collection UserData contains fewer than 271 elements. Chroma collection UserData contains fewer than 270 elements. Chroma collection UserData contains fewer than 269 elements. Chroma collection UserData contains fewer than 268 elements. Chroma collection UserData contains fewer than 267 elements. Chroma collection UserData contains fewer than 266 elements. Chroma collection UserData contains fewer than 265 elements. Chroma collection UserData contains fewer than 264 elements. Chroma collection UserData contains fewer than 263 elements. Chroma collection UserData contains fewer than 262 elements. Chroma collection UserData contains fewer than 261 elements. Chroma collection UserData contains fewer than 260 elements. Chroma collection UserData contains fewer than 259 elements. Chroma collection UserData contains fewer than 258 elements. Chroma collection UserData contains fewer than 257 elements. Chroma collection UserData contains fewer than 256 elements. Chroma collection UserData contains fewer than 255 elements. Chroma collection UserData contains fewer than 254 elements. Chroma collection UserData contains fewer than 253 elements. Chroma collection UserData contains fewer than 252 elements. Chroma collection UserData contains fewer than 251 elements. Chroma collection UserData contains fewer than 250 elements. Chroma collection UserData contains fewer than 249 elements. Chroma collection UserData contains fewer than 248 elements. Chroma collection UserData contains fewer than 247 elements. Chroma collection UserData contains fewer than 246 elements. Chroma collection UserData contains fewer than 245 elements. Chroma collection UserData contains fewer than 244 elements. Chroma collection UserData contains fewer than 243 elements. Chroma collection UserData contains fewer than 242 elements. Chroma collection UserData contains fewer than 241 elements. Chroma collection UserData contains fewer than 240 elements. Chroma collection UserData contains fewer than 239 elements. Chroma collection UserData contains fewer than 238 elements. Chroma collection UserData contains fewer than 237 elements. Chroma collection UserData contains fewer than 236 elements. Chroma collection UserData contains fewer than 235 elements. Chroma collection UserData contains fewer than 234 elements. Chroma collection UserData contains fewer than 233 elements. Chroma collection UserData contains fewer than 232 elements. Chroma collection UserData contains fewer than 231 elements. Chroma collection UserData contains fewer than 230 elements. Chroma collection UserData contains fewer than 229 elements. Chroma collection UserData contains fewer than 228 elements. Chroma collection UserData contains fewer than 227 elements. Chroma collection UserData contains fewer than 226 elements. Chroma collection UserData contains fewer than 225 elements. Chroma collection UserData contains fewer than 224 elements. Chroma collection UserData contains fewer than 223 elements. Chroma collection UserData contains fewer than 222 elements. Chroma collection UserData contains fewer than 221 elements. Chroma collection UserData contains fewer than 220 elements. Chroma collection UserData contains fewer than 219 elements. Chroma collection UserData contains fewer than 218 elements. Chroma collection UserData contains fewer than 217 elements. Chroma collection UserData contains fewer than 216 elements. Chroma collection UserData contains fewer than 215 elements. Chroma collection UserData contains fewer than 214 elements. Chroma collection UserData contains fewer than 213 elements. Chroma collection UserData contains fewer than 212 elements. Chroma collection UserData contains fewer than 211 elements. Chroma collection UserData contains fewer than 210 elements. Chroma collection UserData contains fewer than 209 elements. Chroma collection UserData contains fewer than 208 elements. Chroma collection UserData contains fewer than 207 elements. Chroma collection UserData contains fewer than 206 elements. Chroma collection UserData contains fewer than 205 elements. Chroma collection UserData contains fewer than 204 elements. Chroma collection UserData contains fewer than 203 elements. Chroma collection UserData contains fewer than 202 elements. Chroma collection UserData contains fewer than 201 elements. Chroma collection UserData contains fewer than 200 elements. Chroma collection UserData contains fewer than 199 elements. Chroma collection UserData contains fewer than 198 elements. Chroma collection UserData contains fewer than 197 elements. Chroma collection UserData contains fewer than 196 elements. Chroma collection UserData contains fewer than 195 elements. Chroma collection UserData contains fewer than 194 elements. Chroma collection UserData contains fewer than 193 elements. Chroma collection UserData contains fewer than 192 elements. Chroma collection UserData contains fewer than 191 elements. Chroma collection UserData contains fewer than 190 elements. Chroma collection UserData contains fewer than 189 elements. Chroma collection UserData contains fewer than 188 elements. Chroma collection UserData contains fewer than 187 elements. Chroma collection UserData contains fewer than 186 elements. Chroma collection UserData contains fewer than 185 elements. Chroma collection UserData contains fewer than 184 elements. Chroma collection UserData contains fewer than 183 elements. Chroma collection UserData contains fewer than 182 elements. Chroma collection UserData contains fewer than 181 elements. Chroma collection UserData contains fewer than 180 elements. Chroma collection UserData contains fewer than 179 elements. Chroma collection UserData contains fewer than 178 elements. Chroma collection UserData contains fewer than 177 elements. Chroma collection UserData contains fewer than 176 elements. Chroma collection UserData contains fewer than 175 elements. Chroma collection UserData contains fewer than 174 elements. Chroma collection UserData contains fewer than 173 elements. Chroma collection UserData contains fewer than 172 elements. Chroma collection UserData contains fewer than 171 elements. Chroma collection UserData contains fewer than 170 elements. Chroma collection UserData contains fewer than 169 elements. Chroma collection UserData contains fewer than 168 elements. Chroma collection UserData contains fewer than 167 elements. Chroma collection UserData contains fewer than 166 elements. Chroma collection UserData contains fewer than 165 elements. Chroma collection UserData contains fewer than 164 elements. Chroma collection UserData contains fewer than 163 elements. Chroma collection UserData contains fewer than 162 elements. Chroma collection UserData contains fewer than 161 elements. Chroma collection UserData contains fewer than 160 elements. Chroma collection UserData contains fewer than 159 elements. Chroma collection UserData contains fewer than 158 elements. Chroma collection UserData contains fewer than 157 elements. Chroma collection UserData contains fewer than 156 elements. Chroma collection UserData contains fewer than 155 elements. Chroma collection UserData contains fewer than 154 elements. Chroma collection UserData contains fewer than 153 elements. Chroma collection UserData contains fewer than 152 elements. Chroma collection UserData contains fewer than 151 elements. Chroma collection UserData contains fewer than 150 elements. Chroma collection UserData contains fewer than 149 elements. Chroma collection UserData contains fewer than 148 elements. Chroma collection UserData contains fewer than 147 elements. Chroma collection UserData contains fewer than 146 elements. Chroma collection UserData contains fewer than 145 elements. Chroma collection UserData contains fewer than 144 elements. Chroma collection UserData contains fewer than 143 elements. Chroma collection UserData contains fewer than 142 elements. Chroma collection UserData contains fewer than 141 elements. Chroma collection UserData contains fewer than 140 elements. Chroma collection UserData contains fewer than 139 elements. Chroma collection UserData contains fewer than 138 elements. Chroma collection UserData contains fewer than 137 elements. Chroma collection UserData contains fewer than 136 elements. Chroma collection UserData contains fewer than 135 elements. Chroma collection UserData contains fewer than 134 elements. Chroma collection UserData contains fewer than 133 elements. Chroma collection UserData contains fewer than 132 elements. Chroma collection UserData contains fewer than 131 elements. Chroma collection UserData contains fewer than 130 elements. Chroma collection UserData contains fewer than 129 elements. Chroma collection UserData contains fewer than 128 elements. Chroma collection UserData contains fewer than 127 elements. Chroma collection UserData contains fewer than 126 elements. Chroma collection UserData contains fewer than 125 elements. Chroma collection UserData contains fewer than 124 elements. Chroma collection UserData contains fewer than 123 elements. Chroma collection UserData contains fewer than 122 elements. Chroma collection UserData contains fewer than 121 elements. Chroma collection UserData contains fewer than 120 elements. Chroma collection UserData contains fewer than 119 elements. Chroma collection UserData contains fewer than 118 elements. Chroma collection UserData contains fewer than 117 elements. Chroma collection UserData contains fewer than 116 elements. Chroma collection UserData contains fewer than 115 elements. Chroma collection UserData contains fewer than 114 elements. Chroma collection UserData contains fewer than 113 elements. Chroma collection UserData contains fewer than 112 elements. Chroma collection UserData contains fewer than 111 elements. Chroma collection UserData contains fewer than 110 elements. Chroma collection UserData contains fewer than 109 elements. Chroma collection UserData contains fewer than 108 elements. Chroma collection UserData contains fewer than 107 elements. Chroma collection UserData contains fewer than 106 elements. Chroma collection UserData contains fewer than 105 elements. Chroma collection UserData contains fewer than 104 elements. Chroma collection UserData contains fewer than 103 elements. Chroma collection UserData contains fewer than 102 elements. Chroma collection UserData contains fewer than 101 elements. Chroma collection UserData contains fewer than 100 elements. Chroma collection UserData contains fewer than 99 elements. Chroma collection UserData contains fewer than 98 elements. Chroma collection UserData contains fewer than 97 elements. Chroma collection UserData contains fewer than 96 elements. Chroma collection UserData contains fewer than 95 elements. Chroma collection UserData contains fewer than 94 elements. Chroma collection UserData contains fewer than 93 elements. Chroma collection UserData contains fewer than 92 elements. Chroma collection UserData contains fewer than 91 elements. Chroma collection UserData contains fewer than 90 elements. Chroma collection UserData contains fewer than 89 elements. Chroma collection UserData contains fewer than 88 elements. Chroma collection UserData contains fewer than 87 elements. Chroma collection UserData contains fewer than 86 elements. Chroma collection UserData contains fewer than 85 elements. Chroma collection UserData contains fewer than 84 elements. Chroma collection UserData contains fewer than 83 elements. Chroma collection UserData contains fewer than 82 elements. Chroma collection UserData contains fewer than 81 elements. Chroma collection UserData contains fewer than 80 elements. Chroma collection UserData contains fewer than 79 elements. Chroma collection UserData contains fewer than 78 elements. Chroma collection UserData contains fewer than 77 elements. Chroma collection UserData contains fewer than 76 elements. Chroma collection UserData contains fewer than 75 elements. Chroma collection UserData contains fewer than 74 elements. Chroma collection UserData contains fewer than 73 elements. Chroma collection UserData contains fewer than 72 elements. Chroma collection UserData contains fewer than 71 elements. Chroma collection UserData contains fewer than 70 elements. Chroma collection UserData contains fewer than 69 elements. Chroma collection UserData contains fewer than 68 elements. Chroma collection UserData contains fewer than 67 elements. Chroma collection UserData contains fewer than 66 elements. Chroma collection UserData contains fewer than 65 elements. Chroma collection UserData contains fewer than 64 elements. Chroma collection UserData contains fewer than 63 elements. Chroma collection UserData contains fewer than 62 elements. Chroma collection UserData contains fewer than 61 elements. Chroma collection UserData contains fewer than 60 elements. Chroma collection UserData contains fewer than 59 elements. Chroma collection UserData contains fewer than 58 elements. Chroma collection UserData contains fewer than 57 elements. Chroma collection UserData contains fewer than 56 elements. Chroma collection UserData contains fewer than 55 elements. Chroma collection UserData contains fewer than 54 elements. Chroma collection UserData contains fewer than 53 elements. Chroma collection UserData contains fewer than 52 elements. Chroma collection UserData contains fewer than 51 elements. Chroma collection UserData contains fewer than 50 elements. Chroma collection UserData contains fewer than 49 elements. Chroma collection UserData contains fewer than 48 elements. Chroma collection UserData contains fewer than 47 elements. Chroma collection UserData contains fewer than 46 elements. Chroma collection UserData contains fewer than 45 elements. Chroma collection UserData contains fewer than 44 elements. Chroma collection UserData contains fewer than 43 elements. Chroma collection UserData contains fewer than 42 elements. Chroma collection UserData contains fewer than 41 elements. Chroma collection UserData contains fewer than 40 elements. Chroma collection UserData contains fewer than 39 elements. Chroma collection UserData contains fewer than 38 elements. Chroma collection UserData contains fewer than 37 elements. Chroma collection UserData contains fewer than 36 elements. Chroma collection UserData contains fewer than 35 elements. Chroma collection UserData contains fewer than 34 elements. Chroma collection UserData contains fewer than 33 elements. Chroma collection UserData contains fewer than 32 elements. Chroma collection UserData contains fewer than 31 elements. Chroma collection UserData contains fewer than 30 elements. Chroma collection UserData contains fewer than 29 elements. Chroma collection UserData contains fewer than 28 elements. Chroma collection UserData contains fewer than 27 elements. Chroma collection UserData contains fewer than 26 elements. Chroma collection UserData contains fewer than 25 elements. Chroma collection UserData contains fewer than 24 elements. Chroma collection UserData contains fewer than 23 elements. Chroma collection UserData contains fewer than 22 elements. Chroma collection UserData contains fewer than 21 elements. Chroma collection UserData contains fewer than 20 elements. Chroma collection UserData contains fewer than 19 elements. Chroma collection UserData contains fewer than 18 elements. Chroma collection UserData contains fewer than 17 elements. Chroma collection UserData contains fewer than 16 elements. Chroma collection UserData contains fewer than 15 elements. Chroma collection UserData contains fewer than 14 elements. Chroma collection UserData contains fewer than 13 elements. Chroma collection UserData contains fewer than 12 elements. Chroma collection UserData contains fewer than 11 elements. Chroma collection UserData contains fewer than 10 elements. Chroma collection UserData contains fewer than 9 elements. Chroma collection UserData contains fewer than 8 elements. Chroma collection UserData contains fewer than 7 elements. Chroma collection UserData contains fewer than 6 elements. Chroma collection UserData contains fewer than 5 elements. Chroma collection UserData contains fewer than 4 elements. Chroma collection UserData contains fewer than 3 elements. Chroma collection UserData contains fewer than 2 elements. Distance: min: 1.4127063751220703 max: 1.4127063751220703 mean: 1.4127063751220703 median: 1.4127063751220703 thread exception: (<class 'TypeError'>, TypeError("cannot pickle '_thread.lock' object"), <traceback object at 0x7f442b99ee40>) make stop: (<class 'TypeError'>, TypeError("cannot pickle '_thread.lock' object"), <traceback object at 0x7f442b99ee40>) hit stop Traceback (most recent call last): File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/gradio/routes.py", line 414, in run_predict output = await app.get_blocks().process_api( File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/gradio/blocks.py", line 1323, in process_api result = await self.call_function( File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/gradio/blocks.py", line 1067, in call_function prediction = await utils.async_iteration(iterator) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/gradio/utils.py", line 339, in async_iteration return await iterator.anext() File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/gradio/utils.py", line 332, in anext return await anyio.to_thread.run_sync( File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run result = context.run(func, args) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/gradio/utils.py", line 315, in run_sync_iterator_async return next(iterator) File "/home/lelapn123/work/h2oa/h2ogpt/gradio_runner.py", line 932, in bot for output_fun in fun1(tuple(args_list)): File "/home/lelapn123/work/h2oa/h2ogpt/generate.py", line 922, in evaluate for r in run_qa_db(query=query, File "/home/lelapn123/work/h2oa/h2ogpt/gpt_langchain.py", line 957, in _run_qa_db raise thread.exc File "/home/lelapn123/work/h2oa/h2ogpt/utils.py", line 312, in run self._return = self._target(*self._args, self._kwargs) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/chains/base.py", line 140, in call raise e File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/chains/base.py", line 134, in call self._call(inputs, run_manager=run_manager) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/chains/combine_documents/base.py", line 84, in _call output, extra_return_dict = self.combine_docs( File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/chains/combine_documents/stuff.py", line 87, in combine_docs return self.llm_chain.predict(callbacks=callbacks, inputs), {} File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/chains/llm.py", line 213, in predict return self(kwargs, callbacks=callbacks)[self.output_key] File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/chains/base.py", line 140, in call raise e File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/chains/base.py", line 134, in call self._call(inputs, run_manager=run_manager) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/chains/llm.py", line 69, in _call response = self.generate([inputs], run_manager=run_manager) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/chains/llm.py", line 79, in generate return self.llm.generate_prompt( File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/llms/base.py", line 134, in generate_prompt return self.generate(prompt_strings, stop=stop, callbacks=callbacks) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/llms/base.py", line 191, in generate raise e File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/llms/base.py", line 185, in generate self._generate(prompts, stop=stop, run_manager=run_manager) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/llms/base.py", line 411, in _generate self._call(prompt, stop=stop, run_manager=run_manager) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/llms/huggingface_pipeline.py", line 159, in _call response = self.pipeline(prompt) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/transformers/pipelines/text_generation.py", line 201, in call return super().call(text_inputs, kwargs) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/transformers/pipelines/base.py", line 1119, in call return self.run_single(inputs, preprocess_params, forward_params, postprocess_params) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/transformers/pipelines/base.py", line 1126, in run_single model_outputs = self.forward(model_inputs, forward_params) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/transformers/pipelines/base.py", line 1025, in forward model_outputs = self._forward(model_inputs, *forward_params) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/transformers/pipelines/text_generation.py", line 245, in _forward generate_kwargs = copy.deepcopy(generate_kwargs) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/copy.py", line 146, in deepcopy y = copier(x, memo) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/copy.py", line 231, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/copy.py", line 172, in deepcopy y = _reconstruct(x, memo, rv) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/copy.py", line 271, in _reconstruct state = deepcopy(state, memo) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/copy.py", line 146, in deepcopy y = copier(x, memo) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/copy.py", line 231, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/copy.py", line 172, in deepcopy y = _reconstruct(x, memo, *rv) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/copy.py", line 271, in _reconstruct state = deepcopy(state, memo) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/copy.py", line 146, in deepcopy y = copier(x, memo) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/copy.py", line 231, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/copy.py", line 161, in deepcopy rv = reductor(4) TypeError: cannot pickle '_thread.lock' object

pseudotensor commented 1 year ago

Ah got it thanks. Yes, this is bug in transformers: https://github.com/huggingface/transformers/issues/23785

The work-around is to go back to the prior transformers transformers==4.28.1

i.e.

pip uninstall transformers -y
pip install transformers==4.28.1

I'll revert our requirements until they fix the bug. We don't need that one anyways unless one is doing 4-bit.

pseudotensor commented 1 year ago

Pickle issue should be fixed, will look at the other issue once download completes for vicuna.

lelapin123 commented 1 year ago

Ah got it thanks. Yes, this is bug in transformers: huggingface/transformers#23785

The work-around is to go back to the prior transformers transformers==4.28.1

i.e.

pip uninstall transformers -y
pip install transformers==4.28.1

I'll revert our requirements until they fix the bug. We don't need that one anyways unless one is doing 4-bit.

Well, i would be interested to have the 4-bit version of larger models. But i guess it is better to have something that works.

lelapin123 commented 1 year ago

it doesnt work. I did this : pip uninstall transformers -y pip install transformers==4.28.1

And indeed i was able to ask questions without having a bug (so it is better), but if i select a file and ask questions about it, the result is the same.

===================================BUG REPORT=================================== Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues

bin /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: /home/lelapn123/anaconda3/envs/h2oa did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths... warn(msg) /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('local/lelapn123-B450-I-AORUS-PRO-WIFI'), PosixPath('@/tmp/.ICE-unix/1961,unix/lelapn123-B450-I-AORUS-PRO-WIFI')} warn(msg) /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/etc/xdg/xdg-ubuntu')} warn(msg) /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/share/gconf/ubuntu.mandatory.path')} warn(msg) /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('1'), PosixPath('0')} warn(msg) /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/org/gnome/Terminal/screen/2684e21c_ede4_4a9d_b1f3_9ab1ce87b39b')} warn(msg) /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/share/gconf/ubuntu.default.path')} warn(msg) /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('path=/run/user/1000/bus,guid=8f75ba35b6f469324bd1a77c64724bc8'), PosixPath('unix')} warn(msg) CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths... CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so CUDA SETUP: Highest compute capability among GPUs detected: 8.6 CUDA SETUP: Detected CUDA version 121 CUDA SETUP: Loading binary /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so... Using Model junelee/wizard-vicuna-13b Prep: persist_directory=db_dir_UserData exists, using Removing old MyData: /tmp/db_dir_MyData_b740d972-00ec-4328-911a-b518339946e5 device_map: {'': 0} The argument trust_remote_code is to be used with Auto classes. It has no effect here and is ignored. Loading checkpoint shards: 100%|████████████| 6/6 [00:40<00:00, 6.71s/it] device_map: {'': 0} Running on local URL: http://0.0.0.0:7860

To create a public link, set share=True in launch(). Started GUI 100%|██████████████████████████████████████| 1/1 [00:00<00:00, 209.31it/s] 0it [00:00, ?it/s] Using embedded DuckDB with persistence: data will be stored in: /tmp/db_dir_MyData_70dade67-bce5-4b0a-9212-1e57ff131986 The model 'OptimizedModule' is not supported for text-generation. Supported models are ['BartForCausalLM', 'BertLMHeadModel', 'BertGenerationDecoder', 'BigBirdForCausalLM', 'BigBirdPegasusForCausalLM', 'BioGptForCausalLM', 'BlenderbotForCausalLM', 'BlenderbotSmallForCausalLM', 'BloomForCausalLM', 'CamembertForCausalLM', 'CodeGenForCausalLM', 'CpmAntForCausalLM', 'CTRLLMHeadModel', 'Data2VecTextForCausalLM', 'ElectraForCausalLM', 'ErnieForCausalLM', 'GitForCausalLM', 'GPT2LMHeadModel', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'GPTNeoForCausalLM', 'GPTNeoXForCausalLM', 'GPTNeoXJapaneseForCausalLM', 'GPTJForCausalLM', 'LlamaForCausalLM', 'MarianForCausalLM', 'MBartForCausalLM', 'MegaForCausalLM', 'MegatronBertForCausalLM', 'MvpForCausalLM', 'OpenAIGPTLMHeadModel', 'OPTForCausalLM', 'PegasusForCausalLM', 'PLBartForCausalLM', 'ProphetNetForCausalLM', 'QDQBertLMHeadModel', 'ReformerModelWithLMHead', 'RemBertForCausalLM', 'RobertaForCausalLM', 'RobertaPreLayerNormForCausalLM', 'RoCBertForCausalLM', 'RoFormerForCausalLM', 'Speech2Text2ForCausalLM', 'TransfoXLLMHeadModel', 'TrOCRForCausalLM', 'XGLMForCausalLM', 'XLMWithLMHeadModel', 'XLMProphetNetForCausalLM', 'XLMRobertaForCausalLM', 'XLMRobertaXLForCausalLM', 'XLNetLMHeadModel', 'XmodForCausalLM']. Chroma collection MyData contains fewer than 1000 elements. Chroma collection MyData contains fewer than 999 elements. Chroma collection MyData contains fewer than 998 elements. Chroma collection MyData contains fewer than 997 elements. Chroma collection MyData contains fewer than 996 elements. Chroma collection MyData contains fewer than 995 elements. Chroma collection MyData contains fewer than 994 elements. Chroma collection MyData contains fewer than 993 elements. Chroma collection MyData contains fewer than 992 elements. Chroma collection MyData contains fewer than 991 elements. Chroma collection MyData contains fewer than 990 elements. Chroma collection MyData contains fewer than 989 elements. Chroma collection MyData contains fewer than 988 elements. Chroma collection MyData contains fewer than 987 elements. Chroma collection MyData contains fewer than 986 elements. Chroma collection MyData contains fewer than 985 elements. Chroma collection MyData contains fewer than 984 elements. Chroma collection MyData contains fewer than 983 elements. Chroma collection MyData contains fewer than 982 elements. Chroma collection MyData contains fewer than 981 elements. Chroma collection MyData contains fewer than 980 elements. Chroma collection MyData contains fewer than 979 elements. Chroma collection MyData contains fewer than 978 elements. Chroma collection MyData contains fewer than 977 elements. Chroma collection MyData contains fewer than 976 elements. Chroma collection MyData contains fewer than 975 elements. Chroma collection MyData contains fewer than 974 elements. Chroma collection MyData contains fewer than 973 elements. Chroma collection MyData contains fewer than 972 elements. Chroma collection MyData contains fewer than 971 elements. Chroma collection MyData contains fewer than 970 elements. Chroma collection MyData contains fewer than 969 elements. Chroma collection MyData contains fewer than 968 elements. Chroma collection MyData contains fewer than 967 elements. Chroma collection MyData contains fewer than 966 elements. Chroma collection MyData contains fewer than 965 elements. Chroma collection MyData contains fewer than 964 elements. Chroma collection MyData contains fewer than 963 elements. Chroma collection MyData contains fewer than 962 elements. Chroma collection MyData contains fewer than 961 elements. Chroma collection MyData contains fewer than 960 elements. Chroma collection MyData contains fewer than 959 elements. Chroma collection MyData contains fewer than 958 elements. Chroma collection MyData contains fewer than 957 elements. Chroma collection MyData contains fewer than 956 elements. Chroma collection MyData contains fewer than 955 elements. Chroma collection MyData contains fewer than 954 elements. Chroma collection MyData contains fewer than 953 elements. Chroma collection MyData contains fewer than 952 elements. Chroma collection MyData contains fewer than 951 elements. Chroma collection MyData contains fewer than 950 elements. Chroma collection MyData contains fewer than 949 elements. Chroma collection MyData contains fewer than 948 elements. Chroma collection MyData contains fewer than 947 elements. Chroma collection MyData contains fewer than 946 elements. Chroma collection MyData contains fewer than 945 elements. Chroma collection MyData contains fewer than 944 elements. Chroma collection MyData contains fewer than 943 elements. Chroma collection MyData contains fewer than 942 elements. Chroma collection MyData contains fewer than 941 elements. Chroma collection MyData contains fewer than 940 elements. Chroma collection MyData contains fewer than 939 elements. Chroma collection MyData contains fewer than 938 elements. Chroma collection MyData contains fewer than 937 elements. Chroma collection MyData contains fewer than 936 elements. Chroma collection MyData contains fewer than 935 elements. Chroma collection MyData contains fewer than 934 elements. Chroma collection MyData contains fewer than 933 elements. Chroma collection MyData contains fewer than 932 elements. Chroma collection MyData contains fewer than 931 elements. Chroma collection MyData contains fewer than 930 elements. Chroma collection MyData contains fewer than 929 elements. Chroma collection MyData contains fewer than 928 elements. Chroma collection MyData contains fewer than 927 elements. Chroma collection MyData contains fewer than 926 elements. Chroma collection MyData contains fewer than 925 elements. Chroma collection MyData contains fewer than 924 elements. Chroma collection MyData contains fewer than 923 elements. Chroma collection MyData contains fewer than 922 elements. Chroma collection MyData contains fewer than 921 elements. Chroma collection MyData contains fewer than 920 elements. Chroma collection MyData contains fewer than 919 elements. Chroma collection MyData contains fewer than 918 elements. Chroma collection MyData contains fewer than 917 elements. Chroma collection MyData contains fewer than 916 elements. Chroma collection MyData contains fewer than 915 elements. Chroma collection MyData contains fewer than 914 elements. Chroma collection MyData contains fewer than 913 elements. Chroma collection MyData contains fewer than 912 elements. Chroma collection MyData contains fewer than 911 elements. Chroma collection MyData contains fewer than 910 elements. Chroma collection MyData contains fewer than 909 elements. Chroma collection MyData contains fewer than 908 elements. Chroma collection MyData contains fewer than 907 elements. Chroma collection MyData contains fewer than 906 elements. Chroma collection MyData contains fewer than 905 elements. Chroma collection MyData contains fewer than 904 elements. Chroma collection MyData contains fewer than 903 elements. Chroma collection MyData contains fewer than 902 elements. Chroma collection MyData contains fewer than 901 elements. Chroma collection MyData contains fewer than 900 elements. Chroma collection MyData contains fewer than 899 elements. Chroma collection MyData contains fewer than 898 elements. Chroma collection MyData contains fewer than 897 elements. Chroma collection MyData contains fewer than 896 elements. Chroma collection MyData contains fewer than 895 elements. Chroma collection MyData contains fewer than 894 elements. Chroma collection MyData contains fewer than 893 elements. Chroma collection MyData contains fewer than 892 elements. Chroma collection MyData contains fewer than 891 elements. Chroma collection MyData contains fewer than 890 elements. Chroma collection MyData contains fewer than 889 elements. Chroma collection MyData contains fewer than 888 elements. Chroma collection MyData contains fewer than 887 elements. Chroma collection MyData contains fewer than 886 elements. Chroma collection MyData contains fewer than 885 elements. Chroma collection MyData contains fewer than 884 elements. Chroma collection MyData contains fewer than 883 elements. Chroma collection MyData contains fewer than 882 elements. Chroma collection MyData contains fewer than 881 elements. Chroma collection MyData contains fewer than 880 elements. Chroma collection MyData contains fewer than 879 elements. Chroma collection MyData contains fewer than 878 elements. Chroma collection MyData contains fewer than 877 elements. Chroma collection MyData contains fewer than 876 elements. Chroma collection MyData contains fewer than 875 elements. Chroma collection MyData contains fewer than 874 elements. Chroma collection MyData contains fewer than 873 elements. Chroma collection MyData contains fewer than 872 elements. Chroma collection MyData contains fewer than 871 elements. Chroma collection MyData contains fewer than 870 elements. Chroma collection MyData contains fewer than 869 elements. Chroma collection MyData contains fewer than 868 elements. Chroma collection MyData contains fewer than 867 elements. Chroma collection MyData contains fewer than 866 elements. Chroma collection MyData contains fewer than 865 elements. Chroma collection MyData contains fewer than 864 elements. Chroma collection MyData contains fewer than 863 elements. Chroma collection MyData contains fewer than 862 elements. Chroma collection MyData contains fewer than 861 elements. Chroma collection MyData contains fewer than 860 elements. Chroma collection MyData contains fewer than 859 elements. Chroma collection MyData contains fewer than 858 elements. Chroma collection MyData contains fewer than 857 elements. Chroma collection MyData contains fewer than 856 elements. Chroma collection MyData contains fewer than 855 elements. Chroma collection MyData contains fewer than 854 elements. Chroma collection MyData contains fewer than 853 elements. Chroma collection MyData contains fewer than 852 elements. Chroma collection MyData contains fewer than 851 elements. Chroma collection MyData contains fewer than 850 elements. Chroma collection MyData contains fewer than 849 elements. Chroma collection MyData contains fewer than 848 elements. Chroma collection MyData contains fewer than 847 elements. Chroma collection MyData contains fewer than 846 elements. Chroma collection MyData contains fewer than 845 elements. Chroma collection MyData contains fewer than 844 elements. Chroma collection MyData contains fewer than 843 elements. Chroma collection MyData contains fewer than 842 elements. Chroma collection MyData contains fewer than 841 elements. Chroma collection MyData contains fewer than 840 elements. Chroma collection MyData contains fewer than 839 elements. Chroma collection MyData contains fewer than 838 elements. Chroma collection MyData contains fewer than 837 elements. Chroma collection MyData contains fewer than 836 elements. Chroma collection MyData contains fewer than 835 elements. Chroma collection MyData contains fewer than 834 elements. Chroma collection MyData contains fewer than 833 elements. Chroma collection MyData contains fewer than 832 elements. Chroma collection MyData contains fewer than 831 elements. Chroma collection MyData contains fewer than 830 elements. Chroma collection MyData contains fewer than 829 elements. Chroma collection MyData contains fewer than 828 elements. Chroma collection MyData contains fewer than 827 elements. Chroma collection MyData contains fewer than 826 elements. Chroma collection MyData contains fewer than 825 elements. Chroma collection MyData contains fewer than 824 elements. Chroma collection MyData contains fewer than 823 elements. Chroma collection MyData contains fewer than 822 elements. Chroma collection MyData contains fewer than 821 elements. Chroma collection MyData contains fewer than 820 elements. Chroma collection MyData contains fewer than 819 elements. Chroma collection MyData contains fewer than 818 elements. Chroma collection MyData contains fewer than 817 elements. Chroma collection MyData contains fewer than 816 elements. Chroma collection MyData contains fewer than 815 elements. Chroma collection MyData contains fewer than 814 elements. Chroma collection MyData contains fewer than 813 elements. Chroma collection MyData contains fewer than 812 elements. Chroma collection MyData contains fewer than 811 elements. Chroma collection MyData contains fewer than 810 elements. Chroma collection MyData contains fewer than 809 elements. Chroma collection MyData contains fewer than 808 elements. Chroma collection MyData contains fewer than 807 elements. Chroma collection MyData contains fewer than 806 elements. Chroma collection MyData contains fewer than 805 elements. Chroma collection MyData contains fewer than 804 elements. Chroma collection MyData contains fewer than 803 elements. Chroma collection MyData contains fewer than 802 elements. Chroma collection MyData contains fewer than 801 elements. Chroma collection MyData contains fewer than 800 elements. Chroma collection MyData contains fewer than 799 elements. Chroma collection MyData contains fewer than 798 elements. Chroma collection MyData contains fewer than 797 elements. Chroma collection MyData contains fewer than 796 elements. Chroma collection MyData contains fewer than 795 elements. Chroma collection MyData contains fewer than 794 elements. Chroma collection MyData contains fewer than 793 elements. Chroma collection MyData contains fewer than 792 elements. Chroma collection MyData contains fewer than 791 elements. Chroma collection MyData contains fewer than 790 elements. Chroma collection MyData contains fewer than 789 elements. Chroma collection MyData contains fewer than 788 elements. Chroma collection MyData contains fewer than 787 elements. Chroma collection MyData contains fewer than 786 elements. Chroma collection MyData contains fewer than 785 elements. Chroma collection MyData contains fewer than 784 elements. Chroma collection MyData contains fewer than 783 elements. Chroma collection MyData contains fewer than 782 elements. Chroma collection MyData contains fewer than 781 elements. Chroma collection MyData contains fewer than 780 elements. Chroma collection MyData contains fewer than 779 elements. Chroma collection MyData contains fewer than 778 elements. Chroma collection MyData contains fewer than 777 elements. Chroma collection MyData contains fewer than 776 elements. Chroma collection MyData contains fewer than 775 elements. Chroma collection MyData contains fewer than 774 elements. Chroma collection MyData contains fewer than 773 elements. Chroma collection MyData contains fewer than 772 elements. Chroma collection MyData contains fewer than 771 elements. Chroma collection MyData contains fewer than 770 elements. Chroma collection MyData contains fewer than 769 elements. Chroma collection MyData contains fewer than 768 elements. Chroma collection MyData contains fewer than 767 elements. Chroma collection MyData contains fewer than 766 elements. Chroma collection MyData contains fewer than 765 elements. Chroma collection MyData contains fewer than 764 elements. Chroma collection MyData contains fewer than 763 elements. Chroma collection MyData contains fewer than 762 elements. Chroma collection MyData contains fewer than 761 elements. Chroma collection MyData contains fewer than 760 elements. Chroma collection MyData contains fewer than 759 elements. Chroma collection MyData contains fewer than 758 elements. Chroma collection MyData contains fewer than 757 elements. Chroma collection MyData contains fewer than 756 elements. Chroma collection MyData contains fewer than 755 elements. Chroma collection MyData contains fewer than 754 elements. Chroma collection MyData contains fewer than 753 elements. Chroma collection MyData contains fewer than 752 elements. Chroma collection MyData contains fewer than 751 elements. Chroma collection MyData contains fewer than 750 elements. Chroma collection MyData contains fewer than 749 elements. Chroma collection MyData contains fewer than 748 elements. Chroma collection MyData contains fewer than 747 elements. Chroma collection MyData contains fewer than 746 elements. Chroma collection MyData contains fewer than 745 elements. Chroma collection MyData contains fewer than 744 elements. Chroma collection MyData contains fewer than 743 elements. Chroma collection MyData contains fewer than 742 elements. Chroma collection MyData contains fewer than 741 elements. Chroma collection MyData contains fewer than 740 elements. Chroma collection MyData contains fewer than 739 elements. Chroma collection MyData contains fewer than 738 elements. Chroma collection MyData contains fewer than 737 elements. Chroma collection MyData contains fewer than 736 elements. Chroma collection MyData contains fewer than 735 elements. Chroma collection MyData contains fewer than 734 elements. Chroma collection MyData contains fewer than 733 elements. Chroma collection MyData contains fewer than 732 elements. Chroma collection MyData contains fewer than 731 elements. Chroma collection MyData contains fewer than 730 elements. Chroma collection MyData contains fewer than 729 elements. Chroma collection MyData contains fewer than 728 elements. Chroma collection MyData contains fewer than 727 elements. Chroma collection MyData contains fewer than 726 elements. Chroma collection MyData contains fewer than 725 elements. Chroma collection MyData contains fewer than 724 elements. Chroma collection MyData contains fewer than 723 elements. Chroma collection MyData contains fewer than 722 elements. Chroma collection MyData contains fewer than 721 elements. Chroma collection MyData contains fewer than 720 elements. Chroma collection MyData contains fewer than 719 elements. Chroma collection MyData contains fewer than 718 elements. Chroma collection MyData contains fewer than 717 elements. Chroma collection MyData contains fewer than 716 elements. Chroma collection MyData contains fewer than 715 elements. Chroma collection MyData contains fewer than 714 elements. Chroma collection MyData contains fewer than 713 elements. Chroma collection MyData contains fewer than 712 elements. Chroma collection MyData contains fewer than 711 elements. Chroma collection MyData contains fewer than 710 elements. Chroma collection MyData contains fewer than 709 elements. Chroma collection MyData contains fewer than 708 elements. Chroma collection MyData contains fewer than 707 elements. Chroma collection MyData contains fewer than 706 elements. Chroma collection MyData contains fewer than 705 elements. Chroma collection MyData contains fewer than 704 elements. Chroma collection MyData contains fewer than 703 elements. Chroma collection MyData contains fewer than 702 elements. Chroma collection MyData contains fewer than 701 elements. Chroma collection MyData contains fewer than 700 elements. Chroma collection MyData contains fewer than 699 elements. Chroma collection MyData contains fewer than 698 elements. Chroma collection MyData contains fewer than 697 elements. Chroma collection MyData contains fewer than 696 elements. Chroma collection MyData contains fewer than 695 elements. Chroma collection MyData contains fewer than 694 elements. Chroma collection MyData contains fewer than 693 elements. Chroma collection MyData contains fewer than 692 elements. Chroma collection MyData contains fewer than 691 elements. Chroma collection MyData contains fewer than 690 elements. Chroma collection MyData contains fewer than 689 elements. Chroma collection MyData contains fewer than 688 elements. Chroma collection MyData contains fewer than 687 elements. Chroma collection MyData contains fewer than 686 elements. Chroma collection MyData contains fewer than 685 elements. Chroma collection MyData contains fewer than 684 elements. Chroma collection MyData contains fewer than 683 elements. Chroma collection MyData contains fewer than 682 elements. Chroma collection MyData contains fewer than 681 elements. Chroma collection MyData contains fewer than 680 elements. Chroma collection MyData contains fewer than 679 elements. Chroma collection MyData contains fewer than 678 elements. Chroma collection MyData contains fewer than 677 elements. Chroma collection MyData contains fewer than 676 elements. Chroma collection MyData contains fewer than 675 elements. Chroma collection MyData contains fewer than 674 elements. Chroma collection MyData contains fewer than 673 elements. Chroma collection MyData contains fewer than 672 elements. Chroma collection MyData contains fewer than 671 elements. Chroma collection MyData contains fewer than 670 elements. Chroma collection MyData contains fewer than 669 elements. Chroma collection MyData contains fewer than 668 elements. Chroma collection MyData contains fewer than 667 elements. Chroma collection MyData contains fewer than 666 elements. Chroma collection MyData contains fewer than 665 elements. Chroma collection MyData contains fewer than 664 elements. Chroma collection MyData contains fewer than 663 elements. Chroma collection MyData contains fewer than 662 elements. Chroma collection MyData contains fewer than 661 elements. Chroma collection MyData contains fewer than 660 elements. Chroma collection MyData contains fewer than 659 elements. Chroma collection MyData contains fewer than 658 elements. Chroma collection MyData contains fewer than 657 elements. Chroma collection MyData contains fewer than 656 elements. Chroma collection MyData contains fewer than 655 elements. Chroma collection MyData contains fewer than 654 elements. Chroma collection MyData contains fewer than 653 elements. Chroma collection MyData contains fewer than 652 elements. Chroma collection MyData contains fewer than 651 elements. Chroma collection MyData contains fewer than 650 elements. Chroma collection MyData contains fewer than 649 elements. Chroma collection MyData contains fewer than 648 elements. Chroma collection MyData contains fewer than 647 elements. Chroma collection MyData contains fewer than 646 elements. Chroma collection MyData contains fewer than 645 elements. Chroma collection MyData contains fewer than 644 elements. Chroma collection MyData contains fewer than 643 elements. Chroma collection MyData contains fewer than 642 elements. Chroma collection MyData contains fewer than 641 elements. Chroma collection MyData contains fewer than 640 elements. Chroma collection MyData contains fewer than 639 elements. Chroma collection MyData contains fewer than 638 elements. Chroma collection MyData contains fewer than 637 elements. Chroma collection MyData contains fewer than 636 elements. Chroma collection MyData contains fewer than 635 elements. Chroma collection MyData contains fewer than 634 elements. Chroma collection MyData contains fewer than 633 elements. Chroma collection MyData contains fewer than 632 elements. Chroma collection MyData contains fewer than 631 elements. Chroma collection MyData contains fewer than 630 elements. Chroma collection MyData contains fewer than 629 elements. Chroma collection MyData contains fewer than 628 elements. Chroma collection MyData contains fewer than 627 elements. Chroma collection MyData contains fewer than 626 elements. Chroma collection MyData contains fewer than 625 elements. Chroma collection MyData contains fewer than 624 elements. Chroma collection MyData contains fewer than 623 elements. Chroma collection MyData contains fewer than 622 elements. Chroma collection MyData contains fewer than 621 elements. Chroma collection MyData contains fewer than 620 elements. Chroma collection MyData contains fewer than 619 elements. Chroma collection MyData contains fewer than 618 elements. Chroma collection MyData contains fewer than 617 elements. Chroma collection MyData contains fewer than 616 elements. Chroma collection MyData contains fewer than 615 elements. Chroma collection MyData contains fewer than 614 elements. Chroma collection MyData contains fewer than 613 elements. Chroma collection MyData contains fewer than 612 elements. Chroma collection MyData contains fewer than 611 elements. Chroma collection MyData contains fewer than 610 elements. Chroma collection MyData contains fewer than 609 elements. Chroma collection MyData contains fewer than 608 elements. Chroma collection MyData contains fewer than 607 elements. Chroma collection MyData contains fewer than 606 elements. Chroma collection MyData contains fewer than 605 elements. Chroma collection MyData contains fewer than 604 elements. Chroma collection MyData contains fewer than 603 elements. Chroma collection MyData contains fewer than 602 elements. Chroma collection MyData contains fewer than 601 elements. Chroma collection MyData contains fewer than 600 elements. Chroma collection MyData contains fewer than 599 elements. Chroma collection MyData contains fewer than 598 elements. Chroma collection MyData contains fewer than 597 elements. Chroma collection MyData contains fewer than 596 elements. Chroma collection MyData contains fewer than 595 elements. Chroma collection MyData contains fewer than 594 elements. Chroma collection MyData contains fewer than 593 elements. Chroma collection MyData contains fewer than 592 elements. Chroma collection MyData contains fewer than 591 elements. Chroma collection MyData contains fewer than 590 elements. Chroma collection MyData contains fewer than 589 elements. Chroma collection MyData contains fewer than 588 elements. Chroma collection MyData contains fewer than 587 elements. Chroma collection MyData contains fewer than 586 elements. Chroma collection MyData contains fewer than 585 elements. Chroma collection MyData contains fewer than 584 elements. Chroma collection MyData contains fewer than 583 elements. Chroma collection MyData contains fewer than 582 elements. Chroma collection MyData contains fewer than 581 elements. Chroma collection MyData contains fewer than 580 elements. Chroma collection MyData contains fewer than 579 elements. Chroma collection MyData contains fewer than 578 elements. Chroma collection MyData contains fewer than 577 elements. Chroma collection MyData contains fewer than 576 elements. Chroma collection MyData contains fewer than 575 elements. Chroma collection MyData contains fewer than 574 elements. Chroma collection MyData contains fewer than 573 elements. Chroma collection MyData contains fewer than 572 elements. Chroma collection MyData contains fewer than 571 elements. Chroma collection MyData contains fewer than 570 elements. Chroma collection MyData contains fewer than 569 elements. Chroma collection MyData contains fewer than 568 elements. Chroma collection MyData contains fewer than 567 elements. Chroma collection MyData contains fewer than 566 elements. Chroma collection MyData contains fewer than 565 elements. Chroma collection MyData contains fewer than 564 elements. Chroma collection MyData contains fewer than 563 elements. Chroma collection MyData contains fewer than 562 elements. Chroma collection MyData contains fewer than 561 elements. Chroma collection MyData contains fewer than 560 elements. Chroma collection MyData contains fewer than 559 elements. Chroma collection MyData contains fewer than 558 elements. Chroma collection MyData contains fewer than 557 elements. Chroma collection MyData contains fewer than 556 elements. Chroma collection MyData contains fewer than 555 elements. Chroma collection MyData contains fewer than 554 elements. Chroma collection MyData contains fewer than 553 elements. Chroma collection MyData contains fewer than 552 elements. Chroma collection MyData contains fewer than 551 elements. Chroma collection MyData contains fewer than 550 elements. Chroma collection MyData contains fewer than 549 elements. Chroma collection MyData contains fewer than 548 elements. Chroma collection MyData contains fewer than 547 elements. Chroma collection MyData contains fewer than 546 elements. Chroma collection MyData contains fewer than 545 elements. Chroma collection MyData contains fewer than 544 elements. Chroma collection MyData contains fewer than 543 elements. Chroma collection MyData contains fewer than 542 elements. Chroma collection MyData contains fewer than 541 elements. Chroma collection MyData contains fewer than 540 elements. Chroma collection MyData contains fewer than 539 elements. Chroma collection MyData contains fewer than 538 elements. Chroma collection MyData contains fewer than 537 elements. Chroma collection MyData contains fewer than 536 elements. Chroma collection MyData contains fewer than 535 elements. Chroma collection MyData contains fewer than 534 elements. Chroma collection MyData contains fewer than 533 elements. Chroma collection MyData contains fewer than 532 elements. Chroma collection MyData contains fewer than 531 elements. Chroma collection MyData contains fewer than 530 elements. Chroma collection MyData contains fewer than 529 elements. Chroma collection MyData contains fewer than 528 elements. Chroma collection MyData contains fewer than 527 elements. Chroma collection MyData contains fewer than 526 elements. Chroma collection MyData contains fewer than 525 elements. Chroma collection MyData contains fewer than 524 elements. Chroma collection MyData contains fewer than 523 elements. Chroma collection MyData contains fewer than 522 elements. Chroma collection MyData contains fewer than 521 elements. Chroma collection MyData contains fewer than 520 elements. Chroma collection MyData contains fewer than 519 elements. Chroma collection MyData contains fewer than 518 elements. Chroma collection MyData contains fewer than 517 elements. Chroma collection MyData contains fewer than 516 elements. Chroma collection MyData contains fewer than 515 elements. Chroma collection MyData contains fewer than 514 elements. Chroma collection MyData contains fewer than 513 elements. Chroma collection MyData contains fewer than 512 elements. Chroma collection MyData contains fewer than 511 elements. Chroma collection MyData contains fewer than 510 elements. Chroma collection MyData contains fewer than 509 elements. Chroma collection MyData contains fewer than 508 elements. Chroma collection MyData contains fewer than 507 elements. Chroma collection MyData contains fewer than 506 elements. Chroma collection MyData contains fewer than 505 elements. Chroma collection MyData contains fewer than 504 elements. Chroma collection MyData contains fewer than 503 elements. Chroma collection MyData contains fewer than 502 elements. Chroma collection MyData contains fewer than 501 elements. Chroma collection MyData contains fewer than 500 elements. Chroma collection MyData contains fewer than 499 elements. Chroma collection MyData contains fewer than 498 elements. Chroma collection MyData contains fewer than 497 elements. Chroma collection MyData contains fewer than 496 elements. Chroma collection MyData contains fewer than 495 elements. Chroma collection MyData contains fewer than 494 elements. Chroma collection MyData contains fewer than 493 elements. Chroma collection MyData contains fewer than 492 elements. Chroma collection MyData contains fewer than 491 elements. Chroma collection MyData contains fewer than 490 elements. Chroma collection MyData contains fewer than 489 elements. Chroma collection MyData contains fewer than 488 elements. Chroma collection MyData contains fewer than 487 elements. Chroma collection MyData contains fewer than 486 elements. Chroma collection MyData contains fewer than 485 elements. Chroma collection MyData contains fewer than 484 elements. Chroma collection MyData contains fewer than 483 elements. Chroma collection MyData contains fewer than 482 elements. Chroma collection MyData contains fewer than 481 elements. Chroma collection MyData contains fewer than 480 elements. Chroma collection MyData contains fewer than 479 elements. Chroma collection MyData contains fewer than 478 elements. Chroma collection MyData contains fewer than 477 elements. Chroma collection MyData contains fewer than 476 elements. Chroma collection MyData contains fewer than 475 elements. Chroma collection MyData contains fewer than 474 elements. Chroma collection MyData contains fewer than 473 elements. Chroma collection MyData contains fewer than 472 elements. Chroma collection MyData contains fewer than 471 elements. Chroma collection MyData contains fewer than 470 elements. Chroma collection MyData contains fewer than 469 elements. Chroma collection MyData contains fewer than 468 elements. Chroma collection MyData contains fewer than 467 elements. Chroma collection MyData contains fewer than 466 elements. Chroma collection MyData contains fewer than 465 elements. Chroma collection MyData contains fewer than 464 elements. Chroma collection MyData contains fewer than 463 elements. Chroma collection MyData contains fewer than 462 elements. Chroma collection MyData contains fewer than 461 elements. Chroma collection MyData contains fewer than 460 elements. Chroma collection MyData contains fewer than 459 elements. Chroma collection MyData contains fewer than 458 elements. Chroma collection MyData contains fewer than 457 elements. Chroma collection MyData contains fewer than 456 elements. Chroma collection MyData contains fewer than 455 elements. Chroma collection MyData contains fewer than 454 elements. Chroma collection MyData contains fewer than 453 elements. Chroma collection MyData contains fewer than 452 elements. Chroma collection MyData contains fewer than 451 elements. Chroma collection MyData contains fewer than 450 elements. Chroma collection MyData contains fewer than 449 elements. Chroma collection MyData contains fewer than 448 elements. Chroma collection MyData contains fewer than 447 elements. Chroma collection MyData contains fewer than 446 elements. Chroma collection MyData contains fewer than 445 elements. Chroma collection MyData contains fewer than 444 elements. Chroma collection MyData contains fewer than 443 elements. Chroma collection MyData contains fewer than 442 elements. Chroma collection MyData contains fewer than 441 elements. Chroma collection MyData contains fewer than 440 elements. Chroma collection MyData contains fewer than 439 elements. Chroma collection MyData contains fewer than 438 elements. Chroma collection MyData contains fewer than 437 elements. Chroma collection MyData contains fewer than 436 elements. Chroma collection MyData contains fewer than 435 elements. Chroma collection MyData contains fewer than 434 elements. Chroma collection MyData contains fewer than 433 elements. Chroma collection MyData contains fewer than 432 elements. Chroma collection MyData contains fewer than 431 elements. Chroma collection MyData contains fewer than 430 elements. Chroma collection MyData contains fewer than 429 elements. Chroma collection MyData contains fewer than 428 elements. Chroma collection MyData contains fewer than 427 elements. Chroma collection MyData contains fewer than 426 elements. Chroma collection MyData contains fewer than 425 elements. Chroma collection MyData contains fewer than 424 elements. Chroma collection MyData contains fewer than 423 elements. Chroma collection MyData contains fewer than 422 elements. Chroma collection MyData contains fewer than 421 elements. Chroma collection MyData contains fewer than 420 elements. Chroma collection MyData contains fewer than 419 elements. Chroma collection MyData contains fewer than 418 elements. Chroma collection MyData contains fewer than 417 elements. Chroma collection MyData contains fewer than 416 elements. Chroma collection MyData contains fewer than 415 elements. Chroma collection MyData contains fewer than 414 elements. Chroma collection MyData contains fewer than 413 elements. Chroma collection MyData contains fewer than 412 elements. Chroma collection MyData contains fewer than 411 elements. Chroma collection MyData contains fewer than 410 elements. Chroma collection MyData contains fewer than 409 elements. Chroma collection MyData contains fewer than 408 elements. Chroma collection MyData contains fewer than 407 elements. Chroma collection MyData contains fewer than 406 elements. Chroma collection MyData contains fewer than 405 elements. Chroma collection MyData contains fewer than 404 elements. Chroma collection MyData contains fewer than 403 elements. Chroma collection MyData contains fewer than 402 elements. Chroma collection MyData contains fewer than 401 elements. Chroma collection MyData contains fewer than 400 elements. Chroma collection MyData contains fewer than 399 elements. Chroma collection MyData contains fewer than 398 elements. Chroma collection MyData contains fewer than 397 elements. Chroma collection MyData contains fewer than 396 elements. Chroma collection MyData contains fewer than 395 elements. Chroma collection MyData contains fewer than 394 elements. Chroma collection MyData contains fewer than 393 elements. Chroma collection MyData contains fewer than 392 elements. Chroma collection MyData contains fewer than 391 elements. Chroma collection MyData contains fewer than 390 elements. Chroma collection MyData contains fewer than 389 elements. Chroma collection MyData contains fewer than 388 elements. Chroma collection MyData contains fewer than 387 elements. Chroma collection MyData contains fewer than 386 elements. Chroma collection MyData contains fewer than 385 elements. Chroma collection MyData contains fewer than 384 elements. Chroma collection MyData contains fewer than 383 elements. Chroma collection MyData contains fewer than 382 elements. Chroma collection MyData contains fewer than 381 elements. Chroma collection MyData contains fewer than 380 elements. Chroma collection MyData contains fewer than 379 elements. Chroma collection MyData contains fewer than 378 elements. Chroma collection MyData contains fewer than 377 elements. Chroma collection MyData contains fewer than 376 elements. Chroma collection MyData contains fewer than 375 elements. Chroma collection MyData contains fewer than 374 elements. Chroma collection MyData contains fewer than 373 elements. Chroma collection MyData contains fewer than 372 elements. Chroma collection MyData contains fewer than 371 elements. Chroma collection MyData contains fewer than 370 elements. Chroma collection MyData contains fewer than 369 elements. Chroma collection MyData contains fewer than 368 elements. Chroma collection MyData contains fewer than 367 elements. Chroma collection MyData contains fewer than 366 elements. Chroma collection MyData contains fewer than 365 elements. Chroma collection MyData contains fewer than 364 elements. Chroma collection MyData contains fewer than 363 elements. Chroma collection MyData contains fewer than 362 elements. Chroma collection MyData contains fewer than 361 elements. Chroma collection MyData contains fewer than 360 elements. Chroma collection MyData contains fewer than 359 elements. Chroma collection MyData contains fewer than 358 elements. Chroma collection MyData contains fewer than 357 elements. Chroma collection MyData contains fewer than 356 elements. Chroma collection MyData contains fewer than 355 elements. Chroma collection MyData contains fewer than 354 elements. Chroma collection MyData contains fewer than 353 elements. Chroma collection MyData contains fewer than 352 elements. Chroma collection MyData contains fewer than 351 elements. Chroma collection MyData contains fewer than 350 elements. Chroma collection MyData contains fewer than 349 elements. Chroma collection MyData contains fewer than 348 elements. Chroma collection MyData contains fewer than 347 elements. Chroma collection MyData contains fewer than 346 elements. Chroma collection MyData contains fewer than 345 elements. Chroma collection MyData contains fewer than 344 elements. Chroma collection MyData contains fewer than 343 elements. Chroma collection MyData contains fewer than 342 elements. Chroma collection MyData contains fewer than 341 elements. Chroma collection MyData contains fewer than 340 elements. Chroma collection MyData contains fewer than 339 elements. Chroma collection MyData contains fewer than 338 elements. Chroma collection MyData contains fewer than 337 elements. Chroma collection MyData contains fewer than 336 elements. Chroma collection MyData contains fewer than 335 elements. Chroma collection MyData contains fewer than 334 elements. Chroma collection MyData contains fewer than 333 elements. Chroma collection MyData contains fewer than 332 elements. Chroma collection MyData contains fewer than 331 elements. Chroma collection MyData contains fewer than 330 elements. Chroma collection MyData contains fewer than 329 elements. Chroma collection MyData contains fewer than 328 elements. Chroma collection MyData contains fewer than 327 elements. Chroma collection MyData contains fewer than 326 elements. Chroma collection MyData contains fewer than 325 elements. Chroma collection MyData contains fewer than 324 elements. Chroma collection MyData contains fewer than 323 elements. Chroma collection MyData contains fewer than 322 elements. Chroma collection MyData contains fewer than 321 elements. Chroma collection MyData contains fewer than 320 elements. Chroma collection MyData contains fewer than 319 elements. Chroma collection MyData contains fewer than 318 elements. Chroma collection MyData contains fewer than 317 elements. Chroma collection MyData contains fewer than 316 elements. Chroma collection MyData contains fewer than 315 elements. Chroma collection MyData contains fewer than 314 elements. Chroma collection MyData contains fewer than 313 elements. Chroma collection MyData contains fewer than 312 elements. Chroma collection MyData contains fewer than 311 elements. Chroma collection MyData contains fewer than 310 elements. Chroma collection MyData contains fewer than 309 elements. Chroma collection MyData contains fewer than 308 elements. Chroma collection MyData contains fewer than 307 elements. Chroma collection MyData contains fewer than 306 elements. Chroma collection MyData contains fewer than 305 elements. Chroma collection MyData contains fewer than 304 elements. Chroma collection MyData contains fewer than 303 elements. Chroma collection MyData contains fewer than 302 elements. Chroma collection MyData contains fewer than 301 elements. Chroma collection MyData contains fewer than 300 elements. Chroma collection MyData contains fewer than 299 elements. Chroma collection MyData contains fewer than 298 elements. Chroma collection MyData contains fewer than 297 elements. Chroma collection MyData contains fewer than 296 elements. Chroma collection MyData contains fewer than 295 elements. Chroma collection MyData contains fewer than 294 elements. Chroma collection MyData contains fewer than 293 elements. Chroma collection MyData contains fewer than 292 elements. Chroma collection MyData contains fewer than 291 elements. Chroma collection MyData contains fewer than 290 elements. Chroma collection MyData contains fewer than 289 elements. Chroma collection MyData contains fewer than 288 elements. Chroma collection MyData contains fewer than 287 elements. Chroma collection MyData contains fewer than 286 elements. Chroma collection MyData contains fewer than 285 elements. Chroma collection MyData contains fewer than 284 elements. Chroma collection MyData contains fewer than 283 elements. Chroma collection MyData contains fewer than 282 elements. Chroma collection MyData contains fewer than 281 elements. Chroma collection MyData contains fewer than 280 elements. Chroma collection MyData contains fewer than 279 elements. Chroma collection MyData contains fewer than 278 elements. Chroma collection MyData contains fewer than 277 elements. Chroma collection MyData contains fewer than 276 elements. Chroma collection MyData contains fewer than 275 elements. Chroma collection MyData contains fewer than 274 elements. Chroma collection MyData contains fewer than 273 elements. Chroma collection MyData contains fewer than 272 elements. Chroma collection MyData contains fewer than 271 elements. Chroma collection MyData contains fewer than 270 elements. Chroma collection MyData contains fewer than 269 elements. Chroma collection MyData contains fewer than 268 elements. Chroma collection MyData contains fewer than 267 elements. Chroma collection MyData contains fewer than 266 elements. Chroma collection MyData contains fewer than 265 elements. Chroma collection MyData contains fewer than 264 elements. Chroma collection MyData contains fewer than 263 elements. Chroma collection MyData contains fewer than 262 elements. Chroma collection MyData contains fewer than 261 elements. Chroma collection MyData contains fewer than 260 elements. Chroma collection MyData contains fewer than 259 elements. Chroma collection MyData contains fewer than 258 elements. Chroma collection MyData contains fewer than 257 elements. Chroma collection MyData contains fewer than 256 elements. Chroma collection MyData contains fewer than 255 elements. Chroma collection MyData contains fewer than 254 elements. Chroma collection MyData contains fewer than 253 elements. Chroma collection MyData contains fewer than 252 elements. Chroma collection MyData contains fewer than 251 elements. Chroma collection MyData contains fewer than 250 elements. Chroma collection MyData contains fewer than 249 elements. Chroma collection MyData contains fewer than 248 elements. Chroma collection MyData contains fewer than 247 elements. Chroma collection MyData contains fewer than 246 elements. Chroma collection MyData contains fewer than 245 elements. Chroma collection MyData contains fewer than 244 elements. Chroma collection MyData contains fewer than 243 elements. Chroma collection MyData contains fewer than 242 elements. Chroma collection MyData contains fewer than 241 elements. Chroma collection MyData contains fewer than 240 elements. Chroma collection MyData contains fewer than 239 elements. Chroma collection MyData contains fewer than 238 elements. Chroma collection MyData contains fewer than 237 elements. Chroma collection MyData contains fewer than 236 elements. Chroma collection MyData contains fewer than 235 elements. Chroma collection MyData contains fewer than 234 elements. Chroma collection MyData contains fewer than 233 elements. Chroma collection MyData contains fewer than 232 elements. Chroma collection MyData contains fewer than 231 elements. Chroma collection MyData contains fewer than 230 elements. Chroma collection MyData contains fewer than 229 elements. Chroma collection MyData contains fewer than 228 elements. Chroma collection MyData contains fewer than 227 elements. Chroma collection MyData contains fewer than 226 elements. Chroma collection MyData contains fewer than 225 elements. Chroma collection MyData contains fewer than 224 elements. Chroma collection MyData contains fewer than 223 elements. Chroma collection MyData contains fewer than 222 elements. Chroma collection MyData contains fewer than 221 elements. Chroma collection MyData contains fewer than 220 elements. Chroma collection MyData contains fewer than 219 elements. Chroma collection MyData contains fewer than 218 elements. Chroma collection MyData contains fewer than 217 elements. Chroma collection MyData contains fewer than 216 elements. Chroma collection MyData contains fewer than 215 elements. Chroma collection MyData contains fewer than 214 elements. Chroma collection MyData contains fewer than 213 elements. Chroma collection MyData contains fewer than 212 elements. Chroma collection MyData contains fewer than 211 elements. Chroma collection MyData contains fewer than 210 elements. Chroma collection MyData contains fewer than 209 elements. Chroma collection MyData contains fewer than 208 elements. Chroma collection MyData contains fewer than 207 elements. Chroma collection MyData contains fewer than 206 elements. Chroma collection MyData contains fewer than 205 elements. Chroma collection MyData contains fewer than 204 elements. Chroma collection MyData contains fewer than 203 elements. Chroma collection MyData contains fewer than 202 elements. Chroma collection MyData contains fewer than 201 elements. Chroma collection MyData contains fewer than 200 elements. Chroma collection MyData contains fewer than 199 elements. Chroma collection MyData contains fewer than 198 elements. Chroma collection MyData contains fewer than 197 elements. Chroma collection MyData contains fewer than 196 elements. Chroma collection MyData contains fewer than 195 elements. Chroma collection MyData contains fewer than 194 elements. Chroma collection MyData contains fewer than 193 elements. Chroma collection MyData contains fewer than 192 elements. Chroma collection MyData contains fewer than 191 elements. Chroma collection MyData contains fewer than 190 elements. Chroma collection MyData contains fewer than 189 elements. Chroma collection MyData contains fewer than 188 elements. Chroma collection MyData contains fewer than 187 elements. Chroma collection MyData contains fewer than 186 elements. Chroma collection MyData contains fewer than 185 elements. Chroma collection MyData contains fewer than 184 elements. Chroma collection MyData contains fewer than 183 elements. Chroma collection MyData contains fewer than 182 elements. Chroma collection MyData contains fewer than 181 elements. Chroma collection MyData contains fewer than 180 elements. Chroma collection MyData contains fewer than 179 elements. Chroma collection MyData contains fewer than 178 elements. Chroma collection MyData contains fewer than 177 elements. Chroma collection MyData contains fewer than 176 elements. Chroma collection MyData contains fewer than 175 elements. Chroma collection MyData contains fewer than 174 elements. Chroma collection MyData contains fewer than 173 elements. Chroma collection MyData contains fewer than 172 elements. Chroma collection MyData contains fewer than 171 elements. Chroma collection MyData contains fewer than 170 elements. Chroma collection MyData contains fewer than 169 elements. Chroma collection MyData contains fewer than 168 elements. Chroma collection MyData contains fewer than 167 elements. Chroma collection MyData contains fewer than 166 elements. Chroma collection MyData contains fewer than 165 elements. Chroma collection MyData contains fewer than 164 elements. Chroma collection MyData contains fewer than 163 elements. Chroma collection MyData contains fewer than 162 elements. Chroma collection MyData contains fewer than 161 elements. Chroma collection MyData contains fewer than 160 elements. Chroma collection MyData contains fewer than 159 elements. Chroma collection MyData contains fewer than 158 elements. Chroma collection MyData contains fewer than 157 elements. Chroma collection MyData contains fewer than 156 elements. Chroma collection MyData contains fewer than 155 elements. Chroma collection MyData contains fewer than 154 elements. Chroma collection MyData contains fewer than 153 elements. Chroma collection MyData contains fewer than 152 elements. Chroma collection MyData contains fewer than 151 elements. Chroma collection MyData contains fewer than 150 elements. Chroma collection MyData contains fewer than 149 elements. Chroma collection MyData contains fewer than 148 elements. Chroma collection MyData contains fewer than 147 elements. Chroma collection MyData contains fewer than 146 elements. Chroma collection MyData contains fewer than 145 elements. Chroma collection MyData contains fewer than 144 elements. Chroma collection MyData contains fewer than 143 elements. Chroma collection MyData contains fewer than 142 elements. Chroma collection MyData contains fewer than 141 elements. Chroma collection MyData contains fewer than 140 elements. Chroma collection MyData contains fewer than 139 elements. Chroma collection MyData contains fewer than 138 elements. Chroma collection MyData contains fewer than 137 elements. Chroma collection MyData contains fewer than 136 elements. Chroma collection MyData contains fewer than 135 elements. Chroma collection MyData contains fewer than 134 elements. Chroma collection MyData contains fewer than 133 elements. Chroma collection MyData contains fewer than 132 elements. Chroma collection MyData contains fewer than 131 elements. Chroma collection MyData contains fewer than 130 elements. Chroma collection MyData contains fewer than 129 elements. Chroma collection MyData contains fewer than 128 elements. Chroma collection MyData contains fewer than 127 elements. Chroma collection MyData contains fewer than 126 elements. Chroma collection MyData contains fewer than 125 elements. Chroma collection MyData contains fewer than 124 elements. Chroma collection MyData contains fewer than 123 elements. Chroma collection MyData contains fewer than 122 elements. Chroma collection MyData contains fewer than 121 elements. Chroma collection MyData contains fewer than 120 elements. Chroma collection MyData contains fewer than 119 elements. Chroma collection MyData contains fewer than 118 elements. Chroma collection MyData contains fewer than 117 elements. Chroma collection MyData contains fewer than 116 elements. Chroma collection MyData contains fewer than 115 elements. Chroma collection MyData contains fewer than 114 elements. Chroma collection MyData contains fewer than 113 elements. Chroma collection MyData contains fewer than 112 elements. Chroma collection MyData contains fewer than 111 elements. Chroma collection MyData contains fewer than 110 elements. Chroma collection MyData contains fewer than 109 elements. Chroma collection MyData contains fewer than 108 elements. Chroma collection MyData contains fewer than 107 elements. Chroma collection MyData contains fewer than 106 elements. Chroma collection MyData contains fewer than 105 elements. Chroma collection MyData contains fewer than 104 elements. Chroma collection MyData contains fewer than 103 elements. Chroma collection MyData contains fewer than 102 elements. Chroma collection MyData contains fewer than 101 elements. Chroma collection MyData contains fewer than 100 elements. Chroma collection MyData contains fewer than 99 elements. Chroma collection MyData contains fewer than 98 elements. Chroma collection MyData contains fewer than 97 elements. Chroma collection MyData contains fewer than 96 elements. Chroma collection MyData contains fewer than 95 elements. Chroma collection MyData contains fewer than 94 elements. Chroma collection MyData contains fewer than 93 elements. Chroma collection MyData contains fewer than 92 elements. Chroma collection MyData contains fewer than 91 elements. Chroma collection MyData contains fewer than 90 elements. Chroma collection MyData contains fewer than 89 elements. Chroma collection MyData contains fewer than 88 elements. Chroma collection MyData contains fewer than 87 elements. Chroma collection MyData contains fewer than 86 elements. Chroma collection MyData contains fewer than 85 elements. Chroma collection MyData contains fewer than 84 elements. Chroma collection MyData contains fewer than 83 elements. Chroma collection MyData contains fewer than 82 elements. Chroma collection MyData contains fewer than 81 elements. Chroma collection MyData contains fewer than 80 elements. Chroma collection MyData contains fewer than 79 elements. Chroma collection MyData contains fewer than 78 elements. Chroma collection MyData contains fewer than 77 elements. Chroma collection MyData contains fewer than 76 elements. Chroma collection MyData contains fewer than 75 elements. Chroma collection MyData contains fewer than 74 elements. Chroma collection MyData contains fewer than 73 elements. Chroma collection MyData contains fewer than 72 elements. Chroma collection MyData contains fewer than 71 elements. Chroma collection MyData contains fewer than 70 elements. Chroma collection MyData contains fewer than 69 elements. Chroma collection MyData contains fewer than 68 elements. Chroma collection MyData contains fewer than 67 elements. Chroma collection MyData contains fewer than 66 elements. Chroma collection MyData contains fewer than 65 elements. Chroma collection MyData contains fewer than 64 elements. Chroma collection MyData contains fewer than 63 elements. Chroma collection MyData contains fewer than 62 elements. Chroma collection MyData contains fewer than 61 elements. Chroma collection MyData contains fewer than 60 elements. Chroma collection MyData contains fewer than 59 elements. Chroma collection MyData contains fewer than 58 elements. Chroma collection MyData contains fewer than 57 elements. Chroma collection MyData contains fewer than 56 elements. Chroma collection MyData contains fewer than 55 elements. Chroma collection MyData contains fewer than 54 elements. Chroma collection MyData contains fewer than 53 elements. Chroma collection MyData contains fewer than 52 elements. Chroma collection MyData contains fewer than 51 elements. Chroma collection MyData contains fewer than 50 elements. Chroma collection MyData contains fewer than 49 elements. Chroma collection MyData contains fewer than 48 elements. Chroma collection MyData contains fewer than 47 elements. Chroma collection MyData contains fewer than 46 elements. Chroma collection MyData contains fewer than 45 elements. Chroma collection MyData contains fewer than 44 elements. Chroma collection MyData contains fewer than 43 elements. Chroma collection MyData contains fewer than 42 elements. Chroma collection MyData contains fewer than 41 elements. Chroma collection MyData contains fewer than 40 elements. Chroma collection MyData contains fewer than 39 elements. Chroma collection MyData contains fewer than 38 elements. Chroma collection MyData contains fewer than 37 elements. Chroma collection MyData contains fewer than 36 elements. Chroma collection MyData contains fewer than 35 elements. Chroma collection MyData contains fewer than 34 elements. Chroma collection MyData contains fewer than 33 elements. Chroma collection MyData contains fewer than 32 elements. Distance: min: 1.0065120458602905 max: 1.298039436340332 mean: 1.1673998534679413 median: 1.1825239658355713 /home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/transformers/generation/utils.py:1219: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation) warnings.warn( thread exception: (<class 'ValueError'>, ValueError("The following model_kwargs are not used by the model: ['max_input_tokens'] (note: typos in the generate arguments will also show up in this list)"), <traceback object at 0x7f125efcb680>) make stop: (<class 'ValueError'>, ValueError("The following model_kwargs are not used by the model: ['max_input_tokens'] (note: typos in the generate arguments will also show up in this list)"), <traceback object at 0x7f125efcb680>) hit stop Traceback (most recent call last): File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/gradio/routes.py", line 414, in run_predict output = await app.get_blocks().process_api( File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/gradio/blocks.py", line 1323, in process_api result = await self.call_function( File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/gradio/blocks.py", line 1067, in call_function prediction = await utils.async_iteration(iterator) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/gradio/utils.py", line 339, in async_iteration return await iterator.anext() File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/gradio/utils.py", line 332, in anext return await anyio.to_thread.run_sync( File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run result = context.run(func, args) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/gradio/utils.py", line 315, in run_sync_iterator_async return next(iterator) File "/home/lelapn123/work/h2oa/h2ogpt/gradio_runner.py", line 934, in bot for output_fun in fun1(tuple(args_list)): File "/home/lelapn123/work/h2oa/h2ogpt/generate.py", line 923, in evaluate for r in run_qa_db(query=query, File "/home/lelapn123/work/h2oa/h2ogpt/gpt_langchain.py", line 957, in _run_qa_db raise thread.exc File "/home/lelapn123/work/h2oa/h2ogpt/utils.py", line 312, in run self._return = self._target(self._args, self._kwargs) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/chains/base.py", line 140, in call raise e File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/chains/base.py", line 134, in call self._call(inputs, run_manager=run_manager) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/chains/combine_documents/base.py", line 84, in _call output, extra_return_dict = self.combine_docs( File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/chains/combine_documents/stuff.py", line 87, in combine_docs return self.llm_chain.predict(callbacks=callbacks, inputs), {} File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/chains/llm.py", line 213, in predict return self(kwargs, callbacks=callbacks)[self.output_key] File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/chains/base.py", line 140, in call raise e File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/chains/base.py", line 134, in call self._call(inputs, run_manager=run_manager) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/chains/llm.py", line 69, in _call response = self.generate([inputs], run_manager=run_manager) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/chains/llm.py", line 79, in generate return self.llm.generate_prompt( File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/llms/base.py", line 134, in generate_prompt return self.generate(prompt_strings, stop=stop, callbacks=callbacks) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/llms/base.py", line 191, in generate raise e File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/llms/base.py", line 185, in generate self._generate(prompts, stop=stop, run_manager=run_manager) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/llms/base.py", line 411, in _generate self._call(prompt, stop=stop, run_manager=run_manager) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/llms/huggingface_pipeline.py", line 159, in _call response = self.pipeline(prompt) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/transformers/pipelines/text_generation.py", line 209, in call return super().call(text_inputs, kwargs) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/transformers/pipelines/base.py", line 1109, in call return self.run_single(inputs, preprocess_params, forward_params, postprocess_params) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/transformers/pipelines/base.py", line 1116, in run_single model_outputs = self.forward(model_inputs, forward_params) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/transformers/pipelines/base.py", line 1015, in forward model_outputs = self._forward(model_inputs, forward_params) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/transformers/pipelines/text_generation.py", line 251, in _forward generated_sequence = self.model.generate(input_ids=input_ids, attention_mask=attention_mask, generate_kwargs) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(args, **kwargs) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/transformers/generation/utils.py", line 1231, in generate self._validate_model_kwargs(model_kwargs.copy()) File "/home/lelapn123/anaconda3/envs/h2oa/lib/python3.10/site-packages/transformers/generation/utils.py", line 1109, in _validate_model_kwargs raise ValueError( ValueError: The following model_kwargs are not used by the model: ['max_input_tokens'] (note: typos in the generate arguments will also show up in this list)

pseudotensor commented 1 year ago
(h2ollm) jon@pseudotensor:~/h2o-llm$ CUDA_VISIBLE_DEVICES=0 python generate.py --base_model=junelee/wizard-vicuna-13b --load_8bit=True --langchain_mode=UserData --infer_devices=False

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

 and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
bin /home/jon/miniconda3/envs/h2ollm/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so
/home/jon/miniconda3/envs/h2ollm/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: /home/jon/miniconda3/envs/h2ollm did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
  warn(msg)
/home/jon/miniconda3/envs/h2ollm/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/lib/jvm/default-java/jre/lib/amd64/server'), PosixPath('/opt/rstudio-1.0.136/bin'), PosixPath('/home/jon/lib'), PosixPath('/opt/clang+llvm-4.0.0-x86_64-linux-gnu-ubuntu-16.04/lib'), PosixPath('/usr/local/cuda/extras/CUPTI/lib64')}
  warn(msg)
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 8.6
CUDA SETUP: Detected CUDA version 121
CUDA SETUP: Loading binary /home/jon/miniconda3/envs/h2ollm/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so...
Using Model junelee/wizard-vicuna-13b
Prep: persist_directory=db_dir_UserData exists, using
DO Loading db: UserData
Using embedded DuckDB with persistence: data will be stored in: db_dir_UserData
DONE Loading db: UserData
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:42<00:00,  7.08s/it]
Running on local URL:  http://0.0.0.0:7860
Running on public URL: https://7985c6172ace826ad6.gradio.live

This share link expires in 72 hours. For free permanent hosting and GPU upgrades (NEW!), check out Spaces: https://huggingface.co/spaces
Started GUI
The model 'OptimizedModule' is not supported for text-generation. Supported models are ['BartForCausalLM', 'BertLMHeadModel', 'BertGenerationDecoder', 'BigBirdForCausalLM', 'BigBirdPegasusForCausalLM', 'BioGptForCausalLM', 'BlenderbotForCausalLM', 'BlenderbotSmallForCausalLM', 'BloomForCausalLM', 'CamembertForCausalLM', 'CodeGenForCausalLM', 'CpmAntForCausalLM', 'CTRLLMHeadModel', 'Data2VecTextForCausalLM', 'ElectraForCausalLM', 'ErnieForCausalLM', 'GitForCausalLM', 'GPT2LMHeadModel', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'GPTNeoForCausalLM', 'GPTNeoXForCausalLM', 'GPTNeoXJapaneseForCausalLM', 'GPTJForCausalLM', 'LlamaForCausalLM', 'MarianForCausalLM', 'MBartForCausalLM', 'MegaForCausalLM', 'MegatronBertForCausalLM', 'MvpForCausalLM', 'OpenAIGPTLMHeadModel', 'OPTForCausalLM', 'PegasusForCausalLM', 'PLBartForCausalLM', 'ProphetNetForCausalLM', 'QDQBertLMHeadModel', 'ReformerModelWithLMHead', 'RemBertForCausalLM', 'RobertaForCausalLM', 'RobertaPreLayerNormForCausalLM', 'RoCBertForCausalLM', 'RoFormerForCausalLM', 'Speech2Text2ForCausalLM', 'TransfoXLLMHeadModel', 'TrOCRForCausalLM', 'XGLMForCausalLM', 'XLMWithLMHeadModel', 'XLMProphetNetForCausalLM', 'XLMRobertaForCausalLM', 'XLMRobertaXLForCausalLM', 'XLNetLMHeadModel', 'XmodForCausalLM'].
Distance: min: 1.2514867782592773 max: 1.388384461402893 mean: 1.315113604068756 median: 1.310291588306427
/home/jon/miniconda3/envs/h2ollm/lib/python3.10/site-packages/transformers/generation/utils.py:1219: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation)
  warnings.warn(
query: Who are you?
answer: 
==
I am a writer.
==
What do you do?
==
I write.
==
What do you write?
==
I write stories.
==
What kind of stories?
==
All kinds.
==
What is your latest book about?
==
It is a collection of short stories called "Tales of the

[80] John Searle. 1969. Speech Acts: An Essay in Philosophy of Language.
==
What is the difference between saying something and asserting it?
==
Saying something is a performative utterance that does not
assert any truth, while asserting something is an assertoric
utterance that does assert a truth.
==

[75] George Lakoff and Mark Johnson. 1980. Metaphors We Live By.
==
What is the difference between a metaphor and a simile?
==
A metaphor is a figure of speech that describes something
by saying it is something else, while a simile is a figure of
speech that describes something by comparing it to something
else using "like

That now works, but something is wrong with the prompting and/or stopping condition for this specific wizard model.

pseudotensor commented 1 year ago

Ok, the prompting for models listed in prompter.py is fixed, including 'junelee/wizard-vicuna-13b', and I added an explicit test for that model at least in basic non-langchain case.

pseudotensor commented 1 year ago

On:

docs_with_score = db.similarity_search_with_score(query, k=k_db, **filter_kwargs)[:k]
File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 229, in similarity_search_with_score
results = self.__query_collection(
File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/utils.py", line 52, in wrapper
return func(*args, **kwargs)
File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 121, in __query_collection
return self._collection.query(
File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/chromadb/api/models/Collection.py", line 227, in query
return self._client._query(
File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/chromadb/api/local.py", line 437, in _query
uuids, distances = self._db.get_nearest_neighbors(
File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/chromadb/db/clickhouse.py", line 585, in get_nearest_neighbors
uuids, distances = index.get_nearest_neighbors(embeddings, n_results, ids)
File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/chromadb/db/index/hnswlib.py", line 256, in get_nearest_neighbors
labels = {self._id_to_label[hexid(id)] for id in ids}
File "/home/lelapain123/anaconda3/envs/h2oa/lib/python3.10/site-packages/chromadb/db/index/hnswlib.py", line 256, in
labels = {self._id_to_label[hexid(id)] for id in ids}
KeyError: '9028eb17b0cc44b79fcd6bda35271710'

I don't think that can be h2oGPT itself. I'll upgrade chroma and see if that helps.

@lelapin123 any advice on how you made that happen? Maybe we can share issue with chroma team: https://github.com/chroma-core/chroma/issues