microsoft / LLaVA-Med

Large Language-and-Vision Assistant for Biomedicine, built towards multimodal GPT-4 level capabilities.
Other
1.29k stars 148 forks source link

Error while Serving for testing purpose #19

Open atultiwari opened 7 months ago

atultiwari commented 7 months ago

Hi,

I wanted to try it out for testing purpose. For that I downloaded both the llava_med_in_text_60k_delta.zip and llama weights. but when I tried to run following command -

python3 -m llava.model.apply_delta \
    --base llama \
    --target output \
    --delta weights/llava_med_in_text_60k_delta

I got the following error - llama does not appear to have a config.json file

llava-med

In this command

Kindly let me know where I am making the mistake. Thank you Regards, Dr. Atul

HaotianHuang commented 7 months ago

Need to convert llama weights to hugging face format. Give that a go

atultiwari commented 7 months ago

Need to convert llama weights to hugging face format. Give that a go

Hi, Thank you for letting me know. So I need to run following command, after downloading the above python file, am I correct?

python convert_llama_weights_to_hf.py \
    --input_dir /path/to/downloaded/llama/weights --model_size 7B --output_dir /output/path

and is only 7B model would be enough for this, because otherwise to download all model weights takes a lot of time. Is there anyplace from where I can directly get the converted model weight

thank you for your help

atultiwari commented 7 months ago

Need to convert llama weights to hugging face format. Give that a go

Hi, I tried with the 7B Llama weights and faced another error with the python script - NameError: name 'false' is not defined. Did you mean: 'False'?

I tested it both in google colab as well as my laptop, same error occured in the both the cases. I used the following command -

python convert_llama_weights_to_hf.py --input_dir 7B --model_size 7B --output_dir hf_output

Here -

Any solutions please. Thank you.

The complete error log is -

Traceback (most recent call last):
  File "/content/drive/MyDrive/llama/convert_llama_weights_to_hf.py", line 1, in <module>
    {"payload":{"allShortcutsEnabled":false,"fileTree":{"src/transformers/models/llama":{"items":[{"name":"__init__.py","path":"src/transformers/models/llama/__init__.py","contentType":"file"},{"name":"configuration_llama.py","path":"src/transformers/models/llama/configuration_llama.py","contentType":"file"},{"name":"convert_llama_weights_to_hf.py","path":"src/transformers/models/llama/convert_llama_weights_to_hf.py","contentType":"file"},{"name":"modeling_llama.py","path":"src/transformers/models/llama/modeling_llama.py","contentType":"file"},{"name":"tokenization_llama.py","path":"src/transformers/models/llama/tokenization_llama.py","contentType":"file"},{"name":"tokenization_llama_fast.py","path":"src/transformers/models/llama/tokenization_llama_fast.py","contentType":"file"}],"totalCount":6},"src/transformers/models":{"items":[{"name":"albert","path":"src/transformers/models/albert","contentType":"directory"},{"name":"align","path":"src/transformers/models/align","contentType":"directory"},{"name":"altclip","path":"src/transformers/models/altclip","contentType":"directory"},{"name":"audio_spectrogram_transformer","path":"src/transformers/models/audio_spectrogram_transformer","contentType":"directory"},{"name":"auto","path":"src/transformers/models/auto","contentType":"directory"},{"name":"autoformer","path":"src/transformers/models/autoformer","contentType":"directory"},{"name":"bark","path":"src/transformers/models/bark","contentType":"directory"},{"name":"bart","path":"src/transformers/models/bart","contentType":"directory"},{"name":"barthez","path":"src/transformers/models/barthez","contentType":"directory"},{"name":"bartpho","path":"src/transformers/models/bartpho","contentType":"directory"},{"name":"beit","path":"src/transformers/models/beit","contentType":"directory"},{"name":"bert","path":"src/transformers/models/bert","contentType":"directory"},{"name":"bert_generation","path":"src/transformers/models/bert_generation","contentType":"directory"},{"name":"bert_japanese","path":"src/transformers/models/bert_japanese","contentType":"directory"},{"name":"bertweet","path":"src/transformers/models/bertweet","contentType":"directory"},{"name":"big_bird","path":"src/transformers/models/big_bird","contentType":"directory"},{"name":"bigbird_pegasus","path":"src/transformers/models/bigbird_pegasus","contentType":"directory"},{"name":"biogpt","path":"src/transformers/models/biogpt","contentType":"directory"},{"name":"bit","path":"src/transformers/models/bit","contentType":"directory"},{"name":"blenderbot","path":"src/transformers/models/blenderbot","contentType":"directory"},{"name":"blenderbot_small","path":"src/transformers/models/blenderbot_small","contentType":"directory"},{"name":"blip","path":"src/transformers/models/blip","contentType":"directory"},{"name":"blip_2","path":"src/transformers/models/blip_2","contentType":"directory"},{"name":"bloom","path":"src/transformers/models/bloom","contentType":"directory"},{"name":"bridgetower","path":"src/transformers/models/bridgetower","contentType":"directory"},{"name":"bros","path":"src/transformers/models/bros","contentType":"directory"},{"name":"byt5","path":"src/transformers/models/byt5","contentType":"directory"},{"name":"camembert","path":"src/transformers/models/camembert","contentType":"directory"},{"name":"canine","path":"src/transformers/models/canine","contentType":"directory"},{"name":"chinese_clip","path":"src/transformers/models/chinese_clip","contentType":"directory"},{"name":"clap","path":"src/transformers/models/clap","contentType":"directory"},{"name":"clip","path":"src/transformers/models/clip","contentType":"directory"},{"name":"clipseg","path":"src/transformers/models/clipseg","contentType":"directory"},{"name":"clvp","path":"src/transformers/models/clvp","contentType":"directory"},{"name":"code_llama","path":"src/transformers/models/code_llama","contentType":"directory"},{"name":"codegen","path":"src/transformers/models/codegen","contentType":"directory"},{"name":"conditional_detr","path":"src/transformers/models/conditional_detr","contentType":"directory"},{"name":"convbert","path":"src/transformers/models/convbert","contentType":"directory"},{"name":"convnext","path":"src/transformers/models/convnext","contentType":"directory"},{"name":"convnextv2","path":"src/transformers/models/convnextv2","contentType":"directory"},{"name":"cpm","path":"src/transformers/models/cpm","contentType":"directory"},{"name":"cpmant","path":"src/transformers/models/cpmant","contentType":"directory"},{"name":"ctrl","path":"src/transformers/models/ctrl","contentType":"directory"},{"name":"cvt","path":"src/transformers/models/cvt","contentType":"directory"},{"name":"data2vec","path":"src/transformers/models/data2vec","contentType":"directory"},{"name":"deberta","path":"src/transformers/models/deberta","contentType":"directory"},{"name":"deberta_v2","path":"src/transformers/models/deberta_v2","contentType":"directory"},{"name":"decision_transformer","path":"src/transformers/models/decision_transformer","contentType":"directory"},{"name":"deformable_detr","path":"src/transformers/models/deformable_detr","contentType":"directory"},{"name":"deit","path":"src/transformers/models/deit","contentType":"directory"},{"name":"deprecated","path":"src/transformers/models/deprecated","contentType":"directory"},{"name":"deta","path":"src/transformers/models/deta","contentType":"directory"},{"name":"detr","path":"src/transformers/models/detr","contentType":"directory"},{"name":"dialogpt","path":"src/transformers/models/dialogpt","contentType":"directory"},{"name":"dinat","path":"src/transformers/models/dinat","contentType":"directory"},{"name":"dinov2","path":"src/transformers/models/dinov2","contentType":"directory"},{"name":"distilbert","path":"src/transformers/models/distilbert","contentType":"directory"},{"name":"dit","path":"src/transformers/models/dit","contentType":"directory"},{"name":"donut","path":"src/transformers/models/donut","contentType":"directory"},{"name":"dpr","path":"src/transformers/models/dpr","contentType":"directory"},{"name":"dpt","path":"src/transformers/models/dpt","contentType":"directory"},{"name":"efficientformer","path":"src/transformers/models/efficientformer","contentType":"directory"},{"name":"efficientnet","path":"src/transformers/models/efficientnet","contentType":"directory"},{"name":"electra","path":"src/transformers/models/electra","contentType":"directory"},{"name":"encodec","path":"src/transformers/models/encodec","contentType":"directory"},{"name":"encoder_decoder","path":"src/transformers/models/encoder_decoder","contentType":"directory"},{"name":"ernie","path":"src/transformers/models/ernie","contentType":"directory"},{"name":"ernie_m","path":"src/transformers/models/ernie_m","contentType":"directory"},{"name":"esm","path":"src/transformers/models/esm","contentType":"directory"},{"name":"falcon","path":"src/transformers/models/falcon","contentType":"directory"},{"name":"flaubert","path":"src/transformers/models/flaubert","contentType":"directory"},{"name":"flava","path":"src/transformers/models/flava","contentType":"directory"},{"name":"fnet","path":"src/transformers/models/fnet","contentType":"directory"},{"name":"focalnet","path":"src/transformers/models/focalnet","contentType":"directory"},{"name":"fsmt","path":"src/transformers/models/fsmt","contentType":"directory"},{"name":"funnel","path":"src/transformers/models/funnel","contentType":"directory"},{"name":"fuyu","path":"src/transformers/models/fuyu","contentType":"directory"},{"name":"git","path":"src/transformers/models/git","contentType":"directory"},{"name":"glpn","path":"src/transformers/models/glpn","contentType":"directory"},{"name":"gpt2","path":"src/transformers/models/gpt2","contentType":"directory"},{"name":"gpt_bigcode","path":"src/transformers/models/gpt_bigcode","contentType":"directory"},{"name":"gpt_neo","path":"src/transformers/models/gpt_neo","contentType":"directory"},{"name":"gpt_neox","path":"src/transformers/models/gpt_neox","contentType":"directory"},{"name":"gpt_neox_japanese","path":"src/transformers/models/gpt_neox_japanese","contentType":"directory"},{"name":"gpt_sw3","path":"src/transformers/models/gpt_sw3","contentType":"directory"},{"name":"gptj","path":"src/transformers/models/gptj","contentType":"directory"},{"name":"gptsan_japanese","path":"src/transformers/models/gptsan_japanese","contentType":"directory"},{"name":"graphormer","path":"src/transformers/models/graphormer","contentType":"directory"},{"name":"groupvit","path":"src/transformers/models/groupvit","contentType":"directory"},{"name":"herbert","path":"src/transformers/models/herbert","contentType":"directory"},{"name":"hubert","path":"src/transformers/models/hubert","contentType":"directory"},{"name":"ibert","path":"src/transformers/models/ibert","contentType":"directory"},{"name":"idefics","path":"src/transformers/models/idefics","contentType":"directory"},{"name":"imagegpt","path":"src/transformers/models/imagegpt","contentType":"directory"},{"name":"informer","path":"src/transformers/models/informer","contentType":"directory"},{"name":"instructblip","path":"src/transformers/models/instructblip","contentType":"directory"},{"name":"jukebox","path":"src/transformers/models/jukebox","contentType":"directory"},{"name":"kosmos2","path":"src/transformers/models/kosmos2","contentType":"directory"},{"name":"layoutlm","path":"src/transformers/models/layoutlm","contentType":"directory"},{"name":"layoutlmv2","path":"src/transformers/models/layoutlmv2","contentType":"directory"},{"name":"layoutlmv3","path":"src/transformers/models/layoutlmv3","contentType":"directory"},{"name":"layoutxlm","path":"src/transformers/models/layoutxlm","contentType":"directory"},{"name":"led","path":"src/transformers/models/led","contentType":"directory"},{"name":"levit","path":"src/transformers/models/levit","contentType":"directory"},{"name":"lilt","path":"src/transformers/models/lilt","contentType":"directory"},{"name":"llama","path":"src/transformers/models/llama","contentType":"directory"},{"name":"longformer","path":"src/transformers/models/longformer","contentType":"directory"},{"name":"longt5","path":"src/transformers/models/longt5","contentType":"directory"},{"name":"luke","path":"src/transformers/models/luke","contentType":"directory"},{"name":"lxmert","path":"src/transformers/models/lxmert","contentType":"directory"},{"name":"m2m_100","path":"src/transformers/models/m2m_100","contentType":"directory"},{"name":"marian","path":"src/transformers/models/marian","contentType":"directory"},{"name":"markuplm","path":"src/transformers/models/markuplm","contentType":"directory"},{"name":"mask2former","path":"src/transformers/models/mask2former","contentType":"directory"},{"name":"maskformer","path":"src/transformers/models/maskformer","contentType":"directory"},{"name":"mbart","path":"src/transformers/models/mbart","contentType":"directory"},{"name":"mbart50","path":"src/transformers/models/mbart50","contentType":"directory"},{"name":"mega","path":"src/transformers/models/mega","contentType":"directory"},{"name":"megatron_bert","path":"src/transformers/models/megatron_bert","contentType":"directory"},{"name":"megatron_gpt2","path":"src/transformers/models/megatron_gpt2","contentType":"directory"},{"name":"mgp_str","path":"src/transformers/models/mgp_str","contentType":"directory"},{"name":"mistral","path":"src/transformers/models/mistral","contentType":"directory"},{"name":"mluke","path":"src/transformers/models/mluke","contentType":"directory"},{"name":"mobilebert","path":"src/transformers/models/mobilebert","contentType":"directory"},{"name":"mobilenet_v1","path":"src/transformers/models/mobilenet_v1","contentType":"directory"},{"name":"mobilenet_v2","path":"src/transformers/models/mobilenet_v2","contentType":"directory"},{"name":"mobilevit","path":"src/transformers/models/mobilevit","contentType":"directory"},{"name":"mobilevitv2","path":"src/transformers/models/mobilevitv2","contentType":"directory"},{"name":"mpnet","path":"src/transformers/models/mpnet","contentType":"directory"},{"name":"mpt","path":"src/transformers/models/mpt","contentType":"directory"},{"name":"mra","path":"src/transformers/models/mra","contentType":"directory"},{"name":"mt5","path":"src/transformers/models/mt5","contentType":"directory"},{"name":"musicgen","path":"src/transformers/models/musicgen","contentType":"directory"},{"name":"mvp","path":"src/transformers/models/mvp","contentType":"directory"},{"name":"nat","path":"src/transformers/models/nat","contentType":"directory"},{"name":"nezha","path":"src/transformers/models/nezha","contentType":"directory"},{"name":"nllb","path":"src/transformers/models/nllb","contentType":"directory"},{"name":"nllb_moe","path":"src/transformers/models/nllb_moe","contentType":"directory"},{"name":"nougat","path":"src/transformers/models/nougat","contentType":"directory"},{"name":"nystromformer","path":"src/transformers/models/nystromformer","contentType":"directory"},{"name":"oneformer","path":"src/transformers/models/oneformer","contentType":"directory"},{"name":"openai","path":"src/transformers/models/openai","contentType":"directory"},{"name":"opt","path":"src/transformers/models/opt","contentType":"directory"},{"name":"owlv2","path":"src/transformers/models/owlv2","contentType":"directory"},{"name":"owlvit","path":"src/transformers/models/owlvit","contentType":"directory"},{"name":"pegasus","path":"src/transformers/models/pegasus","contentType":"directory"},{"name":"pegasus_x","path":"src/transformers/models/pegasus_x","contentType":"directory"},{"name":"perceiver","path":"src/transformers/models/perceiver","contentType":"directory"},{"name":"persimmon","path":"src/transformers/models/persimmon","contentType":"directory"},{"name":"phi","path":"src/transformers/models/phi","contentType":"directory"},{"name":"phobert","path":"src/transformers/models/phobert","contentType":"directory"},{"name":"pix2struct","path":"src/transformers/models/pix2struct","contentType":"directory"},{"name":"plbart","path":"src/transformers/models/plbart","contentType":"directory"},{"name":"poolformer","path":"src/transformers/models/poolformer","contentType":"directory"},{"name":"pop2piano","path":"src/transformers/models/pop2piano","contentType":"directory"},{"name":"prophetnet","path":"src/transformers/models/prophetnet","contentType":"directory"},{"name":"pvt","path":"src/transformers/models/pvt","contentType":"directory"},{"name":"qdqbert","path":"src/transformers/models/qdqbert","contentType":"directory"},{"name":"rag","path":"src/transformers/models/rag","contentType":"directory"},{"name":"realm","path":"src/transformers/models/realm","contentType":"directory"},{"name":"reformer","path":"src/transformers/models/reformer","contentType":"directory"},{"name":"regnet","path":"src/transformers/models/regnet","contentType":"directory"},{"name":"rembert","path":"src/transformers/models/rembert","contentType":"directory"},{"name":"resnet","path":"src/transformers/models/resnet","contentType":"directory"},{"name":"roberta","path":"src/transformers/models/roberta","contentType":"directory"},{"name":"roberta_prelayernorm","path":"src/transformers/models/roberta_prelayernorm","contentType":"directory"},{"name":"roc_bert","path":"src/transformers/models/roc_bert","contentType":"directory"},{"name":"roformer","path":"src/transformers/models/roformer","contentType":"directory"},{"name":"rwkv","path":"src/transformers/models/rwkv","contentType":"directory"},{"name":"sam","path":"src/transformers/models/sam","contentType":"directory"},{"name":"seamless_m4t","path":"src/transformers/models/seamless_m4t","contentType":"directory"},{"name":"segformer","path":"src/transformers/models/segformer","contentType":"directory"},{"name":"sew","path":"src/transformers/models/sew","contentType":"directory"},{"name":"sew_d","path":"src/transformers/models/sew_d","contentType":"directory"},{"name":"speech_encoder_decoder","path":"src/transformers/models/speech_encoder_decoder","contentType":"directory"},{"name":"speech_to_text","path":"src/transformers/models/speech_to_text","contentType":"directory"},{"name":"speech_to_text_2","path":"src/transformers/models/speech_to_text_2","contentType":"directory"},{"name":"speecht5","path":"src/transformers/models/speecht5","contentType":"directory"},{"name":"splinter","path":"src/transformers/models/splinter","contentType":"directory"},{"name":"squeezebert","path":"src/transformers/models/squeezebert","contentType":"directory"},{"name":"swiftformer","path":"src/transformers/models/swiftformer","contentType":"directory"},{"name":"swin","path":"src/transformers/models/swin","contentType":"directory"},{"name":"swin2sr","path":"src/transformers/models/swin2sr","contentType":"directory"},{"name":"swinv2","path":"src/transformers/models/swinv2","contentType":"directory"},{"name":"switch_transformers","path":"src/transformers/models/switch_transformers","contentType":"directory"},{"name":"t5","path":"src/transformers/models/t5","contentType":"directory"},{"name":"table_transformer","path":"src/transformers/models/table_transformer","contentType":"directory"},{"name":"tapas","path":"src/transformers/models/tapas","contentType":"directory"},{"name":"time_series_transformer","path":"src/transformers/models/time_series_transformer","contentType":"directory"},{"name":"timesformer","path":"src/transformers/models/timesformer","contentType":"directory"},{"name":"timm_backbone","path":"src/transformers/models/timm_backbone","contentType":"directory"},{"name":"transfo_xl","path":"src/transformers/models/transfo_xl","contentType":"directory"},{"name":"trocr","path":"src/transformers/models/trocr","contentType":"directory"},{"name":"tvlt","path":"src/transformers/models/tvlt","contentType":"directory"},{"name":"umt5","path":"src/transformers/models/umt5","contentType":"directory"},{"name":"unispeech","path":"src/transformers/models/unispeech","contentType":"directory"},{"name":"unispeech_sat","path":"src/transformers/models/unispeech_sat","contentType":"directory"},{"name":"upernet","path":"src/transformers/models/upernet","contentType":"directory"},{"name":"videomae","path":"src/transformers/models/videomae","contentType":"directory"},{"name":"vilt","path":"src/transformers/models/vilt","contentType":"directory"},{"name":"vision_encoder_decoder","path":"src/transformers/models/vision_encoder_decoder","contentType":"directory"},{"name":"vision_text_dual_encoder","path":"src/transformers/models/vision_text_dual_encoder","contentType":"directory"},{"name":"visual_bert","path":"src/transformers/models/visual_bert","contentType":"directory"},{"name":"vit","path":"src/transformers/models/vit","contentType":"directory"},{"name":"vit_hybrid","path":"src/transformers/models/vit_hybrid","contentType":"directory"},{"name":"vit_mae","path":"src/transformers/models/vit_mae","contentType":"directory"},{"name":"vit_msn","path":"src/transformers/models/vit_msn","contentType":"directory"},{"name":"vitdet","path":"src/transformers/models/vitdet","contentType":"directory"},{"name":"vitmatte","path":"src/transformers/models/vitmatte","contentType":"directory"},{"name":"vits","path":"src/transformers/models/vits","contentType":"directory"},{"name":"vivit","path":"src/transformers/models/vivit","contentType":"directory"},{"name":"wav2vec2","path":"src/transformers/models/wav2vec2","contentType":"directory"},{"name":"wav2vec2_conformer","path":"src/transformers/models/wav2vec2_conformer","contentType":"directory"},{"name":"wav2vec2_phoneme","path":"src/transformers/models/wav2vec2_phoneme","contentType":"directory"},{"name":"wav2vec2_with_lm","path":"src/transformers/models/wav2vec2_with_lm","contentType":"directory"},{"name":"wavlm","path":"src/transformers/models/wavlm","contentType":"directory"},{"name":"whisper","path":"src/transformers/models/whisper","contentType":"directory"},{"name":"x_clip","path":"src/transformers/models/x_clip","contentType":"directory"},{"name":"xglm","path":"src/transformers/models/xglm","contentType":"directory"},{"name":"xlm","path":"src/transformers/models/xlm","contentType":"directory"},{"name":"xlm_prophetnet","path":"src/transformers/models/xlm_prophetnet","contentType":"directory"},{"name":"xlm_roberta","path":"src/transformers/models/xlm_roberta","contentType":"directory"},{"name":"xlm_roberta_xl","path":"src/transformers/models/xlm_roberta_xl","contentType":"directory"},{"name":"xlnet","path":"src/transformers/models/xlnet","contentType":"directory"},{"name":"xmod","path":"src/transformers/models/xmod","contentType":"directory"},{"name":"yolos","path":"src/transformers/models/yolos","contentType":"directory"},{"name":"yoso","path":"src/transformers/models/yoso","contentType":"directory"},{"name":"__init__.py","path":"src/transformers/models/__init__.py","contentType":"file"}],"totalCount":228},"src/transformers":{"items":[{"name":"benchmark","path":"src/transformers/benchmark","contentType":"directory"},{"name":"commands","path":"src/transformers/commands","contentType":"directory"},{"name":"data","path":"src/transformers/data","contentType":"directory"},{"name":"generation","path":"src/transformers/generation","contentType":"directory"},{"name":"integrations","path":"src/transformers/integrations","contentType":"directory"},{"name":"kernels","path":"src/transformers/kernels","contentType":"directory"},{"name":"models","path":"src/transformers/models","contentType":"directory"},{"name":"onnx","path":"src/transformers/onnx","contentType":"directory"},{"name":"pipelines","path":"src/transformers/pipelines","contentType":"directory"},{"name":"sagemaker","path":"src/transformers/sagemaker","contentType":"directory"},{"name":"tools","path":"src/transformers/tools","contentType":"directory"},{"name":"utils","path":"src/transformers/utils","contentType":"directory"},{"name":"__init__.py","path":"src/transformers/__init__.py","contentType":"file"},{"name":"activations.py","path":"src/transformers/activations.py","contentType":"file"},{"name":"activations_tf.py","path":"src/transformers/activations_tf.py","contentType":"file"},{"name":"audio_utils.py","path":"src/transformers/audio_utils.py","contentType":"file"},{"name":"configuration_utils.py","path":"src/transformers/configuration_utils.py","contentType":"file"},{"name":"convert_graph_to_onnx.py","path":"src/transformers/convert_graph_to_onnx.py","contentType":"file"},{"name":"convert_pytorch_checkpoint_to_tf2.py","path":"src/transformers/convert_pytorch_checkpoint_to_tf2.py","contentType":"file"},{"name":"convert_slow_tokenizer.py","path":"src/transformers/convert_slow_tokenizer.py","contentType":"file"},{"name":"convert_slow_tokenizers_checkpoints_to_fast.py","path":"src/transformers/convert_slow_tokenizers_checkpoints_to_fast.py","contentType":"file"},{"name":"convert_tf_hub_seq_to_seq_bert_to_pytorch.py","path":"src/transformers/convert_tf_hub_seq_to_seq_bert_to_pytorch.py","contentType":"file"},{"name":"debug_utils.py","path":"src/transformers/debug_utils.py","contentType":"file"},{"name":"deepspeed.py","path":"src/transformers/deepspeed.py","contentType":"file"},{"name":"dependency_versions_check.py","path":"src/transformers/dependency_versions_check.py","contentType":"file"},{"name":"dependency_versions_table.py","path":"src/transformers/dependency_versions_table.py","contentType":"file"},{"name":"dynamic_module_utils.py","path":"src/transformers/dynamic_module_utils.py","contentType":"file"},{"name":"feature_extraction_sequence_utils.py","path":"src/transformers/feature_extraction_sequence_utils.py","contentType":"file"},{"name":"feature_extraction_utils.py","path":"src/transformers/feature_extraction_utils.py","contentType":"file"},{"name":"file_utils.py","path":"src/transformers/file_utils.py","contentType":"file"},{"name":"generation_flax_utils.py","path":"src/transformers/generation_flax_utils.py","contentType":"file"},{"name":"generation_tf_utils.py","path":"src/transformers/generation_tf_utils.py","contentType":"file"},{"name":"generation_utils.py","path":"src/transformers/generation_utils.py","contentType":"file"},{"name":"hf_argparser.py","path":"src/transformers/hf_argparser.py","contentType":"file"},{"name":"hyperparameter_search.py","path":"src/transformers/hyperparameter_search.py","contentType":"file"},{"name":"image_processing_utils.py","path":"src/transformers/image_processing_utils.py","contentType":"file"},{"name":"image_transforms.py","path":"src/transformers/image_transforms.py","contentType":"file"},{"name":"image_utils.py","path":"src/transformers/image_utils.py","contentType":"file"},{"name":"keras_callbacks.py","path":"src/transformers/keras_callbacks.py","contentType":"file"},{"name":"modelcard.py","path":"src/transformers/modelcard.py","contentType":"file"},{"name":"modeling_attn_mask_utils.py","path":"src/transformers/modeling_attn_mask_utils.py","contentType":"file"},{"name":"modeling_flax_outputs.py","path":"src/transformers/modeling_flax_outputs.py","contentType":"file"},{"name":"modeling_flax_pytorch_utils.py","path":"src/transformers/modeling_flax_pytorch_utils.py","contentType":"file"},{"name":"modeling_flax_utils.py","path":"src/transformers/modeling_flax_utils.py","contentType":"file"},{"name":"modeling_outputs.py","path":"src/transformers/modeling_outputs.py","contentType":"file"},{"name":"modeling_tf_outputs.py","path":"src/transformers/modeling_tf_outputs.py","contentType":"file"},{"name":"modeling_tf_pytorch_utils.py","path":"src/transformers/modeling_tf_pytorch_utils.py","contentType":"file"},{"name":"modeling_tf_utils.py","path":"src/transformers/modeling_tf_utils.py","contentType":"file"},{"name":"modeling_utils.py","path":"src/transformers/modeling_utils.py","contentType":"file"},{"name":"optimization.py","path":"src/transformers/optimization.py","contentType":"file"},{"name":"optimization_tf.py","path":"src/transformers/optimization_tf.py","contentType":"file"},{"name":"processing_utils.py","path":"src/transformers/processing_utils.py","contentType":"file"},{"name":"pytorch_utils.py","path":"src/transformers/pytorch_utils.py","contentType":"file"},{"name":"testing_utils.py","path":"src/transformers/testing_utils.py","contentType":"file"},{"name":"tf_utils.py","path":"src/transformers/tf_utils.py","contentType":"file"},{"name":"time_series_utils.py","path":"src/transformers/time_series_utils.py","contentType":"file"},{"name":"tokenization_utils.py","path":"src/transformers/tokenization_utils.py","contentType":"file"},{"name":"tokenization_utils_base.py","path":"src/transformers/tokenization_utils_base.py","contentType":"file"},{"name":"tokenization_utils_fast.py","path":"src/transformers/tokenization_utils_fast.py","contentType":"file"},{"name":"trainer.py","path":"src/transformers/trainer.py","contentType":"file"},{"name":"trainer_callback.py","path":"src/transformers/trainer_callback.py","contentType":"file"},{"name":"trainer_pt_utils.py","path":"src/transformers/trainer_pt_utils.py","contentType":"file"},{"name":"trainer_seq2seq.py","path":"src/transformers/trainer_seq2seq.py","contentType":"file"},{"name":"trainer_tf.py","path":"src/transformers/trainer_tf.py","contentType":"file"},{"name":"trainer_utils.py","path":"src/transformers/trainer_utils.py","contentType":"file"},{"name":"training_args.py","path":"src/transformers/training_args.py","contentType":"file"},{"name":"training_args_seq2seq.py","path":"src/transformers/training_args_seq2seq.py","contentType":"file"},{"name":"training_args_tf.py","path":"src/transformers/training_args_tf.py","contentType":"file"}],"totalCount":68},"src":{"items":[{"name":"transformers","path":"src/transformers","contentType":"directory"}],"totalCount":1},"":{"items":[{"name":".circleci","path":".circleci","contentType":"directory"},{"name":".github","path":".github","contentType":"directory"},{"name":"docker","path":"docker","contentType":"directory"},{"name":"docs","path":"docs","contentType":"directory"},{"name":"examples","path":"examples","contentType":"directory"},{"name":"model_cards","path":"model_cards","contentType":"directory"},{"name":"notebooks","path":"notebooks","contentType":"directory"},{"name":"scripts","path":"scripts","contentType":"directory"},{"name":"src","path":"src","contentType":"directory"},{"name":"templates","path":"templates","contentType":"directory"},{"name":"tests","path":"tests","contentType":"directory"},{"name":"utils","path":"utils","contentType":"directory"},{"name":".coveragerc","path":".coveragerc","contentType":"file"},{"name":".gitattributes","path":".gitattributes","contentType":"file"},{"name":".gitignore","path":".gitignore","contentType":"file"},{"name":"CITATION.cff","path":"CITATION.cff","contentType":"file"},{"name":"CODE_OF_CONDUCT.md","path":"CODE_OF_CONDUCT.md","contentType":"file"},{"name":"CONTRIBUTING.md","path":"CONTRIBUTING.md","contentType":"file"},{"name":"ISSUES.md","path":"ISSUES.md","contentType":"file"},{"name":"LICENSE","path":"LICENSE","contentType":"file"},{"name":"Makefile","path":"Makefile","contentType":"file"},{"name":"README.md","path":"README.md","contentType":"file"},{"name":"README_es.md","path":"README_es.md","contentType":"file"},{"name":"README_hd.md","path":"README_hd.md","contentType":"file"},{"name":"README_ja.md","path":"README_ja.md","contentType":"file"},{"name":"README_ko.md","path":"README_ko.md","contentType":"file"},{"name":"README_pt-br.md","path":"README_pt-br.md","contentType":"file"},{"name":"README_ru.md","path":"README_ru.md","contentType":"file"},{"name":"README_te.md","path":"README_te.md","contentType":"file"},{"name":"README_zh-hans.md","path":"README_zh-hans.md","contentType":"file"},{"name":"README_zh-hant.md","path":"README_zh-hant.md","contentType":"file"},{"name":"SECURITY.md","path":"SECURITY.md","contentType":"file"},{"name":"awesome-transformers.md","path":"awesome-transformers.md","contentType":"file"},{"name":"conftest.py","path":"conftest.py","contentType":"file"},{"name":"hubconf.py","path":"hubconf.py","contentType":"file"},{"name":"pyproject.toml","path":"pyproject.toml","contentType":"file"},{"name":"setup.py","path":"setup.py","contentType":"file"}],"totalCount":37}},"fileTreeProcessingTime":45.464226000000004,"foldersToFetch":[],"reducedMotionEnabled":null,"repo":{"id":155220641,"defaultBranch":"main","name":"transformers","ownerLogin":"huggingface","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2018-10-29T13:56:00.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/25720743?v=4","public":true,"private":false,"isOrgOwned":true},"symbolsExpanded":false,"treeExpanded":true,"refInfo":{"name":"main","listCacheKey":"v0:1700092121.0","canEdit":false,"refType":"branch","currentOid":"06343b06335a1f8417bd32d3ffc7cf2cca9a24ac"},"path":"src/transformers/models/llama/convert_llama_weights_to_hf.py","currentUser":null,"blob":{"rawLines":["# Copyright 2022 EleutherAI and The HuggingFace Inc. team. All rights reserved.","#","# Licensed under the Apache License, Version 2.0 (the \"License\");","# you may not use this file except in compliance with the License.","# You may obtain a copy of the License at","#","#     http://www.apache.org/licenses/LICENSE-2.0","#","# Unless required by applicable law or agreed to in writing, software","# distributed under the License is distributed on an \"AS IS\" BASIS,","# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.","# See the License for the specific language governing permissions and","# limitations under the License.","import argparse","import gc","import json","import os","import shutil","import warnings","","import torch","","from transformers import LlamaConfig, LlamaForCausalLM, LlamaTokenizer","","","try:","    from transformers import LlamaTokenizerFast","except ImportError as e:","    warnings.warn(e)","    warnings.warn(","        \"The converted tokenizer will be the `slow` tokenizer. To use the fast, update your `tokenizers` library and re-run the tokenizer conversion\"","    )","    LlamaTokenizerFast = None","","\"\"\"","Sample usage:","","```","python src/transformers/models/llama/convert_llama_weights_to_hf.py \\","    --input_dir /path/to/downloaded/llama/weights --model_size 7B --output_dir /output/path","```","","Thereafter, models can be loaded via:","","```py","from transformers import LlamaForCausalLM, LlamaTokenizer","","model = LlamaForCausalLM.from_pretrained(\"/output/path\")","tokenizer = LlamaTokenizer.from_pretrained(\"/output/path\")","```","","Important note: you need to be able to host the whole model in RAM to execute this script (even if the biggest versions","come in several checkpoints they each contain a part of each weight of the model, so we need to load them all in RAM).","\"\"\"","","NUM_SHARDS = {","    \"7B\": 1,","    \"7Bf\": 1,","    \"13B\": 2,","    \"13Bf\": 2,","    \"34B\": 4,","    \"30B\": 4,","    \"65B\": 8,","    \"70B\": 8,","    \"70Bf\": 8,","}","","","def compute_intermediate_size(n, ffn_dim_multiplier=1, multiple_of=256):","    return multiple_of * ((int(ffn_dim_multiplier * int(8 * n / 3)) + multiple_of - 1) // multiple_of)","","","def read_json(path):","    with open(path, \"r\") as f:","        return json.load(f)","","","def write_json(text, path):","    with open(path, \"w\") as f:","        json.dump(text, f)","","","def write_model(model_path, input_base_path, model_size, tokenizer_path=None, safe_serialization=True):","    # for backward compatibility, before you needed the repo to be called `my_repo/model_size`","    if not os.path.isfile(os.path.join(input_base_path, \"params.json\")):","        input_base_path = os.path.join(input_base_path, model_size)","","    os.makedirs(model_path, exist_ok=True)","    tmp_model_path = os.path.join(model_path, \"tmp\")","    os.makedirs(tmp_model_path, exist_ok=True)","","    params = read_json(os.path.join(input_base_path, \"params.json\"))","    num_shards = NUM_SHARDS[model_size]","    n_layers = params[\"n_layers\"]","    n_heads = params[\"n_heads\"]","    n_heads_per_shard = n_heads // num_shards","    dim = params[\"dim\"]","    dims_per_head = dim // n_heads","    base = params.get(\"rope_theta\", 10000.0)","    inv_freq = 1.0 / (base ** (torch.arange(0, dims_per_head, 2).float() / dims_per_head))","    if base > 10000.0:","        max_position_embeddings = 16384","    else:","        max_position_embeddings = 2048","","    tokenizer_class = LlamaTokenizer if LlamaTokenizerFast is None else LlamaTokenizerFast","    if tokenizer_path is not None:","        tokenizer = tokenizer_class(tokenizer_path)","        tokenizer.save_pretrained(model_path)","    vocab_size = tokenizer.vocab_size if tokenizer_path is not None else 32000","","    if \"n_kv_heads\" in params:","        num_key_value_heads = params[\"n_kv_heads\"]  # for GQA / MQA","        num_local_key_value_heads = n_heads_per_shard // num_key_value_heads","        key_value_dim = dim // num_key_value_heads","    else:  # compatibility with other checkpoints","        num_key_value_heads = n_heads","        num_local_key_value_heads = n_heads_per_shard","        key_value_dim = dim","","    # permute for sliced rotary","    def permute(w, n_heads=n_heads, dim1=dim, dim2=dim):","        return w.view(n_heads, dim1 // n_heads // 2, 2, dim2).transpose(1, 2).reshape(dim1, dim2)","","    print(f\"Fetching all parameters from the checkpoint at {input_base_path}.\")","    # Load weights","    if num_shards == 1:","        # Not sharded","        # (The sharded implementation would also work, but this is simpler.)","        loaded = torch.load(os.path.join(input_base_path, \"consolidated.00.pth\"), map_location=\"cpu\")","    else:","        # Sharded","        loaded = [","            torch.load(os.path.join(input_base_path, f\"consolidated.{i:02d}.pth\"), map_location=\"cpu\")","            for i in range(num_shards)","        ]","    param_count = 0","    index_dict = {\"weight_map\": {}}","    for layer_i in range(n_layers):","        filename = f\"pytorch_model-{layer_i + 1}-of-{n_layers + 1}.bin\"","        if num_shards == 1:","            # Unsharded","            state_dict = {","                f\"model.layers.{layer_i}.self_attn.q_proj.weight\": permute(","                    loaded[f\"layers.{layer_i}.attention.wq.weight\"]","                ),","                f\"model.layers.{layer_i}.self_attn.k_proj.weight\": permute(","                    loaded[f\"layers.{layer_i}.attention.wk.weight\"]","                ),","                f\"model.layers.{layer_i}.self_attn.v_proj.weight\": loaded[f\"layers.{layer_i}.attention.wv.weight\"],","                f\"model.layers.{layer_i}.self_attn.o_proj.weight\": loaded[f\"layers.{layer_i}.attention.wo.weight\"],","                f\"model.layers.{layer_i}.mlp.gate_proj.weight\": loaded[f\"layers.{layer_i}.feed_forward.w1.weight\"],","                f\"model.layers.{layer_i}.mlp.down_proj.weight\": loaded[f\"layers.{layer_i}.feed_forward.w2.weight\"],","                f\"model.layers.{layer_i}.mlp.up_proj.weight\": loaded[f\"layers.{layer_i}.feed_forward.w3.weight\"],","                f\"model.layers.{layer_i}.input_layernorm.weight\": loaded[f\"layers.{layer_i}.attention_norm.weight\"],","                f\"model.layers.{layer_i}.post_attention_layernorm.weight\": loaded[f\"layers.{layer_i}.ffn_norm.weight\"],","            }","        else:","            # Sharded","            # Note that attention.w{q,k,v,o}, feed_fordward.w[1,2,3], attention_norm.weight and ffn_norm.weight share","            # the same storage object, saving attention_norm and ffn_norm will save other weights too, which is","            # redundant as other weights will be stitched from multiple shards. To avoid that, they are cloned.","","            state_dict = {","                f\"model.layers.{layer_i}.input_layernorm.weight\": loaded[0][","                    f\"layers.{layer_i}.attention_norm.weight\"","                ].clone(),","                f\"model.layers.{layer_i}.post_attention_layernorm.weight\": loaded[0][","                    f\"layers.{layer_i}.ffn_norm.weight\"","                ].clone(),","            }","            state_dict[f\"model.layers.{layer_i}.self_attn.q_proj.weight\"] = permute(","                torch.cat(","                    [","                        loaded[i][f\"layers.{layer_i}.attention.wq.weight\"].view(n_heads_per_shard, dims_per_head, dim)","                        for i in range(num_shards)","                    ],","                    dim=0,","                ).reshape(dim, dim)","            )","            state_dict[f\"model.layers.{layer_i}.self_attn.k_proj.weight\"] = permute(","                torch.cat(","                    [","                        loaded[i][f\"layers.{layer_i}.attention.wk.weight\"].view(","                            num_local_key_value_heads, dims_per_head, dim","                        )","                        for i in range(num_shards)","                    ],","                    dim=0,","                ).reshape(key_value_dim, dim),","                num_key_value_heads,","                key_value_dim,","                dim,","            )","            state_dict[f\"model.layers.{layer_i}.self_attn.v_proj.weight\"] = torch.cat(","                [","                    loaded[i][f\"layers.{layer_i}.attention.wv.weight\"].view(","                        num_local_key_value_heads, dims_per_head, dim","                    )","                    for i in range(num_shards)","                ],","                dim=0,","            ).reshape(key_value_dim, dim)","","            state_dict[f\"model.layers.{layer_i}.self_attn.o_proj.weight\"] = torch.cat(","                [loaded[i][f\"layers.{layer_i}.attention.wo.weight\"] for i in range(num_shards)], dim=1","            )","            state_dict[f\"model.layers.{layer_i}.mlp.gate_proj.weight\"] = torch.cat(","                [loaded[i][f\"layers.{layer_i}.feed_forward.w1.weight\"] for i in range(num_shards)], dim=0","            )","            state_dict[f\"model.layers.{layer_i}.mlp.down_proj.weight\"] = torch.cat(","                [loaded[i][f\"layers.{layer_i}.feed_forward.w2.weight\"] for i in range(num_shards)], dim=1","            )","            state_dict[f\"model.layers.{layer_i}.mlp.up_proj.weight\"] = torch.cat(","                [loaded[i][f\"layers.{layer_i}.feed_forward.w3.weight\"] for i in range(num_shards)], dim=0","            )","","        state_dict[f\"model.layers.{layer_i}.self_attn.rotary_emb.inv_freq\"] = inv_freq","        for k, v in state_dict.items():","            index_dict[\"weight_map\"][k] = filename","            param_count += v.numel()","        torch.save(state_dict, os.path.join(tmp_model_path, filename))","","    filename = f\"pytorch_model-{n_layers + 1}-of-{n_layers + 1}.bin\"","    if num_shards == 1:","        # Unsharded","        state_dict = {","            \"model.embed_tokens.weight\": loaded[\"tok_embeddings.weight\"],","            \"model.norm.weight\": loaded[\"norm.weight\"],","            \"lm_head.weight\": loaded[\"output.weight\"],","        }","    else:","        state_dict = {","            \"model.norm.weight\": loaded[0][\"norm.weight\"],","            \"model.embed_tokens.weight\": torch.cat(","                [loaded[i][\"tok_embeddings.weight\"] for i in range(num_shards)], dim=1","            ),","            \"lm_head.weight\": torch.cat([loaded[i][\"output.weight\"] for i in range(num_shards)], dim=0),","        }","","    for k, v in state_dict.items():","        index_dict[\"weight_map\"][k] = filename","        param_count += v.numel()","    torch.save(state_dict, os.path.join(tmp_model_path, filename))","","    # Write configs","    index_dict[\"metadata\"] = {\"total_size\": param_count * 2}","    write_json(index_dict, os.path.join(tmp_model_path, \"pytorch_model.bin.index.json\"))","    ffn_dim_multiplier = params[\"ffn_dim_multiplier\"] if \"ffn_dim_multiplier\" in params else 1","    multiple_of = params[\"multiple_of\"] if \"multiple_of\" in params else 256","    config = LlamaConfig(","        hidden_size=dim,","        intermediate_size=compute_intermediate_size(dim, ffn_dim_multiplier, multiple_of),","        num_attention_heads=params[\"n_heads\"],","        num_hidden_layers=params[\"n_layers\"],","        rms_norm_eps=params[\"norm_eps\"],","        num_key_value_heads=num_key_value_heads,","        vocab_size=vocab_size,","        rope_theta=base,","        max_position_embeddings=max_position_embeddings,","    )","    config.save_pretrained(tmp_model_path)","","    # Make space so we can load the model properly now.","    del state_dict","    del loaded","    gc.collect()","","    print(\"Loading the checkpoint in a Llama model.\")","    model = LlamaForCausalLM.from_pretrained(tmp_model_path, torch_dtype=torch.bfloat16, low_cpu_mem_usage=True)","    # Avoid saving this as part of the config.","    del model.config._name_or_path","    model.config.torch_dtype = torch.float16","    print(\"Saving in the Transformers format.\")","    model.save_pretrained(model_path, safe_serialization=safe_serialization)","    shutil.rmtree(tmp_model_path)","","","def write_tokenizer(tokenizer_path, input_tokenizer_path):","    # Initialize the tokenizer based on the `spm` model","    tokenizer_class = LlamaTokenizer if LlamaTokenizerFast is None else LlamaTokenizerFast","    print(f\"Saving a {tokenizer_class.__name__} to {tokenizer_path}.\")","    tokenizer = tokenizer_class(input_tokenizer_path)","    tokenizer.save_pretrained(tokenizer_path)","","","def main():","    parser = argparse.ArgumentParser()","    parser.add_argument(","        \"--input_dir\",","        help=\"Location of LLaMA weights, which contains tokenizer.model and model folders\",","    )","    parser.add_argument(","        \"--model_size\",","        choices=[\"7B\", \"7Bf\", \"13B\", \"13Bf\", \"30B\", \"34B\", \"65B\", \"70B\", \"70Bf\", \"tokenizer_only\"],","        help=\"'f' models correspond to the finetuned versions, and are specific to the Llama2 official release. For more details on Llama2, checkout the original repo: [https://huggingface.co/meta-llama\](https://huggingface.co/meta-llama/)",","    )","    parser.add_argument(","        \"--output_dir\",","        help=\"Location to write HF model and tokenizer\",","    )","    parser.add_argument(\"--safe_serialization\", type=bool, help=\"Whether or not to save using `safetensors`.\")","    args = parser.parse_args()","    spm_path = os.path.join(args.input_dir, \"tokenizer.model\")","    if args.model_size != \"tokenizer_only\":","        write_model(","            model_path=args.output_dir,","            input_base_path=args.input_dir,","            model_size=args.model_size,","            safe_serialization=args.safe_serialization,","            tokenizer_path=spm_path,","        )","    else:","        write_tokenizer(args.output_dir, spm_path)","","","if __name__ == \"__main__\":","    main()"],"stylingDirectives":[[{"start":0,"end":79,"cssClass":"pl-c"}],[{"start":0,"end":1,"cssClass":"pl-c"}],[{"start":0,"end":65,"cssClass":"pl-c"}],[{"start":0,"end":66,"cssClass":"pl-c"}],[{"start":0,"end":41,"cssClass":"pl-c"}],[{"start":0,"end":1,"cssClass":"pl-c"}],[{"start":0,"end":48,"cssClass":"pl-c"}],[{"start":0,"end":1,"cssClass":"pl-c"}],[{"start":0,"end":69,"cssClass":"pl-c"}],[{"start":0,"end":67,"cssClass":"pl-c"}],[{"start":0,"end":74,"cssClass":"pl-c"}],[{"start":0,"end":69,"cssClass":"pl-c"}],[{"start":0,"end":32,"cssClass":"pl-c"}],[{"start":0,"end":6,"cssClass":"pl-k"},{"start":7,"end":15,"cssClass":"pl-s1"}],[{"start":0,"end":6,"cssClass":"pl-k"},{"start":7,"end":9,"cssClass":"pl-s1"}],[{"start":0,"end":6,"cssClass":"pl-k"},{"start":7,"end":11,"cssClass":"pl-s1"}],[{"start":0,"end":6,"cssClass":"pl-k"},{"start":7,"end":9,"cssClass":"pl-s1"}],[{"start":0,"end":6,"cssClass":"pl-k"},{"start":7,"end":13,"cssClass":"pl-s1"}],[{"start":0,"end":6,"cssClass":"pl-k"},{"start":7,"end":15,"cssClass":"pl-s1"}],[],[{"start":0,"end":6,"cssClass":"pl-k"},{"start":7,"end":12,"cssClass":"pl-s1"}],[],[{"start":0,"end":4,"cssClass":"pl-k"},{"start":5,"end":17,"cssClass":"pl-s1"},{"start":18,"end":24,"cssClass":"pl-k"},{"start":25,"end":36,"cssClass":"pl-v"},{"start":38,"end":54,"cssClass":"pl-v"},{"start":56,"end":70,"cssClass":"pl-v"}],[],[],[{"start":0,"end":3,"cssClass":"pl-k"}],[{"start":4,"end":8,"cssClass":"pl-k"},{"start":9,"end":21,"cssClass":"pl-s1"},{"start":22,"end":28,"cssClass":"pl-k"},{"start":29,"end":47,"cssClass":"pl-v"}],[{"start":0,"end":6,"cssClass":"pl-k"},{"start":7,"end":18,"cssClass":"pl-v"},{"start":19,"end":21,"cssClass":"pl-k"},{"start":22,"end":23,"cssClass":"pl-s1"}],[{"start":4,"end":12,"cssClass":"pl-s1"},{"start":13,"end":17,"cssClass":"pl-en"},{"start":18,"end":19,"cssClass":"pl-s1"}],[{"start":4,"end":12,"cssClass":"pl-s1"},{"start":13,"end":17,"cssClass":"pl-en"}],[{"start":8,"end":149,"cssClass":"pl-s"}],[],[{"start":4,"end":22,"cssClass":"pl-v"},{"start":23,"end":24,"cssClass":"pl-c1"},{"start":25,"end":29,"cssClass":"pl-c1"}],[],[{"start":0,"end":3,"cssClass":"pl-s"}],[{"start":0,"end":13,"cssClass":"pl-s"}],[{"start":0,"end":0,"cssClass":"pl-s"}],[{"start":0,"end":3,"cssClass":"pl-s"}],[{"start":0,"end":69,"cssClass":"pl-s"},{"start":68,"end":69,"cssClass":"pl-cce"}],[{"start":0,"end":91,"cssClass":"pl-s"},{"start":0,"end":0,"cssClass":"pl-cce"}],[{"start":0,"end":3,"cssClass":"pl-s"}],[{"start":0,"end":0,"cssClass":"pl-s"}],[{"start":0,"end":37,"cssClass":"pl-s"}],[{"start":0,"end":0,"cssClass":"pl-s"}],[{"start":0,"end":5,"cssClass":"pl-s"}],[{"start":0,"end":57,"cssClass":"pl-s"}],[{"start":0,"end":0,"cssClass":"pl-s"}],[{"start":0,"end":56,"cssClass":"pl-s"}],[{"start":0,"end":58,"cssClass":"pl-s"}],[{"start":0,"end":3,"cssClass":"pl-s"}],[{"start":0,"end":0,"cssClass":"pl-s"}],[{"start":0,"end":119,"cssClass":"pl-s"}],[{"start":0,"end":118,"cssClass":"pl-s"}],[{"start":0,"end":3,"cssClass":"pl-s"}],[],[{"start":0,"end":10,"cssClass":"pl-v"},{"start":11,"end":12,"cssClass":"pl-c1"}],[{"start":4,"end":8,"cssClass":"pl-s"},{"start":10,"end":11,"cssClass":"pl-c1"}],[{"start":4,"end":9,"cssClass":"pl-s"},{"start":11,"end":12,"cssClass":"pl-c1"}],[{"start":4,"end":9,"cssClass":"pl-s"},{"start":11,"end":12,"cssClass":"pl-c1"}],[{"start":4,"end":10,"cssClass":"pl-s"},{"start":12,"end":13,"cssClass":"pl-c1"}],[{"start":4,"end":9,"cssClass":"pl-s"},{"start":11,"end":12,"cssClass":"pl-c1"}],[{"start":4,"end":9,"cssClass":"pl-s"},{"start":11,"end":12,"cssClass":"pl-c1"}],[{"start":4,"end":9,"cssClass":"pl-s"},{"start":11,"end":12,"cssClass":"pl-c1"}],[{"start":4,"end":9,"cssClass":"pl-s"},{"start":11,"end":12,"cssClass":"pl-c1"}],[{"start":4,"end":10,"cssClass":"pl-s"},{"start":12,"end":13,"cssClass":"pl-c1"}],[],[],[],[{"start":0,"end":3,"cssClass":"pl-k"},{"start":4,"end":29,"cssClass":"pl-en"},{"start":30,"end":31,"cssClass":"pl-s1"},{"start":33,"end":51,"cssClass":"pl-s1"},{"start":51,"end":52,"cssClass":"pl-c1"},{"start":52,"end":53,"cssClass":"pl-c1"},{"start":55,"end":66,"cssClass":"pl-s1"},{"start":66,"end":67,"cssClass":"pl-c1"},{"start":67,"end":70,"cssClass":"pl-c1"}],[{"start":4,"end":10,"cssClass":"pl-k"},{"start":11,"end":22,"cssClass":"pl-s1"},{"start":23,"end":24,"cssClass":"pl-c1"},{"start":27,"end":30,"cssClass":"pl-en"},{"start":31,"end":49,"cssClass":"pl-s1"},{"start":50,"end":51,"cssClass":"pl-c1"},{"start":52,"end":55,"cssClass":"pl-en"},{"start":56,"end":57,"cssClass":"pl-c1"},{"start":58,"end":59,"cssClass":"pl-c1"},{"start":60,"end":61,"cssClass":"pl-s1"},{"start":62,"end":63,"cssClass":"pl-c1"},{"start":64,"end":65,"cssClass":"pl-c1"},{"start":68,"end":69,"cssClass":"pl-c1"},{"start":70,"end":81,"cssClass":"pl-s1"},{"start":82,"end":83,"cssClass":"pl-c1"},{"start":84,"end":85,"cssClass":"pl-c1"},{"start":87,"end":89,"cssClass":"pl-c1"},{"start":90,"end":101,"cssClass":"pl-s1"}],[],[],[{"start":0,"end":3,"cssClass":"pl-k"},{"start":4,"end":13,"cssClass":"pl-en"},{"start":14,"end":18,"cssClass":"pl-s1"}],[{"start":4,"end":8,"cssClass":"pl-k"},{"start":9,"end":13,"cssClass":"pl-en"},{"start":14,"end":18,"cssClass":"pl-s1"},{"start":20,"end":23,"cssClass":"pl-s"},{"start":25,"end":27,"cssClass":"pl-k"},{"start":28,"end":29,"cssClass":"pl-s1"}],[{"start":8,"end":14,"cssClass":"pl-k"},{"start":15,"end":19,"cssClass":"pl-s1"},{"start":20,"end":24,"cssClass":"pl-en"},{"start":25,"end":26,"cssClass":"pl-s1"}],[],[],[{"start":0,"end":3,"cssClass":"pl-k"},{"start":4,"end":14,"cssClass":"pl-en"},{"start":15,"end":19,"cssClass":"pl-s1"},{"start":21,"end":25,"cssClass":"pl-s1"}],[{"start":4,"end":8,"cssClass":"pl-k"},{"start":9,"end":13,"cssClass":"pl-en"},{"start":14,"end":18,"cssClass":"pl-s1"},{"start":20,"end":23,"cssClass":"pl-s"},{"start":25,"end":27,"cssClass":"pl-k"},{"start":28,"end":29,"cssClass":"pl-s1"}],[{"start":8,"end":12,"cssClass":"pl-s1"},{"start":13,"end":17,"cssClass":"pl-en"},{"start":18,"end":22,"cssClass":"pl-s1"},{"start":24,"end":25,"cssClass":"pl-s1"}],[],[],[{"start":0,"end":3,"cssClass":"pl-k"},{"start":4,"end":15,"cssClass":"pl-en"},{"start":16,"end":26,"cssClass":"pl-s1"},{"start":28,"end":43,"cssClass":"pl-s1"},{"start":45,"end":55,"cssClass":"pl-s1"},{"start":57,"end":71,"cssClass":"pl-s1"},{"start":71,"end":72,"cssClass":"pl-c1"},{"start":72,"end":76,"cssClass":"pl-c1"},{"start":78,"end":96,"cssClass":"pl-s1"},{"start":96,"end":97,"cssClass":"pl-c1"},{"start":97,"end":101,"cssClass":"pl-c1"}],[{"start":4,"end":94,"cssClass":"pl-c"}],[{"start":4,"end":6,"cssClass":"pl-k"},{"start":7,"end":10,"cssClass":"pl-c1"},{"start":11,"end":13,"cssClass":"pl-s1"},{"start":14,"end":18,"cssClass":"pl-s1"},{"start":19,"end":25,"cssClass":"pl-en"},{"start":26,"end":28,"cssClass":"pl-s1"},{"start":29,"end":33,"cssClass":"pl-s1"},{"start":34,"end":38,"cssClass":"pl-en"},{"start":39,"end":54,"cssClass":"pl-s1"},{"start":56,"end":69,"cssClass":"pl-s"}],[{"start":8,"end":23,"cssClass":"pl-s1"},{"start":24,"end":25,"cssClass":"pl-c1"},{"start":26,"end":28,"cssClass":"pl-s1"},{"start":29,"end":33,"cssClass":"pl-s1"},{"start":34,"end":38,"cssClass":"pl-en"},{"start":39,"end":54,"cssClass":"pl-s1"},{"start":56,"end":66,"cssClass":"pl-s1"}],[],[{"start":4,"end":6,"cssClass":"pl-s1"},{"start":7,"end":15,"cssClass":"pl-en"},{"start":16,"end":26,"cssClass":"pl-s1"},{"start":28,"end":36,"cssClass":"pl-s1"},{"start":36,"end":37,"cssClass":"pl-c1"},{"start":37,"end":41,"cssClass":"pl-c1"}],[{"start":4,"end":18,"cssClass":"pl-s1"},{"start":19,"end":20,"cssClass":"pl-c1"},{"start":21,"end":23,"cssClass":"pl-s1"},{"start":24,"end":28,"cssClass":"pl-s1"},{"start":29,"end":33,"cssClass":"pl-en"},{"start":34,"end":44,"cssClass":"pl-s1"},{"start":46,"end":51,"cssClass":"pl-s"}],[{"start":4,"end":6,"cssClass":"pl-s1"},{"start":7,"end":15,"cssClass":"pl-en"},{"start":16,"end":30,"cssClass":"pl-s1"},{"start":32,"end":40,"cssClass":"pl-s1"},{"start":40,"end":41,"cssClass":"pl-c1"},{"start":41,"end":45,"cssClass":"pl-c1"}],[],[{"start":4,"end":10,"cssClass":"pl-s1"},{"start":11,"end":12,"cssClass":"pl-c1"},{"start":13,"end":22,"cssClass":"pl-en"},{"start":23,"end":25,"cssClass":"pl-s1"},{"start":26,"end":30,"cssClass":"pl-s1"},{"start":31,"end":35,"cssClass":"pl-en"},{"start":36,"end":51,"cssClass":"pl-s1"},{"start":53,"end":66,"cssClass":"pl-s"}],[{"start":4,"end":14,"cssClass":"pl-s1"},{"start":15,"end":16,"cssClass":"pl-c1"},{"start":17,"end":27,"cssClass":"pl-v"},{"start":28,"end":38,"cssClass":"pl-s1"}],[{"start":4,"end":12,"cssClass":"pl-s1"},{"start":13,"end":14,"cssClass":"pl-c1"},{"start":15,"end":21,"cssClass":"pl-s1"},{"start":22,"end":32,"cssClass":"pl-s"}],[{"start":4,"end":11,"cssClass":"pl-s1"},{"start":12,"end":13,"cssClass":"pl-c1"},{"start":14,"end":20,"cssClass":"pl-s1"},{"start":21,"end":30,"cssClass":"pl-s"}],[{"start":4,"end":21,"cssClass":"pl-s1"},{"start":22,"end":23,"cssClass":"pl-c1"},{"start":24,"end":31,"cssClass":"pl-s1"},{"start":32,"end":34,"cssClass":"pl-c1"},{"start":35,"end":45,"cssClass":"pl-s1"}],[{"start":4,"end":7,"cssClass":"pl-s1"},{"start":8,"end":9,"cssClass":"pl-c1"},{"start":10,"end":16,"cssClass":"pl-s1"},{"start":17,"end":22,"cssClass":"pl-s"}],[{"start":4,"end":17,"cssClass":"pl-s1"},{"start":18,"end":19,"cssClass":"pl-c1"},{"start":20,"end":23,"cssClass":"pl-s1"},{"start":24,"end":26,"cssClass":"pl-c1"},{"start":27,"end":34,"cssClass":"pl-s1"}],[{"start":4,"end":8,"cssClass":"pl-s1"},{"start":9,"end":10,"cssClass":"pl-c1"},{"start":11,"end":17,"cssClass":"pl-s1"},{"start":18,"end":21,"cssClass":"pl-en"},{"start":22,"end":34,"cssClass":"pl-s"},{"start":36,"end":43,"cssClass":"pl-c1"}],[{"start":4,"end":12,"cssClass":"pl-s1"},{"start":13,"end":14,"cssClass":"pl-c1"},{"start":15,"end":18,"cssClass":"pl-c1"},{"start":19,"end":20,"cssClass":"pl-c1"},{"start":22,"end":26,"cssClass":"pl-s1"},{"start":27,"end":29,"cssClass":"pl-c1"},{"start":31,"end":36,"cssClass":"pl-s1"},{"start":37,"end":43,"cssClass":"pl-en"},{"start":44,"end":45,"cssClass":"pl-c1"},{"start":47,"end":60,"cssClass":"pl-s1"},{"start":62,"end":63,"cssClass":"pl-c1"},{"start":65,"end":70,"cssClass":"pl-en"},{"start":73,"end":74,"cssClass":"pl-c1"},{"start":75,"end":88,"cssClass":"pl-s1"}],[{"start":4,"end":6,"cssClass":"pl-k"},{"start":7,"end":11,"cssClass":"pl-s1"},{"start":12,"end":13,"cssClass":"pl-c1"},{"start":14,"end":21,"cssClass":"pl-c1"}],[{"start":8,"end":31,"cssClass":"pl-s1"},{"start":32,"end":33,"cssClass":"pl-c1"},{"start":34,"end":39,"cssClass":"pl-c1"}],[{"start":4,"end":8,"cssClass":"pl-k"}],[{"start":8,"end":31,"cssClass":"pl-s1"},{"start":32,"end":33,"cssClass":"pl-c1"},{"start":34,"end":38,"cssClass":"pl-c1"}],[],[{"start":4,"end":19,"cssClass":"pl-s1"},{"start":20,"end":21,"cssClass":"pl-c1"},{"start":22,"end":36,"cssClass":"pl-v"},{"start":37,"end":39,"cssClass":"pl-k"},{"start":40,"end":58,"cssClass":"pl-v"},{"start":59,"end":61,"cssClass":"pl-c1"},{"start":62,"end":66,"cssClass":"pl-c1"},{"start":67,"end":71,"cssClass":"pl-k"},{"start":72,"end":90,"cssClass":"pl-v"}],[{"start":4,"end":6,"cssClass":"pl-k"},{"start":7,"end":21,"cssClass":"pl-s1"},{"start":22,"end":24,"cssClass":"pl-c1"},{"start":25,"end":28,"cssClass":"pl-c1"},{"start":29,"end":33,"cssClass":"pl-c1"}],[{"start":8,"end":17,"cssClass":"pl-s1"},{"start":18,"end":19,"cssClass":"pl-c1"},{"start":20,"end":35,"cssClass":"pl-en"},{"start":36,"end":50,"cssClass":"pl-s1"}],[{"start":8,"end":17,"cssClass":"pl-s1"},{"start":18,"end":33,"cssClass":"pl-en"},{"start":34,"end":44,"cssClass":"pl-s1"}],[{"start":4,"end":14,"cssClass":"pl-s1"},{"start":15,"end":16,"cssClass":"pl-c1"},{"start":17,"end":26,"cssClass":"pl-s1"},{"start":27,"end":37,"cssClass":"pl-s1"},{"start":38,"end":40,"cssClass":"pl-k"},{"start":41,"end":55,"cssClass":"pl-s1"},{"start":56,"end":58,"cssClass":"pl-c1"},{"start":59,"end":62,"cssClass":"pl-c1"},{"start":63,"end":67,"cssClass":"pl-c1"},{"start":68,"end":72,"cssClass":"pl-k"},{"start":73,"end":78,"cssClass":"pl-c1"}],[],[{"start":4,"end":6,"cssClass":"pl-k"},{"start":7,"end":19,"cssClass":"pl-s"},{"start":20,"end":22,"cssClass":"pl-c1"},{"start":23,"end":29,"cssClass":"pl-s1"}],[{"start":8,"end":27,"cssClass":"pl-s1"},{"start":28,"end":29,"cssClass":"pl-c1"},{"start":30,"end":36,"cssClass":"pl-s1"},{"start":37,"end":49,"cssClass":"pl-s"},{"start":52,"end":67,"cssClass":"pl-c"}],[{"start":8,"end":33,"cssClass":"pl-s1"},{"start":34,"end":35,"cssClass":"pl-c1"},{"start":36,"end":53,"cssClass":"pl-s1"},{"start":54,"end":56,"cssClass":"pl-c1"},{"start":57,"end":76,"cssClass":"pl-s1"}],[{"start":8,"end":21,"cssClass":"pl-s1"},{"start":22,"end":23,"cssClass":"pl-c1"},{"start":24,"end":27,"cssClass":"pl-s1"},{"start":28,"end":30,"cssClass":"pl-c1"},{"start":31,"end":50,"cssClass":"pl-s1"}],[{"start":4,"end":8,"cssClass":"pl-k"},{"start":11,"end":49,"cssClass":"pl-c"}],[{"start":8,"end":27,"cssClass":"pl-s1"},{"start":28,"end":29,"cssClass":"pl-c1"},{"start":30,"end":37,"cssClass":"pl-s1"}],[{"start":8,"end":33,"cssClass":"pl-s1"},{"start":34,"end":35,"cssClass":"pl-c1"},{"start":36,"end":53,"cssClass":"pl-s1"}],[{"start":8,"end":21,"cssClass":"pl-s1"},{"start":22,"end":23,"cssClass":"pl-c1"},{"start":24,"end":27,"cssClass":"pl-s1"}],[],[{"start":4,"end":31,"cssClass":"pl-c"}],[{"start":4,"end":7,"cssClass":"pl-k"},{"start":8,"end":15,"cssClass":"pl-en"},{"start":16,"end":17,"cssClass":"pl-s1"},{"start":19,"end":26,"cssClass":"pl-s1"},{"start":26,"end":27,"cssClass":"pl-c1"},{"start":27,"end":34,"cssClass":"pl-s1"},{"start":36,"end":40,"cssClass":"pl-s1"},{"start":40,"end":41,"cssClass":"pl-c1"},{"start":41,"end":44,"cssClass":"pl-s1"},{"start":46,"end":50,"cssClass":"pl-s1"},{"start":50,"end":51,"cssClass":"pl-c1"},{"start":51,"end":54,"cssClass":"pl-s1"}],[{"start":8,"end":14,"cssClass":"pl-k"},{"start":15,"end":16,"cssClass":"pl-s1"},{"start":17,"end":21,"cssClass":"pl-en"},{"start":22,"end":29,"cssClass":"pl-s1"},{"start":31,"end":35,"cssClass":"pl-s1"},{"start":36,"end":38,"cssClass":"pl-c1"},{"start":39,"end":46,"cssClass":"pl-s1"},{"start":47,"end":49,"cssClass":"pl-c1"},{"start":50,"end":51,"cssClass":"pl-c1"},{"start":53,"end":54,"cssClass":"pl-c1"},{"start":56,"end":60,"cssClass":"pl-s1"},{"start":62,"end":71,"cssClass":"pl-en"},{"start":72,"end":73,"cssClass":"pl-c1"},{"start":75,"end":76,"cssClass":"pl-c1"},{"start":78,"end":85,"cssClass":"pl-en"},{"start":86,"end":90,"cssClass":"pl-s1"},{"start":92,"end":96,"cssClass":"pl-s1"}],[],[{"start":4,"end":9,"cssClass":"pl-en"},{"start":10,"end":78,"cssClass":"pl-s"},{"start":59,"end":76,"cssClass":"pl-s1"},{"start":59,"end":60,"cssClass":"pl-kos"},{"start":60,"end":75,"cssClass":"pl-s1"},{"start":75,"end":76,"cssClass":"pl-kos"}],[{"start":4,"end":18,"cssClass":"pl-c"}],[{"start":4,"end":6,"cssClass":"pl-k"},{"start":7,"end":17,"cssClass":"pl-s1"},{"start":18,"end":20,"cssClass":"pl-c1"},{"start":21,"end":22,"cssClass":"pl-c1"}],[{"start":8,"end":21,"cssClass":"pl-c"}],[{"start":8,"end":76,"cssClass":"pl-c"}],[{"start":8,"end":14,"cssClass":"pl-s1"},{"start":15,"end":16,"cssClass":"pl-c1"},{"start":17,"end":22,"cssClass":"pl-s1"},{"start":23,"end":27,"cssClass":"pl-en"},{"start":28,"end":30,"cssClass":"pl-s1"},{"start":31,"end":35,"cssClass":"pl-s1"},{"start":36,"end":40,"cssClass":"pl-en"},{"start":41,"end":56,"cssClass":"pl-s1"},{"start":58,"end":79,"cssClass":"pl-s"},{"start":82,"end":94,"cssClass":"pl-s1"},{"start":94,"end":95,"cssClass":"pl-c1"},{"start":95,"end":100,"cssClass":"pl-s"}],[{"start":4,"end":8,"cssClass":"pl-k"}],[{"start":8,"end":17,"cssClass":"pl-c"}],[{"start":8,"end":14,"cssClass":"pl-s1"},{"start":15,"end":16,"cssClass":"pl-c1"}],[{"start":12,"end":17,"cssClass":"pl-s1"},{"start":18,"end":22,"cssClass":"pl-en"},{"start":23,"end":25,"cssClass":"pl-s1"},{"start":26,"end":30,"cssClass":"pl-s1"},{"start":31,"end":35,"cssClass":"pl-en"},{"start":36,"end":51,"cssClass":"pl-s1"},{"start":53,"end":80,"cssClass":"pl-s"},{"start":68,"end":75,"cssClass":"pl-s1"},{"start":68,"end":69,"cssClass":"pl-kos"},{"start":69,"end":70,"cssClass":"pl-s1"},{"start":74,"end":75,"cssClass":"pl-kos"},{"start":83,"end":95,"cssClass":"pl-s1"},{"start":95,"end":96,"cssClass":"pl-c1"},{"start":96,"end":101,"cssClass":"pl-s"}],[{"start":12,"end":15,"cssClass":"pl-k"},{"start":16,"end":17,"cssClass":"pl-s1"},{"start":18,"end":20,"cssClass":"pl-c1"},{"start":21,"end":26,"cssClass":"pl-en"},{"start":27,"end":37,"cssClass":"pl-s1"}],[],[{"start":4,"end":15,"cssClass":"pl-s1"},{"start":16,"end":17,"cssClass":"pl-c1"},{"start":18,"end":19,"cssClass":"pl-c1"}],[{"start":4,"end":14,"cssClass":"pl-s1"},{"start":15,"end":16,"cssClass":"pl-c1"},{"start":18,"end":30,"cssClass":"pl-s"}],[{"start":4,"end":7,"cssClass":"pl-k"},{"start":8,"end":15,"cssClass":"pl-s1"},{"start":16,"end":18,"cssClass":"pl-c1"},{"start":19,"end":24,"cssClass":"pl-en"},{"start":25,"end":33,"cssClass":"pl-s1"}],[{"start":8,"end":16,"cssClass":"pl-s1"},{"start":17,"end":18,"cssClass":"pl-c1"},{"start":19,"end":71,"cssClass":"pl-s"},{"start":35,"end":48,"cssClass":"pl-s1"},{"start":35,"end":36,"cssClass":"pl-kos"},{"start":36,"end":43,"cssClass":"pl-s1"},{"start":44,"end":45,"cssClass":"pl-c1"},{"start":46,"end":47,"cssClass":"pl-c1"},{"start":47,"end":48,"cssClass":"pl-kos"},{"start":52,"end":66,"cssClass":"pl-s1"},{"start":52,"end":53,"cssClass":"pl-kos"},{"start":53,"end":61,"cssClass":"pl-s1"},{"start":62,"end":63,"cssClass":"pl-c1"},{"start":64,"end":65,"cssClass":"pl-c1"},{"start":65,"end":66,"cssClass":"pl-kos"}],[{"start":8,"end":10,"cssClass":"pl-k"},{"start":11,"end":21,"cssClass":"pl-s1"},{"start":22,"end":24,"cssClass":"pl-c1"},{"start":25,"end":26,"cssClass":"pl-c1"}],[{"start":12,"end":23,"cssClass":"pl-c"}],[{"start":12,"end":22,"cssClass":"pl-s1"},{"start":23,"end":24,"cssClass":"pl-c1"}],[{"start":16,"end":65,"cssClass":"pl-s"},{"start":31,"end":40,"cssClass":"pl-s1"},{"start":31,"end":32,"cssClass":"pl-kos"},{"start":32,"end":39,"cssClass":"pl-s1"},{"start":39,"end":40,"cssClass":"pl-kos"},{"start":67,"end":74,"cssClass":"pl-en"}],[{"start":20,"end":26,"cssClass":"pl-s1"},{"start":27,"end":66,"cssClass":"pl-s"},{"start":36,"end":45,"cssClass":"pl-s1"},{"start":36,"end":37,"cssClass":"pl-kos"},{"start":37,"end":44,"cssClass":"pl-s1"},{"start":44,"end":45,"cssClass":"pl-kos"}],[],[{"start":16,"end":65,"cssClass":"pl-s"},{"start":31,"end":40,"cssClass":"pl-s1"},{"start":31,"end":32,"cssClass":"pl-kos"},{"start":32,"end":39,"cssClass":"pl-s1"},{"start":39,"end":40,"cssClass":"pl-kos"},{"start":67,"end":74,"cssClass":"pl-en"}],[{"start":20,"end":26,"cssClass":"pl-s1"},{"start":27,"end":66,"cssClass":"pl-s"},{"start":36,"end":45,"cssClass":"pl-s1"},{"start":36,"end":37,"cssClass":"pl-kos"},{"start":37,"end":44,"cssClass":"pl-s1"},{"start":44,"end":45,"cssClass":"pl-kos"}],[],[{"start":16,"end":65,"cssClass":"pl-s"},{"start":31,"end":40,"cssClass":"pl-s1"},{"start":31,"end":32,"cssClass":"pl-kos"},{"start":32,"end":39,"cssClass":"pl-s1"},{"start":39,"end":40,"cssClass":"pl-kos"},{"start":67,"end":73,"cssClass":"pl-s1"},{"start":74,"end":113,"cssClass":"pl-s"},{"start":83,"end":92,"cssClass":"pl-s1"},{"start":83,"end":84,"cssClass":"pl-kos"},{"start":84,"end":91,"cssClass":"pl-s1"},{"start":91,"end":92,"cssClass":"pl-kos"}],[{"start":16,"end":65,"cssClass":"pl-s"},{"start":31,"end":40,"cssClass":"pl-s1"},{"start":31,"end":32,"cssClass":"pl-kos"},{"start":32,"end":39,"cssClass":"pl-s1"},{"start":39,"end":40,"cssClass":"pl-kos"},{"start":67,"end":73,"cssClass":"pl-s1"},{"start":74,"end":113,"cssClass":"pl-s"},{"start":83,"end":92,"cssClass":"pl-s1"},{"start":83,"end":84,"cssClass":"pl-kos"},{"start":84,"end":91,"cssClass":"pl-s1"},{"start":91,"end":92,"cssClass":"pl-kos"}],[{"start":16,"end":62,"cssClass":"pl-s"},{"start":31,"end":40,"cssClass":"pl-s1"},{"start":31,"end":32,"cssClass":"pl-kos"},{"start":32,"end":39,"cssClass":"pl-s1"},{"start":39,"end":40,"cssClass":"pl-kos"},{"start":64,"end":70,"cssClass":"pl-s1"},{"start":71,"end":113,"cssClass":"pl-s"},{"start":80,"end":89,"cssClass":"pl-s1"},{"start":80,"end":81,"cssClass":"pl-kos"},{"start":81,"end":88,"cssClass":"pl-s1"},{"start":88,"end":89,"cssClass":"pl-kos"}],[{"start":16,"end":62,"cssClass":"pl-s"},{"start":31,"end":40,"cssClass":"pl-s1"},{"start":31,"end":32,"cssClass":"pl-kos"},{"start":32,"end":39,"cssClass":"pl-s1"},{"start":39,"end":40,"cssClass":"pl-kos"},{"start":64,"end":70,"cssClass":"pl-s1"},{"start":71,"end":113,"cssClass":"pl-s"},{"start":80,"end":89,"cssClass":"pl-s1"},{"start":80,"end":81,"cssClass":"pl-kos"},{"start":81,"end":88,"cssClass":"pl-s1"},{"start":88,"end":89,"cssClass":"pl-kos"}],[{"start":16,"end":60,"cssClass":"pl-s"},{"start":31,"end":40,"cssClass":"pl-s1"},{"start":31,"end":32,"cssClass":"pl-kos"},{"start":32,"end":39,"cssClass":"pl-s1"},{"start":39,"end":40,"cssClass":"pl-kos"},{"start":62,"end":68,"cssClass":"pl-s1"},{"start":69,"end":111,"cssClass":"pl-s"},{"start":78,"end":87,"cssClass":"pl-s1"},{"start":78,"end":79,"cssClass":"pl-kos"},{"start":79,"end":86,"cssClass":"pl-s1"},{"start":86,"end":87,"cssClass":"pl-kos"}],[{"start":16,"end":64,"cssClass":"pl-s"},{"start":31,"end":40,"cssClass":"pl-s1"},{"start":31,"end":32,"cssClass":"pl-kos"},{"start":32,"end":39,"cssClass":"pl-s1"},{"start":39,"end":40,"cssClass":"pl-kos"},{"start":66,"end":72,"cssClass":"pl-s1"},{"start":73,"end":114,"cssClass":"pl-s"},{"start":82,"end":91,"cssClass":"pl-s1"},{"start":82,"end":83,"cssClass":"pl-kos"},{"start":83,"end":90,"cssClass":"pl-s1"},{"start":90,"end":91,"cssClass":"pl-kos"}],[{"start":16,"end":73,"cssClass":"pl-s"},{"start":31,"end":40,"cssClass":"pl-s1"},{"start":31,"end":32,"cssClass":"pl-kos"},{"start":32,"end":39,"cssClass":"pl-s1"},{"start":39,"end":40,"cssClass":"pl-kos"},{"start":75,"end":81,"cssClass":"pl-s1"},{"start":82,"end":117,"cssClass":"pl-s"},{"start":91,"end":100,"cssClass":"pl-s1"},{"start":91,"end":92,"cssClass":"pl-kos"},{"start":92,"end":99,"cssClass":"pl-s1"},{"start":99,"end":100,"cssClass":"pl-kos"}],[],[{"start":8,"end":12,"cssClass":"pl-k"}],[{"start":12,"end":21,"cssClass":"pl-c"}],[{"start":12,"end":117,"cssClass":"pl-c"}],[{"start":12,"end":111,"cssClass":"pl-c"}],[{"start":12,"end":111,"cssClass":"pl-c"}],[],[{"start":12,"end":22,"cssClass":"pl-s1"},{"start":23,"end":24,"cssClass":"pl-c1"}],[{"start":16,"end":64,"cssClass":"pl-s"},{"start":31,"end":40,"cssClass":"pl-s1"},{"start":31,"end":32,"cssClass":"pl-kos"},{"start":32,"end":39,"cssClass":"pl-s1"},{"start":39,"end":40,"cssClass":"pl-kos"},{"start":66,"end":72,"cssClass":"pl-s1"},{"start":73,"end":74,"cssClass":"pl-c1"}],[{"start":20,"end":61,"cssClass":"pl-s"},{"start":29,"end":38,"cssClass":"pl-s1"},{"start":29,"end":30,"cssClass":"pl-kos"},{"start":30,"end":37,"cssClass":"pl-s1"},{"start":37,"end":38,"cssClass":"pl-kos"}],[{"start":18,"end":23,"cssClass":"pl-en"}],[{"start":16,"end":73,"cssClass":"pl-s"},{"start":31,"end":40,"cssClass":"pl-s1"},{"start":31,"end":32,"cssClass":"pl-kos"},{"start":32,"end":39,"cssClass":"pl-s1"},{"start":39,"end":40,"cssClass":"pl-kos"},{"start":75,"end":81,"cssClass":"pl-s1"},{"start":82,"end":83,"cssClass":"pl-c1"}],[{"start":20,"end":55,"cssClass":"pl-s"},{"start":29,"end":38,"cssClass":"pl-s1"},{"start":29,"end":30,"cssClass":"pl-kos"},{"start":30,"end":37,"cssClass":"pl-s1"},{"start":37,"end":38,"cssClass":"pl-kos"}],[{"start":18,"end":23,"cssClass":"pl-en"}],[],[{"start":12,"end":22,"cssClass":"pl-s1"},{"start":23,"end":72,"cssClass":"pl-s"},{"start":38,"end":47,"cssClass":"pl-s1"},{"start":38,"end":39,"cssClass":"pl-kos"},{"start":39,"end":46,"cssClass":"pl-s1"},{"start":46,"end":47,"cssClass":"pl-kos"},{"start":74,"end":75,"cssClass":"pl-c1"},{"start":76,"end":83,"cssClass":"pl-en"}],[{"start":16,"end":21,"cssClass":"pl-s1"},{"start":22,"end":25,"cssClass":"pl-en"}],[],[{"start":24,"end":30,"cssClass":"pl-s1"},{"start":31,"end":32,"cssClass":"pl-s1"},{"start":34,"end":73,"cssClass":"pl-s"},{"start":43,"end":52,"cssClass":"pl-s1"},{"start":43,"end":44,"cssClass":"pl-kos"},{"start":44,"end":51,"cssClass":"pl-s1"},{"start":51,"end":52,"cssClass":"pl-kos"},{"start":75,"end":79,"cssClass":"pl-en"},{"start":80,"end":97,"cssClass":"pl-s1"},{"start":99,"end":112,"cssClass":"pl-s1"},{"start":114,"end":117,"cssClass":"pl-s1"}],[{"start":24,"end":27,"cssClass":"pl-k"},{"start":28,"end":29,"cssClass":"pl-s1"},{"start":30,"end":32,"cssClass":"pl-c1"},{"start":33,"end":38,"cssClass":"pl-en"},{"start":39,"end":49,"cssClass":"pl-s1"}],[],[{"start":20,"end":23,"cssClass":"pl-s1"},{"start":23,"end":24,"cssClass":"pl-c1"},{"start":24,"end":25,"cssClass":"pl-c1"}],[{"start":18,"end":25,"cssClass":"pl-en"},{"start":26,"end":29,"cssClass":"pl-s1"},{"start":31,"end":34,"cssClass":"pl-s1"}],[],[{"start":12,"end":22,"cssClass":"pl-s1"},{"start":23,"end":72,"cssClass":"pl-s"},{"start":38,"end":47,"cssClass":"pl-s1"},{"start":38,"end":39,"cssClass":"pl-kos"},{"start":39,"end":46,"cssClass":"pl-s1"},{"start":46,"end":47,"cssClass":"pl-kos"},{"start":74,"end":75,"cssClass":"pl-c1"},{"start":76,"end":83,"cssClass":"pl-en"}],[{"start":16,"end":21,"cssClass":"pl-s1"},{"start":22,"end":25,"cssClass":"pl-en"}],[],[{"start":24,"end":30,"cssClass":"pl-s1"},{"start":31,"end":32,"cssClass":"pl-s1"},{"start":34,"end":73,"cssClass":"pl-s"},{"start":43,"end":52,"cssClass":"pl-s1"},{"start":43,"end":44,"cssClass":"pl-kos"},{"start":44,"end":51,"cssClass":"pl-s1"},{"start":51,"end":52,"cssClass":"pl-kos"},{"start":75,"end":79,"cssClass":"pl-en"}],[{"start":28,"end":53,"cssClass":"pl-s1"},{"start":55,"end":68,"cssClass":"pl-s1"},{"start":70,"end":73,"cssClass":"pl-s1"}],[],[{"start":24,"end":27,"cssClass":"pl-k"},{"start":28,"end":29,"cssClass":"pl-s1"},{"start":30,"end":32,"cssClass":"pl-c1"},{"start":33,"end":38,"cssClass":"pl-en"},{"start":39,"end":49,"cssClass":"pl-s1"}],[],[{"start":20,"end":23,"cssClass":"pl-s1"},{"start":23,"end":24,"cssClass":"pl-c1"},{"start":24,"end":25,"cssClass":"pl-c1"}],[{"start":18,"end":25,"cssClass":"pl-en"},{"start":26,"end":39,"cssClass":"pl-s1"},{"start":41,"end":44,"cssClass":"pl-s1"}],[{"start":16,"end":35,"cssClass":"pl-s1"}],[{"start":16,"end":29,"cssClass":"pl-s1"}],[{"start":16,"end":19,"cssClass":"pl-s1"}],[],[{"start":12,"end":22,"cssClass":"pl-s1"},{"start":23,"end":72,"cssClass":"pl-s"},{"start":38,"end":47,"cssClass":"pl-s1"},{"start":38,"end":39,"cssClass":"pl-kos"},{"start":39,"end":46,"cssClass":"pl-s1"},{"start":46,"end":47,"cssClass":"pl-kos"},{"start":74,"end":75,"cssClass":"pl-c1"},{"start":76,"end":81,"cssClass":"pl-s1"},{"start":82,"end":85,"cssClass":"pl-en"}],[],[{"start":20,"end":26,"cssClass":"pl-s1"},{"start":27,"end":28,"cssClass":"pl-s1"},{"start":30,"end":69,"cssClass":"pl-s"},{"start":39,"end":48,"cssClass":"pl-s1"},{"start":39,"end":40,"cssClass":"pl-kos"},{"start":40,"end":47,"cssClass":"pl-s1"},{"start":47,"end":48,"cssClass":"pl-kos"},{"start":71,"end":75,"cssClass":"pl-en"}],[{"start":24,"end":49,"cssClass":"pl-s1"},{"start":51,"end":64,"cssClass":"pl-s1"},{"start":66,"end":69,"cssClass":"pl-s1"}],[],[{"start":20,"end":23,"cssClass":"pl-k"},{"start":24,"end":25,"cssClass":"pl-s1"},{"start":26,"end":28,"cssClass":"pl-c1"},{"start":29,"end":34,"cssClass":"pl-en"},{"start":35,"end":45,"cssClass":"pl-s1"}],[],[{"start":16,"end":19,"cssClass":"pl-s1"},{"start":19,"end":20,"cssClass":"pl-c1"},{"start":20,"end":21,"cssClass":"pl-c1"}],[{"start":14,"end":21,"cssClass":"pl-en"},{"start":22,"end":35,"cssClass":"pl-s1"},{"start":37,"end":40,"cssClass":"pl-s1"}],[],[{"start":12,"end":22,"cssClass":"pl-s1"},{"start":23,"end":72,"cssClass":"pl-s"},{"start":38,"end":47,"cssClass":"pl-s1"},{"start":38,"end":39,"cssClass":"pl-kos"},{"start":39,"end":46,"cssClass":"pl-s1"},{"start":46,"end":47,"cssClass":"pl-kos"},{"start":74,"end":75,"cssClass":"pl-c1"},{"start":76,"end":81,"cssClass":"pl-s1"},{"start":82,"end":85,"cssClass":"pl-en"}],[{"start":17,"end":23,"cssClass":"pl-s1"},{"start":24,"end":25,"cssClass":"pl-s1"},{"start":27,"end":66,"cssClass":"pl-s"},{"start":36,"end":45,"cssClass":"pl-s1"},{"start":36,"end":37,"cssClass":"pl-kos"},{"start":37,"end":44,"cssClass":"pl-s1"},{"start":44,"end":45,"cssClass":"pl-kos"},{"start":68,"end":71,"cssClass":"pl-k"},{"start":72,"end":73,"cssClass":"pl-s1"},{"start":74,"end":76,"cssClass":"pl-c1"},{"start":77,"end":82,"cssClass":"pl-en"},{"start":83,"end":93,"cssClass":"pl-s1"},{"start":97,"end":100,"cssClass":"pl-s1"},{"start":100,"end":101,"cssClass":"pl-c1"},{"start":101,"end":102,"cssClass":"pl-c1"}],[],[{"start":12,"end":22,"cssClass":"pl-s1"},{"start":23,"end":69,"cssClass":"pl-s"},{"start":38,"end":47,"cssClass":"pl-s1"},{"start":38,"end":39,"cssClass":"pl-kos"},{"start":39,"end":46,"cssClass":"pl-s1"},{"start":46,"end":47,"cssClass":"pl-kos"},{"start":71,"end":72,"cssClass":"pl-c1"},{"start":73,"end":78,"cssClass":"pl-s1"},{"start":79,"end":82,"cssClass":"pl-en"}],[{"start":17,"end":23,"cssClass":"pl-s1"},{"start":24,"end":25,"cssClass":"pl-s1"},{"start":27,"end":69,"cssClass":"pl-s"},{"start":36,"end":45,"cssClass":"pl-s1"},{"start":36,"end":37,"cssClass":"pl-kos"},{"start":37,"end":44,"cssClass":"pl-s1"},{"start":44,"end":45,"cssClass":"pl-kos"},{"start":71,"end":74,"cssClass":"pl-k"},{"start":75,"end":76,"cssClass":"pl-s1"},{"start":77,"end":79,"cssClass":"pl-c1"},{"start":80,"end":85,"cssClass":"pl-en"},{"start":86,"end":96,"cssClass":"pl-s1"},{"start":100,"end":103,"cssClass":"pl-s1"},{"start":103,"end":104,"cssClass":"pl-c1"},{"start":104,"end":105,"cssClass":"pl-c1"}],[],[{"start":12,"end":22,"cssClass":"pl-s1"},{"start":23,"end":69,"cssClass":"pl-s"},{"start":38,"end":47,"cssClass":"pl-s1"},{"start":38,"end":39,"cssClass":"pl-kos"},{"start":39,"end":46,"cssClass":"pl-s1"},{"start":46,"end":47,"cssClass":"pl-kos"},{"start":71,"end":72,"cssClass":"pl-c1"},{"start":73,"end":78,"cssClass":"pl-s1"},{"start":79,"end":82,"cssClass":"pl-en"}],[{"start":17,"end":23,"cssClass":"pl-s1"},{"start":24,"end":25,"cssClass":"pl-s1"},{"start":27,"end":69,"cssClass":"pl-s"},{"start":36,"end":45,"cssClass":"pl-s1"},{"start":36,"end":37,"cssClass":"pl-kos"},{"start":37,"end":44,"cssClass":"pl-s1"},{"start":44,"end":45,"cssClass":"pl-kos"},{"start":71,"end":74,"cssClass":"pl-k"},{"start":75,"end":76,"cssClass":"pl-s1"},{"start":77,"end":79,"cssClass":"pl-c1"},{"start":80,"end":85,"cssClass":"pl-en"},{"start":86,"end":96,"cssClass":"pl-s1"},{"start":100,"end":103,"cssClass":"pl-s1"},{"start":103,"end":104,"cssClass":"pl-c1"},{"start":104,"end":105,"cssClass":"pl-c1"}],[],[{"start":12,"end":22,"cssClass":"pl-s1"},{"start":23,"end":67,"cssClass":"pl-s"},{"start":38,"end":47,"cssClass":"pl-s1"},{"start":38,"end":39,"cssClass":"pl-kos"},{"start":39,"end":46,"cssClass":"pl-s1"},{"start":46,"end":47,"cssClass":"pl-kos"},{"start":69,"end":70,"cssClass":"pl-c1"},{"start":71,"end":76,"cssClass":"pl-s1"},{"start":77,"end":80,"cssClass":"pl-en"}],[{"start":17,"end":23,"cssClass":"pl-s1"},{"start":24,"end":25,"cssClass":"pl-s1"},{"start":27,"end":69,"cssClass":"pl-s"},{"start":36,"end":45,"cssClass":"pl-s1"},{"start":36,"end":37,"cssClass":"pl-kos"},{"start":37,"end":44,"cssClass":"pl-s1"},{"start":44,"end":45,"cssClass":"pl-kos"},{"start":71,"end":74,"cssClass":"pl-k"},{"start":75,"end":76,"cssClass":"pl-s1"},{"start":77,"end":79,"cssClass":"pl-c1"},{"start":80,"end":85,"cssClass":"pl-en"},{"start":86,"end":96,"cssClass":"pl-s1"},{"start":100,"end":103,"cssClass":"pl-s1"},{"start":103,"end":104,"cssClass":"pl-c1"},{"start":104,"end":105,"cssClass":"pl-c1"}],[],[],[{"start":8,"end":18,"cssClass":"pl-s1"},{"start":19,"end":74,"cssClass":"pl-s"},{"start":34,"end":43,"cssClass":"pl-s1"},{"start":34,"end":35,"cssClass":"pl-kos"},{"start":35,"end":42,"cssClass":"pl-s1"},{"start":42,"end":43,"cssClass":"pl-kos"},{"start":76,"end":77,"cssClass":"pl-c1"},{"start":78,"end":86,"cssClass":"pl-s1"}],[{"start":8,"end":11,"cssClass":"pl-k"},{"start":12,"end":13,"cssClass":"pl-s1"},{"start":15,"end":16,"cssClass":"pl-s1"},{"start":17,"end":19,"cssClass":"pl-c1"},{"start":20,"end":30,"cssClass":"pl-s1"},{"start":31,"end":36,"cssClass":"pl-en"}],[{"start":12,"end":22,"cssClass":"pl-s1"},{"start":23,"end":35,"cssClass":"pl-s"},{"start":37,"end":38,"cssClass":"pl-s1"},{"start":40,"end":41,"cssClass":"pl-c1"},{"start":42,"end":50,"cssClass":"pl-s1"}],[{"start":12,"end":23,"cssClass":"pl-s1"},{"start":24,"end":26,"cssClass":"pl-c1"},{"start":27,"end":28,"cssClass":"pl-s1"},{"start":29,"end":34,"cssClass":"pl-en"}],[{"start":8,"end":13,"cssClass":"pl-s1"},{"start":14,"end":18,"cssClass":"pl-en"},{"start":19,"end":29,"cssClass":"pl-s1"},{"start":31,"end":33,"cssClass":"pl-s1"},{"start":34,"end":38,"cssClass":"pl-s1"},{"start":39,"end":43,"cssClass":"pl-en"},{"start":44,"end":58,"cssClass":"pl-s1"},{"start":60,"end":68,"cssClass":"pl-s1"}],[],[{"start":4,"end":12,"cssClass":"pl-s1"},{"start":13,"end":14,"cssClass":"pl-c1"},{"start":15,"end":68,"cssClass":"pl-s"},{"start":31,"end":45,"cssClass":"pl-s1"},{"start":31,"end":32,"cssClass":"pl-kos"},{"start":32,"end":40,"cssClass":"pl-s1"},{"start":41,"end":42,"cssClass":"pl-c1"},{"start":43,"end":44,"cssClass":"pl-c1"},{"start":44,"end":45,"cssClass":"pl-kos"},{"start":49,"end":63,"cssClass":"pl-s1"},{"start":49,"end":50,"cssClass":"pl-kos"},{"start":50,"end":58,"cssClass":"pl-s1"},{"start":59,"end":60,"cssClass":"pl-c1"},{"start":61,"end":62,"cssClass":"pl-c1"},{"start":62,"end":63,"cssClass":"pl-kos"}],[{"start":4,"end":6,"cssClass":"pl-k"},{"start":7,"end":17,"cssClass":"pl-s1"},{"start":18,"end":20,"cssClass":"pl-c1"},{"start":21,"end":22,"cssClass":"pl-c1"}],[{"start":8,"end":19,"cssClass":"pl-c"}],[{"start":8,"end":18,"cssClass":"pl-s1"},{"start":19,"end":20,"cssClass":"pl-c1"}],[{"start":12,"end":39,"cssClass":"pl-s"},{"start":41,"end":47,"cssClass":"pl-s1"},{"start":48,"end":71,"cssClass":"pl-s"}],[{"start":12,"end":31,"cssClass":"pl-s"},{"start":33,"end":39,"cssClass":"pl-s1"},{"start":40,"end":53,"cssClass":"pl-s"}],[{"start":12,"end":28,"cssClass":"pl-s"},{"start":30,"end":36,"cssClass":"pl-s1"},{"start":37,"end":52,"cssClass":"pl-s"}],[],[{"start":4,"end":8,"cssClass":"pl-k"}],[{"start":8,"end":18,"cssClass":"pl-s1"},{"start":19,"end":20,"cssClass":"pl-c1"}],[{"start":12,"end":31,"cssClass":"pl-s"},{"start":33,"end":39,"cssClass":"pl-s1"},{"start":40,"end":41,"cssClass":"pl-c1"},{"start":43,"end":56,"cssClass":"pl-s"}],[{"start":12,"end":39,"cssClass":"pl-s"},{"start":41,"end":46,"cssClass":"pl-s1"},{"start":47,"end":50,"cssClass":"pl-en"}],[{"start":17,"end":23,"cssClass":"pl-s1"},{"start":24,"end":25,"cssClass":"pl-s1"},{"start":27,"end":50,"cssClass":"pl-s"},{"start":52,"end":55,"cssClass":"pl-k"},{"start":56,"end":57,"cssClass":"pl-s1"},{"start":58,"end":60,"cssClass":"pl-c1"},{"start":61,"end":66,"cssClass":"pl-en"},{"start":67,"end":77,"cssClass":"pl-s1"},{"start":81,"end":84,"cssClass":"pl-s1"},{"start":84,"end":85,"cssClass":"pl-c1"},{"start":85,"end":86,"cssClass":"pl-c1"}],[],[{"start":12,"end":28,"cssClass":"pl-s"},{"start":30,"end":35,"cssClass":"pl-s1"},{"start":36,"end":39,"cssClass":"pl-en"},{"start":41,"end":47,"cssClass":"pl-s1"},{"start":48,"end":49,"cssClass":"pl-s1"},{"start":51,"end":66,"cssClass":"pl-s"},{"start":68,"end":71,"cssClass":"pl-k"},{"start":72,"end":73,"cssClass":"pl-s1"},{"start":74,"end":76,"cssClass":"pl-c1"},{"start":77,"end":82,"cssClass":"pl-en"},{"start":83,"end":93,"cssClass":"pl-s1"},{"start":97,"end":100,"cssClass":"pl-s1"},{"start":100,"end":101,"cssClass":"pl-c1"},{"start":101,"end":102,"cssClass":"pl-c1"}],[],[],[{"start":4,"end":7,"cssClass":"pl-k"},{"start":8,"end":9,"cssClass":"pl-s1"},{"start":11,"end":12,"cssClass":"pl-s1"},{"start":13,"end":15,"cssClass":"pl-c1"},{"start":16,"end":26,"cssClass":"pl-s1"},{"start":27,"end":32,"cssClass":"pl-en"}],[{"start":8,"end":18,"cssClass":"pl-s1"},{"start":19,"end":31,"cssClass":"pl-s"},{"start":33,"end":34,"cssClass":"pl-s1"},{"start":36,"end":37,"cssClass":"pl-c1"},{"start":38,"end":46,"cssClass":"pl-s1"}],[{"start":8,"end":19,"cssClass":"pl-s1"},{"start":20,"end":22,"cssClass":"pl-c1"},{"start":23,"end":24,"cssClass":"pl-s1"},{"start":25,"end":30,"cssClass":"pl-en"}],[{"start":4,"end":9,"cssClass":"pl-s1"},{"start":10,"end":14,"cssClass":"pl-en"},{"start":15,"end":25,"cssClass":"pl-s1"},{"start":27,"end":29,"cssClass":"pl-s1"},{"start":30,"end":34,"cssClass":"pl-s1"},{"start":35,"end":39,"cssClass":"pl-en"},{"start":40,"end":54,"cssClass":"pl-s1"},{"start":56,"end":64,"cssClass":"pl-s1"}],[],[{"start":4,"end":19,"cssClass":"pl-c"}],[{"start":4,"end":14,"cssClass":"pl-s1"},{"start":15,"end":25,"cssClass":"pl-s"},{"start":27,"end":28,"cssClass":"pl-c1"},{"start":30,"end":42,"cssClass":"pl-s"},{"start":44,"end":55,"cssClass":"pl-s1"},{"start":56,"end":57,"cssClass":"pl-c1"},{"start":58,"end":59,"cssClass":"pl-c1"}],[{"start":4,"end":14,"cssClass":"pl-en"},{"start":15,"end":25,"cssClass":"pl-s1"},{"start":27,"end":29,"cssClass":"pl-s1"},{"start":30,"end":34,"cssClass":"pl-s1"},{"start":35,"end":39,"cssClass":"pl-en"},{"start":40,"end":54,"cssClass":"pl-s1"},{"start":56,"end":86,"cssClass":"pl-s"}],[{"start":4,"end":22,"cssClass":"pl-s1"},{"start":23,"end":24,"cssClass":"pl-c1"},{"start":25,"end":31,"cssClass":"pl-s1"},{"start":32,"end":52,"cssClass":"pl-s"},{"start":54,"end":56,"cssClass":"pl-k"},{"start":57,"end":77,"cssClass":"pl-s"},{"start":78,"end":80,"cssClass":"pl-c1"},{"start":81,"end":87,"cssClass":"pl-s1"},{"start":88,"end":92,"cssClass":"pl-k"},{"start":93,"end":94,"cssClass":"pl-c1"}],[{"start":4,"end":15,"cssClass":"pl-s1"},{"start":16,"end":17,"cssClass":"pl-c1"},{"start":18,"end":24,"cssClass":"pl-s1"},{"start":25,"end":38,"cssClass":"pl-s"},{"start":40,"end":42,"cssClass":"pl-k"},{"start":43,"end":56,"cssClass":"pl-s"},{"start":57,"end":59,"cssClass":"pl-c1"},{"start":60,"end":66,"cssClass":"pl-s1"},{"start":67,"end":71,"cssClass":"pl-k"},{"start":72,"end":75,"cssClass":"pl-c1"}],[{"start":4,"end":10,"cssClass":"pl-s1"},{"start":11,"end":12,"cssClass":"pl-c1"},{"start":13,"end":24,"cssClass":"pl-v"}],[{"start":8,"end":19,"cssClass":"pl-s1"},{"start":19,"end":20,"cssClass":"pl-c1"},{"start":20,"end":23,"cssClass":"pl-s1"}],[{"start":8,"end":25,"cssClass":"pl-s1"},{"start":25,"end":26,"cssClass":"pl-c1"},{"start":26,"end":51,"cssClass":"pl-en"},{"start":52,"end":55,"cssClass":"pl-s1"},{"start":57,"end":75,"cssClass":"pl-s1"},{"start":77,"end":88,"cssClass":"pl-s1"}],[{"start":8,"end":27,"cssClass":"pl-s1"},{"start":27,"end":28,"cssClass":"pl-c1"},{"start":28,"end":34,"cssClass":"pl-s1"},{"start":35,"end":44,"cssClass":"pl-s"}],[{"start":8,"end":25,"cssClass":"pl-s1"},{"start":25,"end":26,"cssClass":"pl-c1"},{"start":26,"end":32,"cssClass":"pl-s1"},{"start":33,"end":43,"cssClass":"pl-s"}],[{"start":8,"end":20,"cssClass":"pl-s1"},{"start":20,"end":21,"cssClass":"pl-c1"},{"start":21,"end":27,"cssClass":"pl-s1"},{"start":28,"end":38,"cssClass":"pl-s"}],[{"start":8,"end":27,"cssClass":"pl-s1"},{"start":27,"end":28,"cssClass":"pl-c1"},{"start":28,"end":47,"cssClass":"pl-s1"}],[{"start":8,"end":18,"cssClass":"pl-s1"},{"start":18,"end":19,"cssClass":"pl-c1"},{"start":19,"end":29,"cssClass":"pl-s1"}],[{"start":8,"end":18,"cssClass":"pl-s1"},{"start":18,"end":19,"cssClass":"pl-c1"},{"start":19,"end":23,"cssClass":"pl-s1"}],[{"start":8,"end":31,"cssClass":"pl-s1"},{"start":31,"end":32,"cssClass":"pl-c1"},{"start":32,"end":55,"cssClass":"pl-s1"}],[],[{"start":4,"end":10,"cssClass":"pl-s1"},{"start":11,"end":26,"cssClass":"pl-en"},{"start":27,"end":41,"cssClass":"pl-s1"}],[],[{"start":4,"end":55,"cssClass":"pl-c"}],[{"start":4,"end":7,"cssClass":"pl-k"},{"start":8,"end":18,"cssClass":"pl-s1"}],[{"start":4,"end":7,"cssClass":"pl-k"},{"start":8,"end":14,"cssClass":"pl-s1"}],[{"start":4,"end":6,"cssClass":"pl-s1"},{"start":7,"end":14,"cssClass":"pl-en"}],[],[{"start":4,"end":9,"cssClass":"pl-en"},{"start":10,"end":52,"cssClass":"pl-s"}],[{"start":4,"end":9,"cssClass":"pl-s1"},{"start":10,"end":11,"cssClass":"pl-c1"},{"start":12,"end":28,"cssClass":"pl-v"},{"start":29,"end":44,"cssClass":"pl-en"},{"start":45,"end":59,"cssClass":"pl-s1"},{"start":61,"end":72,"cssClass":"pl-s1"},{"start":72,"end":73,"cssClass":"pl-c1"},{"start":73,"end":78,"cssClass":"pl-s1"},{"start":79,"end":87,"cssClass":"pl-s1"},{"start":89,"end":106,"cssClass":"pl-s1"},{"start":106,"end":107,"cssClass":"pl-c1"},{"start":107,"end":111,"cssClass":"pl-c1"}],[{"start":4,"end":46,"cssClass":"pl-c"}],[{"start":4,"end":7,"cssClass":"pl-k"},{"start":8,"end":13,"cssClass":"pl-s1"},{"start":14,"end":20,"cssClass":"pl-s1"},{"start":21,"end":34,"cssClass":"pl-s1"}],[{"start":4,"end":9,"cssClass":"pl-s1"},{"start":10,"end":16,"cssClass":"pl-s1"},{"start":17,"end":28,"cssClass":"pl-s1"},{"start":29,"end":30,"cssClass":"pl-c1"},{"start":31,"end":36,"cssClass":"pl-s1"},{"start":37,"end":44,"cssClass":"pl-s1"}],[{"start":4,"end":9,"cssClass":"pl-en"},{"start":10,"end":46,"cssClass":"pl-s"}],[{"start":4,"end":9,"cssClass":"pl-s1"},{"start":10,"end":25,"cssClass":"pl-en"},{"start":26,"end":36,"cssClass":"pl-s1"},{"start":38,"end":56,"cssClass":"pl-s1"},{"start":56,"end":57,"cssClass":"pl-c1"},{"start":57,"end":75,"cssClass":"pl-s1"}],[{"start":4,"end":10,"cssClass":"pl-s1"},{"start":11,"end":17,"cssClass":"pl-en"},{"start":18,"end":32,"cssClass":"pl-s1"}],[],[],[{"start":0,"end":3,"cssClass":"pl-k"},{"start":4,"end":19,"cssClass":"pl-en"},{"start":20,"end":34,"cssClass":"pl-s1"},{"start":36,"end":56,"cssClass":"pl-s1"}],[{"start":4,"end":55,"cssClass":"pl-c"}],[{"start":4,"end":19,"cssClass":"pl-s1"},{"start":20,"end":21,"cssClass":"pl-c1"},{"start":22,"end":36,"cssClass":"pl-v"},{"start":37,"end":39,"cssClass":"pl-k"},{"start":40,"end":58,"cssClass":"pl-v"},{"start":59,"end":61,"cssClass":"pl-c1"},{"start":62,"end":66,"cssClass":"pl-c1"},{"start":67,"end":71,"cssClass":"pl-k"},{"start":72,"end":90,"cssClass":"pl-v"}],[{"start":4,"end":9,"cssClass":"pl-en"},{"start":10,"end":69,"cssClass":"pl-s"},{"start":21,"end":47,"cssClass":"pl-s1"},{"start":21,"end":22,"cssClass":"pl-kos"},{"start":22,"end":37,"cssClass":"pl-s1"},{"start":38,"end":46,"cssClass":"pl-s1"},{"start":46,"end":47,"cssClass":"pl-kos"},{"start":51,"end":67,"cssClass":"pl-s1"},{"start":51,"end":52,"cssClass":"pl-kos"},{"start":52,"end":66,"cssClass":"pl-s1"},{"start":66,"end":67,"cssClass":"pl-kos"}],[{"start":4,"end":13,"cssClass":"pl-s1"},{"start":14,"end":15,"cssClass":"pl-c1"},{"start":16,"end":31,"cssClass":"pl-en"},{"start":32,"end":52,"cssClass":"pl-s1"}],[{"start":4,"end":13,"cssClass":"pl-s1"},{"start":14,"end":29,"cssClass":"pl-en"},{"start":30,"end":44,"cssClass":"pl-s1"}],[],[],[{"start":0,"end":3,"cssClass":"pl-k"},{"start":4,"end":8,"cssClass":"pl-en"}],[{"start":4,"end":10,"cssClass":"pl-s1"},{"start":11,"end":12,"cssClass":"pl-c1"},{"start":13,"end":21,"cssClass":"pl-s1"},{"start":22,"end":36,"cssClass":"pl-v"}],[{"start":4,"end":10,"cssClass":"pl-s1"},{"start":11,"end":23,"cssClass":"pl-en"}],[{"start":8,"end":21,"cssClass":"pl-s"}],[{"start":8,"end":12,"cssClass":"pl-s1"},{"start":12,"end":13,"cssClass":"pl-c1"},{"start":13,"end":90,"cssClass":"pl-s"}],[],[{"start":4,"end":10,"cssClass":"pl-s1"},{"start":11,"end":23,"cssClass":"pl-en"}],[{"start":8,"end":22,"cssClass":"pl-s"}],[{"start":8,"end":15,"cssClass":"pl-s1"},{"start":15,"end":16,"cssClass":"pl-c1"},{"start":17,"end":21,"cssClass":"pl-s"},{"start":23,"end":28,"cssClass":"pl-s"},{"start":30,"end":35,"cssClass":"pl-s"},{"start":37,"end":43,"cssClass":"pl-s"},{"start":45,"end":50,"cssClass":"pl-s"},{"start":52,"end":57,"cssClass":"pl-s"},{"start":59,"end":64,"cssClass":"pl-s"},{"start":66,"end":71,"cssClass":"pl-s"},{"start":73,"end":79,"cssClass":"pl-s"},{"start":81,"end":97,"cssClass":"pl-s"}],[{"start":8,"end":12,"cssClass":"pl-s1"},{"start":12,"end":13,"cssClass":"pl-c1"},{"start":13,"end":202,"cssClass":"pl-s"}],[],[{"start":4,"end":10,"cssClass":"pl-s1"},{"start":11,"end":23,"cssClass":"pl-en"}],[{"start":8,"end":22,"cssClass":"pl-s"}],[{"start":8,"end":12,"cssClass":"pl-s1"},{"start":12,"end":13,"cssClass":"pl-c1"},{"start":13,"end":55,"cssClass":"pl-s"}],[],[{"start":4,"end":10,"cssClass":"pl-s1"},{"start":11,"end":23,"cssClass":"pl-en"},{"start":24,"end":46,"cssClass":"pl-s"},{"start":48,"end":52,"cssClass":"pl-s1"},{"start":52,"end":53,"cssClass":"pl-c1"},{"start":53,"end":57,"cssClass":"pl-s1"},{"start":59,"end":63,"cssClass":"pl-s1"},{"start":63,"end":64,"cssClass":"pl-c1"},{"start":64,"end":109,"cssClass":"pl-s"}],[{"start":4,"end":8,"cssClass":"pl-s1"},{"start":9,"end":10,"cssClass":"pl-c1"},{"start":11,"end":17,"cssClass":"pl-s1"},{"start":18,"end":28,"cssClass":"pl-en"}],[{"start":4,"end":12,"cssClass":"pl-s1"},{"start":13,"end":14,"cssClass":"pl-c1"},{"start":15,"end":17,"cssClass":"pl-s1"},{"start":18,"end":22,"cssClass":"pl-s1"},{"start":23,"end":27,"cssClass":"pl-en"},{"start":28,"end":32,"cssClass":"pl-s1"},{"start":33,"end":42,"cssClass":"pl-s1"},{"start":44,"end":61,"cssClass":"pl-s"}],[{"start":4,"end":6,"cssClass":"pl-k"},{"start":7,"end":11,"cssClass":"pl-s1"},{"start":12,"end":22,"cssClass":"pl-s1"},{"start":23,"end":25,"cssClass":"pl-c1"},{"start":26,"end":42,"cssClass":"pl-s"}],[{"start":8,"end":19,"cssClass":"pl-en"}],[{"start":12,"end":22,"cssClass":"pl-s1"},{"start":22,"end":23,"cssClass":"pl-c1"},{"start":23,"end":27,"cssClass":"pl-s1"},{"start":28,"end":38,"cssClass":"pl-s1"}],[{"start":12,"end":27,"cssClass":"pl-s1"},{"start":27,"end":28,"cssClass":"pl-c1"},{"start":28,"end":32,"cssClass":"pl-s1"},{"start":33,"end":42,"cssClass":"pl-s1"}],[{"start":12,"end":22,"cssClass":"pl-s1"},{"start":22,"end":23,"cssClass":"pl-c1"},{"start":23,"end":27,"cssClass":"pl-s1"},{"start":28,"end":38,"cssClass":"pl-s1"}],[{"start":12,"end":30,"cssClass":"pl-s1"},{"start":30,"end":31,"cssClass":"pl-c1"},{"start":31,"end":35,"cssClass":"pl-s1"},{"start":36,"end":54,"cssClass":"pl-s1"}],[{"start":12,"end":26,"cssClass":"pl-s1"},{"start":26,"end":27,"cssClass":"pl-c1"},{"start":27,"end":35,"cssClass":"pl-s1"}],[],[{"start":4,"end":8,"cssClass":"pl-k"}],[{"start":8,"end":23,"cssClass":"pl-en"},{"start":24,"end":28,"cssClass":"pl-s1"},{"start":29,"end":39,"cssClass":"pl-s1"},{"start":41,"end":49,"cssClass":"pl-s1"}],[],[],[{"start":0,"end":2,"cssClass":"pl-k"},{"start":3,"end":11,"cssClass":"pl-s1"},{"start":12,"end":14,"cssClass":"pl-c1"},{"start":15,"end":25,"cssClass":"pl-s"}],[{"start":4,"end":8,"cssClass":"pl-en"}]],"csv":null,"csvError":null,"dependabotInfo":{"showConfigurationBanner":false,"configFilePath":null,"networkDependabotPath":"/huggingface/transformers/network/updates","dismissConfigurationNoticePath":"/settings/dismiss-notice/dependabot_configuration_notice","configurationNoticeDismissed":null,"repoAlertsPath":"/huggingface/transformers/security/dependabot","repoSecurityAndAnalysisPath":"/huggingface/transformers/settings/security_analysis","repoOwnerIsOrg":true,"currentUserCanAdminRepo":false},"displayName":"convert_llama_weights_to_hf.py","displayUrl":"https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/convert_llama_weights_to_hf.py?raw=true","headerInfo":{"blobSize":"12.9 KB","deleteInfo":{"deleteTooltip":"You must be signed in to make or propose changes"},"editInfo":{"editTooltip":"You must be signed in to make or propose changes"},"ghDesktopPath":"https://desktop.github.com/","gitLfsPath":null,"onBranch":true,"shortPath":"8b59307","siteNavLoginPath":"/login?return_to=https%3A%2F%2Fgithub.com%2Fhuggingface%2Ftransformers%2Fblob%2Fmain%2Fsrc%2Ftransformers%2Fmodels%2Fllama%2Fconvert_llama_weights_to_hf.py","isCSV":false,"isRichtext":false,"toc":null,"lineInfo":{"truncatedLoc":"318","truncatedSloc":"279"},"mode":"file"},"image":false,"isCodeownersFile":null,"isPlain":false,"isValidLegacyIssueTemplate":false,"issueTemplateHelpUrl":"https://docs.github.com/articles/about-issue-and-pull-request-templates","issueTemplate":null,"discussionTemplate":null,"language":"Python","languageID":303,"large":false,"loggedIn":false,"newDiscussionPath":"/huggingface/transformers/discussions/new","newIssuePath":"/huggingface/transformers/issues/new","planSupportInfo":{"repoIsFork":null,"repoOwnedByCurrentUser":null,"requestFullPath":"/huggingface/transformers/blob/main/src/transformers/models/llama/convert_llama_weights_to_hf.py","showFreeOrgGatedFeatureMessage":null,"showPlanSupportBanner":null,"upgradeDataAttributes":null,"upgradePath":null},"publishBannersInfo":{"dismissActionNoticePath":"/settings/dismiss-notice/publish_action_from_dockerfile","dismissStackNoticePath":"/settings/dismiss-notice/publish_stack_from_file","releasePath":"/huggingface/transformers/releases/new?marketplace=true","showPublishActionBanner":false,"showPublishStackBanner":false},"rawBlobUrl":"https://github.com/huggingface/transformers/raw/main/src/transformers/models/llama/convert_llama_weights_to_hf.py","renderImageOrRaw":false,"richText":null,"renderedFileInfo":null,"shortPath":null,"tabSize":8,"topBannersInfo":{"overridingGlobalFundingFile":false,"globalPreferredFundingPath":null,"repoOwner":"huggingface","repoName":"transformers","showInvalidCitationWarning":false,"citationHelpUrl":"https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/creating-a-repository-on-github/about-citation-files","showDependabotConfigurationBanner":false,"actionsOnboardingTip":null},"truncated":false,"viewable":true,"workflowRedirectUrl":null,"symbols":{"timedOut":false,"notAnalyzed":false,"symbols":[{"name":"NUM_SHARDS","kind":"constant","identStart":1756,"identEnd":1766,"extentStart":1756,"extentEnd":1899,"fullyQualifiedName":"NUM_SHARDS","identUtf16":{"start":{"lineNumber":55,"utf16Col":0},"end":{"lineNumber":55,"utf16Col":10}},"extentUtf16":{"start":{"lineNumber":55,"utf16Col":0},"end":{"lineNumber":65,"utf16Col":1}}},{"name":"compute_intermediate_size","kind":"function","identStart":1906,"identEnd":1931,"extentStart":1902,"extentEnd":2077,"fullyQualifiedName":"compute_intermediate_size","identUtf16":{"start":{"lineNumber":68,"utf16Col":4},"end":{"lineNumber":68,"utf16Col":29}},"extentUtf16":{"start":{"lineNumber":68,"utf16Col":0},"end":{"lineNumber":69,"utf16Col":102}}},{"name":"read_json","kind":"function","identStart":2084,"identEnd":2093,"extentStart":2080,"extentEnd":2159,"fullyQualifiedName":"read_json","identUtf16":{"start":{"lineNumber":72,"utf16Col":4},"end":{"lineNumber":72,"utf16Col":13}},"extentUtf16":{"start":{"lineNumber":72,"utf16Col":0},"end":{"lineNumber":74,"utf16Col":27}}},{"name":"write_json","kind":"function","identStart":2166,"identEnd":2176,"extentStart":2162,"extentEnd":2247,"fullyQualifiedName":"write_json","identUtf16":{"start":{"lineNumber":77,"utf16Col":4},"end":{"lineNumber":77,"utf16Col":14}},"extentUtf16":{"start":{"lineNumber":77,"utf16Col":0},"end":{"lineNumber":79,"utf16Col":26}}},{"name":"write_model","kind":"function","identStart":2254,"identEnd":2265,"extentStart":2250,"extentEnd":11574,"fullyQualifiedName":"write_model","identUtf16":{"start":{"lineNumber":82,"utf16Col":4},"end":{"lineNumber":82,"utf16Col":15}},"extentUtf16":{"start":{"lineNumber":82,"utf16Col":0},"end":{"lineNumber":275,"utf16Col":33}}},{"name":"permute","kind":"function","identStart":4006,"identEnd":4013,"extentStart":4002,"extentEnd":4152,"fullyQualifiedName":"permute","identUtf16":{"start":{"lineNumber":121,"utf16Col":8},"end":{"lineNumber":121,"utf16Col":15}},"extentUtf16":{"start":{"lineNumber":121,"utf16Col":4},"end":{"lineNumber":122,"utf16Col":97}}},{"name":"write_tokenizer","kind":"function","identStart":11581,"identEnd":11596,"extentStart":11577,"extentEnd":11953,"fullyQualifiedName":"write_tokenizer","identUtf16":{"start":{"lineNumber":278,"utf16Col":4},"end":{"lineNumber":278,"utf16Col":19}},"extentUtf16":{"start":{"lineNumber":278,"utf16Col":0},"end":{"lineNumber":283,"utf16Col":45}}},{"name":"main","kind":"function","identStart":11960,"identEnd":11964,"extentStart":11956,"extentEnd":13181,"fullyQualifiedName":"main","identUtf16":{"start":{"lineNumber":286,"utf16Col":4},"end":{"lineNumber":286,"utf16Col":8}},"extentUtf16":{"start":{"lineNumber":286,"utf16Col":0},"end":{"lineNumber":313,"utf16Col":50}}}]}},"copilotInfo":null,"copilotAccessAllowed":false,"csrf_tokens":{"/huggingface/transformers/branches":{"post":"XF8MjM3ylgJS5dalzAohNn56bP4MCtKdmMZbVD6Fyf1FzYObxppJQGUbnBiLeSWsYdO6Km3DOFOtF8RMoAEQqw"},"/repos/preferences":{"post":"rgD7T5J7WGWcJNt43l_HK6uQ5TuFDODJB3KN1R9Jy-niApRGHBDmVdv2NXBMZAZWs1E3sYL5LfmRjPUxYYfEqQ"}}},"title":"transformers/src/transformers/models/llama/convert_llama_weights_to_hf.py at main · huggingface/transformers"}
NameError: name 'false' is not defined. Did you mean: 'False'?