2024/05/24 22:12:16 - mmengine - WARNING - Failed to search registry with scope "mmengine" in the "builder" registry tree. As a workaround, the current "builder" registry in "xtuner" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmengine" is a correct scope, or whether the registry is initialized.
2024/05/24 22:12:15 - mmengine - INFO -
System environment: sys.platform: linux Python: 3.12.3 (main, Apr 10 2024, 05:33:47) [GCC 13.2.0] CUDA available: True MUSA available: False numpy_random_seed: 1024 GPU 0,1: NVIDIA GeForce RTX 4090 CUDA_HOME: /usr/local/cuda NVCC: Cuda compilation tools, release 12.1, V12.1.66 GCC: x86_64-linux-gnu-gcc (Ubuntu 13.2.0-23ubuntu4) 13.2.0 PyTorch: 2.3.0+cu121 PyTorch compiling details: PyTorch built with:
Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=8.9.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.3.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=1, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF,
TorchVision: 0.18.0+cu121 OpenCV: 4.9.0 MMEngine: 0.10.4
Runtime environment: launcher: none randomness: {'seed': 1024, 'deterministic': False} cudnn_benchmark: False mp_cfg: {'mp_start_method': 'fork', 'opencv_num_threads': 0} dist_cfg: {'backend': 'nccl'} seed: 1024 deterministic: False Distributed launcher: none Distributed training: False GPU number: 1
2024/05/24 22:12:15 - mmengine - INFO - Config: SYSTEM = '' accumulative_counts = 128 batch_size = 1 betas = ( 0.9, 0.999, ) custom_hooks = [ dict( tokenizer=dict( padding_side='right', pretrained_model_name_or_path='./Meta-Llama-3-8B-Instruct', trust_remote_code=True, type='transformers.AutoTokenizer.from_pretrained'), type='xtuner.engine.hooks.DatasetInfoHook'), dict( evaluation_images='https://llava-vl.github.io/static/images/view.jpg', evaluation_inputs=[ '请描述一下这张照片', 'Please describe this picture', ], every_n_iters=50000, image_processor=dict( pretrained_model_name_or_path='./clip-vit-large-patch14-336', trust_remote_code=True, type='transformers.CLIPImageProcessor.from_pretrained'), prompt_template='xtuner.utils.PROMPT_TEMPLATE.llama3_chat', system='', tokenizer=dict( padding_side='right', pretrained_model_name_or_path='./Meta-Llama-3-8B-Instruct', trust_remote_code=True, type='transformers.AutoTokenizer.from_pretrained'), type='xtuner.engine.hooks.EvaluateChatHook'), ] data_path = './data/llava_data/LLaVA-Instruct-150K/llava_v1_5_mix665k.json' data_root = './data/llava_data/' dataloader_num_workers = 0 default_hooks = dict( checkpoint=dict( by_epoch=False, interval=50000, max_keep_ckpts=2, type='mmengine.hooks.CheckpointHook'), logger=dict( interval=10, log_metric_by_epoch=False, type='mmengine.hooks.LoggerHook'), param_scheduler=dict(type='mmengine.hooks.ParamSchedulerHook'), sampler_seed=dict(type='mmengine.hooks.DistSamplerSeedHook'), timer=dict(type='mmengine.hooks.IterTimerHook')) env_cfg = dict( cudnn_benchmark=False, dist_cfg=dict(backend='nccl'), mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0)) evaluation_freq = 50000 evaluation_images = 'https://llava-vl.github.io/static/images/view.jpg' evaluation_inputs = [ '请描述一下这张照片', 'Please describe this picture', ] image_folder = './data/llava_data/llava_images' image_processor = dict( pretrained_model_name_or_path='./clip-vit-large-patch14-336', trust_remote_code=True, type='transformers.CLIPImageProcessor.from_pretrained') launcher = 'none' llava_dataset = dict( data_path='./data/llava_data/LLaVA-Instruct-150K/llava_v1_5_mix665k.json', dataset_map_fn='xtuner.dataset.map_fns.llava_map_fn', image_folder='./data/llava_data/llava_images', image_processor=dict( pretrained_model_name_or_path='./clip-vit-large-patch14-336', trust_remote_code=True, type='transformers.CLIPImageProcessor.from_pretrained'), max_length=1472, pad_image_to_square=True, template_map_fn=dict( template='xtuner.utils.PROMPT_TEMPLATE.llama3_chat', type='xtuner.dataset.map_fns.template_map_fn_factory'), tokenizer=dict( padding_side='right', pretrained_model_name_or_path='./Meta-Llama-3-8B-Instruct', trust_remote_code=True, type='transformers.AutoTokenizer.from_pretrained'), type='xtuner.dataset.LLaVADataset') llm_name_or_path = './Meta-Llama-3-8B-Instruct' load_from = None log_level = 'INFO' log_processor = dict(by_epoch=False) lr = 0.0002 max_epochs = 1 max_length = 1472 max_norm = 1 model = dict( freeze_llm=True, freeze_visual_encoder=True, llm=dict( pretrained_model_name_or_path='./Meta-Llama-3-8B-Instruct', quantization_config=dict( bnb_4bit_compute_dtype='torch.float16', bnb_4bit_quant_type='nf4', bnb_4bit_use_double_quant=True, llm_int8_has_fp16_weight=False, llm_int8_threshold=6.0, load_in_4bit=True, load_in_8bit=False, type='transformers.BitsAndBytesConfig'), torch_dtype='torch.float16', trust_remote_code=True, type='transformers.AutoModelForCausalLM.from_pretrained'), llm_lora=dict( bias='none', lora_alpha=16, lora_dropout=0.05, r=64, task_type='CAUSAL_LM', type='peft.LoraConfig'), pretrained_pth= './work_dirs/llava_llama3_8b_instruct_quant_clip_vit_large_p14_336_e1_gpu1_pretrain/iter_558128.pth', type='xtuner.model.LLaVAModel', visual_encoder=dict( pretrained_model_name_or_path='./clip-vit-large-patch14-336', type='transformers.CLIPVisionModel.from_pretrained')) optim_type = 'torch.optim.AdamW' optim_wrapper = dict( optimizer=dict( betas=( 0.9, 0.999, ), lr=0.0002, type='torch.optim.AdamW', weight_decay=0), type='DeepSpeedOptimWrapper') param_scheduler = [ dict( begin=0, by_epoch=True, convert_to_iter_based=True, end=0.03, start_factor=1e-05, type='mmengine.optim.LinearLR'), dict( begin=0.03, by_epoch=True, convert_to_iter_based=True, end=1, eta_min=0.0, type='mmengine.optim.CosineAnnealingLR'), ] pretrained_pth = './work_dirs/llava_llama3_8b_instruct_quant_clip_vit_large_p14_336_e1_gpu1_pretrain/iter_558128.pth' prompt_template = 'xtuner.utils.PROMPT_TEMPLATE.llama3_chat' randomness = dict(deterministic=False, seed=1024) resume = False runner_type = 'FlexibleRunner' save_steps = 50000 save_total_limit = 2 strategy = dict( config=dict( bf16=dict(enabled=True), fp16=dict(enabled=False, initial_scale_power=16), gradient_accumulation_steps='auto', gradient_clipping='auto', train_micro_batch_size_per_gpu='auto', zero_allow_untested_optimizer=True, zero_force_ds_cpu_optimizer=False, zero_optimization=dict(overlap_comm=True, stage=2)), exclude_frozen_parameters=True, gradient_accumulation_steps=128, gradient_clipping=1, train_micro_batch_size_per_gpu=1, type='xtuner.engine.DeepSpeedStrategy') tokenizer = dict( padding_side='right', pretrained_model_name_or_path='./Meta-Llama-3-8B-Instruct', trust_remote_code=True, type='transformers.AutoTokenizer.from_pretrained') train_cfg = dict(max_epochs=1, type='xtuner.engine.runner.TrainLoop') train_dataloader = dict( batch_size=1, collate_fn=dict(type='xtuner.dataset.collate_fns.default_collate_fn'), dataset=dict( data_path= './data/llava_data/LLaVA-Instruct-150K/llava_v1_5_mix665k.json', dataset_map_fn='xtuner.dataset.map_fns.llava_map_fn', image_folder='./data/llava_data/llava_images', image_processor=dict( pretrained_model_name_or_path='./clip-vit-large-patch14-336', trust_remote_code=True, type='transformers.CLIPImageProcessor.from_pretrained'), max_length=1472, pad_image_to_square=True, template_map_fn=dict( template='xtuner.utils.PROMPT_TEMPLATE.llama3_chat', type='xtuner.dataset.map_fns.template_map_fn_factory'), tokenizer=dict( padding_side='right', pretrained_model_name_or_path='./Meta-Llama-3-8B-Instruct', trust_remote_code=True, type='transformers.AutoTokenizer.from_pretrained'), type='xtuner.dataset.LLaVADataset'), num_workers=0, sampler=dict( length_property='modality_length', per_device_batch_size=128, type='xtuner.dataset.samplers.LengthGroupedSampler')) visual_encoder_name_or_path = './clip-vit-large-patch14-336' visualizer = None warmup_ratio = 0.03 weight_decay = 0 work_dir = './work_dirs/llava_llama3_8b_instruct_qlora_clip_vit_large_p14_336_e1_gpu1_finetune'
2024/05/24 22:12:16 - mmengine - WARNING - Failed to search registry with scope "mmengine" in the "builder" registry tree. As a workaround, the current "builder" registry in "xtuner" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmengine" is a correct scope, or whether the registry is initialized.