lonePatient / albert_pytorch

A Lite Bert For Self-Supervised Learning Language Representations
Apache License 2.0
709 stars 152 forks source link

使用albert.base(英文)finetuning的时候,--gradient_accumulation_steps设置为大于1时直接进入evaluating而不training #24

Open YuxiangLu opened 4 years ago

lonePatient commented 4 years ago

@YuxiangLu 你的意思是说当step 小于gradient_accumulation_steps时,简单看了样,训练跟eval应该都同时进行了,应该时:

            if args.local_rank in [-1, 0] and args.logging_steps > 0 and global_step % args.logging_steps == 0:
                #Log metrics
                if args.local_rank == -1:  # Only evaluate when single GPU otherwise metrics may not average well
                    results = evaluate(args, model, tokenizer)

对结果应该没多大影响,等下我改下,谢了。