search

lessw2020 / Ranger21

Ranger deep learning optimizer rewrite to use newest components
Apache License 2.0
321 stars 45 forks source link

error in warmdown - lr below min lr. current lr = 2.999999999999997e-0518 [07:50<00:04, 4.66s/it] auto handling but please report issue! #28

Open neuronflow opened 3 years ago

neuronflow commented 3 years ago

not sure how to interpret this:

                                                                                                error in warmdown - lr
below min lr. current lr = 2.999999999999997e-0518 [07:50<00:04,  4.66s/it]
auto handling but please report issue!
raff-s commented 3 years ago

got similar issue, my setup

train_dl = model_type.train_dl(train_ds, batch_size=16, num_workers=2, shuffle=True)
valid_dl = model_type.valid_dl(valid_ds, batch_size=16, num_workers=2, shuffle=False)
metrics = [COCOMetric(metric_type=COCOMetricType.bbox)]
n_epochs = 20
opt_func = partial(optimizer.OptimWrapper, opt=Ranger21, num_epochs=n_epochs, num_batches_per_epoch=(len(train_dl))
learn = model_type.fastai.learner(dls=[train_dl, valid_dl], model=model, metrics=metrics, cbs=cbs,opt_func=opt_func)
learn.fit_one_cycle(n_epochs, cbs=cbs).to_fp16

output

error in warmdown - lr below min lr. current lr = 2.999999999999997e-05 auto handling but please report issue!

TolerantChief commented 1 year ago

Did you ever fix this problem? I've got a similar issue and I don't know how to fix it.

TRAIN;18;38.92721739355123;0.6500738552437223;91.424s TEST;18;39.037711521364606;0.6040189125295509;11.692s error in warmdown pct calc. new pct = 1.000928505106778 auto handled but please report issue error in warmdown pct calc. new pct = 1.0018570102135562 auto handled but please report issue error in warmdown pct calc. new pct = 1.0027855153203342 auto handled but please report issue