Closed ultmaster closed 1 year ago
Notice that the training service is being refactored. Should we consider fix #514?
lr_scheduler
, max_steps
, max_epochs
work well in Evaluator.train(...)
. #4950Evaluator
) (deprecation warning) #4992TorchEvaluator
and LightningEvaluator
work well in all pruners #4950 #4992 #5016balance
mode in LevelPruner
#5033dependence aware
mode only generate the same channel-wise masks if two modules have dependence in related pruners #4992global
mode with key max_sparsity_per_layer
can indeed limit the maximum sparsity in related pruners #5017MovementPruner
new API training_epochs
training_steps
regular_scale
movement_mode
sparse_granularity
#5017Linear
, LayerNorm
in both [batch, hidden] and [batch, seq, hidden] senior) #4925 #5017known issue: activation based pruner can not support [batch, seq, hidden]
fit_kwargs
in lightning evaluator.drop_path
and auxiliary_loss
in NASNet.export_probs
to monitor the architecture weights.weighted_sum
.lr_scheduler
, max_steps
, max_epochs
work well in Evaluator.train(...)
. #4950 @QuanluZhangEvaluator
) (deprecation warning) #4992 @QuanluZhangTorchEvaluator
and LightningEvaluator
work well in all pruners #4950 #4992 #5016 @QuanluZhangbalance
mode in LevelPruner
#5033 @zheng-ningxindependence aware
mode only generate the same channel-wise masks if two modules have dependence in related pruners #4992 @Louis-Jglobal
mode with key max_sparsity_per_layer
can indeed limit the maximum sparsity in related pruners #5017 @Louis-JMovementPruner
new API training_epochs
training_steps
regular_scale
movement_mode
sparse_granularity
#5017 @Louis-J Linear
, LayerNorm
in both [batch, hidden] and [batch, seq, hidden] senior) #4925 #5017 @J-shangknown issue: activation based pruner can not support [batch, seq, hidden] replace doc-compression-overview-Compression Pipeline-picture
adaptive_parzen_normal
of TPE. @liuzhe-lz[x] Frameworkcontroller + NFS
[ ] Need more comprehensive tests (either pipeline or manual combinations)
[x] Fix trialConcurrency bug, use ${envId}_run.sh
to replace run.sh
Fix upload dir bug in K8S training service: use a separate working directory for each experiment. @liuzhe-lz
Fix trialConcurrency bug, use ${envId}_run.sh
to replace run.sh. Thanks external contributor @liuzhe-lz
nni.retiarii
code-base to nni.nas
.fit_kwargs
in lightning evaluator (doc)drop_path
and auxiliary_loss
in NASNet (doc)export_probs
to monitor the architecture weights.nni.retiarii
code-base to nni.nas
.weighted_sum
.TorchEvaluator
, LightningEvaluator
, TransformersEvaluator
for easier passing training logic to pruner (doc, API)Evaluator
, the old API is still remained and will be deprecated in v3.0 (doc)lr_scheduler
in pruning by using Evaluator
.ActivationAPoZRankPruner
and ActivationMeanRankPruner
.training_steps
, regular_scale
, movement_mode
, sparse_granularity
for MovementPruner
(doc)GroupNorm
replacement in pruning speedup. Thanks external contributor @cin-xingbalance
mode performance in LevelPruner
.dependency_aware
mode invalid in scheduled pruners.bias
mask cannot be generated.max_sparsity_per_layer
not take effect.Linear
and LayerNorm
speedup replacement in NLP task.LightningModule
failed in pytorch_lightning >= 1.7.0
adaptive_parzen_normal
of TPE.${envId}_run.sh
to replace run.sh
None
rand_like_with_shape
is easy to overflow when dtype=torch.int8
.
Top selling points
Other nominations
Other items
NAS
Compression @J-shang
Speedup @louis-j
Quantization
HPO @liuzhe-lz
Experiment @liuzhe-lz
WebUI
P0 - 15 days
Training service
Pipelines
Deferred from last release