Closed Stannislav closed 2 years ago
@FrancescoCasalegno I'm happy to discuss should there be anything obscure in the changes I made.
LGTM @Stannislav!
This PR is quite large, and maybe in the future we could try to create several smaller PRs to address individual issues, otherwise the reviewing process is a bit complex (11 refactoring changes + 63 files changed = š±).
I tried to document everything in the PR description, and I went through each file to make sure not to miss any issue.
Hopefully we didn't break anything!
Yes, I agree, this was a big one. Partially the reason was that things depended on each other, so they had to be done together as to not leave main
in a broken state. Good thing is most of the changes are just deletions of modules :)
Fixes #21 as part of the clean up.
Description
1.
morphoclass.serialization
serialization
module - unused2.
morphoclass.feature_extractors
>>morphoclass.features
Rename
morphoclass.feature_extractors
asmorphoclass.features
. Also, most of its submodules are removed.feature_extractors.feature_extraction
Removed. It is superseded by the code in the newmorphoclass extract-features
command.feature_extractors.{interneurons,pyramidal_cells}
Removed. They were used to extract features in the oldtrain
CLI command. Since then we changed the way we extract features - now they're pre-extracted using theextract-features
CLI command and the corresponding functionality has been ported from the deleted modules toconsole.cmd_extract_features
. This rendered two modules in question obsolete and it was safe to delete them. In the future one should probably move the feature extraction code fromconsole.cmd_extract_features
to a more appropriate module.feature_extractors.non_graph
Renamed asfeatures.non_graph
.4.
data_preparation
/data-preparation
data-preparation
used to be a DVC stage that got replaced bypreprocess-dataset
dvc.yaml
in the past), so:data_preparation
) fromparams.yaml
anddvc.lock
.gitignore
fordata_preparation/
folderdatasets
is removed fromparams.yaml
. It was used by the stagedata-preparation
and it was already not used anywhere in thedvc.yaml
before this PR.5.
dvc/models/
dvc/models
is where checkpoints used to be in the past, now they're undertraining/checkpoints
, so:params.yaml
anddvc.lock
.gitignore
for folders insidedvc/models/
6.
explain_models
/explain-models
explain-models
used to be a DVC stage that got replaced byxai
(i.e. even before this PR it does not exist anymore in thesrc/
) so:params.yaml
,dvc.yaml
, anddvc.lock
.7.
morphoclass.augmentation
morphoclass.augmentation
was already deprecated in favor of the newmorphoclass.transforms
.morphoclass.augmentation
.morphoclass.augmentation.finding_point_on_branch()
is renamed and moved tomorphoclass.utils.find_point_on_branch()
.8.
dvc/README.md
dvc/README.md
: it contained information about how to run DVC stages, but most of those stages are gone and the information is not useful any more.9. Generation of
params.yaml
and.yaml
training config filesdvc/generate_params.py
- was used in the past to generate training configurations inparams.yaml
; after refactoring we now use separate config files for training configurationdvc/training/make_configs.py
- was used to auto-generate the training config files undertraining/configs
; it was intended to be temporary; the configs have evolved and this script is outdated; given there aren't that many configs it's best to edit them by hand10.
morphoclass performance-report
CLI commandperformance-report
CLI command - it got split and replaced by the following commandsperformance-table
evaluate performance
evaluate latent-features
evaluate outliers
dvc.yaml
,dvc.lock
,params.yaml
+ remove the output directoryreports/performance-report
from the.gitignore
.11.
MorphologyEmbeddingDataset
andMorphologyEmbeddingDataLoader
MorphologyEmbedding{Dataset,DataLoader}
classes - they got superseded by theMorphology{Dataset,DataLoader}
classes which now can handle all kind of data: morphologies, images, diagrams.MorphologyDataset
was only for morphologies andMorphologyEmbeddingDataset
for diagrams and images. Since these two classes were for the most part duplicates of each other it made sense to merge them into one.morphoclass.data.morphology_embedding_data_loader
morphoclass.data.morphology_embedding_dataset
morphology_embedding_*
submodules were used nowhere (seegit grep -rn "morphology_embedding_"
), except for:morphology_dataset.py
ā in methodMorphologyDataset.to_lowerdim_dataset()
, which was only used in the now removedfeature_extractors.{interneurons,pyramidal_cells}
.feature_extractors.{interneurons,pyramidal_cells}.py
ā which have been removedChecklist
setup.cfg
,requirements.txt
, andconstraints.txt
updated with new dependencies. (if needed)