Training notebook

[juancopi81]

Hi Kinyugo,

I find this really interesting, I was hoping to train the code on some of my own audio files. I notices you have a notebook for that. There's this "data_dir": "",

I wanted to ask:

• What types of files should I add here? (mpe would work?)
• How many files would be good?
• Should I use chunks of any length? Or a complete audio file of, say, 5 minutes is ok?

Thanks!

[Kinyugo]

Hello,

You can add any type of audio that is supported by torchaudio. Not sure about the exhaustive list of audio types supported.

There is no limit to the number of files you can use.

Audio is chunked and padded automatically by setting the num_frames parameter, no need to chunk it manually.

[juancopi81]

Hi @Kinyugo,

Thank you very much for you kind and quick response. Sorry for bothering you, I was today giving it a try to the Jupyter Notebook you have in the repo. I keep getting the same error:

ContextualVersionConflict: (Pygments 2.6.1 (/usr/local/lib/python3.8/dist-packages), requirement.parse('pygments<3.0.0,>=2.14.0'), {'rich'})

No matter if I install the package from git and in edit mode. Would you know how could one deal with this? Thank you very much. Below the complete Traceback:

`--------------------------------------------------------------------------- ContextualVersionConflict Traceback (most recent call last) in 3 4 from msanii.config import TrainingConfig ----> 5 from msanii.scripts import run_training

15 frames /usr/local/lib/python3.8/dist-packages/msanii/scripts/init.py in ----> 1 from .inference import ( 2 run_audio2audio, 3 run_inpainting, 4 run_interpolation, 5 run_outpainting,

/usr/local/lib/python3.8/dist-packages/msanii/scripts/inference.py in 17 SamplingConfig, 18 ) ---> 19 from ..data import AudioDataModule 20 from ..pipeline import Pipeline 21 from ..utils import compute_divisible_length

/usr/local/lib/python3.8/dist-packages/msanii/data/init.py in ----> 1 from .audio_datamodule import AudioDataModule

/usr/local/lib/python3.8/dist-packages/msanii/data/audio_datamodule.py in 1 from typing import Optional 2 ----> 3 import lightning as L 4 from torch.utils.data import DataLoader 5

/usr/local/lib/python3.8/dist-packages/lightning/init.py in 39 from lightning.fabric.fabric import Fabric # noqa: E402 40 from lightning.fabric.utilities.seed import seed_everything # noqa: E402 ---> 41 from lightning.pytorch.callbacks import Callback # noqa: E402 42 from lightning.pytorch.core import LightningDataModule, LightningModule # noqa: E402 43 from lightning.pytorch.trainer import Trainer # noqa: E402

/usr/local/lib/python3.8/dist-packages/lightning/pytorch/init.py in 33 34 from lightning.fabric.utilities.seed import seed_everything # noqa: E402 ---> 35 from lightning.pytorch.callbacks import Callback # noqa: E402 36 from lightning.pytorch.core import LightningDataModule, LightningModule # noqa: E402 37 from lightning.pytorch.trainer import Trainer # noqa: E402

/usr/local/lib/python3.8/dist-packages/lightning/pytorch/callbacks/init.py in 12 # See the License for the specific language governing permissions and 13 # limitations under the License. ---> 14 from lightning.pytorch.callbacks.batch_size_finder import BatchSizeFinder 15 from lightning.pytorch.callbacks.callback import Callback 16 from lightning.pytorch.callbacks.checkpoint import Checkpoint

/usr/local/lib/python3.8/dist-packages/lightning/pytorch/callbacks/batch_size_finder.py in 22 23 import lightning.pytorch as pl ---> 24 from lightning.pytorch.callbacks.callback import Callback 25 from lightning.pytorch.tuner.batch_size_scaling import scale_batch_size 26 from lightning.pytorch.utilities.exceptions import _TunerExitException, MisconfigurationException

/usr/local/lib/python3.8/dist-packages/lightning/pytorch/callbacks/callback.py in 23 24 import lightning.pytorch as pl ---> 25 from lightning.pytorch.utilities.types import STEP_OUTPUT 26 27

/usr/local/lib/python3.8/dist-packages/lightning/pytorch/utilities/init.py in 21 from lightning.pytorch.utilities.enums import GradClipAlgorithmType # noqa: F401 22 from lightning.pytorch.utilities.grads import grad_norm # noqa: F401 ---> 23 from lightning.pytorch.utilities.imports import ( # noqa: F401 24 _HIVEMIND_AVAILABLE, 25 _OMEGACONF_AVAILABLE,

/usr/local/lib/python3.8/dist-packages/lightning/pytorch/utilities/imports.py in 30 _OMEGACONF_AVAILABLE = package_available("omegaconf") 31 _PSUTIL_AVAILABLE = package_available("psutil") ---> 32 _RICH_AVAILABLE = package_available("rich") and compare_version("rich", operator.ge, "10.2.2") 33 _TORCH_QUANTIZE_AVAILABLE = bool([eg for eg in torch.backends.quantized.supported_engines if eg != "none"]) 34 _TORCHVISION_AVAILABLE = RequirementCache("torchvision")

/usr/local/lib/python3.8/dist-packages/lightning_utilities/core/imports.py in compare_version(package, op, version, use_base_version) 75 else: 76 # try pkg_resources to infer version ---> 77 pkg_version = Version(pkg_resources.get_distribution(package).version) 78 except TypeError: 79 # this is mocked by Sphinx, so it should return True to generate all summaries

/usr/local/lib/python3.8/dist-packages/pkg_resources/init.py in get_distribution(dist) 464 dist = Requirement.parse(dist) 465 if isinstance(dist, Requirement): --> 466 dist = get_provider(dist) 467 if not isinstance(dist, Distribution): 468 raise TypeError("Expected string, Requirement, or Distribution", dist)

/usr/local/lib/python3.8/dist-packages/pkg_resources/init.py in get_provider(moduleOrReq) 340 """Return an IResourceProvider for the named module or requirement""" 341 if isinstance(moduleOrReq, Requirement): --> 342 return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0] 343 try: 344 module = sys.modules[moduleOrReq]

/usr/local/lib/python3.8/dist-packages/pkg_resources/init.py in require(self, *requirements) 884 included, even if they were already activated in this working set. 885 """ --> 886 needed = self.resolve(parse_requirements(requirements)) 887 888 for dist in needed:

/usr/local/lib/python3.8/dist-packages/pkg_resources/init.py in resolve(self, requirements, env, installer, replace_conflicting, extras) 775 # Oops, the "best" so far conflicts with a dependency 776 dependent_req = required_by[req] --> 777 raise VersionConflict(dist, req).with_context(dependent_req) 778 779 # push the new requirements onto the stack

ContextualVersionConflict: (Pygments 2.6.1 (/usr/local/lib/python3.8/dist-packages), Requirement.parse('pygments<3.0.0,>=2.14.0'), {'rich'})`

[juancopi81]

And if I pass that, I received the following error:

default_config = OmegaConf.structured(TrainingConfig)
custom_config = OmegaConf.create(dict_config)
config = OmegaConf.merge(default_config, custom_config)
ConfigKeyError: Key 'transforms' not in 'DiffusionTrainingConfig'
    full_key: diffusion.transforms
    reference_type=DiffusionTrainingConfig
    object_type=DiffusionTrainingConfig

[Kinyugo]

Hello Juan,

I have fixed the issue with the transforms. You have to delete the transforms keys from the dictionary. However, I recommend that you reinstall the new repo as it fixes some other issues.

Let me know if you have any other concerns.

PS: The model was originally trained using a custom implementation and not diffusers lib and then ported over to diffusers there might be a few bugs here and there.

Regards, Kinyugo Maina

[juancopi81]

Hello @Kinyugo,

Great! Now it is working. Thanks you very much. It is running and logging the results in W&B. Can I use this notebook to resume training from the last checkpoint saved in W&B? How?

Thanks again :)

[juancopi81]

Oh BTY I trained a model and download a cpkt from W&B. Then I copied your HF space and change the checkpoint (I am unsure if I am copying the right file: I am using the checkpoints in the artifacts)

But now it is getting this error:

Traceback (most recent call last):
  File "app.py", line 20, in <module>
    demo = run_demo(config)
  File "/home/user/app/src/msanii/msanii/demo/demo.py", line 51, in run_demo
    pipeline = Pipeline.from_pretrained(config.ckpt_path)
  File "/home/user/app/src/msanii/msanii/pipeline/pipeline.py", line 218, in from_pretrained
    transforms = Pipeline._load_from_checkpoint(
  File "/home/user/app/src/msanii/msanii/pipeline/pipeline.py", line 248, in _load_from_checkpoint
    target_instance = from_config(checkpoint[f"{prefix}_config"], target)
KeyError: 'transforms_config'

[Kinyugo]

Hello,

I am not sure what could be causing the problem. One way to find out if your checkpoint has the transforms_config keys is to load it and inspect the keys.

import torch 

checkpoint = torch.load("path/to/ckpt",  map_location=torch.device("cpu"))
print(checkpoint.keys()) # should have a transforms config key

[juancopi81]

Hello,

Thanks for your response. I just run the code you suggested. It does not have a transforms config key. Maybe I am downloading the wrong cpkt file? This is what I get:

import torch 

checkpoint = torch.load("<path>/model.ckpt",  map_location=torch.device("cpu"))
print(checkpoint.keys()) # should have a transforms config key

dict_keys(['epoch', 'global_step', 'pytorch-lightning_version', 'state_dict', 'loops', 'callbacks', 'optimizer_states', 'lr_schedulers', 'MixedPrecisionPlugin', 'hparams_name', 'hyper_parameters'])