About creating or finding Guitar pre-trained wights for Google Magenta AI music project (DDSP Module)

[soheilpaper]

I am working in one save the earth project (lonely!), Which is described here and one of the Idea is using AI Music codes and one Smart trash bin for encouraging some humankind to do recycling.

So, I am trying to test the Google magenta project for more default models of its demo colab page, one of the most useful musical devices is guitar, and I am trying to add the guitar model in its colab sample code as you can see at below (Google Colaboratory Link for test):

enter image description here

So if possible I hope to have some comment in creating the guitar pretrained weight for it by the below instruction :

Based on GitHub instruction of the DDSP part of the Google Magenta project, it could be domed by creating some pretrained model by gathering around 10-20 minutes of a single monophonic guitar sound source (colab link for test):

enter image description here

Which needs to have some finding some ready mp3 guitar database tags for this purpose and I googled the ready mp3 guitar dataset pre -trained but become low energy! And don't know which guitar pretrained weights or mp3 guitar dataset (like guaitarset form this post would be proper for this part of the above codes?

But in this situation, I have tried to find out the model extension and finding the guitar pretrained weights from that extension and guitar search in google, so I have tried to find out the guitar model for ddsp (the main music model used in the above codes) by searching ddsp guitar terms in google, or by looking in the above codes and find out the model file extension like the below block code:

 #@title Load a model
#@markdown Run for ever new audio input
model = 'Violin' #@param ['Violin', 'Flute', 'Flute2', 'Trumpet', 'Tenor_Saxophone', 'Upload your own (checkpoint folder as .zip)']
MODEL = model

def find_model_dir(dir_name):
  # Iterate through directories until model directory is found
  for root, dirs, filenames in os.walk(dir_name):
    for filename in filenames:
      if filename.endswith(".gin") and not filename.startswith("."):
        model_dir = root
        break
  return model_dir 

if model in ('Violin', 'Flute', 'Flute2', 'Trumpet', 'Tenor_Saxophone'):
  # Pretrained models.
  PRETRAINED_DIR = '/content/pretrained'
  # Copy over from gs:// for faster loading.
  !rm -r $PRETRAINED_DIR &> /dev/null
  !mkdir $PRETRAINED_DIR &> /dev/null
  GCS_CKPT_DIR = 'gs://ddsp/models/timbre_transfer_colab/2021-01-06'
  model_dir = os.path.join(GCS_CKPT_DIR, 'solo_%s_ckpt' % model.lower())

  !gsutil cp $model_dir/* $PRETRAINED_DIR &> /dev/null
  model_dir = PRETRAINED_DIR
  gin_file = os.path.join(model_dir, 'operative_config-0.gin')

else:
  # User models.
  UPLOAD_DIR = '/content/uploaded'
  !mkdir $UPLOAD_DIR
  uploaded_files = files.upload()

  for fnames in uploaded_files.keys():
    print("Unzipping... {}".format(fnames))
    !unzip -o "/content/$fnames" -d $UPLOAD_DIR &> /dev/null
  model_dir = find_model_dir(UPLOAD_DIR)
  gin_file = os.path.join(model_dir, 'operative_config-0.gin')

# Load the dataset statistics.
DATASET_STATS = None
dataset_stats_file = os.path.join(model_dir, 'dataset_statistics.pkl')
print(f'Loading dataset statistics from {dataset_stats_file}')
try:
  if tf.io.gfile.exists(dataset_stats_file):
    with tf.io.gfile.GFile(dataset_stats_file, 'rb') as f:
      DATASET_STATS = pickle.load(f)
except Exception as err:
  print('Loading dataset statistics from pickle failed: {}.'.format(err))

# Parse gin config,
with gin.unlock_config():
  gin.parse_config_file(gin_file, skip_unknown=True)

# Assumes only one checkpoint in the folder, 'ckpt-[iter]`.
ckpt_files = [f for f in tf.io.gfile.listdir(model_dir) if 'ckpt' in f]
ckpt_name = ckpt_files[0].split('.')[0]
ckpt = os.path.join(model_dir, ckpt_name)

# Ensure dimensions and sampling rates are equal
time_steps_train = gin.query_parameter('F0LoudnessPreprocessor.time_steps')
n_samples_train = gin.query_parameter('Harmonic.n_samples')
hop_size = int(n_samples_train / time_steps_train)

time_steps = int(audio.shape[1] / hop_size)
n_samples = time_steps * hop_size

# print("===Trained model===")
# print("Time Steps", time_steps_train)
# print("Samples", n_samples_train)
# print("Hop Size", hop_size)
# print("\n===Resynthesis===")
# print("Time Steps", time_steps)
# print("Samples", n_samples)
# print('')

gin_params = [
    'Harmonic.n_samples = {}'.format(n_samples),
    'FilteredNoise.n_samples = {}'.format(n_samples),
    'F0LoudnessPreprocessor.time_steps = {}'.format(time_steps),
    'oscillator_bank.use_angular_cumsum = True',  # Avoids cumsum accumulation errors.
]

with gin.unlock_config():
  gin.parse_config(gin_params)

# Trim all input vectors to correct lengths 
for key in ['f0_hz', 'f0_confidence', 'loudness_db']:
  audio_features[key] = audio_features[key][:time_steps]
audio_features['audio'] = audio_features['audio'][:, :n_samples]

# Set up the model just to predict audio given new conditioning
model = ddsp.training.models.Autoencoder()
model.restore(ckpt)

# Build model by running a batch through it.
start_time = time.time()
_ = model(audio_features, training=False)
print('Restoring model took %.1f seconds' % (time.time() - start_time))

And I think the new model file extension is .gen and tried to find out some guitar gen file by guitar dataset ".gen" Google search terms, but can not find the made guitar wight file from that Google search result.

So as summarizing this question, if possible hope to have some comments about the two below question:

1. Finding the proper guitar mp3 files for uploading them to this colab page for creating the guitar pretrained weighs (it could be done by finding some dataset or data scraping by some codes like youtube-dl module or ETC).
2. Or finding the pretrained guitar weights for uploading on the main DDSP Colab page.

Also,complaining version of this question could be seen here and if the question is complex for understanding please comment about the parts which needs to be more cleared, and I will edit it.

Also asked below:

https://stackoverflow.com/questions/69762589/about-creating-or-finding-guitar-pre-trained-wights-for-google-magenta-ai-music

https://github.com/magenta/magenta/issues/1960

Thanks.

[bluenote10]

Interesting project! I'm also experimenting with training a guitar model. I may be misunderstanding your question, but regarding "finding the proper guitar mp3 files for uploading": This isn't really about "finding", you can simply create and use what you want, right? In case it helps, here is the file I'm using for training. It is simply a ~14 min recording of me playing some random notes -- not the best sounding guitar and poor recording conditions, but just as an experiment.

[ankur-00007]

Hi @soheilpaper , were you able to get these pre trained models?