A full walkthrough to generate charts?

[xxbidiao]

Hi,

I realized that some of the steps in the setup guide are slightly off and the guides just ends at training stage. I would like to know that how I can get to the final part that evaluates an extra audio piece to generate a stepchart.

Also, it seems that when I do the following 1. Train a step selection (symbolic) model on a dataset:./sml_sym_2_train.sh fraxtil``

I also needed to do ./sml_sym_1_chart.sh fraxtil

But I don't know whether this is intended.

Thank you!

[chrisdonahue]

Thank you for raising this issue! I have added inference code and a trained model to the repository under the infer directory (link)

There are some details in the README about how to run the code to generate step charts. Please let me know if you run into further issues.

[xxbidiao]

Thanks. I'm looking into transferring inference code from tf v0.12 to tf v1 and this seems to involve a retrain. Where can I find the trained net that can be used by the inference code?

[chrisdonahue]

The trained nets (step placement and step selection) are under server_aux. I tried running the inference code in tf v1 and it seemed like the only issue was some RNN variable naming convention had changed in the internal API. You should be able to either 1) fix the checkpoint to use the new naming convention, or 2) fix the code to use the old variable names. Also, you'll need to pull in the newer versions of onset_net.py and sym_net.py from learn/ which update the code for v1 API changes for tf.concat and tf.split. If you get it to work, please send a PR!

[xxbidiao]

The checkpoint problem is not trivial so I'm trying to get the model files corresponding to model_sp and model_ss generated by tf v1 to test whether the only issue lies in the checkpoint. Where can I get these two files if I do a retrain? I did a partial retrain but can't get the correct files to make infer net work.

[chrisdonahue]

The checkpoints should show up in /tmp/train and /tmp/train_sym in alignment with sml_onset_2_train.sh and sml_sym_2_train.sh

[xxbidiao]

I realized that the nfeats in the training module (1) and inference model (2) is not consistent, causing the checkpoint (and the whole thing, tf v1 converted) to fail. Tweaking around the parameters doesn't seem to work; I would like to know if this inconsistency is intended.

Edit: Seems like the inference net is asking for 2 features, but the learning net is only providing 1, so my retrained nets never fit. What parameters should I look into to make these two nets consistent? Thanks!

Also, the onset net phase seems to be working now.

[chrisdonahue]

Ah, great catch! It looks like I forgot to include --feat_time_diff_next from scripts/sml_sym_2_train.sh. This ensures that the selection network will know the delta time to the next note as well as the delta time from the previous note. This was the mystery feature that was missing :)

[xxbidiao]

Nice, things seem to work now. Thanks again!

[chrisdonahue]

No problem. Were you able to modify the checkpoint to fix the inference in tf 1.0? If so, could you make a PR?

[xxbidiao]

An intermediate file is used in my testing (to prove that the file structure is OK) so I don't have a fully trained model yet. The old model won't work in tf 1.x version so I'll make the PR as soon as I have code cleaned up and model retrained from scratch.