Allow synthesizing a mix of simple and complex commands

stanford-oval / thingpedia-common-devices

Thingpedia interface code for commonly used devices

Other

38 stars 28 forks source link

Allow synthesizing a mix of simple and complex commands #378

Closed gcampax closed 3 years ago

gcampax commented 3 years ago

To bias the training data towards simple commands, do two separate generations, with different sets of hyperparameters.

"Simple" generation is optimized for breadth: high pruning size so we don't prune out commands, low depth and number of turns, and compound commands disabled.

"Complex" generation is as before optimized for depth and complexity: relatively low pruning size but high depth and number of turns.

I'm going to try this in Kubeflow to see how it fares, but comments welcome.

coveralls commented 3 years ago

Pull Request Test Coverage Report for Build 1289

0 of 0 changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 54.43%

Totals
Change from base Build 1287:	0%
Covered Lines:	1200
Relevant Lines:	2098

💛 - Coveralls

gcampax commented 3 years ago

Mh, with the hyperparameters in this PR the dataset is too big (~1.3M after augmentation) and OOMs in training. I'm going to try a smaller set of hyperparameters to see if it helps, but we might have to scrap this idea.

gcampax commented 3 years ago

I tested the tuned hyperparameters with build 131, the dataset size looks good, the synthesis is very fast, and the final performance is good. I'm going to merge this.