allenai / pnp

Probabilistic Neural Programming
Apache License 2.0
75 stars 22 forks source link

Feature generation for entities and tokens #50

Closed jayantk closed 7 years ago

jayantk commented 7 years ago

I'm not thrilled with the way this is engineered -- I probably broke the Geoquery semantic parsing example, and there's clearly some kind of missing abstraction around the entity linkings with features -- but we're on a deadline so let's refactor later.

I also updated the preprocessing to load tables, store them as preprocessed json, and use them in the parser's features. I added one feature to the parser, which is 1 for a column and token if the token is contained in one of the column's entities. This change raises dev accuracy to ~30.5.

I also added a flag for training with Learning as Search Optimization. It didn't help, but may be useful in the future.