Figure out if our `switch` function is broken

allenai / deep_qa

A deep NLP library, based on Keras / tf, focused on question answering (but useful for other NLP too)

Apache License 2.0

404 stars 132 forks source link

Figure out if our `switch` function is broken #354

Closed matt-gardner closed 7 years ago

matt-gardner commented 7 years ago

It sure seems like computing gradients through a switch doesn't work. At least it definitely didn't when switch was used in a loss function. We need to figure out if we're actually getting correct gradients for other places where we use switch, and remove the use of switch if we're not.

matt-gardner commented 7 years ago

From the exploration done in #355, I'm reasonably confident this isn't actually an issue, there was just some crazy problem with using switch inside the loss function for some reason. It'd be nice to understand why not, but it's not a P0 bug. I'm closing this one for now.