search

lzamparo / embedding

Learning semantic embeddings for TF binding preferences directly from sequence
Other
0 stars 0 forks source link

Turn samplers for contexts into factor-dependent parameters #7

Open lzamparo opened 7 years ago

lzamparo commented 7 years ago

Had a few thoughts about how to incorporate factor specific information into the context samplers

  1. Turn the sampling kernel into model parameters. The first way to do this is by preferring far away K-mers rather than nearby K-mers. This will be to make sure the model does not simply learn to put adjacent words (with substantial K-mer overlap) together at the expense of learning longer spatial dependencies within probes.
  2. Adapt the samplers by conditioning on K-mers involved and overall statistics for this factor. How this would work is by making one sampler per factor, which has enrichment weights for each word in the subset of the unigram dictionary that appears in that factor. Then for each probe reduced to a sentence, form the sampling probabilities from two components:
    • the K-mer enrichment by this factor
    • the positional preference for further away context kmers