TaggedCorpus refactoring

[zseder]

TaggedCorpus should be split into RawCorpus and TaggedCorpus

• only tag-related content goes into TaggedCorpus
• read should be implemented this way
  • in RawCorpus.read() maybe we should use a needed_fields argument, with a default value of [0] (0th index=word), and TaggedCorpus.read() should have only contain a call to RawCorpus.read(needed_fields=[0,1]), where at first place, there is the tag
  • the only problem is that from now on, words will be a 1-length lists, to be compatible with tags, when those will be 2-length lists (NOT tuples, because lists can be changed in-place, so it's easier to turn them into integers later)
• read() should keep the pre flag, so when featurizer is preprocessing the data, it will only return words
  • if we call featurizer preprocessing in RawCorpus.__init__(), needed_fields flag for read() will be good and only words will be returned, so hopefully no change there

[zseder]

one additional info: no code copying should be used, only code reusing, for example with a good design, add_features can be implemented in a way that it can work with RawCorpus and TaggedCorpus, without a change, so it is enough to do that in RawCorpus. Hint:

new_sen = [word_data + [self.featurizer.featurize(word_data[0])] for word_data in sen]

[zseder]

done in #70