Make clicking on a generated aspect find all relevant sentences

[ChulioZ]

As the aspect extraction has been reworked there are currently two new features preventing a click on a generated aspect from showing all relevant sentences:

1. Negative comparative aspects are mapped to their positive counterpart as it doesn't make sense to say "Python is better than Matlab and of the reasons for that is 'slower'" which could happen for example based on a sentence like "Matlab is slower than Python". The generated aspect from this sentence thus will be "faster" but when clicking on "faster" within the generated aspects tab, this sentence won't show although it should. Suggestion: When searching for all sentences containing the clicked aspect, also search for sentences containing the negative counterpart (you can use the NEGATIVE_MARKER_MAP of pos_link_extracter.py for this).

2. Aspects that are not generated by the sentence itself but instead by its context will lead to the sentence not being shown when the appropriate aspect is being clicked. For example, there could be an aspect "speed" within the top 10 generated aspects that doesn't appear in any sentence itself but in many of their contexts. Clicking on "speed" will show no sentences. Suggestion: When searching for all sentences containing the clicked aspect, also search for sentences containing this aspect within their appropriate contexts. To support this I've already implemented another field for the Sentence object: Each sentence now holds a list of context_aspects which will contain all aspects that have been found within its context.

[mschildw]

Since the filtering happens in the frontend, maybe the best thing would be to adapt the "context_aspect" list used in 2. and put all aspects assigned to the sentence there (also those of 1.). If that is the case in the frontend the filter function easily can look through that list and mark the sentence accordingly.

@alexanderpanchenko Since we don't have that functionality yet, do you think it is useful to use the new aspect extraction for the demo version?

[alexanderpanchenko]

Hm, that is strange. I remember that filtering by new aspects worked fine or I do not understand you.

I find it very important to have new aspects in the production - demo looks much weaker with the old aspects.

On Wed 17. Oct 2018 at 08:26, Matthias Schildwächter < notifications@github.com> wrote:

Since the filtering happens in the frontend, maybe the best thing would be to adapt the "context_aspect" list used in 2. and put all aspects assigned to the sentence there (also those of 1.). If that is the case in the frontend the filter function easily can look through that list and mark the sentence accordingly.

@alexanderpanchenko https://github.com/alexanderpanchenko Since we don't have that functionality yet, do you think it is useful to use the new aspect extraction for the demo version?

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/uhh-lt/cam/issues/93#issuecomment-430505056, or mute the thread https://github.com/notifications/unsubscribe-auth/ABY6voCZdh5eBSU6OZsMNlbQJ7FkO7Z1ks5uls2PgaJpZM4XL1QS .

[mschildw]

I referred to the necessary adaptions described by Julian above. Probably, most of the filtering works, but maybe not if the aspects are taken from context sentences or if the negative counterpart is taken.

In the afternoon I will merge it into master, maybe it is not too bad. Btw. is @ChulioZ not available at the moment?

[alexanderpanchenko]

I think Julian should be available, but maybe with some delay.

On Oct 17, 2018, at 12:20 PM, Matthias Schildwächter notifications@github.com wrote:

I referred to the necessary adaptions described by Julian above. Probably, most of the filtering works, but maybe not if the aspects are taken from context sentences or if the negative counterpart is taken.

In the afternoon I will merge it into master, maybe it is not too bad. Btw. is @ChulioZ https://github.com/ChulioZ not available at the moment?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/uhh-lt/cam/issues/93#issuecomment-430573902, or mute the thread https://github.com/notifications/unsubscribe-auth/ABY6vt1p_vf4QfJ-55pG0h6fypvMyGoVks5ulwRogaJpZM4XL1QS.

[ChulioZ]

Yes, I am, my last few days were pretty full though. Tomorrow I'll have a lot more time and I'll get to all the other questions/issues that arose during the last few days.

Just to answer your question really quick, @alexanderpanchenko : Filtering by the newly found aspects works exactly as fine as with the old aspects. But the filtering process currently only looks at the "real" sentences and not at their contexts, even if the latter is used for creating the aspects.

Take this example:

Say without using the context for aspect generation you'll find "easier" within the top 10 generated aspects, but not "price". With using the context, "easier" is still within the top 10, but now "price" also is.

If you then click on "easier" you will find all sentences containing the word(+++ see below). You won't find any additional sentences that don't contain "easier" even if their context does. The same is true for "price": Not having been in the top 10 before doesn't mean there are 0 sentences containing it, so clicking on "price" will still show those. However, there should also be quite a lot of sentences that don't contain it; only in their context they do. As was the case with "easier", again you won't find these sentences when clicking on "price" because only the sentences themselves are being searched.

The same thing, as @mschild said, is true for aspects that are the negative counterpart of the actual word within the sentences.

The reason for that is simple: Before the new version, aspects were generated exactly as they appear in the sentences. Now, some aspects are adapted (finding "slower" means the actual aspect "faster" is generated) and some aren't from the sentences at all (those from the contexts).

That's why I created this issue to change the search for sentences containg the aspect from "finding the sentences containing the actual word" to "finding the sentences that contributed to this aspect".

+++ I also noted that currently it seems to work via "word in sentence" or something like that as when clicking on something like "price", a sentence containing "priceless" would also be found. I don't know if this is intended, but if it's not it should be gone anyway when using a list of words instead like @mschild suggested.