Running a test data collection

TeMU-BSC / iberifier

2 stars 0 forks source link

Running a test data collection #19

Closed Oliph closed 1 year ago

Oliph commented 2 years ago

Running the data collection for testing purpose (mainly to evaluate the amount of noise)

[x] Twitter
[ ] Meneame
[ ] EDA the data to identify noise

cuquiwi commented 2 years ago

A test for 1000 tweets per claim and 3 claims was done. And I think a lot of noise is collected. Needs some discussion.

Oliph commented 2 years ago

Great, we should take a look at it. Is there a way to track back the tweet collection from the exact claim right now?

cuquiwi commented 2 years ago

Yes, The tweets are now in the twitter_test collection. And each tweet contains a field source that is the id to a claim in maldita collection

Oliph commented 2 years ago

Rerun test with the full keywords with the AND rather than doing an OR between each bigrams

cuquiwi commented 2 years ago

There is a file in the server eda_twitter.json and a collection eda_twitter that will allow to do an easy analysis of the tweets compared with the claim.