search

UP2040499 / auto-osint-v

An automated tool for Validating OSINT. This forms part of the final step of OSINT production as detailed by NATO's open source handbook (2001). This is a research artefact for my Dissertation at the University of Portsmouth.
https://up2040499.github.io/auto-osint-v/
Creative Commons Zero v1.0 Universal
2 stars 0 forks source link

Popular information finder #19

Closed UP2040499 closed 1 year ago

UP2040499 commented 1 year ago

This finds information that is popular amongst the sources found in Source Aggregation #17. Once found, individual (and discrete) entities are stored in a Popular Entity Store. This is accessed by the Priority Manager #20.

UP2040499 commented 1 year ago

Is possible to use the NER from the start for this.

However will need to see if this can be accomplished in good time for ~200-300 sources.

This can be done using BeautifulSoup to parse the HTML.

UP2040499 commented 1 year ago

Could have a threshold for the amount of mentions throughout all sources, for an entity to be added to the popular entity store.

Or use a leaderboard type system, and have a max number of entities to store. E.g. store top 30 most popular entities.

UP2040499 commented 1 year ago

Currently can find popular information, can produce the top x% of popular entities. This value can be changed, more investigation is needed to find an optimum value. To finish:

UP2040499 commented 1 year ago

Closing as popular information finding has been completed. Assigning scores is done by priority manager in #40