search

UP2040499 / auto-osint-v

An automated tool for Validating OSINT. This forms part of the final step of OSINT production as detailed by NATO's open source handbook (2001). This is a research artefact for my Dissertation at the University of Portsmouth.
https://up2040499.github.io/auto-osint-v/
Creative Commons Zero v1.0 Universal
2 stars 0 forks source link

Popular info finder #39

Closed UP2040499 closed 1 year ago

UP2040499 commented 1 year ago

Added the popular info finder Further attempts to optimise source aggregator. Added new functions to entity processor for popular info finder. Created a unit test for the popular info finder.

Took 14 hours 16 minutes

UP2040499 commented 1 year ago

Now need to add functionality for saving the sources that contain the largest number of popular information.

UP2040499 commented 1 year ago

Pylint resutls

Run pylint --fail-under 8 $(git ls-files '*.py')
************* Module auto_osint_v.main
auto_osint_v/main.py:5:0: W0611: Unused import asyncio (unused-import)
************* Module auto_osint_v.popular_information_finder
auto_osint_v/popular_information_finder.py:48:19: W3101: Missing timeout argument for method 'requests.get' can cause your program to hang indefinitely (missing-timeout)
auto_osint_v/popular_information_finder.py:98:12: W0612: Unused variable 'i' (unused-variable)
auto_osint_v/popular_information_finder.py:5:0: C0411: standard import "import itertools" should be placed before "import requests" (wrong-import-order)
************* Module auto_osint_v.sentiment_analyser
auto_osint_v/sentiment_analyser.py:19:0: C0301: Line too long (102/100) (line-too-long)
************* Module auto_osint_v.source_aggregator
auto_osint_v/source_aggregator.py:96:14: E1101: Instance of 'Resource' has no 'cse' member (no-member)
auto_osint_v/source_aggregator.py:152:8: W0612: Unused variable 'join_keywords' (unused-variable)
************* Module auto_osint_v.specific_entity_processor
auto_osint_v/specific_entity_processor.py:30:0: W1404: Implicit string concatenation found in list (implicit-str-concat)
auto_osint_v/specific_entity_processor.py:81:8: W0612: Unused variable 'texts_length' (unused-variable)
auto_osint_v/specific_entity_processor.py:89:4: C0116: Missing function or method docstring (missing-function-docstring)
************* Module auto_osint_v.unit_tests.test_popular_information_finder
auto_osint_v/unit_tests/test_popular_information_finder.py:1:0: C0114: Missing module docstring (missing-module-docstring)
auto_osint_v/unit_tests/test_popular_information_finder.py:[7](https://github.com/UP2040499/auto-osint-v/actions/runs/4832771107/jobs/8611963674?pr=40#step:5:8):0: C0115: Missing class docstring (missing-class-docstring)
auto_osint_v/unit_tests/test_popular_information_finder.py:[8](https://github.com/UP2040499/auto-osint-v/actions/runs/4832771107/jobs/8611963674?pr=40#step:5:9):4: C0116: Missing function or method docstring (missing-function-docstring)
auto_osint_v/unit_tests/test_popular_information_finder.py:11:8: C0415: Import outside toplevel (auto_osint_v.file_handler.FileHandler) (import-outside-toplevel)
auto_osint_v/unit_tests/test_popular_information_finder.py:12:8: C0415: Import outside toplevel (auto_osint_v.specific_entity_processor.EntityProcessor) (import-outside-toplevel)
auto_osint_v/unit_tests/test_popular_information_finder.py:17:8: C0415: Import outside toplevel (auto_osint_v.popular_information_finder.PopularInformationFinder) (import-outside-toplevel)
auto_osint_v/unit_tests/test_popular_information_finder.py:1[9](https://github.com/UP2040499/auto-osint-v/actions/runs/4832771107/jobs/8611963674?pr=40#step:5:10):8: W0[10](https://github.com/UP2040499/auto-osint-v/actions/runs/4832771107/jobs/8611963674?pr=40#step:5:11)5: String statement has no effect (pointless-string-statement)
************* Module auto_osint_v.unit_tests.test_specific_entity_processor
auto_osint_v/unit_tests/test_specific_entity_processor.py:15:4: C0[11](https://github.com/UP2040499/auto-osint-v/actions/runs/4832771107/jobs/8611963674?pr=40#step:5:12)6: Missing function or method docstring (missing-function-docstring)

-----------------------------------
Your code has been rated at 9.44/10
UP2040499 commented 1 year ago

Need to fix unittest directory on this branch. It conflicts with main and others. Move from auto-osint-v/auto_osint_v/unittests -> auto-osint-v/unittests