search

CIRCL / AIL-framework

AIL framework - Analysis Information Leak framework. Project moved to https://github.com/ail-project
https://github.com/ail-project/ail-framework
GNU Affero General Public License v3.0
1.3k stars 282 forks source link

Few Script Errors #465

Closed Phil-ThePower-Pearce closed 8 months ago

Phil-ThePower-Pearce commented 4 years ago

Sentimental Analysis:

Dropped: text/x-c++
Dropped: text/x-c++
Traceback (most recent call last):
  File "./SentimentAnalysis.py", line 168, in <module>
    Analyse(message, server)
  File "./SentimentAnalysis.py", line 65, in Analyse
    sentences = tokenize.sent_tokenize(p_content)
  File "/home/ubuntu/Apps/AIL-framework/AILENV/lib/python3.6/site-packages/nltk/tokenize/__init__.py", line 105, in sent_tokenize
    tokenizer = load('tokenizers/punkt/{0}.pickle'.format(language))
  File "/home/ubuntu/Apps/AIL-framework/AILENV/lib/python3.6/site-packages/nltk/data.py", line 868, in load
    opened_resource = _open(resource_url)
  File "/home/ubuntu/Apps/AIL-framework/AILENV/lib/python3.6/site-packages/nltk/data.py", line 993, in _open
    return find(path_, path + ['']).open()
  File "/home/ubuntu/Apps/AIL-framework/AILENV/lib/python3.6/site-packages/nltk/data.py", line 701, in find
    raise LookupError(resource_not_found)
LookupError:
**********************************************************************
  Resource punkt not found.
  Please use the NLTK Downloader to obtain the resource:

  >>> import nltk
  >>> nltk.download('punkt')

  For more information see: https://www.nltk.org/data.html

Web:

The DNS operation timed out after 2.0002329349517822 second

Webstats:

`Traceback (most recent call last):
  File "./WebStats.py", line 180, in <module>
    analyse(r_serv_trend, 'scheme', date, url_parsed)
  File "./WebStats.py", line 32, in analyse
    field = field.decode('utf8')
AttributeError: 'str' object has no attribute 'decode'
`

Hope the above helps

Phil-ThePower-Pearce commented 4 years ago

I left AIL framework runing over the weekend.

THese were the errors that occured over the weekend `BankAccount Traceback (most recent call last): File "./BankAccount.py", line 107, in paste = Paste.Paste(filename) File "/home/ubuntu/Apps/AIL-framework/bin/packages/Paste.py", line 79, in init self.p_mime = magic.from_buffer(self.get_p_content(), mime=True) File "/home/ubuntu/Apps/AIL-framework/AILENV/lib/python3.6/site-packages/magic.py", line 148, in from_buffer return m.from_buffer(buffer) File "/home/ubuntu/Apps/AIL-framework/AILENV/lib/python3.6/site-packages/magic.py", line 82, in from_buffer return self._handle509Bug(e) File "/home/ubuntu/Apps/AIL-framework/AILENV/lib/python3.6/site-packages/magic.py", line 101, in _handle509Bug raise e File "/home/ubuntu/Apps/AIL-framework/AILENV/lib/python3.6/site-packages/magic.py", line 80, in from_buffer return maybe_decode(magic_buffer(self.cookie, buf)) File "/home/ubuntu/Apps/AIL-framework/AILENV/lib/python3.6/site-packages/magic.py", line 255, in magic_buffer return _magic_buffer(cookie, buf, len(buf)) File "/home/ubuntu/Apps/AIL-framework/AILENV/lib/python3.6/site-packages/magic.py", line 188, in errorcheck_null raise MagicException(err) magic.MagicException: b'cannot allocate 82100232 bytes (Cannot allocate memory)'

IPAddress Traceback (most recent call last): File "./IPAddress.py", line 85, in search_ip(message) File "./IPAddress.py", line 36, in search_ip address = IPv4Address(res) File "/usr/lib/python3.6/ipaddress.py", line 1284, in init self._ip = self._ip_int_from_string(addr_str) File "/usr/lib/python3.6/ipaddress.py", line 1123, in _ip_int_from_string raise AddressValueError("%s in %r" % (exc, ip_str)) from None ipaddress.AddressValueError: Ambiguous (octal/decimal) value in '085' not permitted in '085.196.183.162'

DNS The DNS operation timed out after 2.0001516342163086 seconds""

Indexer bash: line 1: 5279 Killed /home/ubuntu/Apps/AIL-framework//AILENV/bin/python ./Indexer.py

Phone bash: line 1: 5279 Killed /home/ubuntu/Apps/AIL-framework//AILENV/bin/python ./Indexer.py

Decoder Traceback (most recent call last): File "./Decoder.py", line 221, in paste = Paste.Paste(filename) File "/home/ubuntu/Apps/AIL-framework/bin/packages/Paste.py", line 79, in init self.p_mime = magic.from_buffer(self.get_p_content(), mime=True) File "/home/ubuntu/Apps/AIL-framework/AILENV/lib/python3.6/site-packages/magic.py", line 148, in from_buffer return m.from_buffer(buffer) File "/home/ubuntu/Apps/AIL-framework/AILENV/lib/python3.6/site-packages/magic.py", line 82, in from_buffer return self._handle509Bug(e) File "/home/ubuntu/Apps/AIL-framework/AILENV/lib/python3.6/site-packages/magic.py", line 101, in _handle509Bug raise e File "/home/ubuntu/Apps/AIL-framework/AILENV/lib/python3.6/site-packages/magic.py", line 80, in from_buffer return maybe_decode(magic_buffer(self.cookie, buf)) File "/home/ubuntu/Apps/AIL-framework/AILENV/lib/python3.6/site-packages/magic.py", line 255, in magic_buffer return _magic_buffer(cookie, buf, len(buf)) File "/home/ubuntu/Apps/AIL-framework/AILENV/lib/python3.6/site-packages/magic.py", line 188, in errorcheck_null raise MagicException(err) magic.MagicException: b'cannot allocate 82100232 bytes (Cannot allocate memory)' `

Webstats: Traceback (most recent call last): File "./WebStats.py", line 180, in <module> analyse(r_serv_trend, 'scheme', date, url_parsed) File "./WebStats.py", line 32, in analyse field = field.decode('utf8') AttributeError: 'str' object has no attribute 'decode'

Terrtia commented 4 years ago

Hi @Phil-ThePower-Pearce ! Thank you for the report !

The mime type issue seem to be related to https://github.com/ahupp/python-magic/issues/22

Fixed with 4097d95237a81e3c506743397eea4880622a1ccf cf24c59e1d390db39b305b1aec7a17d3b0551b0f and f9856a1589cb49fc6e407587d7609bf4a3e6af23

Please tell me if this solve your issue with the Decoder module

Phil-ThePower-Pearce commented 4 years ago

Sentimental Analysis still has issues after update

`[nltk_data] Downloading package punkt to /root/nltk_data... [nltk_data] Unzipping tokenizers/punkt.zip. Traceback (most recent call last): File "./SentimentAnalysis.py", line 173, in Analyse(message, server) File "./SentimentAnalysis.py", line 76, in Analyse sid = SentimentIntensityAnalyzer(sentiment_lexicon_file) File "/home/ubuntu/Apps/AIL-framework/AILENV/lib/python3.6/site-packages/nltk/sentiment/vader.py", line 334, in init self.lexicon_file = nltk.data.load(lexicon_file) File "/home/ubuntu/Apps/AIL-framework/AILENV/lib/python3.6/site-packages/nltk/data.py", line 868, in load opened_resource = _open(resource_url) File "/home/ubuntu/Apps/AIL-framework/AILENV/lib/python3.6/site-packages/nltk/data.py", line 993, in open return find(path, path + ['']).open() File "/home/ubuntu/Apps/AIL-framework/AILENV/lib/python3.6/site-packages/nltk/data.py", line 701, in find raise LookupError(resource_not_found) LookupError:

Resource vader_lexicon not found. Please use the NLTK Downloader to obtain the resource:

import nltk nltk.download('vader_lexicon')

For more information see: https://www.nltk.org/data.html

Attempted to load sentiment/vader_lexicon.zip/vader_lexicon/vader_lexicon.txt

Searched in:

Terrtia commented 8 months ago

Fixed in v5.0