HurricaneLabs / machinae

Machinae Security Intelligence Collector
MIT License
504 stars 101 forks source link

Multiple errors in /usr/local/bin/machinae, /usr/local/lib/python3.9/dist-packages #66

Closed MindCacher closed 1 year ago

MindCacher commented 2 years ago

Hi. I have installed a couple of fresh 64-bit OS (Kali and Debian) with the necessary Python 3 environment with necessary packages according to the requirements.txt. File "machinae.yml" is fresh and latest. A couple years ago there were no problems when I have used MACHINAE. The problem is reproduced on any Unix-like OS with a classic installation (pip3 install machinae). I run the MACHINAE and get multiple errors like that:

# machinae scanme.org
Traceback (most recent call last):
  File "/usr/local/bin/machinae", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/dist-packages/machinae/cmd.py", line 218, in main
    cmd.run()
  File "/usr/local/lib/python3.9/dist-packages/machinae/cmd.py", line 185, in run
    if not self.conf:
  File "/usr/local/lib/python3.9/dist-packages/machinae/cmd.py", line 71, in conf
    conf = utils.safe_load(f)
  File "/usr/local/lib/python3.9/dist-packages/machinae/utils.py", line 37, in safe_load
    return yaml.load(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
  File "/usr/local/lib/python3.9/dist-packages/yaml/constructor.py", line 49, in get_single_data
    node = self.get_single_node()
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 36, in get_single_node
    document = self.compose_document()
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 55, in compose_document
    node = self.compose_node(None, None)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 64, in compose_node
    if self.check_event(AliasEvent):
  File "/usr/local/lib/python3.9/dist-packages/yaml/parser.py", line 98, in check_event
    self.current_event = self.state()
  File "/usr/local/lib/python3.9/dist-packages/yaml/parser.py", line 449, in parse_block_mapping_value
    if not self.check_token(KeyToken, ValueToken, BlockEndToken):
  File "/usr/local/lib/python3.9/dist-packages/yaml/scanner.py", line 116, in check_token
    self.fetch_more_tokens()
  File "/usr/local/lib/python3.9/dist-packages/yaml/scanner.py", line 227, in fetch_more_tokens
    return self.fetch_alias()
  File "/usr/local/lib/python3.9/dist-packages/yaml/scanner.py", line 610, in fetch_alias
    self.tokens.append(self.scan_anchor(AliasToken))
  File "/usr/local/lib/python3.9/dist-packages/yaml/scanner.py", line 922, in scan_anchor
    raise ScannerError("while scanning an %s" % name, start_mark,
yaml.scanner.ScannerError: while scanning an alias
  in "/etc/machinae.yml", line 1418, column 14
expected alphabetic or numeric character, but found '*'
  in "/etc/machinae.yml", line 1418, column 15

What's wrong? Thank you.

csgaee commented 2 years ago

Hi. I have installed a couple of fresh 64-bit OS (Kali and Debian) with the necessary Python 3 environment with necessary packages according to the requirements.txt. File "machinae.yml" is fresh and latest. A couple years ago there were no problems when I have used MACHINAE. The problem is reproduced on any Unix-like OS with a classic installation (pip3 install machinae). I run the MACHINAE and get multiple errors like that:

# machinae scanme.org
Traceback (most recent call last):
  File "/usr/local/bin/machinae", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/dist-packages/machinae/cmd.py", line 218, in main
    cmd.run()
  File "/usr/local/lib/python3.9/dist-packages/machinae/cmd.py", line 185, in run
    if not self.conf:
  File "/usr/local/lib/python3.9/dist-packages/machinae/cmd.py", line 71, in conf
    conf = utils.safe_load(f)
  File "/usr/local/lib/python3.9/dist-packages/machinae/utils.py", line 37, in safe_load
    return yaml.load(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
  File "/usr/local/lib/python3.9/dist-packages/yaml/constructor.py", line 49, in get_single_data
    node = self.get_single_node()
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 36, in get_single_node
    document = self.compose_document()
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 55, in compose_document
    node = self.compose_node(None, None)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/usr/local/lib/python3.9/dist-packages/yaml/composer.py", line 64, in compose_node
    if self.check_event(AliasEvent):
  File "/usr/local/lib/python3.9/dist-packages/yaml/parser.py", line 98, in check_event
    self.current_event = self.state()
  File "/usr/local/lib/python3.9/dist-packages/yaml/parser.py", line 449, in parse_block_mapping_value
    if not self.check_token(KeyToken, ValueToken, BlockEndToken):
  File "/usr/local/lib/python3.9/dist-packages/yaml/scanner.py", line 116, in check_token
    self.fetch_more_tokens()
  File "/usr/local/lib/python3.9/dist-packages/yaml/scanner.py", line 227, in fetch_more_tokens
    return self.fetch_alias()
  File "/usr/local/lib/python3.9/dist-packages/yaml/scanner.py", line 610, in fetch_alias
    self.tokens.append(self.scan_anchor(AliasToken))
  File "/usr/local/lib/python3.9/dist-packages/yaml/scanner.py", line 922, in scan_anchor
    raise ScannerError("while scanning an %s" % name, start_mark,
yaml.scanner.ScannerError: while scanning an alias
  in "/etc/machinae.yml", line 1418, column 14
expected alphabetic or numeric character, but found '*'
  in "/etc/machinae.yml", line 1418, column 15

What's wrong? Thank you.

did u find any solution ??

cschmidt0121 commented 2 years ago

Hi all, this was a small bug where a placeholder API key value (***YOUR_APIKEY_HERE***) for the greynoise_ent feed was breaking the yml parser. I've removed the asterisks from that placeholder key in the default machinae.yml which fixes this issue.

Thanks for the report!