airbnb / streamalert

StreamAlert is a serverless, realtime data analysis framework which empowers you to ingest, analyze, and alert on data from any environment, using datasources and alerting logic you define.
https://streamalert.io
Apache License 2.0
2.86k stars 334 forks source link

Tests fail when prod cluster is not configured #1282

Closed andreacardaropoli closed 4 years ago

andreacardaropoli commented 4 years ago

Description

When attempting to run the tests without a prod cluster the test scripts fail.

Steps to Reproduce

Delete the prod.json cluster that comes default with StreamAlert, try to run tests against a different cluster:

(venv) andrea@streamalert3 # python manage.py test classifier
[INFO 2020-07-30 10:40:14,181 (streamalert_cli.runner:66)]: Issues? Report here: https://github.com/airbnb/streamalert/issues
Running tests for files found in: rules
Traceback (most recent call last):
  File "manage.py", line 124, in <module>
    sys.exit(main())
  File "manage.py", line 120, in main
    return not cli_runner(options)
  File "/Users/andreacardaropoli/Workspace/streamalert3/streamalert_cli/runner.py", line 70, in cli_runner
    result = cmds[args.command](args)
  File "/Users/andreacardaropoli/Workspace/streamalert3/streamalert_cli/runner.py", line 126, in <lambda>
    command: lambda opts, cmd=cli_command: cmd.handler(opts, config)
  File "/Users/andreacardaropoli/Workspace/streamalert3/streamalert_cli/test/handler.py", line 201, in handler
    result = result and TestRunner(options, config).run()
  File "/Users/andreacardaropoli/Workspace/streamalert3/streamalert_cli/test/handler.py", line 384, in run
    self._process_directory(directory)
  File "/Users/andreacardaropoli/Workspace/streamalert3/streamalert_cli/test/handler.py", line 325, in _process_directory
    self._process_test_file(full_path)
  File "/Users/andreacardaropoli/Workspace/streamalert3/streamalert_cli/test/handler.py", line 340, in _process_test_file
    classifier_result = self._run_classification(event.record)
  File "/Users/andreacardaropoli/Workspace/streamalert3/streamalert_cli/test/handler.py", line 252, in _run_classification
    return _classifier.run(records=[record])
  File "/Users/andreacardaropoli/Workspace/streamalert3/streamalert/classifier/classifier.py", line 250, in run
    self._classify_payload(payload)
  File "/Users/andreacardaropoli/Workspace/streamalert3/streamalert/classifier/classifier.py", line 156, in _classify_payload
    logs_config = self._load_logs_for_resource(payload.service(), payload.resource)
  File "/Users/andreacardaropoli/Workspace/streamalert3/streamalert/classifier/classifier.py", line 91, in _load_logs_for_resource
    resources = self._config['clusters'][self._cluster]['data_sources'].get(service)
KeyError: 'prod'
(venv) andrea@streamalert3 # python manage.py test rules --test-files /Users/andreacardaropoli/Workspace/streamalert3/rules/community/github/github_site_admin_action.json
[INFO 2020-07-30 12:50:06,040 (streamalert_cli.runner:66)]: Issues? Report here: https://github.com/airbnb/streamalert/issues
Running tests for files found in: rules
Summary:
Total Tests: 0
Pass: 0
Fail: 0
[INFO 2020-07-30 12:50:06,109 (streamalert_cli.runner:71)]: Completed

Desired Change

Allow tests when prod cluster is not configured.

ryandeivert commented 4 years ago

fixed with #1286