Installation update and a few thoughts

[germonprez]

I'm at the point of adding the scanners and kibble.localhost is up and running. Success. A few that may or may not be right that I did along the way:

1) Installed PyYAML. There is no yaml (at least that I could find) for Python3 2) I think that the virtual host install could use a bit more description. Loading the modules, setting up proxy, and configuring /etc/hosts is a bit vague. I think I got it to work but it was with a bit of clawing.

Looking ahead to the scanner installs:

1) Regarding git binaries and cloc -- are these required? "require the following optional components" is not clear 2) Should I be putting (git clone https://github.com/apache/kibble-scanners.git) somewhere in particular? 3) This is not clear: Then edit the conf/config.yaml file to match both your ElasticSearch database and the file layout you wish to use on the scanner machine --> I'm guessing that this conf folder is in the clone but could you provide a bit of help on what 'match the db and file layout' are? I'm not very familiar with ES so this is a bit new to me.

I'll add that I'm doing all of this on Ubuntu 16.04 in a Parallels VM.

[Humbedooh]

1) The wording should be precised. git and cloc are used if you want to analyze git repositories. We don't/can't depend on GPL software, so these are explicitly noted as optional components you may wish to install if you want Kibble to work with git repos.

2) you can put the scanners anywhere you like, the file path doesn't matter.

3) matching the db layout basically means you would connect to the same ES instance as the master kibble server. If scanners and the UI are on the same box, you would use the exact same settings as found in kibble.yaml in the UI bit. I can elaborate on this in the documentation.

[germonprez]

Thanks.

1) Isn't there an assumption that I have git installed (point 1) during this entire process (as I do)? Should the directions just reference cloc?

2) How do I turn off certain scanners?

[germonprez]

Also, I think I'm really close :) Getting a pretty big dump on start. Things look good from the get-go. Apache HTTPD server is up and running too.

sudo python3 kibble-scanner.py core: Loaded plugins/scanners/git-sync v/0.1.2 (Sync plugin for Git repositories) core: Loaded plugins/scanners/git-census v/0.1.0 (Census Scanner for Git) core: Loaded plugins/scanners/git-sloc v/0.1.0 (SloC Counter for Git) core: Loaded plugins/scanners/git-evolution v/0.1.0 (Git Evolution Scanner) core: Loaded plugins/scanners/jira v/0.1.0 (Scanner for Atlassian JIRA) core: Loaded plugins/scanners/ponymail v/0.1.0 (Scanner plugin for Apache Pony Mail) core: Loaded plugins/scanners/ponymail-tone v/0.1.0 (Tone/Mood Scanner plugin for Apache Pony Mail) core: Loaded plugins/scanners/ponymail-kpe v/0.1.0 (Key Phrase Extraction plugin for Apache Pony Mail) core: Loaded plugins/scanners/pipermail v/0.1.0 (Scanner for GNU Mailman Pipermail) core: Loaded plugins/scanners/github-issues v/0.1.0 (Scanner for GitHub Issues) core: Loaded plugins/scanners/bugzilla v/0.1.0 (Scanner for BugZilla) core: Loaded plugins/scanners/gerrit v/0.1.1 (Scanner for Gerrit Code Review) core: Kibble Scanner v/0.1.0 starting core: Loaded YAML config from ../conf/config.yaml core: Using direct ElasticSearch broker model core: Connecting to ElasticSearch database at localhost:9200...

HEAD http://localhost:9200/kibble [status:N/A request:0.002s] Traceback (most recent call last): File "/home/parallels/.local/lib/python3.5/site-packages/urllib3/connection.py", line 141, in _new_conn (self.host, self.port), self.timeout, **extra_kw) File "/home/parallels/.local/lib/python3.5/site-packages/urllib3/util/connection.py", line 83, in create_connection raise err File "/home/parallels/.local/lib/python3.5/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/parallels/.local/lib/python3.5/site-packages/elasticsearch/connection/http_urllib3.py", line 149, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, kw) File "/home/parallels/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/home/parallels/.local/lib/python3.5/site-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/home/parallels/.local/lib/python3.5/site-packages/urllib3/packages/six.py", line 686, in reraise raise value File "/home/parallels/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/home/parallels/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, httplib_request_kw) File "/usr/lib/python3.5/http/client.py", line 1106, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1151, in _send_request self.endheaders(body) File "/usr/lib/python3.5/http/client.py", line 1102, in endheaders self._send_output(message_body) File "/usr/lib/python3.5/http/client.py", line 934, in _send_output self.send(msg) File "/usr/lib/python3.5/http/client.py", line 877, in send self.connect() File "/home/parallels/.local/lib/python3.5/site-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/home/parallels/.local/lib/python3.5/site-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f8c1f0555c0>: Failed to establish a new connection: [Errno 111] Connection refused HEAD http://localhost:9200/kibble [status:N/A request:0.000s] Traceback (most recent call last): File "/home/parallels/.local/lib/python3.5/site-packages/urllib3/connection.py", line 141, in _new_conn (self.host, self.port), self.timeout, **extra_kw) File "/home/parallels/.local/lib/python3.5/site-packages/urllib3/util/connection.py", line 83, in create_connection raise err File "/home/parallels/.local/lib/python3.5/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused

--CLIPPED--

[germonprez]

Solved. Closing issue. I did a reinstall of ES and all things seemed to clear up.