search

ecstatic-nobel / OSweep

Don't Just Search OSINT. Sweep It.
https://splunkbase.splunk.com/app/4569/
MIT License
311 stars 65 forks source link

Lookup tables are invalid. #1

Closed MickeyAlton closed 5 years ago

MickeyAlton commented 5 years ago

Hi, i'm having a bit trouble with "osweep". I'm running Splunk on MacOS as an administrator, followed osweep setup instructions. I don't see any data on Splunk and i get "The lookup tale 'XXXXX' is invalid (see attached screen grub). Any ideas? image

Thanks in advance.

ecstatic-nobel commented 5 years ago

Thanks for reporting this issue. I probably should've mentioned that I only tested this on Ubuntu 18.04. Oops! Let's see...

What errors do you get if you run | greynoise feed or | cybercrimeTracker feed in the Search app?

exente commented 5 years ago

I had simmilar issues when I've made this deployment on a CentOS server, finally in my case was required 2 actions on the SPLUNK, fisrt add more resources to the SPLUNK to run several jobs in parallel, second ones was the folder rights

Regarding add resources, you need to add following configuration to the limits.conf file (usually on [SPLUNK installation directory]/etc/system/local/limits.conf), Here is more information about the proper values (https://answers.splunk.com/answers/4888/getting-error-maximum-number-of-concurrent-searches-has-been-reached.html), but I've set follwoing values, but you would set up the best fit for your deployment max_searches_per_cpu = 10 base_max_searches= 4 max_rt_search_multiplier = 2

About the folder rights, you have to check that the application folder has the proper rights to the Splunk user, you could check it under [SPLUNK installation directory]/etc/apps/osweep

If both things are properly set (remeber to restart the Splunk server after deployment), you should start to see the things working and also the CSV files into the [SPLUNK installation directory]/etc/apps/osweep/lookups

If it still not working, you could try to do it manually, to do so, you can download manually i.e. the ranswomware tracker () and left on this directory with the name "ranswomware_tracker_feed.csv", after left the file, you should see the info into the application, if you can't see it, then there is an issue into the splunk configuration.

ecstatic-nobel commented 5 years ago

@exente That's excellent input. I'll look into providing a recommended architecture and including a limits.conf file with the project after I test on some other platforms.

@MickeyAlton hopefully this helps. If not, I'd be interested in seeing what errors you get from trying to manually download the feed.

ecstatic-nobel commented 5 years ago

Release v1.5.3 (commit 8d183485dc90161a2b6c0ba579ed0257c5d2743a) has been pushed to solve any non-default installation path issues. Please pull, install, and test the latest. Let me know if you have any issues.

exente commented 5 years ago

I've just downloaded the latest, seems to work properly, I'll keep you updatedd. Something special you want I test?

On Fri, Dec 21, 2018 at 12:55 AM No Name notifications@github.com wrote:

Release v1.5.3 (commit 8d183485dc90161a2b6c0ba579ed0257c5d2743a) has been pushed to solve any non-default installation path issues. Please pull, install, and test the latest. Let me know if you have any issues.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/leunammejii/osweep/issues/1#issuecomment-449181144, or mute the thread https://github.com/notifications/unsubscribe-auth/AbwGuUWA_iuRVLBxIbC8QsZc2ktJHhdWks5u7CNWgaJpZM4ZRPAu .

ecstatic-nobel commented 5 years ago

@MickeyAlton I'm marking the issue as resolved for now. If you are still having issues, please reopen it.