Issue

[MKKoppula]

Issue1

[MKKoppula]

http://www.logbinder.com/Download/55a61f06-e4ae-4e16-bf92-eab36a1b7f5a

[MKKoppula]

Delete ALL logs older than 2 days. The data is already indexed

02 01 /usr/bin/find /opt/syslog// -type f -mtime +2 -exec rm -fr .gz {} \;

[MKKoppula]

https://docs.google.com/spreadsheets/d/1YJAqNmcXZU-7O9CxVKupOkR6q2S8TXriMeLAUMYmMs4/edit#gid=0

[MKKoppula]

#!/bin/bash

echo "Starting script..."

# First command
ls -la

# Second command
cd /path/to/directory

# Third command
rm *.txt

echo "Script completed."

[MKKoppula]

Splunk can take any data and add it to an intelligent ,searchable ,index. Adding structure to unstructured data, and allowing us to extract all sorts of insights into our business. Not just application issues , but security , user behavior, hardware monitoring , sales totals, etc. We should able to pin point , corelate , and set alerts on specific events. Splunk software enables IT and security teams to get more out of existing security tools, by enabling log aggregation of event data from across the environment into a single repository of critical security insights. Splunk Enterprise can index any kind of data. In particular, any and all IT streaming, machine, and historical data, such as Windows event logs, web server logs, live application logs, network feeds, metrics, change monitoring, message queues, archive files, and so on. 5 main functions of Splunk Enterprise so that way our machine data is available , accessible , and useable Index data - The index collects data from virtually any source and is very important to Splunk.As data enters , Splunk processes data and label sourcetype and standard metadata fields(data and time , index) to the data and then they break the unstructured data to single events. The events are then stored in the index where there are searchable. Search & Investigate - by entering a query into the splunk search bar,we can find events that contains values across multiple data sources allowing us to analyze and run statistics on the vents using Splunk search language. Add knowledge - we can add knowledge objects to the data. These allow us to affect how our data is interpreted, give it classification , add enrichment, normalize it and save reports for future purpose. Monitor & alert - Splunk can proactively monitor all our infrastructure in real-time. To identify issues , problems, and attacks before they impact our customers and services. Report & Analyze - We can create alerts to monitor specific conditions and automatically responds with a variety of actions. Splunk allows us to collect reports and visualizations into dashboards. Empowering groups in organization by giving them the information they need organized into a single pane of glass. Splunk processing Components Indexer - Indexer process incoming machine data, storing the results in indexes as events. As indexers indexes data it creates a number of files organized in sets of directories by age. This organization is important for search. When you search for data , Splunk will only need to open the directories that match the time frame of your search. Making your searches more efficient. Search head. The SH allows users to use the Splunk search Language to search the indexed data, SH handles search requests from users and distribute the requests to the indexers,which performs the searches on the data.SH then consolidate and enriches the results from indexers before returning them to the user. The SH also provides users various tools, such as dashboards, reports and visualizations to assist the search experience Forwarder- Forwarders are Splunk Enterprise instances which consumes data and forward it to the indexers for processing. They require minimal resources, and have very little impact on performance. They reside on the machines where the data originates.As an example if we have a web server we would like to monitor ,We would install the forwarder on that server and have it send data to our indexer. 9:15 Splunk can be deployed in a variety of configurations , it can scale from single instance to a full distributed infrastructure. In a single instance deployment one instance of splunk handles all the functions of Splunk including Input , parsing , indexing and searching of machine data 2 files

image.png image.png 9:15 image.png

image.png 9:15 Monitor processor have many options Files and directories , HTTPO Event Collector , TCP/UDP , Scripts 9:16 image.png

image.png 9:16 image.png

image.png 9:16 First having sepearte indexes can make searches more efficient.being able to use an index as part of search string limits the amount of data Splunk needs to search and returns only the evnts from that index. Multiple indexes also allo us to limit access by user role,letting an admin user control who can see what data. Keeping data in separate indexers will allow us to set retention policies by index. Getting Data In Type of data inputs: Basic searching Search and reporting app allows users to run searches it enables us to create knowledge objects, reports , dashboards and more. Sourcetype - classification of data Source- is the file/directory path , network port , or script from which the event is originated Host-hostname, ip address or FQDN the event from which its originated Using the search bar --> limiting a search by time is key to faster results and is a best practice Once we run a search string , we will have Save as buttot ,search action buttons, 9:16 NDEX MACHINE DATA Index and store any machine data regardless of format or location -- network and endpoint security logs , malware analysis information configurations , sensor data , wire data from networks , change events , data from API's and message queues and even multi line logs from custom applications. With no predefined schema data can be indexed from virtually any source , format or location . SEARCH CORELATE AND INVESTIGATE Search real time and historical data using the same interface . Use familiar search commands to define , limit , or widen your search and corelate events across multiple data sources to reveal new insights . Corelate data based on time , external data , location , sub-searches or joins across multiple data sources . The search assistant offers type-ahead suggestions and contextual help so that you can leverage the full power of the search processing language . DRILL DOWN ANALYSIS Analyze all the data by drilling down , across and back in time quickly using ad-hoc search and timeline controls reveal trends, spikes , and anomalies .Utilize unique field extraction capability to find any value across any field from any data using simple mouse clicks to trace a sequence of events and to quickly find the haystack .Whether your investigating a security alert , responding to an operational outage , or investigating a potential data breach you will get to the answer to minutes rather than hours or days 9:16 Monitor and alert Turn searches into real-time alerts and automatically trigger notifications via email or RSS , generate a ticket at a service desk or execute containment and recovery actions . Alerts can be triggered based on a variety of threshold, trend-based conditions and other complex searches . Gain additional information at the time of alert to assist faster analysis and issue resolution . Reports and dashboards Build reports , advanced graphs and charts to understand important trends , create advanced visualizations , summarize top values and view the frequency of conditions .Create custom dashboards that can integrate multiple charts and views of your real time data. Analyze your data further with chart overlay and pan and zoom controls . Dashboards can be personalized for anyone and allows users to access them from desktops or mobiles . How do I get data in? To get data into your Splunk deployment, point it at a data source. Tell it a bit about the source. That source then becomes a data input. Splunk Enterprise indexes the data stream and transforms it into a series of events. You can view and search those events right away. If the results aren't exactly what you want, you can tweak the indexing process until they are. If you have Splunk Enterprise, the data can be on the same machine as an indexer (local data) or on another machine (remote data). If you have Splunk Cloud, the data resides in your corporate network and you send it to your Splunk Cloud deployment. You can get remote data into your Splunk deployment using network feeds or by installing Splunk forwarders on the hosts where the data originates. For more information on local vs. remote data, see Where is my data? Splunk offers apps and add-ons, with pre-configured inputs for things like Windows- or Linux-specific data sources, Cisco security data, Symantec Blue Coat data, and so on. Look on Splunkbase for an app or add-on that fits your needs. Splunk Enterprise also comes with dozens of recipes for data sources like web server logs, Java 2 Platform, Enterprise Edition (J2EE) logs, or Windows performance metrics. You can get to these from the Add data page in Splunk Web. If the recipes and apps don't cover your needs, then you can use the general input configuration capabilities to specify your particular data source. The categorization of data is done via the index and the sourcetype. If you deal with syslogs, you better check whether your logs conform to the pretrained syslogs described In computing, a log file is a file that records either events that occur in an operating system or other software runs,[1] or messages between different users of a communication software. Logging is the act of keeping a log. In the simplest case, messages are written to a single log file. A transaction log is a file (i.e., log) of the communications (i.e., transactions) between a system and the users of that system,[2] or a data collection method that automatically captures the type, content, or time of transactions made by a person from a terminal with that system.[3] For Web searching, a transaction log is an electronic record of interactions that have occurred during a searching episode between a Web search engine and users searching for information on that Web search engine. Many operating systems, software frameworks and programs include a logging system. A widely used logging standard is syslog, defined in Internet Engineering Task Force (IETF) RFC 5424). The syslog standard enables a dedicated, standardized subsystem to generate, filter, record, and analyze log messages. This relieves software developers of having to design and code their own ad hoc logging systems. 9:17 Event logs record events taking place in the execution of a system in order to provide an audit trail that can be used to understand the activity of the system and to diagnose problems. They are essential to understand the activities of complex systems, particularly in the case of applications with little user interaction (such as server applications). It can also be useful to combine log file entries from multiple sources. This approach, in combination with statistical analysis, may yield correlations between seemingly unrelated events on different servers. Other solutions employ network-wide querying and reporting.[7][8] Transaction logs[edit] Main article: Transaction log Most database systems maintain some kind of transaction log, which are not mainly intended as an audit trail for later analysis, and are not intended to be human-readable. These logs record changes to the stored data to allow the database to recover from crashes or other data errors and maintain the stored data in a consistent state. Thus, database systems usually have both general event logs and transaction logs. [9][10][11][12] Message logs[edit] Internet Relay Chat (IRC), instant messaging (IM) programs, peer-to-peer file sharing clients with chat functions, and multiplayer games (especially MMORPGs) commonly have the ability to automatically log (i.e. save) textual communication, both public (IRC channel/IM conference/MMO public/party chat messages) and private chat messages between users. [13] Message logs are almost universally plain text files, but IM and VoIP clients (which supports textual chat, e.g. Skype) might save them in HTML files or in a custom format to ease reading and encryption. syslog /ˈsɪslɒɡ/ is a standard for message logging. It allows separation of the software that generates messages, the system that stores them, and the software that reports and analyzes them. Each message is labeled with a facility code, indicating the software type generating the message, and assigned a severity level. Application Event Log There are several types of event logs maintained by the Windows operating system. One of these is the Application event log. This log is used to record events written by applications and services. The applications may be commercial applications, like SQL Server or Exchange, or they may be custom applications that you've developed yourself. The events written to the Application event log can run the gamut from application startup events to shutdown events to "heartbeat" events to run-time error events. The same holds true for events written by Windows services. Like events written to other event logs, some of the important elements written to the Application log include the date and time when the event occurred, the event ID, and the event source.

[MKKoppula]

https://www.edureka.co/blog/what-is-splunk/

[MKKoppula]

@version:3.2 @include "scl.conf" @include "/etc/syslog-ng/buckets.d/"

options { chain_hostnames(no); time_reopen(60); flush_lines(5000); log_fifo_size(50000000); flush_lines(5000); flush_timeout(5000); create_dirs(yes); group(splunk); perm(0775); dir_owner(syslog-ng); dir_group(splunk); dir_perm(0755); log_msg_size(32768); stats_freq(300); stats_level(1); mark_freq(300); check_hostname(yes); bad_hostname("^gconfd$"); dns_cache(yes); dns_cache_size(7000); dns_cache_expire(5000); use_dns(yes); keep_hostname(yes); chain_hostnames(yes); use_fqdn(yes); keep_timestamp(no);

};

source s_network {

    udp ( ip() port(514)   so_rcvbuf (805306368) so_sndbuf(8096) time_zone(GMT) keep_timestamp(no) );
    tcp ( ip() port(514)   max-connections(5000)  so_rcvbuf (805306368) so_sndbuf(8096) time_zone(GMT) keep_timestamp(no) );
    tcp ( ip() port(18184) max-connections(5000)  so_rcvbuf (805306368) so_sndbuf(8096) time_zone(GMT) keep_timestamp(no) );
    tcp ( ip() port(9010)  max-connections(5000)  so_rcvbuf (805306368) so_sndbuf(8096) time_zone(GMT) keep_timestamp(no) );
    tcp ( ip() port(9515)  max-connections(5000)  so_rcvbuf (805306368) so_sndbuf(8096) time_zone(GMT) keep_timestamp(no) );
    udp ( ip() port(9010)  so_rcvbuf (805306368) so_sndbuf(8096) time_zone(GMT) keep_timestamp(no) );
    udp ( ip() port(9514)  so_rcvbuf (805306368) so_sndbuf(8096) time_zone(GMT) keep_timestamp(no) );

};

destination d_logstats { file("/opt/syslog/LOGSTATS/logstats.log" owner(syslog-ng) group(splunk) perm(0644) dir_perm(0750) create_dirs(yes)); };

filter f_logstats { match("Log statistics;" value ("MESSAGE")); };