Setup a centralized Log Server with Rsyslog
Every Linux distribution comes equipped with some kind of logging mechanism that logs various types of activities that occurs on that system.
In this tutorial, I'll be walking you through some simple steps to setup a popular remote logging service called Rsyslog. We will setup a central logging server, and send logs from individual servers to this central logging server.
This setup will help you to analyze the log files of all the Servers/ Switches/ Routers in your infrastructure from a central log server.
To get started:
I'll be using a CentOS 6.5 Minimal system which has a static IP address along with a FQDN, however the steps remain the same even for a system based on Debian distributions (Just change yum to sudo and voila!!).
The first thing to do is set a static IP for our Rsyslog Server. Simply append the static IP address and the required details in the ifcfg-eth0 file as shown below:
# vi /etc/sysconfig/network-scripts/ifcfg-eth0
# Add the following info
You will also need to provide a FQDN for your Logging Server. Edit the hosts file and set the server's FQDN as shown:
# vi /etc/hosts
# Append the following line to the file:
192.168.50.150 syslog.cloud.com syslog
In this tutorial, I have disabled the Firewall and SElinux, however I wont recommend doing this on a production server at all.
Once the system is prepped, we can now start with the actual installation and configuration process.
Install and Configure Rsyslog
NOTE: You need to execute the following commands with root privileges
# yum install rsyslog*
NOTE: For Ubuntu systems, run sudo apt-get install rsyslog*
Once the packages are downloaded, edit the rsyslog conf file and add the following few commands herein.
## uncomment the following lines##
## Add the following lines ##
$AllowedSender UDP, 127.0.0.1, 192.168.50.0/24 $AllowedSender TCP, 127.0.0.1, 192.168.50.0/24
Once your done with the basic configurations, you can start running your Rsyslog Server as it is, however, its important to understand how actually the logging does take place as well.
Syslog consists of 3 primary items: Facilities, Priorities and Actions. These can be altered to your choice in the "Rules" section of your rsyslog.conf file as shown below:
The source of a log message is referred to as a Facility.
There is no way to define your own facilities but there are many predefined ones (up to 23 in all, depending on which syslog you use):
|Security events get logged with this|
|User access messages use this|
|For cron, at, and anacron, but not for the programs started by cron|
|Other daemon programs without a facility of their own|
|Used by syslogd to produce timestamps in log files|
|For user programs|
|Obsolete form of networking|
local0 – local7
|Any use; RH uses local7 for boot messages|
When you specify a priority code, all messages with that priority and higher are logged at the specified destination. For example, if you specify a priority code of crit, all messages having alert, panic, emerg, and crit priorities are logged. The following are a list of Priorities that are commonly used:
|A panic condition was reported to all processes.|
|A condition that should be corrected immediately.|
|A critical condition.|
|An error message.|
|A warning message.|
|A condition requiring special handling.|
|A general information message.|
|A message useful for debugging programs.|
|Do not log any messages for the facility.|
|Place holder used to represent all priorities.|
When specifying a priority, that and all higher ones are selected too. A selector is one or more facilities (separated by commas), a dot, then the priority. More complex selectors are possible too) Some example are shown selectors:
|mail facility, any priority|
|mail facility, debug or higher priority (same as *)|
|all messages from mail or news|
|all security messages of warning or higher priority|
|all messages from any facility except debug msgs|
|any facility, info msgs only (and not higher)|
|any facility, pri <= err only|
|any facility, any priority except alert|
Actions refer to what action to take when Log messages don't only have to go to files, you can direct them to user terminals, run them through other programs (with a pipe, to email, pager, or just a log file analyzer), or send them to another host running rsyslog.
Some commonly used examples are as follows:
|/path/of/some/file||A path to a file on the local system|
|/dev/console||This is a link to the system console|
|/complete/path/of/some/file||Don't flush file each time; better performance but risks loss of some log info.|
|username1[,username2 ...]||Log to specific users|
|*||All logged in users|
Once you have made the appropriate changes in your rsyslog file, you can now startup the rsyslog service using the following command:
# service rsyslog start
# chkconfig rsyslog on
In my case, I had made a slight change in my rsyslog.conf file's "rules" section. I had changed the default logging destination for one of the rules to a custom file called "syslogmsgs" (By default, these logs would have been placed in the /var/log/messages file).
You can see from the image below that the syslog was successfully configured and now able to receive logs from my Local System (syslog.cloud.com). But what if I want to enable remote logging? Redirect logs from different Linux distributions to this particular Log Server?
Well, thats a really simple step actually.. most linux distributions today come with the rsyslog package pre-installed in them. If not, all you need to do is simply install the necessary package using the root privileges:
$ sudo apt-get install rsyslog
Once installed, all you need to do is append the remote syslog server's host IP/ FQDN in the client linux machine. The syntax is as follows:
# Run the following commands in your Client Linux System (in my case, my client linux server is named Ubuntu-VM)
$ vi /etc/rsyslog.conf
# Append the following line towards the end of the file
*.* @@192.168.50.150 #syslog.cloud.com IP address
Once entered, save and exit the editor and simply start the rsyslog service on your client:
$ service rsyslog start
You can check your Remote Logging server to see if it is receiving the Logs from your Client Linux or not. In my case, it worked perfectly.
You can even send logs to a remote Logging Server from a Network Router or a Switch as well. In my case, I have a Vyatta Router running in my environment. All I need to do is enable syslog and set the remote logging host to start the logging activity.
NOTE: The following command works ONLY on a vyatta system.
set system syslog host <REMOTE_LOG_SERVER_IP> facility <FACILITY_TYPE> level <LOG_PRIORITY>
Once again, if I check my remote Logging Server, I am now able to receive logs from my Vyatta Router as well.
Well, thats it for this tutorial folks!! Hope you liked it!!
In my next post I shall show you How to enable remote logging for your ESXi and XenServer as well.. so stay tuned!