Vovando Dev

Log.io is a small simple but effective application build on top of Node.js and Socket.io, which allows to monitor Linux servers log files in real time through web interface screen widgets.

How to install and configure log.io:

Step 1: Add Epel Repositories

1. CentOS Epel repositories provides the binary packages for Node.js and NPM – Node Packaged Modules. Install Epel repositories by issuing the following command.

On RHEL/CentOS 6

--------------------- On RHEL/CentOS 6.x - 32 Bit ---------------------
# yum install http://fedora.mirrors.telekom.ro/pub/epel/6/i386/epel-release-6-8.noarch.rpm

--------------------- On RHEL/CentOS 6.x - 64 Bit ---------------------
# yum install http://fedora.mirrors.telekom.ro/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

On RHEL/CentOS 6

# yum install http://fedora.mirrors.telekom.ro/pub/epel/7/x86_64/e/epel-release-7-2.noarch.rpm

Step 2: Install Node.js and NPM Packages

2. Node.js is a Javascript server-side programming platform which allows you to create network applications with backend functionality. NPM (Node Package Manager) is practically the package manager for Node.js. So, on the next step go ahead and install Node.js and NMP binaries on your system through YUM package manager by issuing the following commands.

# curl --silent --location https://rpm.nodesource.com/setup_5.x | bash - 
# yum install -y nodejs

Step 3: Install and Configure Log.io Application

3. Log.io application must be installed on your system through NPM by specifying a valid local system user, through which the installation must take place. While you can use any valid system user to install Log.io, I personally recommend installing the application through root user or other system user with root privileges.

The reason for using this approach is that Log.io must have access to read locally log files and a user with non-privileges root privileges usually can’t access and read some important log files.

So, login with root account and install Log.io application through root account by issuing the following command (if you use other user replace root account with your system user accordingly).

# npm install -g log.io --user “root”

4. Now it’s time to configure Log.io to monitor local log files in real time. Let’s get an inside on how Log.io works.

[root@app .log.io]# ls
harvester.conf  log_server.conf  web_server.conf

1. The harvester file watches for changes in the specified local log files declared in its configuration and sends its output via socket.io TCP
   protocol which further send the messages to Log.io local server or any other remote server declared with its IP Address ( 0.0.0.0 address specified on harvesters broadcasts to all log.io listening servers) – fileharvester.conf
2. Log.io server binds on all network interfaces (if not specified otherwise in log_server.conf file) and waits for messages from locally or remote harvesters nodes and sends their output to log.io Web server (0.0.0.0 means that it waits for messages from any local or remote harvesters) file log_server.conf
3. Log.io Web server binds on all network interfaces, listens for web clients connections on port 28778 and processes and outputs the messages that it receives internally from log.io server – file web_server.conf

First open harvester.conf file for editing, which by default only monitors Apache log files, and replace nodeNamestatement to match your hostname and define the logStreams statements with what internal log files you want to monitor (in this case I’m monitoring multiple log files such as audit, messages and secure logs). Use the below file excerpt as a guide.

exports.config = {
  nodeName: "unicornappsrv",
  logStreams: {
    unicorn_app_log: [
      "/opt/app/current/log/app.log"
    ],
    unicorn_stderr_log: [
      "/opt/app/current/log/unicorn.stderr.log"
    ],
    unicorn_stdout_log: [
      "/opt/app/current/log/unicorn.stdout.log"
    ],
    sidekiq_log: [
      "/opt/app/current/log/sidekiq.log"
    ],
  },
  server: {
    host: '127.0.0.1',
    port: 28777
  }
}

Also if you don’t need harvester output to be sent to a remote Log.io server change the line host on serverstatement to only send its output locally by modifying 0.0.0.0 address with loopback address (127.0.0.1).

5. For security reasons, if you are not expecting remote harvesters output to your local Log.io server openlog_server.conf file and replace 0.0.0.0 address with loopback address (127.0.0.1).

exports.config = {
  host: '127.0.0.1',
  port: 28777
}

6. Other security features such as credentials login, HTTPS or restriction based on IPs to Log.io web server can be applied on web server-side. For this tutorial I will only use as a security measure just credential login.

So, open web_server.conf file, uncomment the entire auth statement by deleting all slashes and asterisks and replace user and pass directives accordingly as suggested on the bottom screenshot.

exports.config = {
  host: '0.0.0.0',
  port: 28778,

  // Enable HTTP Basic Authentication
  auth: {
    user: "admin",
    pass: "admin"
  },

  /*
  // Enable HTTPS/SSL
  ssl: {
    key: '/path/to/privatekey.pem',
    cert: '/path/to/certificate.pem'
  },
  */
  /*
  // Restrict access to websocket (socket.io)
  // Uses socket.io 'origins' syntax
  restrictSocket: '*:*',
  */
  /*
  // Restrict access to http server (express)
  restrictHTTP: [
    "192.168.29.39",
    "10.0.*"
  ]
  */
}

Step 4: Add Firewall Rule and Start Log.io Application

7. In order to gain web access to Log.io server add a rule on RHEL/CentOS 6/7 Firewall to open TCP 28778 port.

Step 5: Start Log.io Application and enter Web Interface

8. In order to start Log.io log monitoring application assure that your current working directory is root’s home .log.io and use the following commands in the following order to start application.

------------ First start server, put it in background and press Enter key ------------
# log.io-server & 

------------ Start log harvester in background ------------
# log.io-harvester & 

9. After the server has been started open a browser, enter your servers IP followed by 28778 port number using HTTP protocol on URL address and a prompt demanding your login credentials should appear.

Enter your user and password configured on step 6 to proceed further and Log.io application should now be visible on your browser presenting monitored log files in real time.

http://X.X.X.X:28778

On Web interface add new screens and organize your streams or nodes accordingly.

10. To stop Log.io application run the following command.

# pkill node

Step 6: Create Log.io Manage Script

11. In order to use a command that manages Log.io application with three switches ( start, stop and status) create the following script named log.io on /usr/local/bin executable directory and append execution permissions to this script.

# nano /usr/local/bin/log.io
# chmod +x /usr/local/bin/log.io

Add the following excerpt to this script file.

#!/bin/bash
                start() {
                echo "Starting log.io process..."
                /usr/local/bin/log.io-server &
                /usr/local/bin/log.io-harvester &
                                         }
                stop() {
                echo "Stopping io-log process..."
                pkill node
                                         }
                status() {
                echo "Status io-log process..."
                netstat -tlp | grep node
                                         }
case "$1" in
                start)
start
        ;;
                stop)
stop
        ;;
                status)
status
                ;;
                *)
echo "Usage: start|stop|status"
        ;;
esac

12. To start, stop or view Log.io status login with root account (or the user that Log.io app has been installed) and just run the following commands to easily manage the application.

# log.io start
# log.io status
# log.io stop

[root@app .log.io]# log.io status
Status io-log process…
tcp 0 0 app.com,:28777 *:* LISTEN 2133/node
tcp 0 0 *:28778 *:* LISTEN 2133/node
[root@qa-app .log.io]#