HAProxy 1.7.X Installation and configuration for IMPALA

Steps:

1. Downloaded HAProxy to Impala server.

           # wget http://www.haproxy.org/download/1.7/src/haproxy-1.7.2.tar.gz

 

2. Installed dependencies

           # yum install  gcc pcre-static pcre-devel -y

3. Untar source and change directory

           #tar xzvf ~/haproxy.tar.gz -C ~/

 Change into the directory.

            #cd ~/haproxy-1.7.2

4. Then compile the program for your system.

             #make TARGET=linux2628

and finally install HAProxy itself.

            # make install

5. To complete the install, use the following commands to copy the settings over.

           # cp /usr/local/sbin/haproxy  /usr/sbin/

           # cp /haproxy-1.7.2/examples/haproxy.init  /etc/init.d/haproxy

           # chmod 755 /etc/init.d/haproxy

6. Create these directories and the statistics file for HAProxy to record in.

         # mkdir -p /etc/haproxy

         # mkdir -p /run/haproxy

        # mkdir -p /var/lib/haproxy

       # touch /var/lib/haproxy/stats

7. Then add a new user for HAProxy.

          # useradd -r haproxy

 

Configuring the load balancer

Setting up HAProxy for load balancing is a quite straight forward process. Basically all you need to do is tell HAProxy what kind of connections it should be listening for and which servers it should relay the connections to. This can be done by creating a configuration file /etc/haproxy/haproxy.cfg with the required settings. For documentation help please go to HAProxy Documentation

make sure you have following setting for HAProxy-1.7.2

 # Vim  /etc/haproxy/haproxy.cfg

______________________________________________
______________________________________________

# HAProxy Server Version 1.7.2
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
    # to have these messages end up in /var/log/haproxy.log you will
    # need to:
    #
    # 1) configure syslog to accept network log events.  This is done
    #    by adding the '-r' option to the SYSLOGD_OPTIONS in
    #    /etc/sysconfig/syslog
    #
    # 2) configure local2 events to go to the /var/log/haproxy.log
    #   file. A line like the following can be added to
    #   /etc/sysconfig/syslog
    #
    #    local2.*                       /var/log/haproxy.log
    #
    log         127.0.0.1 local2
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon
    # turn on stats unix socket
    stats socket /var/lib/haproxy/stats

#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    192s
    timeout queue           1m
    timeout connect         192s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 192s
    timeout check           192s
    maxconn                 3000

#kz : Timeout connect 3600000
    timeout client 3600000
    timeout server 3600000
#####################################################
##Default Timeout Settings###########################
#    timeout connect 5000
#    timeout client 50000
#    timeout server 50000
#####################################################
#
# This sets up the admin page for HA Proxy at port 25002.
#
listen stats
   bind *:25002
    balance
    mode http
#    stats enable
#    stats hide-version
#    stats scope .
#    stats realm Haproxy\ Statistics
#    stats uri /
#    stats auth prni01:haproxy
    log global
    stats enable
    stats hide-version
    stats refresh 30s
    stats show-node
    stats auth haproxy:h@pr0xy
    stats uri  /haproxy?stats
# This is the setup for Impala. Impala client connect to load_balancer_host:25003.
# HAProxy will balance connections among the list of servers listed below.
# The list of Impalad is listening at port 21000 for beeswax (impala-shell) or original ODBC driver.
# For JDBC or ODBC version 2.x driver, use port 21050 instead of 21000.
#Config settings for Impala Shell
listen impalashell
   bind *:25003
    mode tcp
    option tcplog
    balance leastconn
#List of Impala Daemons
server  192.168.16.17  192.168.16.17:21000
server  192.168.16.18  192.168.16.18:21000
server  192.168.16.19  192.168.16.19:21000
server  192.168.16.21  192.168.16.21:21000
server  192.168.16.22  192.168.16.22:21000
server  192.168.16.23  192.168.16.23:21000
server  192.168.16.25  192.168.16.25:21000
server  192.168.16.30  192.168.16.30:21000
server  192.168.16.31  192.168.16.31:21000
server  192.168.16.32  192.168.16.32:21000
server  192.168.16.33  192.168.16.33:21000
server  192.168.16.34  192.168.16.34:21000
server  192.168.16.35  192.168.16.35:21000
server  192.168.16.36  192.168.16.36:21000
server  192.168.16.37  192.168.16.37:21000
server  192.168.16.38  192.168.16.38:21000
server  192.168.16.39  192.168.16.39:21000
server  192.168.16.40  192.168.16.40:21000
server  192.168.16.41  192.168.16.41:21000
server  192.168.16.42  192.168.16.42:21000
server  192.168.16.43  192.168.16.43:21000
server  192.168.16.44  192.168.16.44:21000
server  192.168.16.45  192.168.16.45:21000
server  192.168.16.46  192.168.16.46:21000
server  192.168.16.47  192.168.16.47:21000
#Config settings for Impala JDBC
listen impalajdbc
   bind *:25004
    mode tcp
    option tcplog
    balance leastconn
#List of Impala Daemons
server  192.168.16.17  192.168.16.17:21050
server  192.168.16.18  192.168.16.18:21050
server  192.168.16.19  192.168.16.19:21050
server  192.168.16.21  192.168.16.21:21050
server  192.168.16.22  192.168.16.22:21050
server  192.168.16.23  192.168.16.23:21050
server  192.168.16.25  192.168.16.25:21050
server  192.168.16.30  192.168.16.30:21050
server  192.168.16.31  192.168.16.31:21050
server  192.168.16.32  192.168.16.32:21050
server  192.168.16.33  192.168.16.33:21050
server  192.168.16.34  192.168.16.34:21050
server  192.168.16.35  192.168.16.35:21050
server  192.168.16.36  192.168.16.36:21050
server  192.168.16.37  192.168.16.37:21050
server  192.168.16.38  192.168.16.38:21050
server  192.168.16.39  192.168.16.39:21050
server  192.168.16.40  192.168.16.40:21050
server  192.168.16.41  192.168.16.41:21050
server  192.168.16.42  192.168.16.42:21050
server  192.168.16.43  192.168.16.43:21050
server  192.168.16.44  192.168.16.44:21050
server  192.168.16.45  192.168.16.45:21050
server  192.168.16.46  192.168.16.46:21050
server  192.168.16.47  192.168.16.47:21050

_________________________________________________
_________________________________________________

Change settings as per attached haproxy.cfg file (please note configuration properties may change depends upon the version you are using.)

 

Then start the haproxy service.

# service haproxy start/restart    /// # systemctl start haproxy

Check HA on impala-shell from 'haproxy server' host.

#impala-shell  -I <impala_daemon_host>:25003 (this port is given for impala-shell)

(you should be able to login and check databases)

NOTE: Please check your firewall setting if you are facing any issue


................DONE................