[ULTRAMONKEY-USERS] HA / LB setup - Cannot Access Services via VIP through either Load Balancer

Robert Dudley rob.dudley at gmail.com
Wed Jun 20 20:28:42 EST 2007


Hello all,

I have a bit of a problem with my UM setup that I'm hoping you can  
assist me with. Please understand that I do not claim to be a LB  
expert and may have missed something fundamental when configuring so  
please bear with me.

Simply put, I have a setup (detailed below) where by 2 LBs running in  
HA mode can happily see each other and the shared resource (in this  
case a mail server running Sendmail on port 25). The heartbeat and  
ldirectord daemons are running and fail over between the two load  
balancers appears to work (see LOGS).

The problem occurs when attempting to access the mail server via the  
VIP (using telnet to port 25). Regardless of which LB is currently  
active I can't connect to the server:

$ telnet 192.168.155.150 25
Trying 192.168.155.150...
telnet: connect to address 192.168.155.150: Connection refused
telnet: Unable to connect to remote host

I've scoured the list archives and can't see any one with a similar  
issue. Does anyone have any ideas as to what I need to look at to get  
this working properly?

I've enclosed as much config / log data as I can think to (apologies  
for bloating the message slightly) to give you a better picture of  
what's going on. All IPs and hostnames are real. Only the domain name  
has been changed to protect the innocent :-)

Thank you so much in advance.

Rob

SETUP
-----

2xLBs with dual NICs running UM3 on RHEL 4
1x Mail server running Ubuntu 6.06 LTS and Sendmail
Virtual IP is 192.168.155.150
I am jacked into Switch 1

                        Switch 1
                    (192.168.155.*)
                           |
        ---------------------------------------
       |                                       |
eth0 (155.151)                          eth0 (155.152)
     LBTEMP                                 LBTEMP2
eth1 (200.151)                          eth1 (200.152)
       |                                       |
        ----------------------------------------
                            |
                        Switch 2
                     (192.168.200.*)
                            |
                       Mail Server
                   (real 192.168.200.201)

CONFIG FILES
------------

LBTEMP

/etc/ha.d/ldirectord.cf

# Global Directives
checktimeout=10
checkinterval=10
#fallback=127.0.0.1:80
autoreload=no
logfile="/var/log/ldirectord.log"
#logfile="local0"
quiescent=yes

## Virtual Service for SMTP
virtual=192.168.155.150:25
         #fallback=127.0.0.1:25
         real=192.168.200.201:25 masq 7
         real=192.168.200.202:25 masq 5
         real=192.168.200.203:25 masq 10
         real=192.168.200.204:25 masq 10
         service=smtp
         scheduler=wrr
         #persistent=600
         protocol=tcp

virtual=192.168.155.150:221
        real=192.168.200.201:22 masq
        #real=192.168.7.5:23 masq
        #fallback=127.0.0.1:21
        service=none
        scheduler=wlc
        #request="welcome.msg"
        #receive="test"
        persistent=3600
        #netmask=255.255.255.255
        protocol=tcp

=======================

/etc/ha.d/ha.cf

debugfile /var/log/ha-debug
logfile /var/log/ha-log
logfacility     local0
serial  /dev/ttyS0      # Linux
bcast   eth0            # Linux
mcast eth0 225.0.0.1 694 1 0
mcast eth1 225.0.0.1 694 1 0
auto_failback off
node    lbtemp.mydomain.com
node    lbtemp2.mydomain.com
ping 192.168.155.254
ping 192.168.200.254
respawn hacluster /usr/lib/heartbeat/ipfail
ist.
apiauth ipfail gid=haclient uid=hacluster

=======================

/etc/ha.d/haresources

lbtemp.mydomain.com \
         IPaddr2::192.168.155.150/24/eth0 \
         IPaddr2::192.168.200.150/24/eth1 \
         ldirectord::ldirectord.cf \
         LVSSyncDaemonSwap::master

=======================

LBTEMP2

/etc/ha.d/ldirectord.cf

# Global Directives
checktimeout=10
checkinterval=10
#fallback=127.0.0.1:80
autoreload=no
logfile="/var/log/ldirectord.log"
#logfile="local0"
quiescent=yes

## Virtual Service for SMTP
virtual=192.168.155.150:25
         #fallback=127.0.0.1:25
         real=192.168.200.201:25 masq 7
         real=192.168.200.202:25 masq 5
         real=192.168.200.203:25 masq 10
         real=192.168.200.204:25 masq 10
         service=smtp
         scheduler=wrr
         #persistent=600
         protocol=tcp

virtual=192.168.155.150:221
        real=192.168.200.201:22 masq
        #real=192.168.7.5:23 masq
        #fallback=127.0.0.1:21
        service=none
        scheduler=wlc
        #request="welcome.msg"
        #receive="test"
        persistent=3600
        #netmask=255.255.255.255
        protocol=tcp

=======================

/etc/ha.d/ha.cf

debugfile /var/log/ha-debug
logfile /var/log/ha-log
logfacility     local0
udpport 694
serial  /dev/ttyS0      # Linux
bcast   eth0            # Linux
mcast eth0 225.0.0.1 694 1 0
mcast eth1 225.0.0.1 694 1 0
auto_failback off
node    lbtemp.mydomain.com
node    lbtemp2.mydomain.com
ping 192.168.155.254
ping 192.168.200.254
respawn hacluster /usr/lib/heartbeat/ipfail
apiauth ipfail gid=haclient uid=hacluster

=======================

/etc/ha.d/haresources

=======================

lbtemp.mydomain.com \
         IPaddr2::192.168.155.150/24/eth0 \
         IPaddr2::192.168.200.150/24/eth1 \
         ldirectord::ldirectord.cf \
         LVSSyncDaemonSwap::master

LOGS
----

LBTEMP - ha-log

heartbeat: 2007/06/20_07:14:06 info: **************************
heartbeat: 2007/06/20_07:14:06 info: Configuration validated.  
Starting heartbeat 1.2.3.cvs.20050927
heartbeat: 2007/06/20_07:14:06 info: heartbeat: version 1.2.3.cvs. 
20050927
heartbeat: 2007/06/20_07:14:06 info: Heartbeat generation: 9
heartbeat: 2007/06/20_07:14:06 info: Starting serial heartbeat on  
tty /dev/ttyS0 (19200 baud)
heartbeat: 2007/06/20_07:14:06 info: UDP Broadcast heartbeat started  
on port 694 (694) interface eth0
heartbeat: 2007/06/20_07:14:06 info: UDP multicast heartbeat started  
for group 225.0.0.1 port 694 interface eth0 (ttl=1 loop=0)
heartbeat: 2007/06/20_07:14:06 info: UDP multicast heartbeat started  
for group 225.0.0.1 port 694 interface eth1 (ttl=1 loop=0)
heartbeat: 2007/06/20_07:14:06 info: ping heartbeat started.
heartbeat: 2007/06/20_07:14:06 info: ping heartbeat started.
heartbeat: 2007/06/20_07:14:06 info: pid 6183 locked in memory.
heartbeat: 2007/06/20_07:14:06 info: Local status now set to: 'up'
heartbeat: 2007/06/20_07:14:06 info: pid 6194 locked in memory.
heartbeat: 2007/06/20_07:14:07 info: pid 6186 locked in memory.
heartbeat: 2007/06/20_07:14:07 info: pid 6187 locked in memory.
heartbeat: 2007/06/20_07:14:07 info: pid 6188 locked in memory.
heartbeat: 2007/06/20_07:14:07 info: pid 6189 locked in memory.
heartbeat: 2007/06/20_07:14:07 info: pid 6190 locked in memory.
heartbeat: 2007/06/20_07:14:07 info: Link lbtemp.mydomain.com:eth0 up.
heartbeat: 2007/06/20_07:14:07 info: pid 6191 locked in memory.
heartbeat: 2007/06/20_07:14:07 info: pid 6192 locked in memory.
heartbeat: 2007/06/20_07:14:07 info: pid 6193 locked in memory.
heartbeat: 2007/06/20_07:14:07 info: pid 6195 locked in memory.
heartbeat: 2007/06/20_07:14:07 info: pid 6196 locked in memory.
heartbeat: 2007/06/20_07:14:07 info: pid 6197 locked in memory.
heartbeat: 2007/06/20_07:14:07 info: pid 6198 locked in memory.
heartbeat: 2007/06/20_07:14:23 info: Link lbtemp2.mydomain.com:eth0 up.
heartbeat: 2007/06/20_07:14:23 info: Status update for node  
lbtemp2.mydomain.com: status up
heartbeat: 2007/06/20_07:14:23 info: Running /etc/ha.d/rc.d/status  
status
heartbeat: 2007/06/20_07:14:23 info: Link lbtemp2.mydomain.com:eth1 up.
heartbeat: 2007/06/20_07:14:37 WARN: node 192.168.155.254: is dead
heartbeat: 2007/06/20_07:14:37 WARN: node 192.168.200.254: is dead
heartbeat: 2007/06/20_07:14:37 info: Local status now set to: 'active'
heartbeat: 2007/06/20_07:14:37 info: Starting child client "/usr/lib/ 
heartbeat/ipfail" (1001,104)
heartbeat: 2007/06/20_07:14:37 info: Running /etc/ha.d/rc.d/status  
status
heartbeat: 2007/06/20_07:14:37 info: Starting "/usr/lib/heartbeat/ 
ipfail" as uid 1001  gid 104 (pid 6204)
heartbeat: 2007/06/20_07:14:37 info: Running /etc/ha.d/rc.d/status  
status
heartbeat: 2007/06/20_07:14:52 info: Status update for node  
lbtemp2.mydomain.com: status active
heartbeat: 2007/06/20_07:14:52 info: Running /etc/ha.d/rc.d/status  
status
heartbeat: 2007/06/20_07:15:02 info: remote resource transition  
completed.
heartbeat: 2007/06/20_07:15:02 info: remote resource transition  
completed.
heartbeat: 2007/06/20_07:15:02 info: Initial resource acquisition  
complete (T_RESOURCES(us))
heartbeat: 2007/06/20_07:15:02 info: Local Resource acquisition  
completed.
heartbeat: 2007/06/20_07:15:02 info: Running /etc/ha.d/rc.d/ip- 
request-resp ip-request-resp
heartbeat: 2007/06/20_07:15:02 received ip-request-resp  
IPaddr2::192.168.155.150/24/eth0 OK yes
heartbeat: 2007/06/20_07:15:02 info: Acquiring resource group:  
lbtemp.mydomain.com IPaddr2::192.168.155.150/24/eth0  
IPaddr2::192.168.200.150/24/eth1 ldirectord::ldirectord.cf  
LVSSyncDaemonSwap::master
heartbeat: 2007/06/20_07:15:02 info: Running /etc/ha.d/resource.d/ 
IPaddr2 192.168.155.150/24/eth0 start
heartbeat: 2007/06/20_07:15:02 info: /sbin/ip -f inet addr add  
192.168.155.150/24 brd 192.168.155.255 dev eth0
heartbeat: 2007/06/20_07:15:02 info: /sbin/ip link set eth0 up
heartbeat: 2007/06/20_07:15:02 /usr/lib/heartbeat/send_arp -i 200 -r  
5 -p /var/lib/heartbeat/rsctmp/send_arp/send_arp-192.168.155.150 eth0  
192.168.155.150 auto 192.168.155.150 ffffffffffff
heartbeat: 2007/06/20_07:15:02 info: Running /etc/ha.d/resource.d/ 
IPaddr2 192.168.200.150/24/eth1 start
heartbeat: 2007/06/20_07:15:02 info: /sbin/ip -f inet addr add  
192.168.200.150/24 brd 192.168.200.255 dev eth1
heartbeat: 2007/06/20_07:15:02 info: /sbin/ip link set eth1 up
heartbeat: 2007/06/20_07:15:02 /usr/lib/heartbeat/send_arp -i 200 -r  
5 -p /var/lib/heartbeat/rsctmp/send_arp/send_arp-192.168.200.150 eth1  
192.168.200.150 auto 192.168.200.150 ffffffffffff
heartbeat: 2007/06/20_07:15:03 info: Running /etc/ha.d/resource.d/ 
ldirectord ldirectord.cf start
heartbeat: 2007/06/20_07:15:03 info: Running /etc/ha.d/resource.d/ 
LVSSyncDaemonSwap master start
heartbeat: 2007/06/20_07:15:03 info: ipvs_syncbackup down
heartbeat: 2007/06/20_07:15:03 info: ipvs_syncmaster up
heartbeat: 2007/06/20_07:15:03 info: ipvs_syncmaster obtained


heartbeat: 2007/06/20_07:17:58 info: Heartbeat shutdown in progress.  
(6183)
heartbeat: 2007/06/20_07:17:58 info: Giving up all HA resources.
heartbeat: 2007/06/20_07:17:58 info: Releasing resource group:  
lbtemp.mydomain.com IPaddr2::192.168.155.150/24/eth0  
IPaddr2::192.168.200.150/24/eth1 ldirectord::ldirectord.cf  
LVSSyncDaemonSwap::master
heartbeat: 2007/06/20_07:17:58 info: Running /etc/ha.d/resource.d/ 
LVSSyncDaemonSwap master stop
heartbeat: 2007/06/20_07:17:58 info: ipvs_syncmaster down
heartbeat: 2007/06/20_07:17:58 info: ipvs_syncbackup up
heartbeat: 2007/06/20_07:17:58 info: ipvs_syncmaster released
heartbeat: 2007/06/20_07:17:58 info: Running /etc/ha.d/resource.d/ 
ldirectord ldirectord.cf stop
heartbeat: 2007/06/20_07:17:58 info: Running /etc/ha.d/resource.d/ 
IPaddr2 192.168.200.150/24/eth1 stop
heartbeat: 2007/06/20_07:17:58 info: /sbin/ip -f inet addr delete  
192.168.200.150 dev eth1
heartbeat: 2007/06/20_07:17:58 info: /sbin/ip -o -f inet addr show eth1
heartbeat: 2007/06/20_07:17:58 info: IP Address 192.168.200.150 released
heartbeat: 2007/06/20_07:17:59 info: Running /etc/ha.d/resource.d/ 
IPaddr2 192.168.155.150/24/eth0 stop
heartbeat: 2007/06/20_07:17:59 info: /sbin/ip -f inet addr delete  
192.168.155.150 dev eth0
heartbeat: 2007/06/20_07:17:59 info: /sbin/ip -o -f inet addr show eth0
heartbeat: 2007/06/20_07:17:59 info: IP Address 192.168.155.150 released
heartbeat: 2007/06/20_07:17:59 info: killing /usr/lib/heartbeat/ 
ipfail process group 6204 with signal 15
heartbeat: 2007/06/20_07:17:59 info: All HA resources relinquished.
heartbeat: 2007/06/20_07:17:59 info: killing /usr/lib/heartbeat/ 
ipfail process group 6204 with signal 15
heartbeat: 2007/06/20_07:17:59 WARN: 1 lost packet(s) for  
[lbtemp2.mydomain.com] [225:227]
heartbeat: 2007/06/20_07:17:59 info: No pkts missing from  
lbtemp2.mydomain.com!
heartbeat: 2007/06/20_07:18:00 info: killing HBWRITE process 6193  
with signal 15
heartbeat: 2007/06/20_07:18:00 info: killing HBREAD process 6194 with  
signal 15
heartbeat: 2007/06/20_07:18:00 info: killing HBWRITE process 6195  
with signal 15
heartbeat: 2007/06/20_07:18:00 info: killing HBREAD process 6196 with  
signal 15
heartbeat: 2007/06/20_07:18:00 info: killing HBFIFO process 6186 with  
signal 15
heartbeat: 2007/06/20_07:18:00 info: killing HBWRITE process 6197  
with signal 15
heartbeat: 2007/06/20_07:18:00 info: killing HBWRITE process 6187  
with signal 15
heartbeat: 2007/06/20_07:18:00 info: killing HBREAD process 6198 with  
signal 15
heartbeat: 2007/06/20_07:18:00 info: killing HBREAD process 6188 with  
signal 15
heartbeat: 2007/06/20_07:18:00 info: killing HBWRITE process 6189  
with signal 15
heartbeat: 2007/06/20_07:18:00 info: killing HBREAD process 6190 with  
signal 15
heartbeat: 2007/06/20_07:18:00 info: killing HBWRITE process 6191  
with signal 15
heartbeat: 2007/06/20_07:18:00 info: killing HBREAD process 6192 with  
signal 15
heartbeat: 2007/06/20_07:18:00 info: Core process 6186 exited. 13  
remaining
heartbeat: 2007/06/20_07:18:00 info: Core process 6187 exited. 12  
remaining
heartbeat: 2007/06/20_07:18:00 info: Core process 6188 exited. 11  
remaining
heartbeat: 2007/06/20_07:18:00 info: Core process 6189 exited. 10  
remaining
heartbeat: 2007/06/20_07:18:00 info: Core process 6190 exited. 9  
remaining
heartbeat: 2007/06/20_07:18:00 info: Core process 6191 exited. 8  
remaining
heartbeat: 2007/06/20_07:18:00 info: Core process 6192 exited. 7  
remaining
heartbeat: 2007/06/20_07:18:00 info: Core process 6193 exited. 6  
remaining
heartbeat: 2007/06/20_07:18:00 info: Core process 6194 exited. 5  
remaining
heartbeat: 2007/06/20_07:18:00 info: Core process 6195 exited. 4  
remaining
heartbeat: 2007/06/20_07:18:00 info: Core process 6196 exited. 3  
remaining
heartbeat: 2007/06/20_07:18:00 info: Core process 6197 exited. 2  
remaining
heartbeat: 2007/06/20_07:18:00 info: Core process 6198 exited. 1  
remaining
heartbeat: 2007/06/20_07:18:00 info: Heartbeat shutdown complete.

==========


LBTEMP - ldirectord.log

[Wed Jun 20 07:15:03 2007|ldirectord.cf] ldirectord is stopped for / 
etc/ha.d/ldirectord.cf
[Wed Jun 20 07:15:03 2007|ldirectord.cf] Exiting with exit_status 3:  
Exiting from ldirectord status
[Wed Jun 20 07:15:03 2007|ldirectord.cf] Starting Linux Director  
v1.77.2.32 as daemon
[Wed Jun 20 07:15:03 2007|ldirectord.cf] Added virtual server:  
192.168.155.150:25
[Wed Jun 20 07:15:03 2007|ldirectord.cf] Added virtual server:  
192.168.155.150:221
[Wed Jun 20 07:15:03 2007|ldirectord.cf] Quiescent real server:  
192.168.200.201:25 mapped from 192.168.200.201:25 ( x  
192.168.155.150:25) (Weight set to 0)
[Wed Jun 20 07:15:03 2007|ldirectord.cf] Quiescent real server:  
192.168.200.202:25 mapped from 192.168.200.202:25 ( x  
192.168.155.150:25) (Weight set to 0)
[Wed Jun 20 07:15:03 2007|ldirectord.cf] Quiescent real server:  
192.168.200.204:25 mapped from 192.168.200.204:25 ( x  
192.168.155.150:25) (Weight set to 0)
[Wed Jun 20 07:15:03 2007|ldirectord.cf] Quiescent real server:  
192.168.200.203:25 mapped from 192.168.200.203:25 ( x  
192.168.155.150:25) (Weight set to 0)
[Wed Jun 20 07:15:03 2007|ldirectord.cf] Quiescent real server:  
192.168.200.201:221 mapped from 192.168.200.201:22 ( x  
192.168.155.150:221) (Weight set to 0)
[Wed Jun 20 07:15:03 2007|ldirectord.cf] Restored real server:  
192.168.200.201:25 ( x 192.168.155.150:25) (Weight set to 7)
[Wed Jun 20 07:15:12 2007|ldirectord.cf] Added real server:  
192.168.200.201:22 ( x 192.168.155.150:221) (Weight set to 1)


[Wed Jun 20 07:17:58 2007|ldirectord.cf] Removed real server:  
192.168.200.201:25 ( x 192.168.155.150:25
[Wed Jun 20 07:17:58 2007|ldirectord.cf] Removed virtual server:  
192.168.155.150:25
[Wed Jun 20 07:17:58 2007|ldirectord.cf] Removed real server:  
192.168.200.201:22 ( x 192.168.155.150:221
[Wed Jun 20 07:17:58 2007|ldirectord.cf] Removed virtual server:  
192.168.155.150:221
[Wed Jun 20 07:17:58 2007|ldirectord.cf] Linux Director Daemon  
terminated on signal: TERM

==========

LBTEMP2 - ha-log

heartbeat: 2007/06/20_10:53:34 info: **************************
heartbeat: 2007/06/20_10:53:34 info: Configuration validated.  
Starting heartbeat 1.2.3.cvs.20050404
heartbeat: 2007/06/20_10:53:34 info: heartbeat: version 1.2.3.cvs. 
20050404
heartbeat: 2007/06/20_10:53:35 info: Heartbeat generation: 25
heartbeat: 2007/06/20_10:53:35 info: Starting serial heartbeat on  
tty /dev/ttyS0 (19200 baud)
heartbeat: 2007/06/20_10:53:35 info: UDP Broadcast heartbeat started  
on port 694 (694) interface eth0
heartbeat: 2007/06/20_10:53:35 info: UDP multicast heartbeat started  
for group 225.0.0.1 port 694 interface eth0 (ttl=1 loop=0)
heartbeat: 2007/06/20_10:53:35 info: UDP multicast heartbeat started  
for group 225.0.0.1 port 694 interface eth1 (ttl=1 loop=0)
heartbeat: 2007/06/20_10:53:35 info: ping heartbeat started.
heartbeat: 2007/06/20_10:53:35 info: ping heartbeat started.
heartbeat: 2007/06/20_10:53:35 info: pid 9317 locked in memory.
heartbeat: 2007/06/20_10:53:35 info: pid 9308 locked in memory.
heartbeat: 2007/06/20_10:53:35 info: Local status now set to: 'up'
heartbeat: 2007/06/20_10:53:35 info: Link lbtemp.mydomain.com:eth0 up.
heartbeat: 2007/06/20_10:53:35 info: Status update for node  
lbtemp.mydomain.com: status up
heartbeat: 2007/06/20_10:53:35 info: Running /etc/ha.d/rc.d/status  
status
heartbeat: 2007/06/20_10:53:36 info: pid 9311 locked in memory.
heartbeat: 2007/06/20_10:53:36 info: pid 9312 locked in memory.
heartbeat: 2007/06/20_10:53:36 info: pid 9313 locked in memory.
heartbeat: 2007/06/20_10:53:36 info: pid 9314 locked in memory.
heartbeat: 2007/06/20_10:53:36 info: pid 9322 locked in memory.
heartbeat: 2007/06/20_10:53:36 info: pid 9315 locked in memory.
heartbeat: 2007/06/20_10:53:36 info: pid 9323 locked in memory.
heartbeat: 2007/06/20_10:53:36 info: Link lbtemp2.mydomain.com:eth0 up.
heartbeat: 2007/06/20_10:53:36 info: pid 9321 locked in memory.
heartbeat: 2007/06/20_10:53:36 info: pid 9320 locked in memory.
heartbeat: 2007/06/20_10:53:36 info: pid 9318 locked in memory.
heartbeat: 2007/06/20_10:53:36 info: pid 9319 locked in memory.
heartbeat: 2007/06/20_10:53:36 info: Link lbtemp.mydomain.com:eth1 up.
heartbeat: 2007/06/20_10:53:36 info: pid 9316 locked in memory.
heartbeat: 2007/06/20_10:53:50 info: Status update for node  
lbtemp.mydomain.com: status active
heartbeat: 2007/06/20_10:53:50 info: Running /etc/ha.d/rc.d/status  
status
heartbeat: 2007/06/20_10:54:05 WARN: node 192.168.155.254: is dead
heartbeat: 2007/06/20_10:54:05 WARN: node 192.168.200.254: is dead
heartbeat: 2007/06/20_10:54:05 info: Local status now set to: 'active'
heartbeat: 2007/06/20_10:54:05 info: Starting child client "/usr/lib/ 
heartbeat/ipfail" (500,500)
heartbeat: 2007/06/20_10:54:05 info: Starting "/usr/lib/heartbeat/ 
ipfail" as uid 500  gid 500 (pid 9333)
heartbeat: 2007/06/20_10:54:05 info: Running /etc/ha.d/rc.d/status  
status
heartbeat: 2007/06/20_10:54:05 info: Running /etc/ha.d/rc.d/status  
status
heartbeat: 2007/06/20_10:54:15 info: local resource transition  
completed.
heartbeat: 2007/06/20_10:54:15 info: Initial resource acquisition  
complete (T_RESOURCES(us))
heartbeat: 2007/06/20_10:54:15 info: remote resource transition  
completed.
heartbeat: 2007/06/20_10:54:15 info: No local resources [/usr/lib/ 
heartbeat/ResourceManager listkeys lbtemp2.mydomain.com] to acquire.


heartbeat: 2007/06/20_10:57:12 info: Received shutdown notice from  
'lbtemp.mydomain.com'.
heartbeat: 2007/06/20_10:57:12 info: Resources being acquired from  
lbtemp.mydomain.com.
heartbeat: 2007/06/20_10:57:12 info: acquire local HA resources  
(standby).
heartbeat: 2007/06/20_10:57:12 info: local HA resource acquisition  
completed (standby).
heartbeat: 2007/06/20_10:57:12 info: Standby resource acquisition  
done [all].
heartbeat: 2007/06/20_10:57:12 info: No local resources [/usr/lib/ 
heartbeat/ResourceManager listkeys lbtemp2.mydomain.com] to acquire.
heartbeat: 2007/06/20_10:57:12 info: Running /etc/ha.d/rc.d/status  
status
heartbeat: 2007/06/20_10:57:12 info: Taking over resource group  
IPaddr2::192.168.155.150/24/eth0
heartbeat: 2007/06/20_10:57:12 info: Acquiring resource group:  
lbtemp.mydomain.com IPaddr2::192.168.155.150/24/eth0  
IPaddr2::192.168.200.150/24/eth1 ldirectord::ldirectord.cf  
LVSSyncDaemonSwap::master
heartbeat: 2007/06/20_10:57:12 info: Running /etc/ha.d/resource.d/ 
IPaddr2 192.168.155.150/24/eth0 start
heartbeat: 2007/06/20_10:57:12 info: /sbin/ip -f inet addr add  
192.168.155.150/24 brd 192.168.155.255 dev eth0
heartbeat: 2007/06/20_10:57:12 info: /sbin/ip link set eth0 up
heartbeat: 2007/06/20_10:57:12 /usr/lib/heartbeat/send_arp -i 200 -r  
5 -p /var/lib/heartbeat/rsctmp/send_arp/send_arp-192.168.155.150 eth0  
192.168.155.150 auto 192.168.155.150 ffffffffffff
heartbeat: 2007/06/20_10:57:13 info: Running /etc/ha.d/resource.d/ 
IPaddr2 192.168.200.150/24/eth1 start
heartbeat: 2007/06/20_10:57:13 info: /sbin/ip -f inet addr add  
192.168.200.150/24 brd 192.168.200.255 dev eth1
heartbeat: 2007/06/20_10:57:13 info: /sbin/ip link set eth1 up
heartbeat: 2007/06/20_10:57:13 /usr/lib/heartbeat/send_arp -i 200 -r  
5 -p /var/lib/heartbeat/rsctmp/send_arp/send_arp-192.168.200.150 eth1  
192.168.200.150 auto 192.168.200.150 ffffffffffff
heartbeat: 2007/06/20_10:57:13 info: Running /etc/ha.d/resource.d/ 
ldirectord ldirectord.cf start
heartbeat: 2007/06/20_10:57:14 info: Running /etc/ha.d/resource.d/ 
LVSSyncDaemonSwap master start
heartbeat: 2007/06/20_10:57:14 info: ipvs_syncbackup down
heartbeat: 2007/06/20_10:57:14 info: ipvs_syncmaster up
heartbeat: 2007/06/20_10:57:14 info: ipvs_syncmaster obtained
heartbeat: 2007/06/20_10:57:14 info: /usr/lib/heartbeat/mach_down:  
nice_failback: foreign resources acquired
heartbeat: 2007/06/20_10:57:14 info: mach_down takeover complete.
heartbeat: 2007/06/20_10:57:14 info: mach_down takeover complete for  
node lbtemp.mydomain.com.
heartbeat: 2007/06/20_10:57:43 WARN: node lbtemp.mydomain.com: is dead
heartbeat: 2007/06/20_10:57:43 info: Dead node lbtemp.mydomain.com  
gave up resources.
heartbeat: 2007/06/20_10:57:43 info: Link lbtemp.mydomain.com:eth0 dead.
heartbeat: 2007/06/20_10:57:43 info: Link lbtemp.mydomain.com:eth1 dead.


==========

LBTEMP2 - ldirectord.log

[Wed Jun 20 10:57:13 2007|ldirectord.cf] ldirectord is stopped for / 
etc/ha.d/ldirectord.cf
[Wed Jun 20 10:57:13 2007|ldirectord.cf] Exiting with exit_status 3:  
Exiting from ldirectord status
[Wed Jun 20 10:57:13 2007|ldirectord.cf] Starting Linux Director  
v1.77.2.11 as daemon
[Wed Jun 20 10:57:13 2007|ldirectord.cf] Added virtual server:  
192.168.155.150:25
[Wed Jun 20 10:57:13 2007|ldirectord.cf] Added virtual server:  
192.168.155.150:221
[Wed Jun 20 10:57:13 2007|ldirectord.cf] Quiescent real server:  
192.168.200.201:25 mapped from 192.168.200.201:25 ( x  
192.168.155.150:25) (Weight set to 0)
[Wed Jun 20 10:57:13 2007|ldirectord.cf] Quiescent real server:  
192.168.200.202:25 mapped from 192.168.200.202:25 ( x  
192.168.155.150:25) (Weight set to 0)
[Wed Jun 20 10:57:13 2007|ldirectord.cf] Quiescent real server:  
192.168.200.204:25 mapped from 192.168.200.204:25 ( x  
192.168.155.150:25) (Weight set to 0)
[Wed Jun 20 10:57:13 2007|ldirectord.cf] Quiescent real server:  
192.168.200.203:25 mapped from 192.168.200.203:25 ( x  
192.168.155.150:25) (Weight set to 0)
[Wed Jun 20 10:57:13 2007|ldirectord.cf] Quiescent real server:  
192.168.200.201:221 mapped from 192.168.200.201:22 ( x  
192.168.155.150:221) (Weight set to 0)
[Wed Jun 20 10:57:14 2007|ldirectord.cf] Restored real server:  
192.168.200.201:25 ( x 192.168.155.150:25) (Weight set to 7)
[Wed Jun 20 10:57:23 2007|ldirectord.cf] Added real server:  
192.168.200.201:22 ( x 192.168.155.150:221) (Weight set to 1)



More information about the Ultramonkey-users mailing list