HA클라스터구축(Heartbeat+DRBD+Apache)

HA클라스터구축(Heartbeat+DRBD+Apache)

 

 

1.개요

Heartbeat+DRBD로Active・Standby형의HA(High Availability:고가용성)클라스터 시스템을 구축한다. 여기에서는、2대의 서버로 클라스터 시스템을 구축해서、ACTIVE측 서버다운시에 자동으로STANDBY측 서버에 서비스를 전환(failover)、STANDBY측에서 서비스를 계속할수 있도록 한다。대상 서비스는 DRBD와 Apache로 한다。 【heartbeat+DRBD에 의한 클라스터 시스템 구성】

 

1-1.DRBD 전제

DRBD는 마운트레벨(/home등)이 아닌、디바이스 레벨(/dev/VolGroup00/lvol0등)에데이터의 이중화를 하기위해 ,하드 디스크 추가 또는、기존 논리 볼륨사이즈 축소해서 DRBD용으로 빈 논리볼륨을 추가한다. ※빈 논리볼륨에는 파일 시스템을 작성안 함

 

[root@cl1 ~]# df　←　마운트 상황조회 Filesystem           1K-blocks    Used   Available Use%  Mount on /dev/mapper/VolGroup00-LogVol00                        3428080   1557756   1693380   48%   /　←　논리볼륨(/dev/mapper/VolGroup00-LogVol00)으로 /가 할당 /dev/sda1               101086     17832     78035  19% /boot tmpfs                   127796         0    127796   0% /dev/shm   [root@cl1 ~]# grep swap /etc/fstab　←　swap할당으로 볼륨조회 /dev/VolGroup00/LogVol01 swap                    swap    defaults        0 0　←　논리볼륨(/dev/VolGroup00/LogVol01)으로 swap이 할당   [root@cl1 ~]# lvdisplay -C　←　논리볼륨 일람   LV       VG         Attr   LSize   Origin Snap%  Move Log Copy%  Convert   LogVol00 VolGroup00 -wi-ao   3.38G　←　/에 할당됨※상기조회결과로부터   LogVol01 VolGroup00 -wi-ao 512.00M　←　swap에할당됨※상기조회결과로부터   lvol0    VolGroup00 -wi-a- 992.00M　←　어디에도 할당되어있지 않은 논리볼륨※상기조회결과로부터

1-2.DRBD 인스톨

(１)DRBD 인스톨

 

[root@cl1 ~]# yum -y install drbd82 kmod-drbd82　←　DRBD인스톨

 

(２)DRBD 대응 kernel인스톨 DRBD에 필요한 kmod-drbd82와 kernel은 버전의 동기가 필요하므로、kmod-drbd82대응 kernel버전을 확인해서、필요에 따라서kmod-drbd82에 대응하는 버전의 kernel을 인스톨해서、해당 버전의 kernel로 시스템을 기동한다。

 

[root@cl1 ~]# rpm -qi kmod-drbd82　←　kmod-drbd82 대응 kernel버전확인 Name        : kmod-drbd82                  Relocations: (not relocatable) Version     : 8.2.6                             Vendor: CentOS Release     : 2                             Build Date: 2008년10월04일 01시46분57초 Install Date: 2009년02월10일 23시01분54초      Build Host: c5-i386-build Group       : System Environment/Kernel     Source RPM: drbd82-kmod-8.2.6-2.src.rpm Size        : 2047377                          License: GPL Signature   : DSA/SHA1, 2008년10월04일 01시47분52초, Key ID a8a447dce8562897 URL         : http://www.drbd.org/ Summary     : drbd82 kernel module(s) Description : This package provides the drbd82 kernel modules built for the Linux kernel 2.6.18-92.1.13.el5 for the i686 family of processors.　←　kernel-2.6.18-92.1.13.el5대응   [root@cl1 ~]# uname -r　←　기동중인 kernel 버전 확인 2.6.18-92.1.22.el5　←　kmod-drbd82 대응 kernel버전과다름   [root@cl1 ~]# rpm -q kernel　←　인스톨 한 커널 버전 확인 kernel-2.6.18-92.el5 kernel-2.6.18-92.1.22.el5 kmod-drbd82 대응 kernel버전은 인스톨 되어있지 않다.   [root@cl1 ~]# yum -y install yum-allowdowngrade　←　오래된 버전의 kernel을 인스톨하도록 한다.   [root@cl1 ~]# yum --allow-downgrade -y install kernel-2.6.18-92.1.13.el5　←　kmod-drbd82 대응 kernel 인스톨   [root@cl1 ~]# reboot　←　재기동   부트로더 GRUB의 메뉴에 kmod-drbd82  대응 kernel 버전을  선택해서 시스템을기동한다.   [root@cl1 ~]# uname -r　←　기동중 kernel버젼 확인 2.6.18-92.1.13.el5　←　kmod-drbd82대응 kernel 버전

 

※이후、yum에 의한 커널업데이트시는 、kernel、kmod-drbd82의 버전이  갖춰지면、해당 커널 버전에 시스템을 재기동한다.

1-3.DRBD설정

[root@cl1 ~]# vi /etc/drbd.conf　←　DRBD설정 파일작성 # # please have a a look at the example configuration file in # /usr/share/doc/drbd82/drbd.conf # global { usage-count yes; } common { syncer { rate 10M; } } resource r0 {     protocol C;     startup {          degr-wfc-timeout 120;     }     net {          cram-hmac-alg sha1;          shared-secret "FooFunFactory";     }     on cl1.centossrv.com {　←　노드 １호스트명(｢uname -n｣명령어로 표시되는 이름)          device    /dev/drbd0;          disk      /dev/VolGroup00/lvol0;　←　노드１논리볼륨명          address   192.168.1.100:7789;　←　 노드１IP어드레스          meta-disk  internal;     }     on cl2.centossrv.com {　←　노드２호스트명(｢uname -n｣명령어로 표시되는 이름)          device    /dev/drbd0;          disk      /dev/VolGroup00/lvol0;　←　노드２ 논리볼륨명          address   192.168.1.200:7789;　←　노드２IP 어드레스          meta-disk  internal;     } }   [root@cl1 ~]# drbdadm create-md r0　←　DRBD메터 데이터의 작성 md_offset 1040183296 al_offset 1040150528 bm_offset 1040117760   Found some data  ==> This might destroy existing data! <==   Do you want to proceed? [need to type 'yes' to confirm] yes　←　yes 선택   v07 Magic number not found v07 Magic number not found You want me to create a v08 style flexible-size internal meta data block. There apears to be a v08 flexible-size internal meta data block already in place on /dev/VolGroup00/lvol0 at byte offset 1040183296 Do you really want to overwrite the existing v08 meta-data? [need to type 'yes' to confirm] yes　←　yes 선택   Writing meta data... initialising activity log NOT initialized bitmap New drbd meta data block sucessfully created.

1-4.DRBD디스크  초기동기스킵

DRBD디스크 초회사용시는 노드간 디스크의 풀동기가 행해진다。디스크의 불량개소 검지도 되지만、디스크 사이즈에 있어서는 대량의 시간이 걸리기(1GB로 약10분)때문에、실질적으로 DRBD디스크 동기로 됨。또、DRBD디스크 초기동기를 하는 경우에는 본항을 스킵한다.

 

[root@cl1 ~]# drbdadm -- 6::::1 set-gi r0　←　DRBD메타 데이터상 디스크 동기로 됨 previously 0000000000000004:0000000000000000:0000000000000000:0000000000000000:0:0:0:0:0:0 set GI to  0000000000000006:0000000000000000:0000000000000000:0000000000000000:1:0:0:0:0:0   Write new GI to disk? [need to type 'yes' to confirm] yes　←　yes 선택   [root@cl1 ~]# drbdadm dump-md r0 > /tmp/md　←　DRBD 메타 데이터를 /tmp/md에 보존   [root@cl1 ~]# sed -i -r -e 's/0xF{16}/0x0000000000000000/g' /tmp/md　←　/tmp/md상 디스크 동기로 됨   [root@cl1 ~]# drbdmeta /dev/drbd0 v08 /dev/VolGroup00/lvol0 internal restore-md /tmp/md ; rm -f /tmp/md 　←　/tmp/md를 DRBD메타데이터에 덮어씀   Valid meta-data in place, overwrite? [need to type 'yes' to confirm] yes　←　yes 선택   Successfully restored meta data

1-5.DRBD 기동

[root@cl1 ~]# /etc/rc.d/init.d/drbd start　←　DRBD 기동 Starting DRBD resources:    [ d(r0) s(r0) n(r0) ]. .......... ***************************************************************  DRBD's startup script waits for the peer node(s) to appear.  - In case this node was already a degraded cluster before the    reboot the timeout is 120 seconds. [degr-wfc-timeout]  - If the peer was available before the reboot the timeout will    expire after 0 seconds. [wfc-timeout]    (These values are for resource 'r0'; 0 sec -> wait forever)  To abort waiting enter 'yes' [  29]:yes　←　yes 선택※노드１만   [root@cl1 ~]# chkconfig drbd on　←　DRBD자동기동설정

 

DRBD 인스톨 ～DRBD기동까지를 노드 ２측에도 실시한다.

1-6.DRBD디스크 초기동기

※｢DRBD디스크 초기동기 스킵｣을 실시한 경우는 본항은 실시불요。

 

[root@cl1 ~]# drbdadm -- --overwrite-data-of-peer primary all　←　노드１측을 Primary화&DRBD 디스크 초기동기 실시 ※시간이 걸림(1GB로 약10분)   [root@cl1 ~]# /etc/rc.d/init.d/drbd status　←　DRBD상태 확인 drbd driver loaded OK; device status: version: 8.2.6 (api:88/proto:86-88) GIT-hash: 3e69822d3bb4920a8c1bfdf7d647169eba7d2eb4 build by buildsvn@c5-i386-build, 2008-10-03 11:42:32 m:res  cs          st                 ds                     p  mounted  fstype 0:r0   SyncSource  Primary/Secondary  UpToDate/Inconsistent  C ...    sync'ed:    7.3%               (942492/1015740)K　←　동기중   ・ ・ ・   [root@cl1 ~]# /etc/rc.d/init.d/drbd status　←　DRBD상태확인 drbd driver loaded OK; device status: version: 8.2.6 (api:88/proto:86-88) GIT-hash: 3e69822d3bb4920a8c1bfdf7d647169eba7d2eb4 build by buildsvn@c5-i386-build, 2008-10-03 11:42:32 m:res  cs         st                 ds                 p  mounted  fstype 0:r0   Connected  Primary/Secondary  UpToDate/UpToDate  C　←　동기완료

1-7.DRBD확인

[root@cl1 ~]# drbdadm primary all　←　노드１측을 Primary화 ※DRBD디스크초기동기로 Primary화 한 경우는 실시불요   [root@cl1 ~]# /etc/rc.d/init.d/drbd status　←　노드１측 DRBD상태확인 drbd driver loaded OK; device status: version: 8.2.5 (api:88/proto:86-88) GIT-hash: 9faf052fdae5ef0c61b4d03890e2d2eab550610c build by buildsvn@c5-i386-build, 2008-03-09 10:27:36 m:res  cs         st                 ds                 p  mounted  fstype 0:r0   Connected  Primary/Secondary  UpToDate/UpToDate  C　←노드 １측이 Primary   [root@cl2 ~]# /etc/rc.d/init.d/drbd status　←　노드２측DRBD상태확인 drbd driver loaded OK; device status: version: 8.2.5 (api:88/proto:86-88) GIT-hash: 9faf052fdae5ef0c61b4d03890e2d2eab550610c build by buildsvn@c5-i386-build, 2008-03-09 10:27:36 m:res  cs         st                 ds                 p  mounted  fstype 0:r0   Connected  Secondary/Primary  UpToDate/UpToDate  C　←　노드 ２측이 secondary   [root@cl1 ~]# mkfs -t ext3 /dev/drbd0　←　노드１측 DRBD디스크에 파일시스템 작성 mke2fs 1.39 (29-May-2006) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) 126976 inodes, 253935 blocks 12696 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=260046848 8 block groups 32768 blocks per group, 32768 fragments per group 15872 inodes per group Superblock backups stored on blocks:         32768, 98304, 163840, 229376   Writing inode tables: done Creating journal (4096 blocks): done Writing superblocks and filesystem accounting information: done   This filesystem will be automatically checked every 23 mounts or 180 days, whichever comes first.  Use tune2fs -c or -i to override.   [root@cl1 ~]# mount /dev/drbd0 /mnt　← 노드１측에DRBD디스크를 /mnt에   마운트   [root@cl1 ~]# df　←노드 １측에 DRBD디스크 마운트 확인 Filesystem           1K-blocks    Used   UsedAvailable Used% Mounted on /dev/mapper/VolGroup00-LogVol00                        3428080   1600832   1650304  50% / /dev/sda1               101086     23800     72067  25% /boot tmpfs                   127796         0    127796   0% /dev/shm /dev/drbd0              999764     17652    931328   2% /mnt　←　마운트한 DRBD디스크   [root@cl1 ~]# touch /mnt/test.txt　← 노드 １측에 /mnt(DRBD디스크)에 테스트 파일 작성   [root@cl1 ~]# ll /mnt/　←　노드１측에 /mnt(DRBD디스크)조회 合計 16 drwx------ 2 root root 16384  2月 11 00:01 lost+found -rw-r--r-- 1 root root     0  2月 11 00:02 test.txt　←　작성한 파일   [root@cl1 ~]# umount /mnt　←　노드１측에 DRBD디스크 마운트해제   [root@cl1 ~]# df　←　노드１측에 DRBD 디스크마운트 해제확인 Filesystem           1K-blocks    Used   Available  Use%  Mount on /dev/mapper/VolGroup00-LogVol00                        3428080   1600832   1650304  50% / /dev/sda1               101086     23800     72067  25% /boot tmpfs                   127796         0    127796   0% /dev/shm DRBD 디스크가 없다.   [root@cl1 ~]# drbdadm secondary all　←　노드１측을 Secondary화   [root@cl2 ~]# drbdadm primary all　← 노드 ２측을 Primary화   [root@cl1 ~]# /etc/rc.d/init.d/drbd status　←　노드１측 DRBD 상태확인 drbd driver loaded OK; device status: version: 8.2.6 (api:88/proto:86-88) GIT-hash: 3e69822d3bb4920a8c1bfdf7d647169eba7d2eb4 build by buildsvn@c5-i386-build, 2008-10-03 11:42:32 m:res  cs         st                 ds                 p  mounted  fstype 0:r0   Connected  Secondary/Primary  UpToDate/UpToDate  C　←　노드１측이 Secondary   [root@cl2 ~]# /etc/rc.d/init.d/drbd status　←　노드２측 DRBD 상태확인 drbd driver loaded OK; device status: version: 8.2.6 (api:88/proto:86-88) GIT-hash: 3e69822d3bb4920a8c1bfdf7d647169eba7d2eb4 build by buildsvn@c5-i386-build, 2008-10-03 11:42:32 m:res  cs         st                 ds                 p  mounted  fstype 0:r0   Connected  Primary/Secondary  UpToDate/UpToDate  C　←　노드 ２측이  Primary [root@cl2 ~]# mount /dev/drbd0 /mnt　←　노드２측에 DRBD디스크를 /mnt에 마운트   [root@cl2 ~]# ll /mnt/　←　노드２측에 /mnt(DRBD디스크)조회 合計 16 drwx------ 2 root root 16384  2月 11 00:01 lost+found -rw-r--r-- 1 root root     0  2月 11 00:02 test.txt　←　노드 １측에 작성한 파일   [root@cl2 ~]# umount /mnt　←　노드２측에 DRBD디스크 마운트 해제   [root@cl2 ~]# drbdadm secondary all　←　노드 ２측을 Secondary화   [root@cl1 ~]# drbdadm primary all　← 노드 １측을 Primary화   [root@cl1 ~]# /etc/rc.d/init.d/drbd status　←　노드 １측 DRBD상태확인 drbd driver loaded OK; device status: version: 8.2.5 (api:88/proto:86-88) GIT-hash: 9faf052fdae5ef0c61b4d03890e2d2eab550610c build by buildsvn@c5-i386-build, 2008-03-09 10:27:36 m:res  cs         st                 ds                 p  mounted  fstype 0:r0   Connected  Primary/Secondary  UpToDate/UpToDate  C　←　노드１측이 Primary   [root@cl2 ~]# /etc/rc.d/init.d/drbd status　← 노드２측 DRBD상태확인 drbd driver loaded OK; device status: version: 8.2.5 (api:88/proto:86-88) GIT-hash: 9faf052fdae5ef0c61b4d03890e2d2eab550610c build by buildsvn@c5-i386-build, 2008-03-09 10:27:36 m:res  cs         st                 ds                 p  mounted  fstype 0:r0   Connected  Secondary/Primary  UpToDate/UpToDate  C　←노드 ２측이 Secondary

2-1.heartbeat 인스톨

[root@cl1 ~]# yum -y install heartbeat　←　heartbeat 인스톨   ※이하와 같은 에러 메시지가 출력되어 heartbeat만 인스톨이 실패해서 다시한번 인스톨을 실시한다. useradd: 유저  hacluster 는 존재합니다. error: %pre(heartbeat-2.1.3-3.el5.centos.i386) scriptlet failed, exit status 9 error:   install: %pre scriptlet failed (2), skipping heartbeat-2.1.3-3.el5.centos   [root@cl1 ~]# yum -y install heartbeat　←　heartbeat인스톨 재시행

2-2.heartbeat 설정

[root@cl1 ~]# cp /usr/share/doc/heartbeat-*/ha.cf /etc/ha.d/　←　ha.cf의 샘플을 복사 [root@cl1 ~]# vi /etc/ha.d/ha.cf　←　ha.cf편집 keepalive 2　←　comment해제 (노드간 사활감시시간(초)) deadtime 30　←　 comment해제 (노드 다운검지시간(초)) initdead 120　←　 comment해제 (기동시 노드다운 검지시간(초)) udpport        694　←　 comment해제 (Broadcast사용포트번호) bcast  eth0            # Linux　←　 comment해제 (Broadcast사용 네트워크 인터페이스명) watchdog /dev/watchdog　←　comment out(heartbeat자신의 이상시에 시스템을 재기동한다.) node cl1.centossrv.com　←　추가(노드１호스트 명 (｢uname -n｣명령어로 표시되는 이름)) node cl2.centossrv.com　←　추가(노드2호스트 명 (｢uname -n｣ 명령어로 표시되는 이름) use_logd yes　←　추가(로그 데몬의 유효화)※로그 데몬의 설정은 후술의logd.cf에서 실시 crm yes　←　추가(클라스터 매니져의 유효화)   [root@cl1 ~]# cp /usr/share/doc/heartbeat-*/logd.cf /etc/　←　logd.cf의 샘플을 복사   [root@cl1 ~]# vi /etc/logd.cf　←　logd.cf편집 debugfile /var/log/ha-debug　←　comment 해제(디버그 메시지 용 화일명) logfile        /var/log/ha-log　←　 comment 해제(로그화일명) logfacility    none　←　 comment 해제 & 변경(syslog에 로그를 출력하지 않음)   [root@cl1 ~]# cp /usr/share/doc/heartbeat-*/authkeys /etc/ha.d/　←　authkeys의 샘플을 복사   [root@cl1 ~]# vi /etc/ha.d/authkeys　←　authkeys편집 auth 1　←　comment해제(노드간 인증방식 1은(crc)) 1 crc　←　comment해제(노드 인증방식1은crc))   [root@cl1 ~]# chmod 600 /etc/ha.d/authkeys　←　authkeys 퍼미션 변경 [root@cl1 ~]# vi haresources　←　haresources 신규작성※노드１ cl1.centossrv.com \　←　노드１호스트명(｢uname -n｣명령어로 표시되는 이름)         IPaddr2::192.168.1.80/24/eth0/192.168.1.255 \　←　가상IP어드레스 (예:192.168.1.80/24)를eth0로 설정         MailTo::root::centossrv.com \　←　Failover를 메일 통지하는 경우         drbddisk::r0 \　←　DRBD디스크         Filesystem::/dev/drbd0::/mnt　←　/dev/drbd0를/mnt에 마운트   [root@cl2 ~]# vi haresources　←　haresources 신규작성※노드2 cl2.centossrv.com \　←　노드２호스트명(｢uname -n｣명령어로 표시되는 이름)         IPaddr2::192.168.1.80/24/eth0/192.168.1.255 \　←　가상IP어드레스(예:192.168.1.80/24)를eth0로 설정         MailTo::root::centossrv.com \　←　 Failover를 메일 통지하는 경우 drbddisk::r0 \　←　DRBD디스크         Filesystem::/dev/drbd0::/mnt　←　/dev/drbd0를/mnt에 마운트   [root@cl1 ~]# rm -f /var/lib/heartbeat/crm/cib.xml* ;\ /usr/lib/heartbeat/haresources2cib.py haresources　←　haresources에서 cib.xml작성   [root@cl1 ~]# vi /var/lib/heartbeat/crm/cib.xml　←　cib.xml편집                <nvpair id="IPaddr2_1_attr_0" name="ip" value="192.168.1.80"/>                <nvpair id="IPaddr2_1_attr_1" name="nic" value="24"/>                ↓                <nvpair id="IPaddr2_1_attr_1" name="nic" value="eth0"/>　←　nic의value를eth0에 수정                <nvpair id="IPaddr2_1_attr_2" name="cidr_netmask" value="eth0"/>                ↓                <nvpair id="IPaddr2_1_attr_2" name="cidr_netmask" value="24"/>　←　cidr_netmask의value를24에 수정                <nvpair id="IPaddr2_1_attr_3" name="broadcast" value="192.168.1.255"/>   [root@cl1 ~]# crm_verify -x /var/lib/heartbeat/crm/cib.xml　←　cib.xml체크   [root@cl1 ~]# rm -f haresources　←　haresources해제

2-3.heartbeat기동

[root@cl1 ~]# /etc/rc.d/init.d/heartbeat start　←　heartbeat기동 Starting High-Availability services:                                                            [  OK  ]   [root@cl1 ~]# chkconfig heartbeat on　←　heartbeat자동기동 설정

 

heartbeat인스톨～heartbeat기동까지를 노드２측에도 실시。

2-4.heartbeat확인

（１）heartbeat확인

 

[root@cl1 ~]# ip addr show eth0　←　노드１측IP어드레스 확인 2: eth0:  mtu 1500 qdisc pfifo_fast qlen 1000     link/ether 00:0c:29:33:f8:9f brd ff:ff:ff:ff:ff:ff     inet 192.168.1.100/24 brd 192.168.1.255 scope global eth0     inet 192.168.1.80/24 brd 192.168.1.255 scope global secondary eth0　←　가상IP어드레스(예:192.168.1.80/24)가 설정되어있음     inet6 fe80::20c:29ff:fe33:f89f/64 scope link        valid_lft forever preferred_lft forever   [root@cl1 ~]# df　←　노드１측DRBD 디스크 마운트 확인 Filesystem           1K-blocks    Used   Available  Used% Mounted on /dev/mapper/VolGroup00-LogVol00                        3428080   1665096   1586040  52% / /dev/sda1               101086     23800     72067  25% /boot tmpfs                   127796         0    127796   0% /dev/shm /dev/drbd0              999764     18780    930200   2% /mnt　←　마운트된 DRBD디스크   [root@cl1 ~]# crm_mon　←　노드１측heartbeat상태 확인 Defaulting to one-shot mode You need to have curses available at compile time to enable console mode     ============ Last updated: Wed Feb 11 00:42:17 2009 Current DC: cl1.centossrv.com (b431343b-5a3a-4ab6-9ad1-d1c21584eefd) 2 Nodes configured. 1 Resources configured. ============   Node: cl1.centossrv.com (b431343b-5a3a-4ab6-9ad1-d1c21584eefd): online Node: cl2.centossrv.com (1e6245d4-01f6-42f0-b2aa-5811a88cc9be): online   Resource Group: group_1     IPaddr2_1   (heartbeat::ocf:IPaddr2):       Started cl1.centossrv.com　←　노드１측에 가상IP어드레스가 Active     MailTo_2    (heartbeat::ocf:MailTo):        Started cl1.centossrv.com　←　노드１측에 메일 통지가 Active     drbddisk_3  (heartbeat:drbddisk):   Started cl1.centossrv.com　←　노드１측에 DRBD디스크가 Active     Filesystem_4        (heartbeat::ocf:Filesystem):    Started cl1.centossrv.com　←　노드１측에 파일 시스템이 Active   [root@cl2 ~]# ip addr show eth0　←　노드２측IP어드레스 확인 2: eth0:  mtu 1500 qdisc pfifo_fast qlen 1000     link/ether 00:0c:29:c6:44:03 brd ff:ff:ff:ff:ff:ff     inet 192.168.1.200/24 brd 192.168.1.255 scope global eth0     inet6 fe80::20c:29ff:fec6:4403/64 scope link        valid_lft forever preferred_lft forever 가상 IP어드레스는 설정되어있지않다.   [root@cl2 ~]# df　←　노드２측 DRBD디스크 마운트 확인 Filesystem           1K-blocks    Used   Available  Use% Mounted on /dev/mapper/VolGroup00-LogVol00                        3428080   1669496   1581640  52% / /dev/sda1               101086     23800     72067  25% /boot tmpfs                   127796         0    127796   0% /dev/shm DRBD디스크는 마운트되어있지 않다.   [root@cl2 ~]# crm_mon　←　노드２측heartbeat상태 확인 Defaulting to one-shot mode You need to have curses available at compile time to enable console mode     ============ Last updated: Wed Feb 11 00:43:54 2009 Current DC: cl1.centossrv.com (b431343b-5a3a-4ab6-9ad1-d1c21584eefd) 2 Nodes configured. 1 Resources configured. ============   Node: cl1.centossrv.com (b431343b-5a3a-4ab6-9ad1-d1c21584eefd): online Node: cl2.centossrv.com (1e6245d4-01f6-42f0-b2aa-5811a88cc9be): online   Resource Group: group_1     IPaddr2_1   (heartbeat::ocf:IPaddr2):       Started cl1.centossrv.com　←　노드１측에 가상IP어드레스가 Active     MailTo_2    (heartbeat::ocf:MailTo):        Started cl1.centossrv.com　←　노드１측에 메일통지가 Active     drbddisk_3  (heartbeat:drbddisk):   Started cl1.centossrv.com　←　노드１측에 DRBD디스크가 Active     Filesystem_4        (heartbeat::ocf:Filesystem):    Started cl1.centossrv.com　←　노드１측에 파일시스템이 Active

 

（２）Failover 확인

 

노드１측 시스템정지   [root@cl2 ~]# ip addr show eth0　←　노드２측IP어드레스 확인 2: eth0:  mtu 1500 qdisc pfifo_fast qlen 1000     link/ether 00:0c:29:c6:44:03 brd ff:ff:ff:ff:ff:ff     inet 192.168.1.200/24 brd 192.168.1.255 scope global eth0     inet 192.168.1.80/24 brd 192.168.1.255 scope global secondary eth0　←　가상IP 어드레스(예:192.168.1.80/24)가 설정되어있다.     inet6 fe80::20c:29ff:fec6:4403/64 scope link        valid_lft forever preferred_lft forever   [root@cl2 ~]# df　←　노드２측DRBD디스크 마운트확인 Filesystem           1K-blocks    Used   UsedAvailable Used% Mounted on /dev/mapper/VolGroup00-LogVol00                        3428080   1669496   1581640  52% / /dev/sda1               101086     23800     72067  25% /boot /dev/drbd0              999764     18780    930200   2% /mnt　←　마운트된 DRBD디스크   [root@cl2 ~]# crm_mon　←　노드２측 heartbeat상태확인 Defaulting to one-shot mode You need to have curses available at compile time to enable console mode     ============ Last updated: Wed Feb 11 00:43:54 2009 Current DC: cl2.centossrv.com (b431343b-5a3a-4ab6-9ad1-d1c21584eefd) 2 Nodes configured. 1 Resources configured. ============   Node: cl2.centossrv.com (1e6245d4-01f6-42f0-b2aa-5811a88cc9be): online Node: cl1.centossrv.com (b431343b-5a3a-4ab6-9ad1-d1c21584eefd): OFFLINE   Resource Group: group_1     IPaddr2_1   (heartbeat::ocf:IPaddr2):       Started cl2.centossrv.com　←　노드２측에 가상IP어드레스가 Active     MailTo_2    (heartbeat::ocf:MailTo):        Started cl2.centossrv.com　←　노드２측에 메일통지가 Active     drbddisk_3  (heartbeat:drbddisk):   Started cl2.centossrv.com　←　노드２측에 DRBD디스크가 Active     Filesystem_4        (heartbeat::ocf:Filesystem):    Started cl2.centossrv.com　←　노드２측에 파일 시스템이 Active

 

（３） Failback 확인

 

노드１측 시스템기동   [root@cl1 ~]# ip addr show eth0　←　노드１측IP어드레스 확인 2: eth0:  mtu 1500 qdisc pfifo_fast qlen 1000     link/ether 00:0c:29:33:f8:9f brd ff:ff:ff:ff:ff:ff     inet 192.168.1.100/24 brd 192.168.1.255 scope global eth0     inet 192.168.1.80/24 brd 192.168.1.255 scope global secondary eth0　←　가상IP어드레스(예:192.168.1.80/24)가 설정되어있다.     inet6 fe80::20c:29ff:fe33:f89f/64 scope link        valid_lft forever preferred_lft forever   [root@cl1 ~]# df　←　노드1측DRBD디스크 마운트확인 Filesystem           1K-blocks    Used   Available Used% Mounted on /dev/mapper/VolGroup00-LogVol00                        3428080   1665096   1586040  52% / /dev/sda1               101086     23800     72067  25% /boot tmpfs                   127796         0    127796   0% /dev/shm /dev/drbd0              999764     18780    930200   2% /mnt　←　마운트된 DRBD디스크   [root@cl1 ~]# crm_mon　←　노드１측heartbeat상태확인 Defaulting to one-shot mode You need to have curses available at compile time to enable console mode     ============ Last updated: Fri Feb 20 15:59:11 2009 Current DC: cl2.centossrv.com (1e6245d4-01f6-42f0-b2aa-5811a88cc9be) 2 Nodes configured. 1 Resources configured. ============   Node: cl2.centossrv.com (1e6245d4-01f6-42f0-b2aa-5811a88cc9be): online Node: cl1.centossrv.com (b431343b-5a3a-4ab6-9ad1-d1c21584eefd): online   Resource Group: group_1     IPaddr2_1   (heartbeat::ocf:IPaddr2):       Started cl1.centossrv.com　←　노드１측에 가상IP어드레스가 Active     MailTo_2    (heartbeat::ocf:MailTo):        Started cl1.centossrv.com　←　노드１측에 메일 통지가 Active     drbddisk_3  (heartbeat:drbddisk):   Started cl1.centossrv.com　←　노드１측에 DRBD디스크가 Active     Filesystem_4        (heartbeat::ocf:Filesystem):    Started cl1.centossrv.com　←　노드１측에 파일시스템이 Active   [root@cl2 ~]# ip addr show eth0　←　노드２IP어드레스 확인 2: eth0:  mtu 1500 qdisc pfifo_fast qlen 1000     link/ether 00:0c:29:c6:44:03 brd ff:ff:ff:ff:ff:ff     inet 192.168.1.200/24 brd 192.168.1.255 scope global eth0     inet6 fe80::20c:29ff:fec6:4403/64 scope link        valid_lft forever preferred_lft forever 가상IP어드레스는 설정되어있지 않음(Failback에 의해 해제됨)   [root@cl2 ~]# df　←　노드２측DRBD디스크 마운트 확인 Filesystem           1K-blocks    Used   UsedAvailable Used% Mounted on /dev/mapper/VolGroup00-LogVol00                        3428080   1669496   1581640  52% / /dev/sda1               101086     23800     72067  25% /boot tmpfs                   127796         0    127796   0% /dev/shm DRBD디스크는 마운트 되어있지 않다(Failback에 의해 해제됨)   [root@cl2 ~]# crm_mon　←　노드２측 heartbeat상태확인 Defaulting to one-shot mode You need to have curses available at compile time to enable console mode     ============ Last updated: Fri Feb 20 15:55:02 2009 Current DC: cl2.centossrv.com (1e6245d4-01f6-42f0-b2aa-5811a88cc9be) 2 Nodes configured. 1 Resources configured. ============   Node: cl2.centossrv.com (1e6245d4-01f6-42f0-b2aa-5811a88cc9be): online Node: cl1.centossrv.com (b431343b-5a3a-4ab6-9ad1-d1c21584eefd): OFFLINE   Resource Group: group_1     IPaddr2_1   (heartbeat::ocf:IPaddr2):       Started cl1.centossrv.com　←　노드１측에 가상IP어드레스가 Active     MailTo_2    (heartbeat::ocf:MailTo):        Started cl1.centossrv.com　←　노드１측에 메일 통지가 Active     drbddisk_3  (heartbeat:drbddisk):   Started cl1.centossrv.com　←　노드１측에 DRBD디스크가 Active     Filesystem_4        (heartbeat::ocf:Filesystem):    Started cl1.centossrv.com　←　노드１측에 파일 시스템이 Active

 

（４）자동 Failback 설정

 

자동Failback을무효화하는 경우(노드１이 복구해도 자동으로 Failback하지 않음)※수동에 의한Failback은 다음항 참조 [root@cl1 ~]# cibadmin -o crm_config -U -X '<nvpair id="cib-bootstrap-options-default-resource-stickiness" name="default-resource-stickiness" value="INFINITY"/>'　←　자동Failback무효화   자동Failbsck을 유효화하는 경우※디폴트 [root@cl1 ~]# cibadmin -o crm_config -U -X '<nvpair id="cib-bootstrap-options-default-resource-stickiness" name="default-resource-stickiness" value="0"/>'　←　자동Failback유효화

 

（５）Active／Standby상태 수동전환

 

[root@cl1 ~]# crm_standby -U `hostname` -v on　←　노드１측을 Standby상태로 전환 crm_standby[5608]: 2008/01/25_11:38:51 info: Invoked: crm_standby -U cl1.centossrv.com -v on   [root@cl1 ~]# crm_mon　←　노드１측 heartbeat상태확인 Defaulting to one-shot mode You need to have curses available at compile time to enable console mode     ============ Last updated: Wed Feb 11 00:44:49 2009 Current DC: cl1.centossrv.com (b431343b-5a3a-4ab6-9ad1-d1c21584eefd) 2 Nodes configured. 1 Resources configured. ============   Node: cl1.centossrv.com (b431343b-5a3a-4ab6-9ad1-d1c21584eefd): standby Node: cl2.centossrv.com (1e6245d4-01f6-42f0-b2aa-5811a88cc9be): online   Resource Group: group_1     IPaddr2_1   (heartbeat::ocf:IPaddr2):       Started cl2.centossrv.com　←　노드２측에가상IP어드레스가 Active     MailTo_2    (heartbeat::ocf:MailTo):        Started cl2.centossrv.com　←　노드２측에 메일 통지가 Active     drbddisk_3  (heartbeat:drbddisk):   Started cl2.centossrv.com　←　노드２측에 DRBD디스크가 Active     Filesystem_4        (heartbeat::ocf:Filesystem):    Started cl2.centossrv.com　←　노드２측에 파일 시스템이 Active   [root@cl1 ~]# crm_standby -U `hostname` -v off　←　노드１측을 Active 상태에 전환 crm_standby[5604]: 2008/01/25_11:38:24 info: Invoked: crm_standby -U cl1.centossrv.com -v off   [root@cl1 ~]# crm_mon　←　노드１측heartbeat상태확인 Defaulting to one-shot mode You need to have curses available at compile time to enable console mode     ============ Last updated: Wed Feb 11 00:46:38 2009 Current DC: cl1.centossrv.com (b431343b-5a3a-4ab6-9ad1-d1c21584eefd) 2 Nodes configured. 1 Resources configured. ============   Node: cl1.centossrv.com (b431343b-5a3a-4ab6-9ad1-d1c21584eefd): online Node: cl2.centossrv.com (1e6245d4-01f6-42f0-b2aa-5811a88cc9be): online   Resource Group: group_1     IPaddr2_1   (heartbeat::ocf:IPaddr2):       Started cl1.centossrv.com　←　노드１측에 가상IP어드레스가 Active     MailTo_2    (heartbeat::ocf:MailTo):        Started cl1.centossrv.com　←　노드１측에 메일 통지가 Active     drbddisk_3  (heartbeat:drbddisk):   Started cl1.centossrv.com　←　노드１측에 DRBD디스크가 Active     Filesystem_4        (heartbeat::ocf:Filesystem):    Started cl1.centossrv.com　←　노드１측에 파일시스템이 Active

3-1.Apache인스톨

노드１、노드２양쪽에 Apache인스톨

3-2.Apache감시설정

（１）Apache설정

 

[root@cl1 ~]# /etc/rc.d/init.d/httpd stop　←　노드１측Apache정지 Stopping httpd:                                            [  OK  ]   [root@cl1 ~]# chkconfig httpd off　←　노드１측Apache자동 기동해제   [root@cl2 ~]# /etc/rc.d/init.d/httpd stop　←　노드２측Apache정지 Stopping httpd:                                            [  OK  ]   [root@cl2 ~]# chkconfig httpd off　←　노드２측 Apache 자동기동해제   [root@cl1 ~]# rsync -avz -e ssh --delete /var/www/ /mnt/　←도큐먼트 루트의 데이터를  DRBD디스크(/mnt)에 복사

 

（２）heartbeat설정※노드１측만 작업

 

[root@cl1 ~]# vi haresources　←　haresources신규작성 cl1.centossrv.com \         IPaddr2::192.168.1.80/24/eth0/192.168.1.255 \         MailTo::root::centossrv.com \         drbddisk::r0 \         Filesystem::/dev/drbd0::/var/www \　←　/dev/drbd0を/var/www에 마운트         httpd　←　Apache   [root@cl1 ~]# /usr/lib/heartbeat/haresources2cib.py --stdout haresources > /tmp/cib.xml　←　haresources로부터 /tmp/cib.xml작성   [root@cl1 ~]# vi /tmp/cib.xml　←　/tmp/cib.xml편집                <nvpair id="IPaddr2_1_attr_0" name="ip" value="192.168.1.80"/>                <nvpair id="IPaddr2_1_attr_1" name="nic" value="24"/>                ↓                <nvpair id="IPaddr2_1_attr_1" name="nic" value="eth0"/>　←　nic의 value를 eth0에 수정  <nvpair id="IPaddr2_1_attr_2" name="cidr_netmask" value="eth0"/>                ↓                <nvpair id="IPaddr2_1_attr_2" name="cidr_netmask" value="24"/>　←　cidr_netmask의 value를24에 수정                <nvpair id="IPaddr2_1_attr_3" name="broadcast" value="192.168.1.255"/>   [root@cl1 ~]# crm_verify -x /tmp/cib.xml　←　/tmp/cib.xml체크   [root@cl1 ~]# cibadmin -R -o resources -x /tmp/cib.xml　←　/tmp/cib.xml반영   [root@cl1 ~]# rm -f haresources　←　haresources삭제

3-3.Apache감시확인

[root@cl1 ~]# echo test > /var/www/html/index.html　←　테스트페이지작성   [root@cl1 ~]# crm_mon　←　노드１측 heartbeat상태확인 Defaulting to one-shot mode You need to have curses available at compile time to enable console mode     ============ Last updated: Fri Feb 20 16:32:15 2009 Current DC: cl2.centossrv.com (1e6245d4-01f6-42f0-b2aa-5811a88cc9be) 2 Nodes configured. 1 Resources configured. ============   Node: cl2.centossrv.com (1e6245d4-01f6-42f0-b2aa-5811a88cc9be): online Node: cl1.centossrv.com (b431343b-5a3a-4ab6-9ad1-d1c21584eefd): online   Resource Group: group_1     IPaddr2_1   (heartbeat::ocf:IPaddr2):       Started cl1.centossrv.com     MailTo_2    (heartbeat::ocf:MailTo):        Started cl1.centossrv.com     drbddisk_3  (heartbeat:drbddisk):   Started cl1.centossrv.com     Filesystem_4        (heartbeat::ocf:Filesystem):    Started cl1.centossrv.com     httpd_5     (lsb:httpd):    Started cl1.centossrv.com　←　노드１측에 httpd가  Active

 

http://192.168.1.80에 액세스해서 web페이지가 표시되는 것을 확인

 

노드１측 시스템 정지   [root@cl2 ~]# crm_mon　←　노드２측heartbeat상태 확인 Defaulting to one-shot mode You need to have curses available at compile time to enable console mode     ============ Last updated: Fri Feb 20 15:55:02 2009 Current DC: cl2.centossrv.com (1e6245d4-01f6-42f0-b2aa-5811a88cc9be) 2 Nodes configured. 1 Resources configured. ============   Node: cl2.centossrv.com (1e6245d4-01f6-42f0-b2aa-5811a88cc9be): online Node: cl1.centossrv.com (b431343b-5a3a-4ab6-9ad1-d1c21584eefd): OFFLINE   Resource Group: group_1     IPaddr2_1   (heartbeat::ocf:IPaddr2):       Started cl2.centossrv.com     MailTo_2    (heartbeat::ocf:MailTo):        Started cl2.centossrv.com     drbddisk_3  (heartbeat:drbddisk):   Started cl2.centossrv.com     Filesystem_4        (heartbeat::ocf:Filesystem):    Started cl2.centossrv.com     httpd_5     (lsb:httpd):    Started cl2.centossrv.com　←　노드２측에httpd가 Active

 

http://192.168.1.80에 엑세스해서web페이지가 표시되는 것을 확인

 

노드１측 시스템기동   [root@cl1 ~]# crm_mon　←　노드１측 heartbeat상태확인 Defaulting to one-shot mode You need to have curses available at compile time to enable console mode     ============ Last updated: Fri Feb 20 16:01:22 2009 Current DC: cl2.centossrv.com (1e6245d4-01f6-42f0-b2aa-5811a88cc9be) 2 Nodes configured. 1 Resources configured. ============   Node: cl2.centossrv.com (1e6245d4-01f6-42f0-b2aa-5811a88cc9be): online Node: cl1.centossrv.com (b431343b-5a3a-4ab6-9ad1-d1c21584eefd): online   Resource Group: group_1     IPaddr2_1   (heartbeat::ocf:IPaddr2):       Started cl1.centossrv.com     drbddisk_2  (heartbeat:drbddisk):   Started cl1.centossrv.com     Filesystem_3        (heartbeat::ocf:Filesystem):    Started cl1.centossrv.com     httpd_4     (lsb:httpd):    Started cl1.centossrv.com　←　노드１측에 httpd가 Active

 

http://192.168.1.80에 액세스해서  web페이지가 표시되는 것을 확인 라우터측에 외부로부터의 TCP80번 포트에의 액세스를 192.168.1.80에 전송하도록 설정