Commercial Linux Clusters

Avi Alkalay <avix@br.ibm.com>

Linux, Open Standards Consultant

IBM Corporation

A guide for everyday e-business clusters

Internal use, but not confidential. Last update 2006-03-07.

About this Document

This document highlights High Availability Linux Clusters on IBM hardware and software, covering different cluster solutions

It is intended to be used by architects, sales people and customers

Last version available at: http://avi.alkalay.net/linux/docs/ha/

The e-server logo uses a special true-type font that can be installed from http://watgsa.ibm.com/~avibrz/public/c4eb/ibmfonts

For Linux viewers, you should install Microsoft fonts for better results from http://avi.alkalay.net/software/webcore-fonts

High Availability Status

Who wants Low-Availability systems?

Why are there so few High-Availability systems?

Linux High Availability Potential

Old
HA

HA on
Linux

Expensive Hardware

Expensive HA Software

Complex

Exclusive systems

Unexpensive Hardware

Free HA Software

Simple

All systems

How High Availability Works ?

Host 1 :: Active

Host 2 :: Passive

Host 2 monitors

Host 1

High Availability Cluster

Host 1 is the main application server. HA Software on Host 2 monitors Host 1 availability

When Host 1 fails, the HA software automatically takes the following actions

HA Software makes the Application Data Files available on Host 2

HA Software sets Application service IP addresses on Host 2

HA Software starts Application on Host 2

Run in Screen Show Mode

Which Applications I Can Cluster ?

Any Server Software can be HA clustered

Any DB, Web-Server, WebSphere, Domino, Samba or other File Server, Mail Server, ERP, etc. Application doesn't have to be aware of its HA cluster context

If some server software can't be included in an HA context, the client protocol with the server must be reviewed. This is very rare.

An application (and filesystem) must be able to recover from crashes – preferably quickly. Any regular DB executes this action when started.

All nodes may run different active applications simultaneously. The peer node will be passive for this particular app. These are Active-Active clusters

A Note About Parallel Applications

There is some special apps that were built to use other nodes' computing capacity in a parallel way

Example: DB2 Parallel Server, Oracle RAC, Domino replicas, etc

These apps use to have a Controlling Process that runs on a unique node and dispatches activity to slave nodes

High-Availability must be provided for the Controlling Process in the same way as for regular apps

Parallel Applications use to require (but not always) special storage configurations, for simultaneous multinode shared data access

HA Common
Architectures

Cluster Architecture Perspectives

Array B

Array C

HeartBeat

10.23.0.1

10.23.0.2

Active: DB2

Passive: WebSphere

Active: WebSphere

Passive: DB2

Diamond

200.10.20.1

200.10.20.11 (service)

Wind

200.10.20.2

200.10.20.12 (service)

DB2 Service

Initially active on Host Diamond

Service hostname: db2.domain.com

Service IP: 200.10.20.11

WebSphere Service

Initially active on Host Wind

Service hostname: was.domain.com

Service IP: 200.10.20.12

Service Oriented

diamond.domain.com

IPs: 200.10.20.11 (service), 200.10.20.1

HB: 10.23.0.1

Initial service: DB2

wind.domain.com

IPs: 200.10.20.12 (service), 200.10.20.2

HB: 10.23.0.2

Initial service: WebSphere

Host Oriented

Array A :: Linux OS, Application Software

Internal storage. Each machine has its own.

Array B :: Database

Online for Diamond. Offline for Wind (on if Diamond fails)

Array C :: Application business logic files

Online for Wind. Offline for Diamond (on if Wind fails)

Array Oriented

ServeRAID and SCSI Clusters Overview

Array A

Array B

Array C

DS300/DS400

~

xSeries

Ethernet cross cable: Heartbeat

SCSI connections

~

xSeries

LAN: Service and Heartbeat

Host 1 runs AppA. Optionally, Host 2 may run AppB

Host 2 monitors availability of Host 1 through some Heartbeat software

In case of Active-Active cluster, Host 1 may monitor Host 2 simultaneously

If Host 1 fails, a set of automatic actions on Host 2 will:

Take control of Host-1-owned disk arrays;

Configure Host 2's network interfaces to respond as Host 1 and Host 2 simultaneously

Activate AppA on Host 2

ServeRAID and SCSI Clusters Details

Support for Active-Passive and Active-Active clusters

Red Hat Advanced Server's Cluster Manager is not supported with clustered ServeRAID

Hardware feature: each logical array can be owned by one node at a time (this is why Red Hat's solution will not work)

RAID 5 not supported for clustered environment, in any platform. See ServeRAID manual page 74

Supported high availability software:

System Automation for Multi Platforms (aka TSA)

Linux-HA

SteelEye's LifeKeeper

Data Replication Clusters Overview

Any ^

Host 1 runs AppA. Optionally, Host 2 may run AppB

AppA reads and writes to local storage via DRBD. Each byte written in Host 1's local storage is replicated online to DRBD on Host 2.

Host 2 monitors availability of Host 1 through some Heartbeat software

In case of Active-Active cluster, Host 1 can monitor Host 2 simultaneously

If Host 1 fails, a set of automatic actions on Host 2 will:

Activate Host 2's replica of the storage (mount the volume);

Configure Host 2's network interfaces to respond as Host 1 and Host 2 simultaneously

Activate AppA on Host 2

Any ^

Linux Host 2

Non active

Application

DRBD

Writes

Linux Host 1

Application

DRBD

Read/Write

Read/Write

Gigabit Ethernet cross:
Heartbeat and Replication

LAN: Service and Heartbeat

Data Replication Clusters Details

DRBD stands for Data Replication Block Device. It is a low level replication software that operates on the OS level. http://www.drbd.org/

DRBD is completely transparent to the application and users. No special support for replication is required in application

Support for Active-Passive and Active-Active clusters

Supports LVM and HW RAID. Linux on any ^ family, virtual Linux instances or mixed

DRBD can be used for cross-site replication. SW RAID is a waste of time and CPU when DRBD is used

Customer reference regarding replication bandwith: Hospital running Informix on a DRBD cluster with observed peak load of 6MBits/s over the 100MBits/s fast ethernet interface used for replication

zSeries High Availability Red Paper: http://www.redbooks.ibm.com/abstracts/redp0220.html

Supported high availability software:

Linux-HA for all platforms

SteelEye's LifeKeeper for ^ xSeries

DRBD is not compiled by default on Red Hat AS. But IGS can easily provide this functionality on RHEL

SAN Based Linux Clusters

Fully shared storage clusters brings a new level of clustering to Linux

Leverages high performance parallel applications

While ServeRAID requires that each node keeps its own private logical array (transferable in a fail-over situation), SAN don't. Each storage byte is truly shared across nodes.

When shared storage is needed, a software layer is required to manage simultaneous multiple nodes access. This is provided by things like GPFS, OpenGFS, GFS, Oracle File System (included in RHAS), etc.

Due to higher price, SAN storage should be chosen when simultaneous data access is a must. Other way, SCSI may provide a better price/benefit relation.

Simplest SAN Cluster

FAStT

~

xSeries

Ethernet cross: Heartbeat

~

xSeries

LAN: Service and Heartbeat

Array A

Array B

Array C

Fiber Optic Cables

One FC cable per node is also supported

Both nodes may be active, running the same parallel application. But generally one node runs a special application process to balance requests: This is the Controller Process

Some High Availability software is needed to move the Controlling Process from one node to another, when in a failure situation

Support for Active-Passive and Active-Active clusters, but Parallel Active-Active applications takes more benefit from the HW.

When each node has 2 FC boards, special software, provided with the product, is needed to run in each node

FAStT support:
http://www.storage.ibm.com/disk/fastt/supserver.htm

Supported high availability software:

System Automation for Multi Platforms (aka TSA)

Linux-HA

Red Hat Advanced Server

SteelEye's LifeKeeper

Polyserve

SAN Clusters Details

Advanced SAN Cluster

~

xSeries

FAStT

Array A

Array B

Array C

Fiber Optic Cables

~

xSeries

SAN Switch

Trunk

SAN Switch

n-nodes

n-nodes

LAN

LAN

Which Storage to Use

The best (price/perf) storage solution is 100% dependent on the application that will be hosted

A Note About Storage Sizing

Oversized architectures are very common. Customers of new applications don't have a clue of their storage needs. Here are some real world examples.

Source: IBM Strategic Outsourcing / Web Hosting delivery team and Customers. 02H03

Minimum Red Hat Linux installation (best for a server): 800MB

Storage needed to install OS + DB2 + WebSphere or Oracle software files: 1GB

Database size of one of brazilian biggest B2B portal running Ariba: 20GB

General Motors in Brazil all portals (B2B, B2C) web content size: 2.8GB

Same portals total database size on disk: 670MB

Same portals compressed web server logs for 1 year: 40GB

Bookseller B2B web content: 3GB

DB size on disk of a brazilian online yellow pages service: 18GB

Web content of same: 300MB

Portal web content (EARs, JARs) of a small bank: 10MB

Brazil's large SAP customer's HR database size: 60GB

A big call center CRM database size: 5GB

Oracle ERP DB size for a company in the automotive sector: 111GB

Single SCSI HD as of 02H03: 140GB

Popular Software
Details

DB2 on Linux Clusters

DB2 UDB parallel server on n-node Intel Linux Cluster is the more cost effective, high performance database. Intel hardware is a commodity, and horizontal scalability is virtually infinite

Even when used as a parallel application, DB2 is a Share-Nothing database. No simultaneous shared storage like SAN is required. DS300/400 (SCSI) or even internal storage with replication may be used for inexpensive configurations

Storage sizing for DB2 (or any database) must contemplate backup and transaction log sizes, not only the pure data size. Sizing must be done by the customer DBA that knows the application

DB2 Parallel needs HA software to take-over the Controlling Process. SA/MP (aka TSA) is included for free on DB2 Parallel on Linux and AIX, for any number of nodes

Detailed DB2 on DRBD Linux cluster configuration manual:
http://www-3.ibm.com/software/data/pubs/papers/#db2halinux

Oracle RAC on Linux Clusters

Oracle 9i Real Application Cluster is a complete solution for n-node Linux Clusters. It is quite popular today

Oracle is a share-all database, so only FC configurations are supported

Oracle filesystem (included in Red Hat Enterprise Advanced Server 3.0) is the software layer that will guarantee shared data integrity

No HA software is needed. Oracle provides HA for the controlling process. Oracle on Red Hat AS + xSeries servers + SAN storage is a complete solution

Storage sizing for DB2 (or any database) must contemplate backup and transaction log sizes, not only the pure data size. Sizing must be done by the customer DBA that knows the application

Special cluster software may be used, but not necessary:

Polyserve

Lotus on Linux Clusters

From an HA Linux clustering perspective, Lotus Domino is a regular application with no parallel characteristics

Domino provides database replication capabilities though, that can be used to eliminate shared storage and reduce solution price

If external storage still needed, DS300/DS400 (SCSI) will do the job. No SAN is required

System Automation for Multiplatforms

Also known as Tivoli System Automation, is the IBM strategic solution for High Availability

Since version 2.1, supports Linux DRBD (replication), ServeRAID and Fiber Channel storages, due to its storage-agnostic features

Powerful high level policy and rules creation with grouping and relationships

Reduce implementation time, coding and support effort with Automation Policy

Can be integrated in a Tivoli-managed data center

HA infrastructure derived from RSCT, from IBM AIX's HACMP

Mainframe-like High Availability for Linux, toward autonomic end-to-end automation

Currently supported on ^ xSeries, pSeries, AIX and zSeries

HA Linux Clusters Contacts

Mark Price/Beaverton/IBM@IBMUS
IGS L3 contact (only if local IGS contracted)

Alan Robertson/Denver/IBM@IBMUS
LTC HA Linux Clusters specialist

Carol Carson/Poughkeepsie/IBM@IBMUS
xSeries Linux Sales Enablement

Douglas McGuire/Lexington/IBM@IBMUS
Americas Linux Clusters

Joachim Schmalzried/Germany/IBM@IBMDE
IBM System Automation for Multiplatforms

High Availability Software:

Tivoli System Automation: http://ibm.com/software/tivoli/products/sys-auto-linux

Linux-HA: http://www.linux-ha.org

Red Hat Advanced Server: http://www.redhat.com

SteelEye's LifeKeeper: http://www.steeleye.com

Polyserve: http://www.polyserve.com

Linux, Open Standards Consultant

IBM Corporation

Avi Alkalay <avix@br.ibm.com>

+55-11-2132-2327

Thank You !

People on This Document

Alan Robertson: HA Linux Clusters LTC specialist

Alcino Bras: FAStT and SAN info

João Marcos: DB2 review

Moacir Malemont: Lotus review

Joachim Schmalzried: Tivoli System Automation awareness

Jeferson Moia: xSeries part numbers

Ana Maria Bezerra: Strategic Outsourcing real-world sizing numbers

Commercial Linux Clusters

Linux HA Clusters Architectures

Linux-HA HA ServeRAID FAStT high availability DRBD DB2 Lotus Oracle RAC

Avi Alkalay

2003-11-02T04:47:41

Avi Alkalay