Linux/Unix - Redundant web servers

Author

Redundant web servers

ccieToBe

2002-10-18, 12:10 pm

I have two servers that I'd like to setup in a redundant configuration. They'll be providing web, ftp, dns and email services. The hardware in each server is identical (900MHz Duron, 80GB ATA66 HD, 64MB RAM). Any ideas on how I can pull this off? I know that I could setup DNS to load balance betwen the servers and am fine with doing that...but what about keeping them sychronized? For example, let's say someone logs into one of the servers and uploads a new page to their web site. Is there any way for the two servers to then automatically synchronize? This is an even bigger issue with email. Here's what I've come up with so far:

Use something like rsync - this works for the web end, but I don't think it would do anything for me as far as email goes.

Setting up a cluster - it looks like this should work?

I'm open to using any open source solution. Any suggestions are appreciated.

ccieToBe

2002-10-18, 5:32 pm

Let me add to what I'm trying to accomplish...

There will be a primary server located on my network. DNS will be setup so that all traffic goes to it as long as it's up. What I'd like to do is then have a secondary server that would be located on someone else's network (for redundancy reasons). I guess it all boils down to wanting to syncronize certain directories (/var/mail and /home) between these systems so that if the primary goes down, no data is lost and the secondary picks up the load immediately.

I guess one way that this could be accomplished would be by using Samba's DFS implementation through an SSH tunnel? Or is there a better way to pull this off? I might just use one server with a RAID array for simplicity's sake.

Mr. Linux Guy

2002-10-19, 10:43 pm

First off, let me mention that RH has a product called Piranha which may be what you are looking for. I do not know of any ports to BSD, but it is open source and you can try compiling it and see if you can get it t work on your network. The officuao version with support sells for like 3, 000 bucks, but there's no way I would spend that much cash inless I couldn't handle it on my own. There is a short review of the RH solution here:

http://www.unixreview.com/documents/s=1245/urm0008c/

Polyserve and Understudy are other products which may be of use to you. Seems like the ideal solution if you could afford it would be to mirror your disks, or at the very least, to kludge up a script that will copy your data from the main server to the failover, so as to have it ready in case of a problem. This can be expensive but it is simple.

If you have a dual setup you could make sure that any request that one sever gets is also directed to another machine. If the secondary machine can be made to monitor the primary to see if it is doing its job right (or wrong), it can take over the function of the primary machine if the processes begin to thrash. I have set our HP-UX boxes to do so at the college, but there will be a slight delay. Not sure if this has given you any ideas. If not let me know and will try to come up with something more usefulo and within your operating budget.

R.

One thing I wouod also mention on this note is that if possible, it may be a good idea to set aside some dedicated servers for mail, dns, etc., depending on how heavily they may be used. This simplifies configuration and troubleshooting. It may not always be necessary or affordable, but if possilbe, it is an ideal situation.

ZFletche

2002-10-22, 1:21 am

You could also look into using rsync and a cron job. though your Data would be good to the last sync

This was taken fron the rsync homepage

rsync is a file transfer program for Unix systems. rsync uses the "rsync algorithm" which provides a very fast method for bringing remote files into sync. It does this by sending just the differences in the files across the link, without requiring that both sets of files are present at one of the ends of the link beforehand.
Some features of rsync include

can update whole directory trees and filesystems
optionally preserves symbolic links, hard links, file ownership, permissions, devices and times
requires no special privilages to install
internal pipelining reduces latency for multiple files
can use rsh, ssh or direct sockets as the transport
supports anonymous rsync which is ideal for mirroring

ccieToBe

2002-10-24, 11:43 pm

Thanks for the info Mr. Linux Guy and ZFletche. A cluster looks like the ultimate solution, but the administrative and bandwidth overhead doesn't look too appealing until I start doing this on a much large scale. This should be a fairly low utilization server - I just want to have high uptime.

I think what I'll do is setup a RAID 1 array on the primary server, then have a secondary that uses rsync to syncronize the web and ftp sites. My thought is if the primary goes down, the only things the secondary server will be lacking are recently updated files, and mail that was sitting in the queue. Once the primary comes back up it should be able to process what was left in its queue. It would then take over again and the secondary would go back into its passive mode. As far as user accounts go - it should be easy enough to write a shell script that modifies both servers when I add, remove or modify accounts.

I am splitting up tasks onto multiple serers to a certain extent. I have 2 web servers, 2 VPN servers and 1 game server, 1 caching server. The VPN servers are going to run internal DNS anyway (for load balancing/failover purposes), so it would probably make a lot of sense for me to just offload the internal DNS to those and use my ISP's DNS servers for external DNS.

Mr. Linux Guy

2002-10-28, 3:04 pm

Sounds like an interesting setup. I haven't done much with clustering but will be taking a few CS/EE classes at LSU this spring, may get to do some fun stuff then.

rlrouns

2002-10-29, 12:57 pm

Go to www.redbooks.ibm.com and do a search on SG24-5994 this is a redbook dealing with Linux on Netfinity and Xseries servers (basically Intel based servers). The first part deals with High Availability Clustering on a Linux platform. It goes into the different types of clustering and the software out there to do it. You can view the redbook in html or PDF format. Here is a direct link to the book: http://publib-b.boulder.ibm.com/cgi...query=SG24-5994

I hope this helps. I know that it will clear up some issues and at least help you get steered in the right direction!

Roberto

ccieToBe

2002-10-29, 1:25 pm

Thanks rlouns.

Sponsored Links