|
Table of contents
About
Network details
Software details
About
Kluster is a 16 node OpenMosix based distributed computing environment. The software used is a patched, revised implementation of the Red Hat 9.0 distribution and is named Green Planet Cluster Linux (RH 9.0). It has LAM-MPI software support for MPI- programming and full Red Hat Linux 9.0 compatibility. Kluster was developed and configured by the MSc students (2001-2004 batch) from St. Thomas College in Kozhencherry under the guidance of Ninan Sajeeth Philip who is an associate of IUCAA from the same institution. Of the 16 machines, one acts as a file server and the rest are etherbooted using the tftp protocol. To get Red Hat 9.0 compatibility on all the nodes, the exported file system was custom configured based on the concepts utilized by the Linux Terminal Server Project. The OpenMosix Linux patch for the Linux vanilla kernel 2.3.20 is used as the base system on all the 16 nodes and are networked by 100 Mbps fast Ethernet cards and a 24 port switch. The nodes are grouped into 4 units and are connected via a KVM switch for display, keyboard and mouse connectivity. These connections are not mandatory but is done for the ease of testing/monitoring.
[Back to top]Network details
Kluster is configured as an internal vlan with IP address ranging from 172.16.10.1-172.16.10.16 that is visible only within Kluster. The rest of the network has to enter Kluster through the network IP 192.168.19.1. Separate user accounts are provided to each user who intend to work on the cluster. Users need to ssh into the system. Once logged in, rsh is enabled on all nodes that MPI software may run without requesting for passwords.
/usr and /home areas are exported to all the nodes in the cluster. If node based user space is required, separate 40 GB is provided on each node under /usr2. 128 Mb space is mounted on the master node as /usr1 and is visible on all the nodes. The / area on the diskless nodes are identical except for a few machine dependent areas that are separately mounted for each node on the file server. This include the ssh keys, hardware details, NIC configuration details etc.
Services can be started or stopped through /etc/rc.d/init.d on all nodes independently as per requirement of the user. Root privileges are required to run these options.
[Back to top]Software details
Kluster has OpenMosix patched kernel Linux 2.4.20 on the server and Green Planet Cluster Linux (RH 9.0) on all the nodes. Green Planet Cluster Linux is an experimental Open Source Software for running diskless clusters developed by Ninan Sajeeth Philip for this project. Although OpenMosix is a kernel patch and is running on the Cluster, its functions can be inhibited locally on any node. The basic command to start, stop or restart OpenMosix services on the cluster is:
/etc/rc.d/init.d/openmosix {start,stop,restart}.
LAM-MPI is installed on the system. A sample lamhosts file is available under /etc/lamhosts. One can lamboot the entire cluster by issuing the command:
lamboot -v /etc/lamhosts
[Back to top]OpenMosix Tutorial
Here are links to OpenMosix Tutorial
MPI Tutorial
Here are links to MPI Tutorial
How To Write and Run programmes on the Kluster
Writing programs for the cluster requires some extra work as compared to conventional programming. The major difference is that we have to do things in parallel to achieve best performance from the Kluster. Most of the things can be parallelized and the programmer has to decide how exactly his code should be organized that the results would be optimal. Many good tutors exists on the web that can help you develop codes for distributed computing environments.
As mentioned earlier, the Kluster supports both OpenMosix and LAM-MPI programming environments. These two environments are different in many respects but have their own merits and demerits. OpenMosix allows automated migration of computational loads across nodes in the cluster. MPI users sometimes consider it as a disadvantage while a few points out this as an advantage since this automatically takes care of all load balancing issues. Options are available to stop any of the services if the user wants so.
For those who want things in Numbers, a performance analysis of the Kluster using skampi benchmarking software with OpenMosix a nd LAM running simultaneously it is available here. To understand the numbers you might want to read this manual. In compute intensive clusters with more than one user running the codes, it is in general observed that MPI on top of OpenMosix only improves performance. We expect some realistic evaluation of this postulate at the end of the Alpha test of Kluster.
A very useful and basic introduction to MPI programming can be found here.
A simple primer to Fork() can be found here.
Those who do not like the syntax of C++ or Fortran, it is also possible to write MPI code in Python.
Once you are ready with a code to be run on the Kluster, you need to get a user account on the machine. The system administrator creates an account for you with the command:
addkuser your_new_ID
This command creates an account for you on the server and on all nodes of the Kluster. You are now ready to run your code.
ssh to 192.168.19.1 and login with your user name and password. To copy your files from IUCAA network to the cluster, use sftp user@192.168.19.1 with user replaced by your Kluster user name.
If your code has many procedures that can be executed in parallel by allocating and deallocating its own memory space, you can use the OpenMosix features to parallelize the computation. You need not even have to recompile the code. Just issue the command:
mosrun your_programme_name
OpenMosix will automatically migrate your procedures across the machines!! If you want to
see a simple example, here is one.
If you are running the code from Xwindows,
you can actually see the load on each node by executing:
openmosixview &
OR , if you need a graphical interface that allows you to see and even drag procedures across nodes, execute:
openmosixmigmon &
For terminals,
mosmon
can show the load on each system.
Here is a code to run N-body simulation using fork.
If your code uses MPI, you need to recompile your code on Kluster with the command:
mpicc your_c_program
OR
mpic++ your_c++_program
OR
mpif77 your_f77_program
Since your home area is ported to all the nodes in the cluster, the compiled version of the code is also visible to all nodes. The exporting is done through oMFS, the OpenMosix file system.
Let us assume that your code does simultaneously N procedures in parallel. This means that you would like to run N things in parallel. To make this possible, first you need to lamboot your system. You can do this by issuing the command:
lamboot -v /etc/lamhosts
As the program runs, you can see the nodes coming up. If you are running mosmon in a terminal, the load on each node can be visualized.
Since the system is ready to run MPI codes as soon as lamboot is run, you may now run your code with all N procedures running simultaneously by issuing the command:
mpirun -np N name_of_your_executable
As your program picksup load, it can be seen in the mosmon window.
A WORD OF CAUTION
Since the same /home is ported to all the nodes, if your nodes write output files instead of passing the results to the originating node, make sure that adequate steps are taken to prevent the nodes from overwriting the files it generate. One simple method is to save them with separate names.
Finally, if you have some questions or difficulty using Kluster, you are most welcome to e-mail me...
All the Best and Happy Klustering!!
Benchmarking Tests
Here are links to benchmarking tests
This HTML page was created by Joe Philip Ninan. Visit my website at http://www.geocities.com/joephilipninan