Documentation Tips

Setting Up

Compiling and Running the Code

Some Simple Trouble Shooting

Return to User Information

Return to Beowulf Home Page

1. The ScaLAPACK routines have a systematic naming scheme which tells
you alot about a function. For example `PDDBSV` is a double
precision routine (PD) that solves a linear system (SV).

2. The ScaLAPACK routines are grouped into single precision, double
precision, complex, complex16 categories at the bottom of the page.
These links connect to an index of routines.
So to see the code for `PDDBSV` click on the double precision
link, and search for `PDDBSV` or to find single precision
eigenvalue routines go to the single precision index and search on the
key word eigenvalues.

3. The ScaLAPACK User's Guide is difficult to use, since the index and table of contents are not complete and often the active links don't have any contents. I found that doing a search on google was often an effect way to find information.

4.Finally, the site gives instructions for getting started with
ScaLAPACK; Please note that **all necessary libraries have already been
installed on the cluster**, so it is not necessary for you to
compile any libraries in your home directory. Our installation of
ScaLAPACK uses the ATLAS libraries installed on the cluster
(these are optimized BLAS libraries).

**SLmake.inc : ** This defines the necessary variables for the
Makefile and will be the same for any ScaLAPACK program that you
write. It contains all the library and header information that is used by the
Makefile when linking your program.

**Makefile :** This make file has targets that create
the executable xsmsolve xsolve xeig. If you are new to makefiles
you can easily modify this simple example
to accommodate any ScaLAPACK program that you create.

**pdlaread.f: ** One specified process reads a matrix
stored as a text file
and either writes the data to its own submatrix or broadcasts the data to
the appropriate process. On completion each process in the process grid
has a submatrix. The global matrix exists only as a collection of
submatrices in the grid, in other words no process in the grid ever
has the whole global matrix as defined in the file -- it is a 2D block
cyclically distributed matrix.

**pdlawrite.f : ** If a 2D block
cyclically distributed matrix is stored on the processes in a process grid
one can use pdlawrite to have one process receive the values of the matrix from
the other process and write them to a file. The result is the
whole global matrix goes into one file.

**pdscaexinfo.f :** This simple program uses one process in the
grid to read the header
information from HEADER.dat and broadcasts this information to all
other processes
in the grid.

**solve.f : ** This code calls pdlaread and
scaexinfo to get the data needed to solve a large square system. It solves the
system with a call to `pdgesv`, and writes the answer to the
file `sol.dat` with a call to `pdlawrite`. The answer is
also checked and the result is reported in the log file.
It also shows how to make ScaLAPACK calls to calculate
norms and perform Matrix vector multiplications.

**smallsolve.f : ** This code does the same things as solve.f,
but it also writes the whole matrix A, b and x to the log file returned
at runtime. Memory problems occur if this is used for large files.

**eig.f : ** This code calls `pdlaread` and
scaexinfo to get the Matrix. It finds some of the eigenvalues for a
general dense matrix. It calls `PDGEHRD` to reduce a real general
matrix A to
upper Hessenberg form H by an orthogonal similarity
transformation H = Q^T*A*Q. It then calculates the eigenvalues of H
with a call to the subroutine `PDGEHQR`.
`PDGEHQR` takes the upper Hessenberg form H of the real general matrix A
and calculates H's eigenvalues
which are the same as those of A. The eigenvalues are written to two
files realeig.dat and imageig.dat where each file contains the
respective parts of a complex vector;

eigenvalue = realeig + i*imageig.

ScaLAPACK documentation
requires the block size (NB) >= 6 otherwise `PDGEHQR` doesn't
work properly.

**RHS.dat: ** This contains the write hand side of the system of
equations. The first line contains the row and column size of the vector,
each additional line contains one element from the vector.

**MAT.dat: ** This contains the matrix of the linear system or
the matrix whose eigenvalues are to be computed. The first line contains
the row and column size of the Matrix, each additional line contains
one element from the Matrix, organized a column at a time; the first n
elements are the first column of A, the second n elements are the
second column of A,....

**HEADER.dat: ** A small file that contains basic information needed by all codes
listed here, including the process grid info. The file format is self
explanatory and an
explanation of the input/output parameters follows:

N : The order of the matrix A.

NRHS : The number of right hand sides.

NB : The size of the square blocks the matrices A and B are split
into.

P : The number of process rows.

Q : The number of process columns.

**write.m: ** This is a simple matlab code that will take a
matrix or vector from Matlab and write it to a file that is readable
by `pdlaread.f`

**Compiling: **
`make [eig, solve, smsolve]`

**Running: **
`scasub -mpimon -np [process columns * process rows]
[xeig, xsolve, xsmsolve]`

**Error Codes:**
When a ScaLAPACK routine returns the error code is stored in the
variable INFO. The programs listed above print out this code in the log
file for each of the major ScaLAPACK computations. INFO = 0 means the
call was successfully completed, otherwise something is wrong.

For example you might get the message:

`PDGESV para 605 illegal value`

This mens that INFO = -605 = -(100*i) + j
so i = 6 and j = 5 implies that there is a problem with the 5th
element of the array DESCA which is the 6th argument passed to
`PDGESV`.

**Work Space:**
A segmentation violation and `INFO = 0` indicates that one of the ScaLAPACK routines needs more workspace.
A fast and reliable method is to have the routine do an LWORK Query.

Created February 21, 2003 by Janine Kennedy.