Introduction

This document describes how to run your simulations on the CCS oneSIS cluster.

Model

You cannot run simulations directly on the compute nodes. To use the cluster, ssh into the head node ares, then submit your jobs using the PBS queuing system command qsub. Both ares and the compute nodes share /scratch00 and /scratch02 through NFS.

The program qsub allocates compute nodes and runs programs on the compute CPUs. Each compute node can run 2 to 16 simulations depending on the number of CPU cores it has. Memory usage is also a consideration when requesting CPUs during job submission.

NOTE: THE COMPUTE NODES CAN ONLY WRITE TO /scratch00 and /scratch02!! Keep this in mind when writing your scripts.

Getting Started

The compute nodes have no shared libraries and no executables. Any executable you run on the compute nodes must be statically linked for this reason. Here is a sample compile to run on the oneSIS cluster:

1. Choose a compiler.
   
   

    >module load intel
   
   

2. Compile the code staticall
   
   

    >icc -static program.c -o a.out
   
   

3. Now that you have your staticly linked executable a.out you can copy the executable and any data you might need to start the run to the shared file system /scratch00.
   
   

    >mkdir -p /scratch00/john/run1
   
    >cp ./a.out /scratch00/john/run1/
   
    >cp ./data.txt /scratch00/john/run1/
   
   

4. Now that you have your executable and all your data in /scratch00 you need to use the PBS queuing system to submit your job. For more details look at the PBS page.