Jötunn and Garpur

Jötunn at a glance

nodes: 4cpu: 2x Intel Xeon CPU E5-2690 v3 @ 2.60GHz (2.6 GHz, 12 core)memory: 128GB DDR4interconnect: 10 Gb/s ethernet

Application for an account on the Jötunn cluster must be made via the Univesity of Iceland intranet, https://ugla.hi.is

Support

Support on specific subjects will be added below.

• Basic Usage
• Running Jobs on Garpur and Jötunn
• Garpur advanced job script
• Software

For all staff and students in HÍ, then please use the service portal to for help and include HPC in the subject. There you can also monitor the status of the request.

For people outside HÍ then please send all requests about garpur or jotunn to help@hi.is and include HPC in the subject.

Basic Usage

First step in connecting to a IHPC cluster is setting up a SSH client. Most Linux system have this installed from the get-go, while Windows users are recommended to use the free Windows SSH client PuTTY.
All computations on our HPC systems are performed through a command line interface.
Users should always run their computations on the compute nodes, not the login nodes. This is done by using the queuing system.

Basic HPC Concepts

High performance Computing (HPC): http://en.wikipedia.org/wiki/High-performance_computing

Cluster Computing and High Performace Computing: What these types of resources can and cannot do and how they are different from a desktop computing environment.
http://en.wikipedia.org/wiki/Cluster_computing

Basic Unix/Linux : Working with the shell and command line, file editing, setting file and directory rights etc.

SSH : Tool to used to login into the cluster. http://en.wikipedia.org/wiki/Secure_Shell

Basic Shell Scripting : Users create and edit scripts which launch compute jobs.  These scripts are submitted to a queue and run your job when you reach the head of the line: http://en.wikipedia.org/wiki/Shell_script

Running Jobs on Garpur and Jötunn

Garpur and Jötunn both use the Slurm workload manager.

Users should always run their computations on the compute nodes, not the login nodes. This is done by using the queuing system.

If you have previously been using torque then the following site might be useful http://www.nersc.gov/users/computational-systems/cori/running-jobs/for-edison-users/torque-moab-vs-slurm-comparisons/

To run a job on the system, you need to create a job script. A job script is a reqular shell script either bash or csh with some directives which specifies number of cpus, memory etc. Then, this will be interpreted by the batch system on submission. Below is a very basic job sample script:

#!/bin/bash
#SBATCH -n 1
#SBATCH -J ExampleJobName

#print hostname on which the job is running
echo $HOSTNAME

Once you have your job script ready, you can use qsub command as follows:

sbatch <your job script filename>

To check the status of your job you can then do:

squeue -u <username>

Other options of note for job scripts

• -N nodes= 2 –ntasks-per-node=4
• Requests 2 nodes with 4 cores each for the job
• -p himem
• Requests to use the high memory nodes or gpu nodes [Garpur only]

If you want to run a job on the compute nodes interactivly, you can use the command
salloc -N 1

ssh $SLURM_NODELIST
To leave this interactive session use the “exit” command.

Software

Environment Modules

Software management is done by modules on IHPC systems. Once you logged into one of our clusters, module command is available in your environment.

1. To see which modules are loaded in your session, use the command:
   module list
2. To see which modules are available in the system or to list the complete application, use the command:
   module avail (or “module avail python” to only see what python packages are installed)
3. Loading module to your environment, you can use following command for instance to load boost/1.48.0
   module load intel/compiler/2016.3
   This will load the Intel compilers version 2016.3
4. To unload the module, use the following command
   module unload intel/compiler/2016.3
   This will unload the intel/compiler/2016.3 version from your environment

For additional software requests, please contact support

help@hi.is

Python

You can use virtualenv in python and it will install all packages you need into your virtualenv directory wherever you create it. Something like this should work (obviously you can call the directories whatever you want):

1. mkdir python_venv (or what ever you want to call the directory)
2. cd python_venv
3. module load python/3.8.1 (or python/3.6.1 or some other python you need)
4. python3 -m venv python381
5. source python381/bin/activate
6. pip install hello_world (pip, pip3 and pip3.8 is all the same here as you can see in the newly create bin directory)
7. And when you are done you can deactivate it with python381/bin/deactivate or just log out (shortcut: ctrl+d)

The above example will create a python virtual environment where you can customize it like you want and have different versions of python etc. So it copies a small portion of python to your home directory where you can customize it. Then the module hello_world is installed with pip which in your case would probably be something more useful to your project.

You can read more about it here (and by google-ing) https://docs.python.org/3/tutorial/venv.html