Transition from Torque(PBS) to SLURM

We have already transitioned all HPC systems from Torque(PBS)/Moab to SLURM.

This is for a few reasons, most notably that Torque is not longer opensource and the cost for Moab is increasing.  SLURM combines these capabilities and is free open-source software, meaning we can customize and optimize it for our purposes.  For these reasons it is becoming the prevailing job management system for scientific clusters around the world and will be the new standard going forward for UPitt Structural Biology.

This page is a brief primer on some differences, not a complete resource.  For a complete SLURM reference, see the page dedicated to SLURM or the SLURM documentation.

Current scheduler status:
Ultron – SLURM
Vision – SLURM
Archer – SLURM
Executor – SLURM

SLURM PBS compatibility layer

Slurm will attempt to understand all PBS options in a batch script as well as command line options (like qsub) via wrapper scripts.  This means that you may be able to run your existing scripts as-is without modification.  This is not guaranteed however, and you are encouraged to become familiar with SLURM and create native SLURM scripts.

Comparison between SLURM and Torque (PBS)

A Torque “queue” = a SLURM “partition

Some common commands in SLURM and Torque(PBS)

TaskTorque/PBSSLURM
Submit a jobqsub myjob.shsbatch myjob.sh
Delete a jobqdel 123scancel 123
Show job statusqstatsqueue
Show expected job start time - (showstart in Maui/Moab)squeue --start
Show queue infoqstat -qsinfo
Show queue detailsqstat -Q -f scontrol show partition
mdiag -c
Show job detailsqstat -f 123scontrol show job 123
Show queue detailsqstat -Q -fscontrol show partition
Show node detailspbsnode n0000scontrol show node n0000
Show QoS details -(mdiag -q in Maui/Moab)sacctmgr show qos

A simple example of a PBS script converted for SLURM:

run1.pbs
—————————————————-

#!/bin/bash
#PBS -N germline
#PBS -m be
#PBS -k oe

cd $PBS_O_WORKDIR
germline -bits 50 -min_m 1 -err_hom 2 <<EOF
1
CEU.22.map
CEU.22.ped
generated
EOF
—————————————————–

run1.slurm
—————————————————–

#!/bin/bash
#SBATCH –job-name=”germline”
#SBATCH –mail-type=BEGIN,END

cd $SLURM_SUBMIT_DIR
germline -bits 50 -min_m 1 -err_hom 2 <<EOF
1
CEU.22.map
CEU.22.ped
generated
EOF
—————————————————–

Note that “#PBS -k oe” is not present in the slurm version.  It is not necessary, and there is not equivalent.  Slurm will by default write a single stderr/stdout file to the directory from which the job was submitted.