Difference between revisions of "SLURM GPU"
| (12 intermediate revisions by 2 users not shown) | |||
| Line 10: | Line 10: | ||
For basic GPU jobs, where you will be using a single CPU thread and a single GPU, the following will be sufficient:<br /> | For basic GPU jobs, where you will be using a single CPU thread and a single GPU, the following will be sufficient:<br /> | ||
<pre style="white-space:pre-wrap; width:22%; border:1px solid lightgrey; background:#E0E0E0; color:black;">#SBATCH --name=gpu_job | <pre style="white-space:pre-wrap; width:22%; border:1px solid lightgrey; background:#E0E0E0; color:black;">#SBATCH --job-name=gpu_job | ||
#SBATCH --time=01:00:00 | #SBATCH --time=01:00:00 | ||
#SBATCH --partition=snsm_itn19 | |||
#SBATCH --qos=openaccess | |||
#SBATCH --gres=gpu:1 | #SBATCH --gres=gpu:1 | ||
./GpuEnabledExe</pre> | ./GpuEnabledExe</pre> | ||
You can request up to two GPUs be used by a single node job. This can be requested by calling<br /> | You can request up to two GPUs be used by a single node job. This can be requested by calling<br /> | ||
| Line 19: | Line 22: | ||
== MPI GPU Jobs == | == MPI GPU Jobs == | ||
On CIRCE, MPI parallel jobs with GPU support require extra care when crafting your request. You MUST use the nodes,ntasks-per-node parameters for your job request. In this case, every node you select (up to 20) will get you an additional GPU. So, lets say I want to use 8 GPUs with MPI, each with a single CPU thread: | On the CIRCE/Student clusters, MPI parallel jobs with GPU support require extra care when crafting your request. You MUST use the nodes,ntasks-per-node parameters for your job request. In this case, every node you select (up to 20) will get you an additional GPU. So, lets say I want to use 8 GPUs with MPI, each with a single CPU thread: | ||
<pre style="white-space:pre-wrap; width:25%; border:1px solid lightgrey; background:#E0E0E0; color:black;">#SBATCH --name=myMPIGPUjob | <pre style="white-space:pre-wrap; width:25%; border:1px solid lightgrey; background:#E0E0E0; color:black;">#SBATCH --job-name=myMPIGPUjob | ||
#SBATCH --time=01:00:00 | #SBATCH --time=01:00:00 | ||
#SBATCH --nodes=8 | #SBATCH --nodes=8 | ||
#SBATCH --ntasks-per-node=1 | #SBATCH --ntasks-per-node=1 | ||
#SBATCH --partition=snsm_itn19 | |||
#SBATCH --qos=openaccess | |||
#SBATCH --gres=gpu:1 | #SBATCH --gres=gpu:1 | ||
./MpiGpuExe</pre> | ./MpiGpuExe</pre> | ||
The largest possible GPU job is expressed below (this may take quite some time to dispatch) | The largest possible GPU job is expressed below (this may take quite some time to dispatch) | ||
<pre style="white-space:pre-wrap; width:25%; border:1px solid lightgrey; background:#E0E0E0; color:black;">#SBATCH --name=myMPIGPUjob | <pre style="white-space:pre-wrap; width:25%; border:1px solid lightgrey; background:#E0E0E0; color:black;">#SBATCH --job-name=myMPIGPUjob | ||
#SBATCH --time=01:00:00 | #SBATCH --time=01:00:00 | ||
#SBATCH --nodes=20 | #SBATCH --nodes=20 | ||
#SBATCH --ntasks-per-node=2 | #SBATCH --ntasks-per-node=2 | ||
#SBATCH --partition=snsm_itn19 | |||
#SBATCH --qos=openaccess | |||
#SBATCH --gres=gpu:2 | #SBATCH --gres=gpu:2 | ||
./MpiGpuExe</pre> | ./MpiGpuExe</pre> | ||
This will request the use of 40 GPUs for an MPI application. | This will request the use of 40 GPUs for an MPI application. | ||
| Line 39: | Line 48: | ||
== Specify a GPU model == | == Specify a GPU model == | ||
The available GPU versions on CIRCE are listed below: | |||
{| class="wikitable" | {| class="wikitable" | ||
|- style="background-color:#f1edbe;" | |- style="background-color:#f1edbe;" | ||
| '''Model''' | | '''Model''' | ||
| '''Constraint''' | | '''Constraint''' | ||
| '''Number Available''' | | '''Number Available''' | ||
| '''Partition Availability''' | |||
|- | |- | ||
|- | |- | ||
| | |GeForce GTX 1080 Ti | ||
| | |gpu_gtx1080ti | ||
| | |49 | ||
| | |snsm_itn19 | ||
|} | |} | ||
However, for inquisitive users who would like to specify a GPU model using a SLURM "Constraint", your submission script would look similar to: | |||
<pre style="white-space:pre-wrap; width:22%; border:1px solid lightgrey; background:#E0E0E0; color:black;">#SBATCH --job-name=gpu_job | |||
#SBATCH --time=01:00:00 | |||
#SBATCH --partition=snsm_itn19 | |||
#SBATCH --qos=openaccess | |||
#SBATCH --gres=gpu:1 | |||
#SBATCH -C "gpu_K20" | |||
</pre> | |||
Latest revision as of 17:39, 25 May 2021
GPU Jobs
Research Computing hosts 28 compute nodes with GPU capabilities. Requesting GPU resources requires additional parameters to be added to your job script in order to land on the right equipment and ensure that your job is dispatched correctly.
Basic GPU Job
For basic GPU jobs, where you will be using a single CPU thread and a single GPU, the following will be sufficient:
#SBATCH --job-name=gpu_job #SBATCH --time=01:00:00 #SBATCH --partition=snsm_itn19 #SBATCH --qos=openaccess #SBATCH --gres=gpu:1 ./GpuEnabledExe
You can request up to two GPUs be used by a single node job. This can be requested by calling
#SBATCH --gres=gpu:2
MPI GPU Jobs
On the CIRCE/Student clusters, MPI parallel jobs with GPU support require extra care when crafting your request. You MUST use the nodes,ntasks-per-node parameters for your job request. In this case, every node you select (up to 20) will get you an additional GPU. So, lets say I want to use 8 GPUs with MPI, each with a single CPU thread:
#SBATCH --job-name=myMPIGPUjob #SBATCH --time=01:00:00 #SBATCH --nodes=8 #SBATCH --ntasks-per-node=1 #SBATCH --partition=snsm_itn19 #SBATCH --qos=openaccess #SBATCH --gres=gpu:1 ./MpiGpuExe
The largest possible GPU job is expressed below (this may take quite some time to dispatch)
#SBATCH --job-name=myMPIGPUjob #SBATCH --time=01:00:00 #SBATCH --nodes=20 #SBATCH --ntasks-per-node=2 #SBATCH --partition=snsm_itn19 #SBATCH --qos=openaccess #SBATCH --gres=gpu:2 ./MpiGpuExe
This will request the use of 40 GPUs for an MPI application.
Specify a GPU model
The available GPU versions on CIRCE are listed below:
| Model | Constraint | Number Available | Partition Availability |
| GeForce GTX 1080 Ti | gpu_gtx1080ti | 49 | snsm_itn19 |
However, for inquisitive users who would like to specify a GPU model using a SLURM "Constraint", your submission script would look similar to:
#SBATCH --job-name=gpu_job #SBATCH --time=01:00:00 #SBATCH --partition=snsm_itn19 #SBATCH --qos=openaccess #SBATCH --gres=gpu:1 #SBATCH -C "gpu_K20"