Tutorial
Using the HPC for the first time can be daunting, so we have put together a simple tutorial for you! It is important to note that you will not be able to follow along with the tutorial if you do not have an HPC account set up, if that is the case fill out the HPC Request Form. You also need to be connected to UNI's network to remotely connect to the HPC, if you are off campus consider requesting access to UNI's VPN.
Explanation
Our goal with this project is to show the benefits of the HPC. In this example, we have a program where we want to calculate five large factorials. Our python program runs two functions, one of them will calculate all of our factorials one at a time. This is how python programs are normally written and executed. The second function will request multiple processes to calculate all of the factorials at once, so the amount of time will drastically decrease. Once the program finishes running, it will output a file with the runtimes of each task.
The function that calculates our factorials one at a time will only require one thread for all of the calculations. The function that calculates our factorials in parallel will require 5 threads (one for each factorial function). This brings the total number of threads for this process up to 6, so we need to request 6 CPUs in the job script.
Steps | Screen Capture |
---|---|
1. First you will need to start by downloading the program file along with the job script. We strongly recommend you look through the python code to get an understanding of how to write code that will run multiple processes. There are comments to help you understand what is going on. | ![]() |
2. When you download the program file and the job script make sure you remove the .txt file extension and save as Type: "All files (*.*)". You should end up with the two following files: "tutorial.py", "job.script". You might not have the same icons if you don't have python installed but as long as it does not say "Text Document" under Type, you will be fine. | ![]() |
3. Open File Explorer and navigate to your downloads folder. Click on the "job.script" file and right click. On the menu that pops up click on 'Copy as Path'. | ![]() |
4. Now you will need to open Terminal on your computer and type the following command: scp "" [CatID]@login.hpc.uni.edu:/home/[CatID] You will need to replace [CatID] with your CatID and inside the quotations you will need to paste the path that you copied in step 3. | ![]() |
5. Enter the command and you should get prompted for your CatID password. When you type your password, nothing will show on screen. After you enter your password, you should see the file transfer to the HPC. | ![]() |
6. Now Copy the previous command and replace "job.script" in the quotations with "tutorial.py". Enter the command and retype your password. You should now have both files uploaded to the HPC under your account. | ![]() |
7. In your terminal type the next command: ssh [CatID]@login.hpc.uni.edu When you enter you will be asked for your CatID password again. Once you submit it you will be remotely connected to the HPC, and you can now submit commands! | ![]() |
8. If you type the "ls" command, you should see both of the files you just uploaded. Before we can run the python code we need to make the file executable. Type the following command: chmod +x tutorial.py Type "ls" again and you should see that the tutorial.py file is now green, this means it is executable and ready to run. | ![]() |
9. You are now ready to submit the python code as a job to run, run the following command: sbatch job.script Now if you run the "squeue" command, you should see your job running on the cluster. | ![]() |
10. Once the job is done, it should take around 75 seconds, run the "ls" command. You should see a new file called "run-times.txt". Type the following command to open the file with the emacs text editor: emacs run-times.txt | ![]() |
11. You can see that when the program runs in parallel it finishes running in less than half the time it would normally. The speed gains from running in parallel depends on what you are trying to do, but it will save you time and utilize the resources that the HPC has to offer. You have finished the tutorial and can now delete the files from your home directory! | ![]() |