Introduction to Parallel Computing

This is the main page of a graduate-level course in parallel computing being taught in 2012/2 at the Polytechnic Institute IPRJ/UERJ. It is generally useful for programmers at the advanced level in the fields of scientific and multimedia programming.

General Info

• Meeting times: Tues 12:30pm-2pm, Thursdays 2:20pm - 4pm
• Evaluation criteria: 1 quizz at the end of the term (60%), plus practical projects (40%).
• Forum for file exchange and discussion: uerj.tk

Pre-requisites

• Linux - intermediate to advanced (will be reviewed as needed) - read Literatura recomendada pela equipe
• C/C++ - intermediate to advanced (will be reviewed) - read Literatura recomendada pela equipe

Approximate Content

The course focuses on software techniques for parallel computing. We are aiming at a comprehensive treatment on different types of practical parallel programming techniques

• process-oriented parallel programming
• thread programming/thread safety
• single-core vector instructions
• multi-processor and multi-core programming
• mapreduce/hadoop
• MPI
• Cuda

Each of the above techniques would merit a course of their own, as done in many of the best universities. Therefore we aim at attaining a practical familiarity with each, in the first half of the course, and we will specialize in the later part of the course as to help the students' research.

Main Resources

• Textbooks
  • 1st part of the course: "Is Parallel Programming Hard, and, if so, what can you do about it?" - Paul E. McKenney / IBM (editor).
  • For MPI: "An Introduction to Parallel Programming" , by Peter Pacheco[1]
  • For Cuda: Programming Massively Parallel Processors: A Hands-On Approach[2]
  • General textbook: A. Grama, A. Gupta, G. Karypis, V. Kumar, Introduction to Parallel Computing, Second Edition, Addison-Wesley, 2003.
  • Algorithms-oriented textbook: Algorithms: sequential, parallel, and distributed, Kenneth A. Berman, Jerome L. Paul
    • For more theoretical info, see the chapter on parallel algorits in Cormen's classical algorithms book.
• Mapreduce, GFS, and Bigtable:
  • Papers from Google
  • http://code.google.com/edu/parallel/mapreduce-tutorial.html
  • Wikipedia
  • Hadoop documentation
• Presentations from IBM and Intel: Cilk, etc.
  • http://software.intel.com/en-us/videos/channel/parallel-programming
• Wikipedia pages
  • http://en.wikipedia.org/wiki/Vectorization_%28parallel_computing%29
  • http://en.wikipedia.org/wiki/Mapreduce
  • http://en.wikipedia.org/wiki/Hadoop
• Rice lecture notes on Parallel Computing [3]
• Other Resources
  • http://www-users.cs.umn.edu/~karypis/parbook/
  • http://www.cse.iitd.ernet.in/~subodh/courses/CSL860/

Lectures

Partial listing & Tentative Outline

1. Overview of parallel computing: https://computing.llnl.gov/tutorials/parallel_comp/
2. Review of Linux:
   1. See the book Running Linux http://wiki.nosdigitais.teia.org.br/Literatura_recomendada_pela_equipe
3. Review of C/C++
4. Fundamental programming techniques: processes and threads
   1. Read The Unix Programming Environment for some classic multi-process programming[4]
5. Mapreduce/Hadoop
6. MPI
7. Cuda

Homework

Project 1

• The class will be divided into interest groups
• The interest groups will each pick one of the following technologies or else propose another one of their liking.
  • mapreduce/hadoop
  • MPI
  • Cuda
  • Hybrid technologies (Distributed + Cuda)
  • Languages designed for concurrency: Google Go, Erlang, etc.
  • Programming for the Playstation 3 parallel architecture[5]
• The project will consist in a series of presentations by the group members

individually.

• Grade will be asigned to each presentation individually

Project 2

• This project will consist on a practical programming problem from the

student's research. The student is required to describe a problem from his research and present a parallel implementation of some aspect of it.

• This will be evaluated through actual code and acompanying monograph.