Every so often people come up to us and ask, “Aren’t compilers and compiler directives good enough for HPC applications?” or “Won’t a compiler accomplish that for us?” While compilers have made massive progress in the last two decades, they are still nowhere near the point of putting us and many other HPC programmers out of business. Compilers are still a “one-size-fits-all” solution that needs to be able to deal with any and all input, whereas HPC programmers can be thought of as a designer-fitted solution. Application expertise brings a lot to the table that compilers cannot compete with: Our past experiences have helped us optimize applications that have irregular memory access patterns. While some applications such as matrix applications have regular and simple …
No Free Lunch for GPU Compiler Directives
Last week, Steve Scott at NVIDIA put up a viral post entitled, “No Free Lunch for Intel MIC (or GPU’s).” It was a great read and a big hit in technical computing circles. The centrepiece of Scott’s piece was to say that there are no magic compilers. GPUs don’t have them, and neither will MIC. No compiler will be able to automatically recompile existing code and get great performance from MIC or GPUs. Rather, it takes a good amount of elbow grease to write high-performance code. We totally agree. The problem Scott addresses is real. Despite marketing spin to the contrary, developing code for GPUs requires work. However, we don’t agree with Scott’s conclusion that compiler directives are a good solution. You can’t fight …