FAQ: High Performance

Please note that the SMP and GPU technologies discussed here have been incorporated into our V7 client, available on our Home Page.

NVIDIA GeForce™ GTX 590

Introduction

The performance of the Folding@home (FAH) software is critical to the success of the Folding@home project. In order to study many of the problems of interest (especially related to protein misfolding and aggregation, such as in Alzheimer’s disease), we need to not just have lots of computers participating, but we need results returned more quickly so that we can simulate trajectories of sufficient length. When FAH first started, we achieved this by running simulations for many months or even years (indeed, our first Alzheimer’s Disease simulations ran for almost two years straight). However, we want to tackle problems that could take even longer, but those projects wouldn’t be practical if we had to wait many years for all the results to come back. This suggests the need to find methods which can perform the simulations even faster.

Multi-core CPUs and powerful graphics cards are now common, and we’ve seen significant gains in our capabilities by utilizing these technologies in our latest software, the V7 client. In the past, they were each supported by separate software programs. Now, if your computer supports these technologies, the V7 client can use them by default.

Our goal is to apply our simulations to further our knowledge of protein folding, misfolding, and related diseases, including Alzheimer’s disease, Huntington’s disease, and certain forms of cancer. These new technologies give us considerably longer trajectories in the same wall clock time, allowing us to turn what used to take years to simulate even on FAH to a few weeks to months. Thanks to your help and these new technologies, Folding@home is now steadily in the five and six petaFLOP range. In the years following our breakthroughs in 2007, it has remained one of the world’s most powerful computing systems.

We’ve been able to study very complex proteins and perform many drug design and viral entry simulations that would have been impractical before. This power has also allowed us to make our models more accurate, since such accuracy often requires more complex computations. Through a combination of new algorithms, new hardware, and your help, we’ve increased our capabilities by a million fold from when we first started. Every five years the length of the proteins we’ve been able to study have doubled, while the simulation timescales increase by 1000x. Just breaking past a microsecond was a big deal, and we’ve already performed several simulations out to several milliseconds – the timescale for most proteins. We’re really excited about where this appears to be leading, allowing us to tackle really challenging and important problems.

Folding using SMP on multi-cored computers

Introduction

It’s now very common for modern computers to contain multiple CPU cores, which allow the computer to process multiple sets of information in parallel. Most computers contain dual- or quad-core CPUs, and we expect to see more and more CPU cores as time goes on. In FAH, we use these multiple CPU cores together to speed up our simulations. For example, a quad-core CPU can complete Work Units nearly four times faster than a single-core CPU. Now that these are common, a project that would have taken a year to complete now takes us only a few months. Initially, it was a challenge to scale the GROMACS core — the highly optimized software that performs the actual protein folding simulations behind the scenes — to fully utilize multiple CPU cores, but our methods are now quite efficient at using them.

We’re very excited about what the multi-core processors has been able to do so far. One of our papers (#53 in our Papers page) would have been impossible without the multi-core processors and represents a landmark calculation in the simulation of protein folding. We’re looking forward to more exciting results like that in the years to come! Please see the SMP FAQ for more information.

How can I run FAH on my multi-core computer?

For several years following its introduction in late 2006, the SMP client was separate from the other software in FAH. (Our main recommended v6 client only ran on one CPU core). This required a separate download, installation, and configuration. Now that multi-core computers are very common, V7 — the latest generation of our software — can automatically set up SMP during installation as a “slot”. Thus, if your computer contains multiple CPU cores, V7 can install it for you. Please see the V7 FAQ and the Installation Guides for more information.

Folding on graphics cards

What are GPU’s and how can they help FAH?

GPU’s are Graphics Processing Units — chips used in the graphics cards of today’s PCs to help speed high performance graphics, such as 3D games or 3D scientific visualization. GPUs have the possibility to perform an enormous number of Floating Point OPerations (FLOPs). However, they achieve this high performance by losing generality — there are only certain types of calculations which would be well-suited to GPUs (see our GPU FAQs for more information). However, after much work, we have been able to write a highly optimized molecular dynamics code for GPU’s, achieving a 20x to 40x speed increase over comparable CPU code for certain types of calculations in FAH. This means that we will be able to make an enormous advance over what we could do only just a few years ago.

Which GPUs will be supported?

We support CUDA-class GPUs for NVIDIA (primarily G80 and later) and R6xx or later GPUs for ATI. This essentially encapsulates most of the more modern GPUs.

How can I run FAH on my GPU?

Just as with our SMP client, originally the GPU client was separate. Over time, drivers and other underlying software matured, and running FAH on the GPU became easier and more reliable. Now our V7 software will check for supported hardware and software, and it can automatically set up a GPU “slot” for you. Please see the V7 FAQ and the Installation Guides for more information.

Credits

There have been several incarnations of our GPU code for molecular dynamics. The earliest version resulted from our collaborators Professor Pat Hanrahan (Computer Science Department at Stanford) and Prof. Eric Darve (Mechanical Engineering Department at Stanford) and their groups; this version was never released on Folding@home. The next version was written and optimized by V. Vishal and released as GPU1 on Folding@home. Our second generation GPU code (GPU2) on Folding@home was written by V. Vishal, Mark Friedrichs (general code and ATI), Scott LeGrand (NVIDIA port), and Mike Houston (ATI optimizations). They then continued our methods to release our latest iteration, GPU3.

Folding on the Sony Playstation 3 (PS3)

The PS3 is a powerful system for scientific calculation. In 2007, it was more powerful than CPUs, but less powerful than GPUs, and took a middle path between scientific flexibility and speed. Other benefits are the uniformity of PS3s (all have the same processor, GPU, RAM, etc) and the ability to stream data quickly to the GPU, allowing for real time visualization of the protein. Since it’s induction, over 15 million PS3 users have participated in FAH, in total donating more than 100 million computational hours.

The PS3 system was a game changer for Folding@home, as it opened the door for new methods and new processors, eventually also leading to the use of GPUs. We have had numerous successes in recent years. Specifically, in a paper just published in the Journal of Medicinal Chemistry, we report on tests of predictions from earlier Folding@home simulations, and how these predictions have led to a new strategy to fight Alzheimer’s disease. The next steps, now underway at Stanford, are to take this lead compound and help push it towards a viable drug. It’s too early to report on our preliminary results there, but we’re very excited that the directions set out in this paper do appear to be bearing fruit in terms of a viable drug (not just a drug candidate).

Following discussions with Sony, we retired the PS3 client on November 6, 2012. Please see the PS3 FAQ for more information about the PS3 client.

History

We have been working since early 2003 to give the FAH software advanced features to drive the performance of the Folding@home project. We have been trying several different approaches. We are doing this work through collaboration with other groups because there are limits to what the FAH team can do; through collaborations, we can also take advantage of expertise not natively found within our group. Currently, all of our attention has been directed towards our main core, GROMACS. We are working with the GROMACS development team to take the very latest computational methods out of the lab and add them into FAH.

In 2006 the FAH team (with key collaborators in industry and at Stanford University) began a new initiative to take Folding@home the next to the petaFLOP scale. This is 1,000,000,000,000,000 floating-point operations per second, a level of performance that was unmatched by even the fastest supercomputers. To reach this, we needed new technology and new computational methods. We recognized that common inexpensive consumer electronics often contained powerful streaming processors, which had advanced to the point where they could be used for scientific purposes. Although each platform has its limits, we knew that they could work together to give FAH considerably more computational power than ever before.

We pursued the Cell processor found in Sony’s PlayStation 3, and the ATI and Nvidia graphics cards used in personal computers for rendering video games. Our goal was to apply these new technologies to push Folding@home into a new level of capabilities. After much work, we released our first GPU client on October 2006, the SMP client in November, and our PS3 client the following March. Thanks to these platforms, six months later on September 16, 2007, Folding@home became the first distributed computing project to cross the petaFLOP barrier, and we were recognized by Guinness World Records. The PS3s and GPUs were so fast that we started looking at the some results several times a day instead of a couple times a month!

Other platforms

GROMACS on Clearspeed

Clearspeed offered a CPU architecture which, while similar in architecture to GPUs in terms of many parallel FPUs, goes beyond GPUs by having a more flexible architecture and many other scientifically useful properties, such as double precision Floating Point Units. In 2005, Clearspeed worked on porting GROMACS and has done some public demos. The later completed the port and we started testing its feasibility to be used in FAH. We launched this port in July 2005 and saw a significant increase over CPUs. However, as of 2006, we are turning our attention to the other platforms, and do not have any further updates on Clearspeed for the time being.

Intel Xeon Phi

In late 2012, Intel brought to market the Xeon Phi Coprocessor, based on their Many Integrated Core (MIC) architecture. The basis of the Intel MIC architecture is to leverage x86 legacy by creating a x86-compatible multiprocessor architecture that can utilize existing parallelization software tools.

However, there is no direct compatibility with GROMACS, as the Pentium class chips lack modern chip optimizations like SSE, but a SIMD or OPENCL port is not out of the question at some future point. The hardware cost puts this out of the reach of most consumers, and will mostly see an institutional customer base, futher limiting its possible adoption. This platform is being watched as it matures.

FAH on BOINC

BOINC is an infrastructure for doing distributed computing. BOINC is produced by the same group at LBNL which has done SETI@Home and has become a useful tool for distributed computing. Several years ago we explored using BOINC as an additional client in addition to the existing FAH clients in order to give donors a choice, especially those who currently run BOINC.

In January 2006 we launched an initial release BOINC client which we alpha tested in a small group, but we ran into some significant issues with the client. In April we updated much of the code, but we had to deal with a staff turnover in the BOINC part of the development team, which slowed development. As of June 2006 we are putting this platform on hold, as until such time as our staffing situation changes, and the incompatibilities on both sides are resolved, further development has been shelved.

Folding@home is supported primarily from grants from the National Institutes of Health and the National Science Foundation.

For More Information, Please See:

Last Updated on January 24, 2013, at 02:00 PM

Add your computer's power to over 327,000 others that are helping us find cures to Alzheimer's, Huntington's, Parkinson's and many cancers ...

... in just 5 minutes.

Step 1.

Download protein folding simulation software called

Folding@home

.

Step 2.

Run the installation. The software will automatically start up and open a web browser with your control panel.

Step 3.

Follow the instructions to Start Folding.

Stanford University

will send your computer a folding problem to solve. When your first job is completed, your computer will swap the results for a new job.

Download the protein folding simulation software that fits your machine.

()


Installation guide