Visualization of a small portion of the computational results of an astrophysical simulation of 8.8 million gravitating bodies run on 512 nodes of the Concurrent Supercomputing Consortium's Intel Touchstone Delta System. The image shows about 137,000 gravitating bodies that form a single halo about 400 megaparsecs across. Color represents projected density along the line of sight, i.e., different colors correspond to pixels that "contain" different numbers of bodies. Densities are much higher at the center of the object. The entire simulation contained about 700 halos like the one depicted here.
The process by which galaxies form is among the most important unsolved problems in physics. There is a wealth of observational data -- modern observations span the electromagnetic spectrum from radio frequency to gamma rays. However, a firm theoretical understanding is yet to be acquired. Questions as simple as, "Why are there two families of galaxies, spiral and elliptical?" remain unanswered.
The first step in understanding how galaxies and stars form is to understand the environment in which their formation occurs. Scientists infer that approximately 90 percent of the mass of the Universe is in the form of dark matter, that which can be observed only indirectly, through its gravitational influence on observable matter, such as stars, gas clouds, and nebulae. Simulations are used to study the shapes and dynamics of dark matter halos that are known to surround observed galaxies.
One obstacle to the development of efficient codes to simulate the galaxy formation process is the sheer size of the problem. To obtain useful information, the software must simulate the interactions of millions of bodies. Improved algorithms for N-body calculations have been developed and adapted to this problem. Even with the new algorithms, however, conventional supercomputers lack the computational power to perform simulations in a reasonable period of time at an acceptable level of resolution. A few hundred thousand bodies is the most that conventional systems can model at a time, about two orders of magnitude too few to obtain new scientific results. Hence, scientists must turn to parallel computers -- and very large ones at that -- to perform the computations.
Parallel versions of simulation code are generally more difficult to develop than those for sequential computing systems because of the inherent complexity of parallel systems. In this particular case, code development was further complicated by the requirements for internal data organization in the program that needed to adapt as the computation progressed to reflect changes in the structure of the evolving universe. Concurrent Supercomputing Consortium (CSCC) scientists overcame these programming difficulties and recently developed a parallel simulation code for the 512 node Intel Touchstone Delta System, which achieved speedups in excess of 400 over the single processor speed.
In March 1992, researchers ran a simulation with almost 8.8 million bodies for 780 timesteps on 512 processors of the Touchstone Delta system. The simulation was of a spherical region of space of diameter 10 megaparsecs (or, about 30 million light years); a region large enough to contain several hundred typical galaxies. The simulation ran continuously for 16.7 hours, and carried out 3.24 x 1,014 floating point operations, for a sustained rate of 5.4 gigaflops per second. Algorithm improvements accounted for nearly a 3,000- fold improvement in execution time over traditional calculation methods. Subsequently, the research team ran two 17.15 million body simulations using the Cold Dark Matter model of the Universe. These simulations, the largest N-body simulations ever run, took into account recently acquired microwave background measurements gathered by the COBE satellite.