The observational evidence for cosmic acceleration is strong, but understanding the origin of this acceleration is one of the greatest unsolved problems in contemporary science. Either Einstein's theory of gravity has to be modified, or space is permeated by a mysterious form of energy, called dark energy, that causes the acceleration.
The two basic models for dark energy are that it is either energy associated with empty space (vacuum energy) and is constant throughout space and time — equivalent to the so-called "cosmological constant," or it is an energy field that varies over space and time — called a "scalar field," or "quintessence." Coming up with a theory to explain how this works has proved elusive in both cases.
Vacuum EnergyVacuum energy is the most straightforward explanation for dark energy. Mathematically it is equivalent to the addition of a constant term, called the cosmological constant, in the equation that describes the expansion of the Universe. So far, the various probes of dark energy are consistent with a constant value for the vacuum energy.
However, the physical basis for the vacuum energy is a mystery. The cosmological constant was introduced by Einstein in order to produce a static Universe, then dropped when Edwin Hubble and others showed that the Universe is expanding.
Later, with the development of quantum mechanics, it was realized that the Heisenberg Uncertainty Principle allowed for particles to blink into and out of existence on extremely short times scales (about a trillionth of a nanosecond for electrons), so that empty space, or the vacuum, is not truly empty. The effects of these so-called virtual particles has been measured in the shift of energy levels of hydrogen atoms and in particle masses.
Attempts to estimate the energy density associated with the quantum vacuum lead to the absurd result that the vacuum energy density should be at 60 -120 orders of magnitude larger than is observed: that is a factor ~ 1,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000 or more! No satisfactory explanation for resolving this enormous discrepancy has been put forward. Advances in understanding the nature of elementary particles, perhaps stimulated by discoveries with the Large Hadron Collider at CERN may shed light on the vacuum energy in the near future.
Another idea is that the Universe is but one of many Universes, each with different values of he vacuum energy. Only those Universes with a small value of the vacuum energy would allow for the formation of stars and galaxies. In essence, we could not exist in a Universe that has a large value of the vacuum energy, so the fact that we exist and are worrying about it means that it must be small, so we shouldn't worry about it. It just is.
Scalar Fields or QuintessenceVacuum energy, or the cosmological constant is, as the name implies, constant in space and time. A more general approach assumes that the vacuum energy can vary over space and time due to the existence of a new force field which is called a scalar field, or quintessence. One class of such models assumes that the scalar field energy density tracks the energy density of radiation and matter at very early times and then comes to dominate the energy density of the Universe at later times. This tracking property could provide an explanation for the cosmic coincidence that we live in an epoch when the energy densities of dark matter and dark energy are so nearly equal.
Many versions of scalar fields have been proposed, but as yet none has emerged as a favorite. In particular, as with vacuum energy, no satisfactory physical explanations exist for why the strength of the scalar field is so small, or for the relation, if any, between dark matter and dark energy.
An important goal of future research is to distinguish between vacuum energy and scalar fields as dark energy candidates. The most promising way is to use the methods described above to determine the exact relation between the density and pressure of the dark energy. This relationship is expressed as pressure = w x density, where w is called the "equation of state parameter". For vacuum energy w = -1, whereas for scalar fields, w can be less than or greater than -1, and it can vary with time. To date, all observations are consistent with w = -1, but other values, as well as variation with time, are also possible.