Posted by 
As I finished my PhD in 1992, the universe was full of mystery – we didn’t even know exactly what it is made of. One could argue that cosmologists had made little progress in our understanding of these basic facts since the discovery of the cosmic microwave background (CMB), the afterglow of the Big Bang, in the 1960s.
I left the UK after my doctoral studies to begin a research career in the US, where I was lucky to be recruited to work on a new experiment called the Sloan Digital Sky Survey (SDSS). This new survey embraced advances in digital technologies with the ambition of measuring the “redshifts” (how light becomes more red if a source appears to move away from you) of a million galaxies.
These redshifts were then used to measure distances, and allowed cosmologists to map the three-dimensional structure of the universe.
One cosmic puzzle in the 1980s, based on the pioneering CfA Redshift Survey of Margaret Geller and John Huchra, was the significant lumpiness of galaxies, and therefore matter, in our cosmic neighbourhood. Galaxies were clustered together across a wide range of scales, with evidence for coherent “superclusters” of galaxies spanning over 30 million light years in length.
This article is part of our series Cosmology in crisis? which uncovers the greatest problems facing cosmologists today – and discusses the implications of solving them.
It was important to know how such superclusters could have formed from the smooth CMB, as it would tell us the total amount of matter in the universe and, more intriguingly, what that matter was made of. That was assuming the only force in play was gravity.
By the end of the first phase of the SDSS, we had achieved our goal of a million redshifts. This data was used to discover many superclusters across the universe, including the amazing “Sloan Great Wall”, which remains one of the largest known coherent structures in the universe, over a billion light years in length.
I am lucky to have lived through this amazing era of cosmic discovery around the turn of the century. Surveys like SDSS, combined with new observations of the CMB and searches for distant exploding stars known as Type Ia Supernovae (SNeIa), coincided to deliver an emphatic answer to the question: “What is the universe made of?”
The discovery of dark energy
From 1999 to 2004, the cosmological community came together to agree that the universe was 5% normal (baryonic) matter, 25% dark matter (unknown, invisible matter), and 70% “dark energy” (an expansive force) – essentially a cosmological constant, which was first postulated by Einstein. The discovery that the universe was dominated by this constant energy shocked everyone, especially as Einstein had called the cosmological constant his “biggest blunder”.
Today, cosmologists still agree this is the most likely make-up of our universe. But observational cosmologists like me have refined our measurements of these cosmic variables significantly – reducing the errors on these quantities.
The latest numbers from the Dark Energy Survey (DES) indicate that 31.5% of the universe is matter (a combination of dark and normal), with the remainder being dark energy assuming a cosmological constant. The error on this measurement is just 3%.
Knowing these numbers to higher precision will hopefully help cosmologists understand why the universe is like this. Why would we expect to have 70% of the universe today as “dark” (can’t be seen via electromagnetic radiation) and not associated with “matter” like everything else in the universe?
The origin of this dark energy remains the biggest challenge to physics, even after 20 years of intense study.
Intriguing measurements
Like me, a few cosmologists have become distracted by other problems over the last two decades. However, 2024 could be the start of a new era of discovery. This year, cosmologists published new results based on two of our best cosmological probes.
The first probe consists of exploding stars dubbed “SNeIa”. As these stars have a narrow range of masses, their explosions can be well calibrated, giving cosmologists a predictable brightness that can be seen far away. By comparing the known brightness of these SNeIa to their redshifts, we can determine the expansion history of the universe. These objects were, in fact, critical for discovering that the expansion of our universe is accelerating.
The second probe works by looking at Baryon Acoustic Oscillations (BAO) – relics of predictable sound waves in the plasma (charged gas) of the early universe, before the CMB. These are now frozen into the large-scale structure of galaxies around us. Like SNeIa, their predictable size can be compared with their observed size today to measure the expansion history of the universe.
Recently, DES reported its final SNeIa results from over a decade of work, detecting and characterising many thousands of supernova events. While these SNeIa results are consistent with the orthodox view that the universe is dominated by a cosmological constant, they do leave open the tantalising possibility of new physics – namely, that the dark energy could be varying with cosmic time.
That said, scientists are trained to be sceptical, and there are many reasons to distrust a single experiment, single observation, or even a single set of cosmologists!
Cosmologists now go to extraordinary lengths to “blind” their results from themselves during analysis of the data, only revealing the answer at the last moment. This blinding is done to avoid unconscious human biases affecting the work, which could possibly encourage people to get the answer they believe they should see.
This is why repeatability of results is at the heart of all science. In cosmology, we cherish the need for multiple experiments checking and challenging each other.
The second result to turn heads was the first BAO measurements from the Dark Energy Spectroscopic Instrument (DESI), successor to the SDSS. The first DESI map of the cosmos is deeper and denser than the original SDSS. Its first BAO results are intriguing – the data alone is still consistent with a cosmological constant, but with hints of a possible time-varying dark energy when combined with other data sources.
In particular, when DESI analyses the combination of its BAO results with the final DES SNeIa data, the significance of a time-varying dark energy increases to 3.9 sigma (a measure of how unusual a set of data is if a hypothesis is true) – only 0.6% chance of being a statistical fluke.
Most of us would take such odds, but scientists have been hurt before by systematic errors within their data that can mimic such statistical certainty. Particle physicists therefore demand a discovery standard of 5 sigma for any claims of new physics – or less than a one in a million chance of being wrong!
As scientists will say: “Extraordinary claims require extraordinary evidence.”
Mindboggling implications
Are we entering a new era of cosmological discovery? If so, what would it mean?
The answer to my first question is probably yes. The next few years will be fun for cosmologists, with new data and results due from the European Space Agency’s Euclid mission. Launched last year, it is already scanning the sky with unprecedented accuracy.
Likewise, DESI will get more and better data, while the European Southern Observatory starts its own massive redshift survey in 2025. Then you have the Rubin Observatory in Chile coming online soon. Combining these datasets should prove beyond doubt if dark energy varies with cosmic time.
If it does, it implies there is less dark energy now than in the past. This could be caused by many things but, interestingly, it could signify the end of a present, accelerated phase of the expansion of the universe.
It also implies that dark energy is probably not a cosmological constant thought to be due to the background energy associated with empty space. According to quantum mechanics, empty space isn’t really empty, with particles popping in and out of existence creating something we call “vacuum energy”. Ironically, predictions of this vacuum energy do not agree with our cosmological observations by many orders of magnitude.
So, if we did discover that dark energy varies over time, it might explain why observations are at odds with quantum mechanics, which is an extremely well-tested theory. This would suggest the assumption in the standard model of cosmology, that dark energy is constant, needs a rethink. Such a realisation may help solve other mysteries about the universe – or pose new ones.
In short, the new cosmological observations coming this decade will stimulate a new era of physical thinking. Congratulations to my younger cosmologists: it is your era to have fun.
Robert Nichol receives funding from STFC for work on 4MOST.
