Don’t Stop Believing: Einstein’s Quest for a New Theory of Relativity

From a lecture series presented by The Great Courses

By 1912 or so, Einstein had, more or less, the major conceptual pieces in place for what would ultimately become his general theory of relativity. But he did not have a working theory, yet. He still had a long way to go before he—or anyone else—would be able to devise a workable theory that could connect the force that we call gravity with the geometry of space and time. Delve into Einstein’s missteps as he worked toward a general theory of relativity.

Albert Einstein
Albert Einstein in his office at the University of Berlin, published in the United States in 1920.

To complete his theory, Einstein needed to produce an equation—or perhaps a set of equations—that could be used to relate the distribution of matter and energy with the geometry of space and time. These equations are known as the gravitational field equations, or sometimes just the field equations.

With the correct gravitational field equations, one could calculate things like how objects should move through space under the influence of gravity. Without these equations, you can’t do much at all with Einstein’s theory. The missing equations were essential, and Einstein knew it.

Let’s Get Physical

Marcel Grossmann
Marcel Grossmann (1878-1936) worked with Einstein in the beginning to develop field equations for general theory of relativity.

Einstein spent much of 1912 working with his friend and colleague Marcel Grossmann on precisely this problem. In doing so, they found themselves taking two very different approaches. At times, Einstein adopted a mostly “physical strategy.” In doing so, he relied primarily on his intuition for physics—something Einstein had in spades. He thought it was important that he come up with a set of field equations that mimicked the Newtonian equations of gravity under certain circumstances, and that respected some basic and longstanding physical principles, like the laws of conservation of energy, and the conservation of momentum. Einstein also insisted that the equivalence principle must somehow be manifest in these equations.

This is a transcript from the video series What Einstein Got Wrong. Watch it now, on The Great Courses Plus.

At other times, however, Einstein took a very different and much less physical approach. In these instances, Einstein focused instead on the formal mathematics of the problem. It was no ordinary math problem. To incorporate gravity into a system of non-Euclidean geometry is an incredibly difficult task, and involves what is known as tensor analysis. Graduate students in physics departments sometimes take a whole course on this topic, and that course is often seen as one of the most difficult. The students are just trying to learn the math. Einstein was trying to invent it.

In any case, neither of these two strategies worked out particularly well for Einstein. His physical strategy led to equations that had some features that he liked, but that had serious mathematical problems. In particular, these equations were not covariant, which means that they couldn’t be self-consistently applied in all frames of reference.

Any equations that were not covariant couldn’t be the right equations, and Einstein knew it. From the more mathematical approach, Einstein came up with some very elegant, and entirely covariant, field equations. In fact, these equations were quite similar—but yet different—from those that would ultimately appear in the final version of Einstein’s theory.

Another Theory Bites the Dust

But at this point in time, Einstein became convinced that these equations didn’t align well enough with the predictions of Newtonian gravity. If this had been true, these new field equations would lead to erroneous predictions for some well-measured things—like the orbits of planets, for example. We now know, however, that Einstein was wrong about this. This early set of field equations does, in fact, mimic the Newtonian predictions in the correct limit—but Einstein didn’t know that at the time. Einstein also objected on the grounds that these equations don’t respect the conservation of energy or momentum. For these and other reasons, Einstein jettisoned this set of field equations.

Learn more about Newton’s hardest problem

Considering how close they were to the right answer, this was almost certainly a mistake. Instead, Einstein embraced the equations that came from his physical strategy—which were in fact much more problematic that the ones he had decided to throw out.

So, in 1913, Einstein and Grossmann published a paper titled, “Outline of a Generalized Theory of Relativity and of a Theory of Gravitation.” Conceptually, this paper contained all of the major elements that would later make up the general theory of relativity. But in this version, many of the details were far from correct.

And most important, this version of the theory was not covariant and, thus, was not mathematically self-consistent. By calling this paper an “outline,” Einstein seems to have been acknowledging that this couldn’t be the final answer. But it still represented an important landmark on the way to Einstein’s ultimate theory.

What’s Mercury Got to Do with It?

The precession of the perihelion of Mercury’s orbit

For decades, scientists had noticed that the orbit of the planet Mercury doesn’t precisely agree with the behavior that is predicted by Newtonian gravity. It’s close, but it’s not in perfect agreement. More specifically, the orientation of the ellipse that makes up Mercury’s orbit rotates a small amount each year. This is called the precession of the perihelion of Mercury’s orbit. And by Einstein’s day, the rate of this precession had been measured to be off—or in disagreement with the Newtonian prediction—by about 43 arcseconds per century, or about 0.01 degrees per century.

Some scientists had even imagined that there might be another planet somewhere nearby that was gently tugging on Mercury and slightly altering its orbit—a planet they called Vulcan. But Vulcan, it turns out, doesn’t exist.

We now know that Mercury’s orbit doesn’t agree with the Newtonian prediction because the Newtonian prediction is slightly wrong. To make a more accurate prediction, we need Einstein’s theory of general relativity. But the version of this theory that Einstein published in 1913 doesn’t lead to the right answer to this question either. Instead of the correct rate of 43 arcseconds per century, this version of Einstein’s theory predicted only 18. Einstein worked out this calculation himself, and he knew that it was a problem for this theory.

As time went on, Einstein also became increasingly concerned that his theory wasn’t covariant—and, therefore, wasn’t internally self-consistent. In early 1914, Einstein wrote something in a letter that does a good job of capturing his feelings at the time. He said: “Nature shows us only the tail of the lion. But, I have no doubt that the lion belongs with it, even if he cannot reveal himself all at once.”

Einstein was confident that there was, in fact, a great theory out there to be discovered—a theory that would connect the geometry of space and time with the force of gravity. But he also knew that he hadn’t found that theory yet. He’d seen the lion’s tail, but not yet the lion.

At this point in the story, Einstein had spent three years searching for the correct field equations that would complete his theory of gravity, geometry, and acceleration. And now it was 1914, the year of that the solar eclipse that was predicted to take place. With this event, Einstein thought he could measure the deflection of starlight around the Sun with enough accuracy to test his notion of the equivalence principle, and to test the basis of the general theory of relativity.

Learn more about the introduction to astrophysics

Total Eclipse of the Sun

With this goal in mind, a group of astronomers set out on an expedition to Crimea, where the eclipse of the Sun would be total. But only a few weeks before the eclipse, the First World War broke out, and the astronomers were captured by the Russian army. They must have seemed like likely spies to the Russians. But in any case, they were held as prisoners for a matter of weeks, which was long enough to make it impossible for them to make any measurements of that year’s solar eclipse.

Einstein, of course, was very disappointed by this missed opportunity; but in reality, he had just dodged a bullet. Einstein didn’t know it at the time, the equations he was using were incorrect, leading him to predict the wrong amount of deflection by the Sun. The correct amount of deflection was actually twice as large as the value that Einstein had calculated and published.

If the team of astronomers had been able to carry out their measurement, they very likely would have shown that Einstein was wrong, discrediting him and all the work that he had done up to that point.

But of course, no one could have known this at the time. Einstein did know, however, that the current version of his field equations had problems. They were still not covariant. And on top of this, Einstein knew that they predicted the wrong behavior for Mercury’s orbit. But despite these problems, Einstein gradually became more—instead of less—confident in the validity of his incorrect result. A full decade had passed since he first published his special theory of relativity, and he must have been very frustrated and exhausted after so many years of effort.

Albert Einstein speaking
In early 1915, Einstein began to present publicly the incorrect version of his theory.

It was around this time that Einstein began to present publicly the incorrect version of his theory. In a week-long series of lectures in June 1915, Einstein presented the incorrect version of his theory to a group of physicists and mathematicians at a university in Germany, going into considerable detail. Among those in attendance was David Hilbert—one of the world’s most brilliant mathematicians and perhaps one of the greatest and most influential mathematicians of all time. Hilbert immediately took a great interest in Einstein’s new theory.

From the lecture series What Einstein Got Wrong, taught by Professor Dan Hooper

Images courtesy of:
by Unknown photographer [Public domain] via Wikimedia Commons
By adfadfdf [Public domain] via Wikimedia Public Domain
By Harris & Ewing, photographer. (ca. 1940) Albert Einstein speaking. , ca. 1940. [Photograph] Retrieved from the Library of Congress
By Benutzer:Rainer Zenz via Wikimedia Public Domain