Response to review of The Trouble with Physics by Joe Polchinski

Lee Smolin
April 2007


A number of people have asked me to reply to a review of The Trouble with Physics by Joe Polchinski. I had been hesitant to because it is generally considered rude for an author to reply to a review-the author has had his or her say and so has the reviewer; any one who wants to can compare them and decide who to believe.   The only exception is if the review misquotes or grossly misrepresents the book, but this was not one of those cases.  Polchinski’s review was one of those that treated my  and Peter Woit’s books respectfully and replied with a discussion of the evidence, without indulging in ad hominum attacks or misrepresentations. Given this, I felt no need to respond. 

However, a number of people have commented on blogs or said to me directly that the lack of a response to Polchinski’s review was being taken as a concession on my part that I agreed with his criticisms of my book.  Given this, I have concluded that it would not be inappropriate to make a few remarks in reply to what I understand to be factual disagreements.  Beyond these there are of course matters of differing scientific judgments about open problems. However, a major point of my book is that such disagreements are to be respected as necessary for the progress of science, so I certainly have no need or desire to take issue with Polchinski on these points.

Polchinski begins,  perhaps inadvertently, with one of the most perceptive remarks made by a reviewer of TTWP:  “Smolin presents the rise and fall of string theory as a morality play.”   I thank him for this, as he correctly sees that the key idea of the book is that the success of science is due to the formation of communities tied together by adherence to ethical principles.  This ethical communities theory of what science is and how it works is, to my mind, the major theme of the book and its presentation, in Chapter 17, is the key to reading the book. As Polchinski perceives, the story of string theory serves this theme as a case study. 

With regard to these broader issues, Polchinski thinks that Woit and I exaggerated the sociological issues, “such influences are not as strong as these authors posit…”  Very sadly,  I have to say that my impression is that some of the response to the books show the opposite. In the book I raised the idea of “groupthink” and then explained why I did not think it applied completely to the string community.    But I have had to revise my views  as the responses of a few string theorists, such as at George Johnson’s seminars at KITP, and certain online critiques and debates,  offered textbook examples of groupthink.  Rather than regarding criticism as an opportunity for reflection and response,  colleagues in these settings were driven to demonize us, calling us cranks and worse,  questioning our integrity and motives, while proudly insisting on not reading the books. These unfortunate responses gave a very unflattering portrait of our community, its openness to criticism from experts and its welcoming of a diversity of approaches.

Of course I have been very gratified that the vast majority of string theorists I have encountered or communicated with since TTWP was published have been friendly and, if the book came up, respectful and, in some cases even supportive and complementary.   Also not surpringly, a few have expressed their disagreements, in some cases strong, with the content of the book, and when there was interest this led to an exchange of views, always friendly and professional.  

Now onto some of the scientific issues:

Polchinski raises the issue of a positive cosmological constant and says my treatment of it is “based on a myth”   In his words: ‘Smolin claims that string theorists had predicted that the energy of the vacuum — something often called dark energy — could not be positive and that the surprising 1998 discovery of the accelerating expansion of the universe (which implies the existence of positive dark energy) caused a hasty retreat. There was, in fact, no such prediction. Although his book is for the most part thoroughly referenced, Smolin cites no source on this point.” 

Now, here is what I say, and please note the careful wording, “One of the few things we could conclude from the string theories then known was that the cosmological constant could only be zero or negative. I don’t know of any particular string theorist who predicted that the cosmological constant could not be a positive number, but it was widely understood to be a consequence of string theory. The reasons are too technical to do justice to them here.’ p 153.    Please note the qualification “then known”-and note that I explain why there is no citation. 

For experts, let me mention the technical argument I had in mind.  It begins with the fact that supersymmetry requires that the cosmological constant be zero or negative.  Supersymmetry, however, appears necessary in perturbative string theories to cancel the tachyonic instabilities. Now there has been some work that suggests that these may cancel at least to leading order with a weaker condition like Fermi-boson mass matching- but to my knowledge, there is no demonstration of consistent perturbative string theory without some such condition.  This remains true even more than 3 years after the KKLT-which only gave evidence for string theories with positive cosmological constant at the semi-classical level. Indeed, we still have no evidence for any of these theories past the semi-classical level so it is in fact still an open problem whether there are any consistent string theories on positive cosmological constant backgrounds. 

I did quote Witten saying, “I don’t know any clear-cut way to get de Sitter space from string theory or M-theory. This last statement is not very surprising given the classical no go theorem. For, in view of the usual problems in stabilizing moduli, it is hard to get de Sitter space in a reliable fashion at the quantum level given that it does not arise classically.” E. Witten, “Quantum Gravity in de Sitter Space,” hep-th/0106109.

Polchinski asserts that this was addressed to another context;  if he has reason to believe this he ought to explain it, as Witten’s paper-which is highly interesting-seems to read pretty straightforwardly.  

 Polchinski goes on in his notes to assert “It is obvious that there could have been no such prediction. From 1995-98, string theorists were discovering a host of new nonperturbative tools: dualities, branes, black hole entropy counting, matrix theory, and AdS/CFT duality. These were at the time studied almost exclusively in the context of supersymmetry. The problem of moduli stabilization, necessary for any nonsupersymmetric compactification (and positive energy density states are necessarily nonsupersymmetric) was left for the future; there were no general results or predictions.”

If Polchinksi says that he and other string theorists were not concerned because they believed that the problem of moduli stabilization would eventually-when studied-lead to a theory with positive cosmological constant, I cannot disagree.  But I can report that other people I spoke with were worried.  Not the least because the problem of moduli stabilization was unsolved and known to be difficult.   

It is indeed, characteristic of the style of research I criticize in TTWP, that a difficult problem whose solution was absolutely necessary to the success of string theory as a physical theory-moduli stabilization-could be happily left for the future.  I do not recall any string theory talk or paper stating “string theory will be an interesting candidate for a physical theory if the very difficult problem of moduli stabilization can be successfully solved.”  But this is what they ought to have said, (here is my ethics coming in here) had they given a true account of the situation.

At present we have evidence that the problem can be solved, at least at the semi-classical level.  All the discussions that rely on the KKLT and related mechanisms assume that a still more difficult problem can be solved, which is to develop a fully stable, consistent and finite perturbation theory around the semi-classical vacua which have been stabilized.  

If moduli stabilization and the problem of making a positive cosmological constant string theory were not seen as crises among the people Joe talks with, this seems to me just to confirm that they work in a closed intellectual universe in which an optimistic slant is put on everything and hopes are confused with results.  But from the point of view of others there was a big problem.  I recall many discussions about the difficulty string theory faced accounting for the observations of positive dark energy.   At this time I was working on a series of results on LQG with positive cosmological constant.  At some of the talks I gave on these results the issue of whether string theory could do the same came up and no one ever disagreed with my assertion that this was a problem. 

Regarding  the Ads/CFT conjecture, let me first emphasize that no claim  in my book is contradicted by what Polchinksi says here.  I stated that there is lots of evidence for some form of the AdS/CFT conjecture.  And I am not surprised that evidence has continued to accumulate since the book was finished.  What I questioned in the book-and continue to question-is that there is proof for the strongest form of the conjecture, which posits an exact equivalence between string theory on AdS5 x S5 and N=4 SYM.  There are two points here, The first is that evidence is not proof, and for there to be a proof of equivalence between two mathematical objects each must be well defined.  You cannot prove X is isomorphic to Y without having explicit definitions of X, Y and the isomorphism.  There is no rigorous non-perturbative definition of “string theory on an asymptotically AdS5 x S5 background”, and there is no rigorous non-perturbative definition of N=4 SYM in d=4.  Without them we cannot be sure that there are even well defined mathematical structures that correspond to these words. 

The second point is that a weaker form of the conjecture might be true.  Indeed, this must be the case given what we have just said about the lack of a complete non-perturbative formulation of either objects of the conjecture.  What is at issue is whether in addition the strong form postulated by Maldacena is true.

Regarding  background independence,  Polchinski claims that,  “(as Smolin belatedly notes),  Maldacena duality provides a solution to this problem, one that is unexpected  and powerful.”   This exaggerates and distorts the situation.  What is true-and what I acknowledge,  is that if the strong form of the AdS/CFT conjecture is shown to be correct, then a very weak, and limited form of background will have been achieved.  But for reasons just mentioned, which I explain in detail in the book, this is still a big if. 

What has been shown so far relies on the fact that one can use the fact that SUSY N=4 Yang-Mills has the same global super-symmetry as perturbative  physics on a background AdS5 X S5 spacetime, to express some physical quantities in the latter in terms of observables of the former.   This is great mathematical  physics and a great achievement, but the whole point of general relativity and quantum gravity is that the generic solutions are governed by no global symmetries because the geometry of spacetime is completely dynamical.   This has two implications.  First it makes it very non-trivial to show the strong form of the Maldacena conjecture, because  it must extend to solutions of supergravity arbitrarily far from those with global symmetries in the bulk.  However,  if this is possible at all it will be because the full algebra of global super-symmetries remain on the boundary.    The case of asymptotically flat will be much harder because there the asymptotic symmetries of the generic case are very different from the global symmetries of the ground state, and indeed there are no proposals for a gauge-gravity duality in this case.  The case of positive cosmological constant-which appears to be the physical case-is harder still.  And we have not even yet touched the real meaning of background independence, which is that fixed classical fields or global symmetries play absolutely no role in the formulation of the dynamics or observables of the theory. 

The latter is what is meant by background independence  in the rest of the classical and quantum gravity world, and so far string theory and the AdS/CFT conjectures do not come close to addressing it.   It was in fairness to string theory that I was willing to acknowledge  that the strong form of the AdS/CFT conjecture,  if true, would provide a very limited and weak  form of background independence. One would hope that in fairness to the truth string theorists who make this point would also hasten to acknowledge  how far this would be from the real, full meaning of background independence.  Brian Greene does acknowledge this when he proposes that the latter idea be distinguished by calling it “manifest background independence.”

Polchinski also acknowledges the difference, when he says, “In string theory it has always been clear that the physics is background-independent even if the language being used is not, and the search for a more suitable language continues.”  But this is not the most accurate way to put it.  It would be more accurate to say, “Some string theorists believe that the formulations of perturbative string theories and dualities between them that they study concretely are approximations to a deeper, background independent formulation.  This missing background independent formulation is not just a different t language  for the theory, it is hoped to be the statement of the principles and laws that define the theory, from which everything studied so far would be derived as an approximation.  Despite this belief, only a few concrete proposals have been made for the laws and principles of this conjectural  background independent formulation of string theory and none has gained wide support.”

The difference between how Polchinski puts it and I put it are I think indicative of some of the issues TTWP raises about over-statements of results.  

Following this, Polchinski goes further and  says something that seems just false,  “But his principal candidate (loop quantum gravity) is, as yet, much more background-dependent than the current form of string theory.”  In a note he explains this as “I am referring here to the problem of the constraints. Until these are solved, one does not really have background independence: there is an enormous Hilbert space, most of which is unphysical.”    This is confusing and misleading, for two reasons.  First we have the exact and general solution to the spatial diffeomorphism constraints.  We also have-with certain choices of orderings of the Hamiltonian constraint-several infinite classes of solutions to all the constraints.  We also have many versions of dynamical quantum geometries  in which all the constraints are satisfied.  So I don’t know what “until these are solved” could refer to.  Second, the enormous Hilbert space he refers to is a consequence of the background independence,  it is enormous because it permits all possible backgrounds.  By now it is well understood that this expansion of the Hilbert space is a necessary step to constructing a subspace of states invariant under the exact diffeomorphisms. 

The best interpretation I can put on this remark is that Polchinski is ignorant of basic and well established results of LQG.   That such a good scientist could appear to be so ignorant of the basic results of a major research program rival to his own is part of the problem the title of my book refers to. 

On the role of mathematics, Polchinski asserts that “Much of Smolin’s criticism of string theory deals with its lack of mathematical rigor. But physics is not mathematics. Physicists work by calculation, physical reasoning, modeling and cross-checking more than by proof, and what they can understand is generally much greater than what can be rigorously demonstrated.”  Certainly, but the point is that the missing elements and demonstrations are not missing only at a rigorous level. Even at a physicists non-rigorous level we have no proof of perturbative finiteness, S-duality, the existence of perturbative string theories with positive cc backgrounds, or the strong form of the Maldacena conjecture. And, as many string theorists have noted, we have no statement of the principles of the theory and no succinct set of equations, analogous to the Einstein or Schroedinger  equations that define the theory.  This is not a matter of rigor.

It is also false that there are no mathematically rigorous results in physics.  There are lots of rigorous results and theorems in classical general relativity, classical mechanics, quantum mechanics and statistical mechanics. There are even a number in quantum field theory. There are also rigorous results in LQG.   It is true that it has so far proved impossible to rigorously define the standard model, but that may be because that theory does not exist.   From this point of view, the lack of rigorous results in a well studied subject can be taken as evidence that the approximate methods used do not define a real theory-because, I would hope everyone would agree that any real physical theory must sooner or later admit a formulation in terms of rigorous mathematics. 

With regard to heavy ion physics, yes the applications of the AdS/CFT duality to this are interesting and important. But they should not be exaggerated.  Polchinski  does so when he says, “And so the quantum gravity that is manifesting itself in dual form at Brookhaven is likely to be the same one that operates everywhere else in the universe.”    First because there is no quantum gravity here, in this particular application  only the correspondence with classical supergravity arises.  Second, what is the basis for the “likely” here?   I can imagine  an aether theorist making the same argument: aether theory must be right because after a lot of work the principle of relativity of inertial frames was shown to be a consequence of the dynamics of the aether, therefore since nature   “uses a small number of principles in diverse ways”, the aether must be the right explanation for why this principle is observed in nature.  Further, it remains the case that the calculations behind these claims are done with an extended super-symmetric theory, when real QCD has no super-symmetries at all.  It may be that they get some things approximately right for reasons that have nothing to do with string theory, such as the use of a scale invariant theory to provide a rough approximation of a non-scale invariant theory, in an experimental  regime which has approximate scaling. 

On cosmological application of string theory:  “A further development over the past few years, as our understanding has deepened, has been the extensive study of the experimental consequences of specific kinds of string theory. Many of these make distinctive predictions for particle physics and cosmology. Most or all of these may well be falsified by experiment (which is, after all, the fate of most new models). The conclusive test of string theory may still be far off, but in the meantime, science proceeds through many small steps.”  Given the infinite number of string theories, most with a large number of free parameters, it is not surprising that some can  be found that predict phenomena that, with appropriate adjustments of free parameters, put some effects just at the threshold of observability.  This is just a consequence of the lack of falsifiability of the theory.   Of course, if some new phenomena were discovered experimentally that could only be explained on the assumption that string theory is the fundamental theory of nature, that would be all the proof that is needed.  But in most of these cases, such as cosmic strings, there are already on the table alternative explanations for such effects which do not involve fundamental string theory.

Polchinski’s piece is a spirited defense of string theory and, in particular, of the view that the right thing to do in the face of the issues Woit as I raise is to continue to follow the theory for what may  be a very long time into the future, given that he agrees that  “conclusive test of string theory may still be far off.”   Of course, the key point on which good scientists differ in their judgments is precisely how long is too long to invest a large portion of our resources in fundamental theory on such a long and risky bet.

This is fine so far as it goes, but nonetheless in retrospect it is disappointing that Polchinski has chosen to not engage with the broader arguments of the book.   The reason why string theory occupies one of four parts of the book was to give a context in which to raise some broad and fundamental  questions about how science works, and how well it works in the present academic environment-compared  to earlier times when there were many fewer scientists, they were far less organized and professionalized and yet, progress was faster.   The issues that Polchinski chooses not to deal with include:

-Unification through higher dimensions is an old idea, going back to 1914.  It failed over and over again, for reasons that are key to the issues string theory now faces-stabilization of compactifications and the vast freedom of choices for the higher dimensional geometry and topology.  Thus, what is in trouble now is not just string theory, it is an almost century old idea which suffers those two fatal flaws. 

-The present situation is anomalous in the light of history.  Almost always, when the right idea or explanation was put on the table it succeeded quickly to make contact with experiment.   In some cases the new theory inspired new proposals for experimental  tests that within a decade confirmed the new theory’s predictions.  

(Polchinski alludes to atomism as a counter-example, but it is not much of one.  There is a difference between  a general philosophical idea-that the world is made of atoms-and a detailed theory-such as the Rutherford,  Bohr or Schroedinger atoms.  My discussion concerns theories, and indeed these theories of the structure of the atom found experimental support immediately.  If one wants to say that there is a general idea that the true degrees of freedom in nature are extended objects then that is fine, but that general idea does not distinguish string theory from loop quantum gravity and spin foam models.)

My hope in writing this book was not to “kill” string theory, and indeed I emphasize that string theory is among the ideas I believe are worth still exploring, in the context of a lively, diverse and critical environment in which different ideas compete to prove themselves.   What I did hope to do was to kill thc complacent  “only game in town”,  groupthink attitude  towards string theory-which  I argue in the book was held for reasons that are both factually false and inimical to the progress of science.     By doing so I hoped to bring about a lively, open minded debate within the field in which we all asked ourselves how it could happen that our best and brightest would seize on an apparently unique theory that turned out decades later to have still no complete and coherent formulation and to come in an apparent infinite number of versions and so make no falsifiable predictions.   What I hoped for was a detailed debate on the scientific and sociological issues, which took the current situation,  not as a public relations crisis for one research program, but as a genuine intellectual  puzzle and challenge to all of us who hope to contribute to the progress of physics.

I did this because I thought and still think such a debate would be good for progress. Among other reasons, because it would open up the field to new and better ideas by people not committed already to a single research program.  Polchinski’s review is so far the best public response from a string theorist to my book, but it falls far short of taking on the real debate and issues raised in the book.  Perhaps there will be soon miraculous developments that will render such a debate unnecessary, but in their absence I, and the many colleagues who  havc  responded very positively to my book, stand ready to have it.