Entropy 1

Prospects are good for laboratory construction and testing of this solid state Maxwell demon in the near future.

Thermodynamics is a funny subject. The first time you go through it, you don’t understand it at all. The second time you go through it, you think you understand it, except for one or two small points. The third time you go through it, you know you don’t understand it, but by that time you are so used to it, it doesn’t bother you any more.

This quote by Arnold Sommerfeld summarizes my state of mind from the moment I had taught thermodynamics for the third or fourth time. I could do all the exercises and answer almost all student questions (for them it was the first time, so it did not matter what the answer was anyway), but now that I don’t have to do this any more my lack of  understanding does bother me. So I did the stupidest thing you can do: go back to the original literature, Reflections on the Motive Power of Fire, by S. Carnot.

Every student of the natural sciences has to take a course in thermodynamics, and rightly so. In most of these courses, and in most textbooks, an introduction of the gas laws and the first law of thermodynamics quickly leads up to a thing called ‘the Carnot cycle’ which is then used to get the (maximum) thermodynamic efficiency of a heat engine and the concept of entropy, and to establish the thermodynamic (absolute) temperature scale. However, in aforementioned textbooks a modern approach is taken, using concepts and properties unfamiliar to Carnot. For instance, the first law — energy cannot be created or destroyed — was unknown at the time and was not formulated until some thirty years later. He believed heat to be a conserved quantity, and that the fall of heat from a high to low temperature could somehow create useful work in a way similar to the fall of water in a watermill.

Although the book2 is filled with errors and misconceptions, it is nevertheless considered the birth of thermodynamics, eventually leading to the concept of entropy and the second law. And Carnot does introduce a number of concepts that we still use today. He invents reversible processes (although he does not call it that), and realizes that you need a hot and a cold reservoir to get useful work. And he makes an essential foundational assumption: there are no perpetual motion machines (pmm’s), or as Clapeyron3 phrases it ten years later: “his demonstrations are founded on the absurdity of the possibility of creating motive power of heat out of nothing.”

This leads to the first question I asked myself, and for which I so far have no clear answer: If perpetual motion machines exist, do they violate the second law, and  can we still define entropy as a state function? And, maybe also important: if we constructed such a machine, would it destroy the fabric of the universe as we know it?4

The search for perpetual motion machines and free energy has a long history. Carnot himself remarks about it that (p.12)

[….] this would be not only perpetual motion, but an unlimited creation of motive power without consumption either of caloric or of any other agent whatever. Such a creation is entirely contrary to ideas now accepted, to the laws of mechanics and of sound5 physics. It is inadmissible.

Which appears to imply several things I want to explore later. He seems to have some idea of energy conservation, even though that did not come up explicitly, and caloric can be consumed.  In contradiction with his believe that caloric is a conserved quantity. I’ll devote a post to his ideas about caloric and how it was viewed in his time later.

Even though this might seem to leave open the option to get useful work from just the high temperature reservoir6, he uses the non-existence of pmm’s to show both a high and a low temperature reservoir are needed to get useful work7,8. It is this assumption that also leads to the existence of a maximum efficiency for the heat engine, which is what Carnot was after. But it has other consequences as well, that did not become clear until much later: the existence of a new state function, and the possibility of an absolute temperature scale. Both these topics need separate discussion.

The quote at the top of this post comes from a paper in which this assumption is challenged. J.C. Maxwell was the first to suggest the possibility, based on his statistical interpretation of gases:

Or in short if heat is the motion of finite portions of matter and if we can apply tools to such portions of matter so as to deal with them separately then we can take advantage of the different motion of different portions to restore a uniformly hot system to unequal temperatures or to motions of large masses. Only we can’t, not being clever enough.9

The emphasis is by Maxwell himself. His “neat fingered  being” has given rise to a vast literature, to connections with information and computing, and also warrants a separate discussion.10 Many people think we are clever enough now. On the left is a picture of the growing number of papers devoted to the actual creation of a Maxwell demon. Suffice it to say that no one has succeeded in actually building such an engine, and I would not hold my breath for it either.

I am running ahead of myself a little here, but I’d like to formulate a question which seems legitimate: is it indeed impossible to use equilibrium fluctuations to get useful work? The first law does not prohibit it, and maybe even the second law doesn’t, if we believe Landauer11. It would not truly be perpetual, since in order to save the first law the system needs to cool: suppose we can separate hot and cold molecules, thus creating a temperature gradient. We can run a reversible machine on this gradient until it has equilibrated. The temperature will now be lower than that of the original reservoir. And we can continue doing this, all the time lowering the temperature, taking the heat energy from the single reservoir and converting that to useful work, or information, or whatever takes your fancy. I am not sure if this would violate the third law of thermodynamics: zero temperature cannot be reached in a finite number of steps. There seems to be no limit to the temperature difference a demon might create.

But before going into the details of this and related questions, I’d like to explore Carnot’s book a little more, and find out if you can indeed only define entropy as a state function when no pmm’s exist. Before even doing that, it is interesting to investigate what Carnot thinks about heat and caloric.

[1] D.P. Sheehan, A.R. Putnam, and J.H. Wright, A Solid–State Maxwell Demon, Found. Physics, 32, (2002), 1557–1595. So far (15 years later) no such machine has been constructed. The ratchet picture is taken from: Z.C. Tu, Efficiency at maximum power of Feynman’s ratchet as a heat engine, J. Phys. A Math. Theor. 41, (2008), 312003. In chapter 46 of R.P. Feynman’s The Feynman Lectures on Physics, Volume I you can read why it does not work when T1=T2.

[2] I use  S. Carnot, Reflections on the Motive Power of Fire, and on Machines Fitted to Develop that Power, (1824). Translated and edited by R.H. Thurston(1890), Dover Publications, Inc., 1988, but sometimes check the French original if I’m not certain of his intention. Usually that does not help much.

[3] É. Clapeyron, Memoir on the Motive Power of Heat, (1834). Translated and edited by E. Mendoza, Dover Publications, Inc., 1988.

[4] The short science fiction story The Last Question by Isaac Asimov appears to take this possibility serious, although it does not happen until the end of time.

[5] The original French states “aux lois de la mécanique et de la saine physique” (p. 21) so he means sane, not propagation of density waves.

[6] We now distinguish pmm’s of the first and of the second kind. Those of the first kind would violate what we now call the first law (even though it came second): create energy from nothing. A pmm of the second kind violates the second law (the entropy of the universe cannot decrease), but at the moment I am not even sure if a pmm based on a Maxwell demon would violate that law. It would not really be perpetual, however. I’ll get to that later as well.

[7] The concept of work or useful work was well established. Carnot himself states: “We use here the expression motive power to express the useful effect that a motor is capable of producing. This effect can always be likened to the elevation of a weight to a certain height. It has, as we know, as a measure, the product of the weight multiplied by the height to which it is raised.” (footnote, p. 5).

[8] Planck’s formulation of the second law directly derives from this: “It is impossible to construct an engine which will work in a complete cycle, and produce no effect except the raising of a weight, and the cooling of a heat reservoir”, M. Planck, Treatise on Thermodynamics, (1945) Dover Publications, Inc., p. 89.

[9] J.C. Maxwell, Scientific Letters 2, 331–332.

[10] H.S. Leff and A.F. Rex, Maxwell’s demon. Entropy, Information, Computing. Adam Hilger, Bristol, 1990.

[11] R. Landauer, Irreversibility and heat generation in the computing process, IBM J. Res. Dev., 5, (1961), 183.