Seeing Maximum Entropy From The Principle Of Virtual Work

seeing maximum entropy from the principle of virtual work qiuping a. wang institut supérieur des matériaux et mécaniques avancées du mans

Seeing maximum entropy from the principle of virtual work
Qiuping A. Wang
Institut Supérieur des Matériaux et Mécaniques Avancées du Mans,
44 Av. Bartholdi, 72000 Le Mans, France
Abstract
We propose an application of the principle of virtual work of
mechanics to random dynamics of mechanical systems. The total virtual
work of the interacting forces and inertial forces on every particle
of the system is calculated by considering the motion of each
particle. Then according to the principle of Lagrange-d’Alembert for
dynamical equilibrium, the vanishing virtual work gives rise to the
thermodynamic equilibrium state with maximization of thermodynamic
entropy with suitable constraints. This approach establishes a close
relationship between the maximum entropy approach for statistical
mechanics and a fundamental principle of mechanics, and constitutes an
attempt to give the maximum entropy approach, considered by many as
only an inference principle based on the subjectivity of probability
and entropy, the status of fundamental physics law.
PACS numbers : 05.20.-y, 05.70.-a, 02.30.Xx,
1)Introduction
==============
The principle of maximum entropy (maxent) is widely used in the
statistical sciences and engineering as a powerful tool and
fundamental rule. The maxent approach in statistical mechanics can be
traced back to the works of Boltzmann and Gibbs[0] and finally be
given the status of principle thanks to the work of Jaynes[0] who used
it with Boltzmann-Gibbs-Shannon (BGS) entropy (see below) to derive
the canonical probability distribution for statistical mechanics in a
simple manner. However, in spite of its success and popularity, maxent
has always been at the center of scientific and philosophical
discussions and has raised many questions and controversies[0][0][0].
A central question is why a thermodynamic system chooses the
equilibrium microstates such that the BGS entropy gets to maximum. As
a basic assumption of scientific theory, maxent is not directly or
indirectly related to observation and undoubted facts. In the
literature, maxent is postulated as such or justified either a priori
by the second laws with additional hypothesis such as the entropy
functional (Boltzmann or Shannon entropy)[0], or a posteriori by the
correctness of the probability distributions derived from it[0]. In
statistical inference theory, it was often justified by intuitive
arguments based on the subjectivity of probability[0] or by relating
it to other principles such as the consistency requirement and the
principle of insufficient reason of Laplace, which have been the
object of considerable criticisms[0].
Another important question about maxent is whether or not the BGS
entropy is unique as the measure of uncertainty or disorder that can
be maximized in order to determine probability distributions. This was
already an question raised 40 years ago by the scientists who tried to
generalize the Shannon entropy by mathematical considerations [0][0].
Nowadays, the answer to this question becomes much more urgent and
waited due to the controversy and debate surrounding the development
of the statistical theories using maxent with different entropy
functionals [0].
In the present work, we try to contribute to the debate around maxent
by an attempt to derive maxent from a well known fundamental principle
of classical mechanics, the virtual work principle or
Lagrange-d’Alembert principle (LAP) [0][0] without additional
hypotheses to LAP and about entropy property. LAP is widely used in
physical sciences as well as in mechanical engineering. It is a basic
principle capable of yielding all the basic laws of statics and of
dynamics of mechanical systems. It is in addition a simple, clearly
defined, easily understandable and palpable law of physics. It is
hoped that this derivation is scientifically and pedagogically
beneficial for the understanding of maxent and of the relevant
questions and controversies around it. In this work, the term entropy,
denoted by S, is used in the sense of the second law of thermodynamics
for equilibrium system.
The paper is organized as follows. In the first section, we recall the
principle of virtual work before applying it to equilibrium
thermodynamic system to derive maxent for the thermodynamic entropy of
equilibrium state. Then we will briefly mention a previous result in
order to show that other maximizable entropy functionals different
from the BGS forms are possible even for equilibrium system.
2)Principle of virtual work
===========================
The variational calculus in mechanics has a long history which may be
traced back to Galilei and other physicists of his time who studied
the equilibrium problem of statics with LAP (or virtual displacement1).
LAP gets unified and concise mathematical forms thanks to Lagrange[0]
and d’Alembert[0] and is considered as a most basic principle of
mechanics from which all the fundamental laws of statics and dynamics
can be understood thoroughly.
LAP says that the total work done by all forces acting on a system in
static equilibrium is zero on all possible virtual displacements which
are consistent with the constraints of the system. Let us suppose a
simple case of a system of N points of mass in equilibrium under the
action of N forces Fi (i=1,2,…N) with Fi on the point i, and imagine
virtual displacement of each point for the point i. According
to viwop, the virtual work of all the forces Fi on all
cancels itself for static equilibrium, i.e.

(0)
This principle for statics was extended to dynamical equilibrium by
d’Alembert[0] in the LAP by adding the initial force on each
point:

(0)
where mi is the mass of the poin i and its acceleration. From
this principle, we can not only derive Newtonian equation of dynamics,
but also other fundamental principles such as least action principle.
3)Why maximum thermodynamic entropy ?
=====================================
We suppose that the mechanics laws are usable not only for mechanical
system containing small number of particles in regular motion, but
also for large number of particles in random and stochastic motion for
which one has to use statistical approach introducing probability
distribution of mechanical states. Let us first consider a canonical
ensemble with equilibrium systems, each composed of N particles in
random motion with the velocity of the particle i. It will be
shown that the result for canonical ensemble can be easily extended to
microcanonical ensemble and grand-canonical ensemble. Without loss of
generality, let us look at a system without macroscopic motion, i.e.,
.
We imagine that the system in thermodynamic equilibrium leaves the
equilibrium state by a reversible infinitesimal virtual process. Let
be the force on a particle i of the system at that moment.
includes all the interacting forces particles-particles and
particles-walls of the container. During the virtual process, each
particle with acceleration has a virtual displacement
. The total virtual work on this displacement is given by

(0)
Although the sum of the accelerations of all the particles vanishes,
i.e., , the acceleration on each particle can be
nonzero. So in general . As a matter of fact, we have
where eki is the kinetic energy of the particle. On the other hand, we
suppose these are no dissipative forces in the system or on the
particles. It means that the energy of the system will not change if
the system is completely closed and isolated. Let epi be the potential
energy of a particle i subjet to the force , we should have
and

(0)
So finally it follows that
.
(0)
where is a virtual variation of the total energy of
the particle i.
At this stage, no statistics has been done. The particles are treated
as if they had a regular dynamics. As a matter of fact, for a
canonical system, the random dynamics can leads the N particles to
different microstates j with different energy Ej and probability pj (j=1,2
… w). A microstate j is a distribution of the N particles over the one
particle states k with energy k where k varies from, say, 1 to g. For
classical discernable particles, g can be very large and k undergoes
continuous variation from 1 to g. We can imagine N identical particles
distributed over the g states. A microstate j is a combination of g
numbers nk of particles over the g states, i.e., j={n1, n2, … ng, }.
For a given j with probability pj, the virtual energy variation
due to the virtual work can be given by since
a work does not affect the population. Finally, the sum of the
energy variation of each particle can be written as the statistical
average (over different microstates j) of the energy change of
the system due to the virtual displacements, i.e.
.
(0)
This is a well known relationship in statistical mechanics. Here we
have derived it from the microscopic consideration of virtual work on
each particle of the system. A simple calculation shows that
which means
.
(0)
where is the total average energy with a virtual change
and is a virtual heat transfer. This is an expression
of the first law of thermodynamics in the virtual sense for canonical
ensemble. If we consider a reversible virtual process, we have
where is the infinitesimal virtual change of the thermodynamic
entropy and the inverse absolute temperature according to the
second law of thermodynamics.
Now let us use LAP in Eq.(0), it follows that
.
(0)
Then we should add the constraint due to the normalization
into the variational expression with a Lagrange multiplier ,
the viwop in Eq.(0) becomes

(0)
which is the variational calculus of maxent applied to thermodynamic
entropy for canonical ensemble. Note that at this stage the entropy
functional S(pj) is not yet determined. Note that in Eq.(0), the
average energy as a constraint for the maximization of entropy S is a
natural consequence of LAP, in contrast to the introduction of this
constraint in the inference theory or inferential statistical
mechanics[0] by the argument that an averaged value of an observable
quantity represents a factual information to be put into the
maximization of information in order to derive unbiased probability
distribution[0].
For microcanonical ensemble, the system is completely closed and
isolated with constant energy so that . When the virtual
displacements occur, the total virtual work would be transformed into
virtual heat such that Eq.(0) becomes . According to LAP
, Eq.(0) reads

(0)
which necessarily leads to uniform probability distribution over the
different microstates j, i.e., pj =1/w whatever is the form of the
entropy S. Note that here the uniform distribution over the
microstates is not an a priori assumption, but a consequence of LAP.
For grand-canonical ensemble, according to the first law ,
Eq.(0) should be , where  is the chemical potential and N the
average particle number of the system given by with Nj the
particle number of the microstate j. According to LAP, we have

(0)
which is the usual calculus of maxent for grand-canonical ensemble.
The conclusion of this section is that, at thermodynamic equilibrium,
the maxent under suitable constraints is a consequence of the
equilibrium condition LAP of mechanical systems subject to random
motion. From the above discussion, one notices that maxent can be
written in a concise form such as
.
(0)
We stress that in the above derivation, the only essential assumptions
or fundamental physical hypotheses used before the LAP are the first
and second laws of thermodynamics for equilibrium system and
reversible process. Hence the three derived expressions of maxent from
Eg.(0) to (0) for the three statistical ensembles are in principle
valid for all systems for which the first and second laws are valid,
whatsoever is the form of the entropy S. The maximizable entropy form
is an issue we will briefly discuss below.
4)Maximizable entropy functionals
=================================
In general, the entropy functionals are given either as a first
hypothesis or from physical or mathematical considerations about the
entropy property. The standard approach in statistical mechanics is to
use maxent for given entropy in order to derive the probability
distribution. As a matter of fact, in the standard textbook, the only
entropy form widely used in BGS is the Shannon entropy. This is in
addition a claim of uniqueness of the Shannon entropy as maximizable
entropy to be used in maxent[0]. Of course this uniqueness claim
becomes questionable when other entropy forms are used with maxent for
systems in equilibrium or out of equilibrium.
In order to contribute to this discussion and to see the possible
entropy forms, we will inverse the reasoning of maxent which is to
yield probability distributions by maximizing entropy. Here we will
derive the entropy forms from known probability distribution by a
definition of entropy implying maxent. This is possible thanks to the
Eq.(0) which is nothing but
.
(0)
This is just the definition of thermodynamic entropy for reversible
process of canonical system. Obviously, this definition entails maxent
if . But before application of LAP or , let us write
Eq.(0) as follows

(0)
which will enable us to derive the entropy form directly from
probability distribution.
It is well known that is a well known and confirmed
distribution of thermostatistics for canonical ensemble. This
distribution can be directly derived from the Maxwell velocity
distribution of dilute gas if we suppose additivity of energy for the
particles in the gas. As shown in many textbook, if , it is
easy to derive the Gibbs-Shannon entropy using Eq.(0) (with Boltzmann
constant kB=1):

(0)
whose optimization by obviously gives .
Another possible probability distribution for canonical system in
thermodynamic equilibrium is derived for finite system in
equilibrium with finite heat bath where and N2 is the particle
number of the heat bath[0]. It was proved[0] that the entropy defined
by Eq.(0) is the Tsallis one where q=1+a and .
Obviously, when N2 is very large (a thermodynamic limit), a tends to 0
or q tends to 1. In this case, Sq becomes the Shannon one and the
probability distribution becomes exponential. The maxent applied to Sq
naturally gives . This means that the Shannon entropy form as
maximizable one is non unique and subject to the condition such as
additivity and thermodynamic limit in this investigation.
As a mathematical study, we have tried to see other maximizable
entropy forms with other probability distributions[0]. In this case,
the entropy, as a generic measure of probabilistic uncertainty (not
only the thermodynamic entropy), is defined by where x is a
random variable of a probability distribution p (xj) and is a
generic parameter. We have obtained the maximizable entropies
for the distribution (Zipf law or large xj Lévy flight), and
for for the -statistics[0].
5)Concluding remarks
====================
This work shows that the maximum entropy principle has a close
connection with the fundamental principle of classical mechanics, the
principle of virtual work, i.e., for a mechanical system to be in
thermodynamics equilibrium with maximum entropy, the total virtual
work of all the forces on all the elements (particles) of the system
should vanish. Indeed, if one admits that thermodynamic entropy is a
measure of dynamical disorder and randomness, it is natural to say
that this disorder must get to maximum in order that all the random
forces act on each degree of freedom of the motion in such a way that
over any possible (virtual) displacement, the work of all the forces
is zero. In other words, this vanishing work can be obtained if and
only if the randomness of the forces is at maximum over all degree of
freedom allowed by the constraints to get stable equilibrium state.
To our opinion, the present result is helpful not only for the
understanding of maxent derived from a more basic and well understood
mechanical principle, it also shows that entropy in physics is not
necessarily a subjective quantity reaching maximum for correct
inference, and that maximum entropy is a law of physics but not merely
an inference principle.
After finishing this paper, the author became aware of the works of
Plastino and Curado[0] on the equivalence between the particular
thermodynamic relation and maxent in the derivation of
probability distribution. They consider the particular thermodynamic
process affecting only the microstate population in order to find a
different way from maxent to derive probability. The work part is not
considered in their work. Their analysis is pertinent and
consequential. The present work provides a substantial support of
their reasoning from a basic principle of mechanics.
References
[0]
J.L. Lagrange, Mécanique analytique, Blanchard, reprint , Paris (1965)
(Also: Oeuvres, Vol. 11.)
[0]
J. D’Alembert, Traité de dynamique, Editions Jacques Cabay , Sceaux
(1990)
[0]
J. Willard Gibbs, Principes élémentaires de mécanique statistique
(Paris, Hermann, 1998)
[0]
E.T. Jaynes, The evolution of Carnot's principle, The opening talk at
the EMBO
Workshop on Maximum Entropy Methods in x-ray crystallographic and
biological macromolecule structure determination, Orsay, France, April
24-28, 1984; Gibbs vs Boltzmann entropies, American Journal of Physics,
33,391(1965) ; Where do we go from here? in Maximum entropy and
Bayesian methods in inverse problems, pp.21-58, eddited by C. Ray
Smith and W.T. Grandy Jr., D. Reidel, Publishing Company (1985)
[0]
Jos Uffink, Can the maximum entropy principle be explained as a
consistency requirement, Studies in History and Philosophy of Modern
Physics, 26B (1995): 223-261
[0]
L.M. Martyushev and V.D. Seleznev, Maximum entropy production
principle in physics, chemistry and biology, Physics Reports, 426,
1-45 (2006)
[0]
Y.P. Terletskii, Statistical physics, North-Holland Publishing
Company, Amsterdam, 1971
[0]
Q.A. Wang, Some invariant probability and entropy as a maximizable
measure of uncertainty, cond-mat/0612076
[0]
A. Rényi, Calcul de probabilité, Paris, Dunod, 1966, P522
A. Wehrl, Rev. Mod. Phys., 50(1978)221
[0]
M.D. Esteban, Kybernetika, 31(1995)337
[0]
R. Balian and M. Nauenberg, Europhysics news, 37/2 (2006)9; R. Luzzy,
A.R. Vasconcellos and J.G. Ramos, Europhysics news, 37/2 (2006)11; M.
Nauenberg, Phys. Rev. E, 67 (2003) 036114; Phys. Rev. E, 69 (2004)
038102; cond-mat/0305365v1; See also the special issue of Comptes
Rendus Physique, 7, (2006) and the special issue of Europhysics news,
36/6 (2005)
[0]
G. Kaniadakis, Physica A, 296(2001)405; Phys. Rev. E, 66(2002)056125
[0]
A. Plastino and E.F.M. Curado, Phys. Rev. E, 72(2005)047103;
E.F.M. Curado and A. Plastino, arXiv:cond-mat/0601076; also
cond-mat/0509070
1 In mechanics, the virtual displacement of a system is a kind of
imaginary infinitesimal displacement with no time passage and no
influence on the forces. It should be perpendicular to the constraint
forces.
12