Читать книгу Informatics and Machine Learning. From Martingales to Metaheuristics онлайн

78 страница из 101

If no constraint on probabilities, other than that they sum to 1, the Lagrangian form for the optimization is as follows:

kkk

If you have as prior information the existence of the mean, μ, of some quantity x, then you have the Lagrangian:

kkkkkk

If you have as prior information the existence of the mean and variance of some quantity (the first and second statistical moments), then you have the Lagrangian:

k

With the introduction of Shannon entropy above, c. 1948, a reformulation of statistical mechanics was indicated (Jaynes [112] ) whereby entropy could be made the starting point for the entire theory by way of maximum entropy with whatever system constraints – immediately giving rise to the classic distributions seen in nature for various systems (itself an alternate derivation starting point for statistical mechanics already noted by Maxwell over 100 years ago). So instead of introducing other statistical mechanics concepts (ergodicity, equal a priori probabilities, etc.) and matching the resulting derivations to phenomenological thermodynamics equations to get entropy, with the Jaynes derivation we start with entropy and maximize it directly to obtain the rest of the theory.