Репост из: AGI
https://openreview.net/forum?id=BJij4yg0Z
However, it is my opinion that Bayesian statistics is not, at present, a theory that can be used to explain why a learning algorithm works. The Bayesian theory is too optimistic: you introduce a prior and model and then trust both implicitly. Relative to any particular prior and model (likelihood), the Bayesian posterior is the optimal summary of the data, but if either part is misspecified, then the Bayesian posterior carries no optimality guarantee. The prior is chosen for convenience here. And the model (a neural network feeding into cross entropy) is clearly misspecified.
However, it is my opinion that Bayesian statistics is not, at present, a theory that can be used to explain why a learning algorithm works. The Bayesian theory is too optimistic: you introduce a prior and model and then trust both implicitly. Relative to any particular prior and model (likelihood), the Bayesian posterior is the optimal summary of the data, but if either part is misspecified, then the Bayesian posterior carries no optimality guarantee. The prior is chosen for convenience here. And the model (a neural network feeding into cross entropy) is clearly misspecified.