LogDensity's and Likelihoods

Actually, the most general interface in YADAS for converting arguments into logs of densities is called Likelihood. This is a poor name, since these are used in prior terms as well as likelihood terms, and they actually computes the logs of densities rather than the densities themselves. A better name would have been LogDensity, and possibly making things worse, we now use that name to describe a subtype of ``Likelihood''s that operate row-wise on rectangular arrays. A small number of Likelihoods (most of them log-densities) enable the bulk of analyses. For example, consider the Gaussian log-density (see the code in Gaussian.java). A log-density's vital characteristic is its compute method, which takes a two-dimensional array of reals and an integer as input and returns a single real. In Gaussian, the first ``column'' in the input array is the vector of ``data'' (call them $y_i$), the second column is the vector of means (call them $\mu_i$), and the third column is the vector of standard deviations (call them $\sigma_i$). The log-density computes the function $$\{-\log(\sigma_i) - (y_i-\mu_i)^2/2\sigma_i^2\}.$$ , and the ``likelihood'' is responsible for summing over $i$. In Gaussian, the function requires the same number of data points, means, and standard deviations, and most LogDensitys behave similarly, even though in many problems, all the means or standard deviations are identical. ``Constants'' are typically computed in LogDensity functions. This leads to some loss in performance, but that is the price of generality. In the Gaussian example, if one is considering a change to the mean parameter, the term involving $-\log(\sigma_i)$ is the same for the current parameter set and for the proposed parameter set, and will just be canceled out, but we compute it anyway. A more extreme example is if the standard deviation is known, in which case most Bayesians will think of this term as a constant and will recoil at the idea of computing it and subtracting it from itself. In today's analysis, a term may be a function of constants, but in tomorrow's analysis, some of those constants may be unknown parameters. Computing these constant terms allows us to get by with a single Gaussian likelihood function. (Users should feel free to write your own GaussianFixedSD likelihood function that refrains from computing the constant term if they wish.) Here are the log-density functions that users are likely to need. All of these take rectangular arrays. We describe the meanings of the columns of the array. The output of the compute method is the sum, over rows, of the function applied to the elements in each row. Gaussian. Three arguments: data $y$, mean $\mu$, and standard deviation $\sigma$. Computes $-\log \sigma - (y-\mu)^2/2\sigma^2$. Again, we stress that the Gaussian distribution is parameterized in terms of its standard deviation. Gamma. The three arguments are data $y$, shape parameter $\alpha$, and scale parameter $\theta$, so that the mean of $y$ is $\alpha\theta$. Another version, GammaMeanAlpha, parameterizes the density in terms of its mean and shape parameter. Poisson. The two arguments are the data $y$ and the mean $\lambda$. A source of confusion for Poisson and Binomial is that all likelihood functions expect all their arguments to be real, not integer, valued. This holds even for Poisson data, and Binomial data and sample sizes. On the other hand, it is legal to use noninteger values for these should you feel the urge. Binomial. The three arguments are, in order, the number of successes $x$, the sample size $n$, and the probability $p$. See the note for the Poisson distribution for a warning. Beta. The three arguments are the data $y$ and the two parameters $a$ and $b$. The mean of $y$ is $a/(a+b)$. StudentT. This distribution takes four arguments: the data $y$, the mean $\mu$, the scale parameter $\sigma$, and the number of degrees of freedom $\nu$. For large $\nu$, the standard deviation is approximately $\sigma$. Note that this includes the Cauchy distribution, when $\nu=1$. Uniform. The first argument is the data $y$, the second argument is the lower limit $a$, and the second argument is the upper limit $b$. The uniform likelihood can be used to state that a parameter is bounded between two values, for example in interval censoring problems. NegativeBinomial. The negative binomial distribution can be used to model count data that are overdispersed relative to the Poisson distribution; see McCullagh and Nelder (1985, §6.2.3 in the second edition) and Graves and Picard (2002). The first argument is the data $y$, the second argument is the mean $\mu$, and the third argument is an index parameter $\phi$. The variance of $y$ is equal to $\mu(1+\phi)/\phi$. Weibull. The Weibull takes three arguments: data $y$, scale parameter $\sigma$, and index parameter $\phi$. We also supply a separate class Weibull3 that allows an additional argument (the left endpoint of the distribution). Dirichlet. The Dirichlet distribution takes two arguments: the vectors of ``data'' (the probabilities) and the exponents in the Dirichlet prior. See also our SeveralDirichlets class. InverseGamma. The Inverse Gamma family is often used in conjugate models. It is here as well: the arguments are the data, shape and scale parameters. We also have InverseRootGamma, the density of $\sigma$ when $\sigma^{-2}$ has a Gamma distribution. Hypergeometric. For sampling from finite populations, the hypergeometric distribution is available. Its arguments are the number of successes in the sample, the size of the sample, the size of the population, and the number of successes in the population. A type of Likelihood that is not a LogDensity is MultivariateNormal. We have experimented with some multivariate normal applications, but we have not bundled the code with this distribution because it uses third party software for matrix manipulation. Ask the author if you are interested in MultivariateNormal functionality.