• Thesis to Book with ISBN
• Aim And Scope
• Article in Press
• Editorial Board
• Current Issue
• Archives
• Publication Charges
• Authors Information
• Contact Us
• Paper Format

[Abstract] [PDF] [HTML] [Linked References]

Inference in a Curved Binomial Distribution

Sanchayita Sadhu^*, Babulal Seal^†

^*Department of Statistics, Burdwan University, West Bengal, INDIA.

^†Associate Professor, Department of Statistics, Burdwan University, West Bengal, INDIA.

Corresponding Addresses:

^*[email protected], ^†[email protected]

Research Article

Introduction

                Curved exponential families of distributions play an important role in theory of statistics. In case of inference, for curved exponential family we can not find test statistic as good, powerful and efficient as the exponential family. But there is a ray of hope due to Bradly Efron. He has shown in his paper, “Defining the Curvature of a Statistical Problem (with Application to Second Order Efficiency)” [1] that families with small curvature enjoy the good statistical properties of exponential families.

                In curved exponential family, variance of maximum likelihood estimator (MLE) exceeds the Cramer-Rao lower bound. For curved exponential family MLE is not the sufficient statistic in general. Nonetheless, with small curvature, curved exponential family enjoys good statistical properties. Firstly it is to be found that the values of the involved parameters of the distributions, for which, curvature have small values. Hence, after finding such values, it can be suggested that for the values of the parameter, the test statistic of certain hypothesis is equivalent to that of the corresponding exponential family. In this case it is found that some inference procedures are available fragmentally for continuous distributions. But for discrete cases we don’t find any procedure. Motivated by my (Sanchayita Sadhu) project work (submitted during the period of my post graduate study), this paper wants to find some techniques of inference in binomial distribution – a discrete distribution.

Curved Exponential Family:-

                Let X = (X₁, X₂,…, X_d) have a distribution P_q, q Î Θ Í R^q. Suppose P_q has a density (pmf)  of the form

,

where k > q. Then the family {P_q, q Î Θ} is called curved exponential family.

Some Examples of Curved Exponential Family:

1. A set of independently and identically distributed random variables which follow N (θ, θ²), θ is the unknown parameter involved in the distribution [4].
2. A random variable, say, X that follows gamma (a, 1/a), a is the unknown parameter involved in the distribution.
3. X and Y, say, are two random variables such that

           where p is the unknown parameter involved in both the distributions [4].

4. Let a random variable, say, X = (X₁,X₂) has a Bivariate normal distribution with zero mean, standard deviation equal to 1 and correlation parameter r, r is the involved unknown parameter [4].
5. Suppose Z_i = z_i, i = 1,2,…,n. X_i are independent  Poi(λz_i) variables and Z₁,Z₂,…,Z_n have some joint p.m.f p(z₁,z₂,…,z_n). It is implicitly assumed that each Z_i > 0 with probability 1. Then the joint p.m.f of (X₁,X₂,…,X_n,Z₁,Z₂,…,Z_n) is

             N₀ = set of non-negative integer, N₁ = set of positive integers [4].

6. Equicorrelation Multivariate Normal distribution: suppose (X₁,X₂,…,X_n) are jointly multivariate normal with general mean m_i, variances all 1 and a common pair wise correlation r. This is an example of curved exponential family [4].

Applications of curved exponential family:-

First of all, some real life data, in which the random variables belong to curved exponential family, is discussed. The analysis, done in this paper, hope to be applied to the data to draw certain inference.

                Some practical examples of curved exponential family are discussed here.

ØMixed Ancestral Graph (MAG):-

                The family of distributions represented by a linear MAG M over a set of k variables is a locally parameterized curved exponential family of dimension equal to k(k+1)/2 minus the number of pairs of variables in M that are not adjacent to each other. (Theorem 4; Parameterizing and Scoring Mixed Ancestral Graphs by Thomas Richdson & Peter Spirtes [2]).  

ØLazega’s Lawyer dataset:-

Lazega’s Lawyer dataset [5] is another example where the random variables of the dataset are from curved family.

ØUse in Social Networking:-

David. R. Hunter, in his paper, “Curved Exponential Family Models for Social Networks”, [3] has stated on the usefulness of curved exponential family models on generalization of exponential random graph model (ERGMs).

 Concept of Statistical Curvature:-

                The concept of mathematical curvature extends to the curved lines in Euclidian k-space, E^k, say, £= {h_q, qÎΘ}, where Θ is the interval of real line. For each q, h_q is a vector in E^k whose component wise derivatives with respect to q is denoted by

                &                    

                These derivatives are assumed to exist continuously in neighborhood of a value of q where it is wished to define the curvature. Let us also suppose that a k´k symmetric non-negative definite matrix S_q is defined continuously in q.

  Let M_q be a 2´2 matrix, with entries denoted

                                 ν₂₀(q), ν₁₁(q), ν₀₂(q),

defined by

  M_q      (1)

and let

γ_q     (2)

Then γ_q is “the curvature of £ at q with respect to inner productS_q”.

Statistical Curvature:-

γ_q (given in (2)), the statistical curvature of ₣ at q, is the geometric curvature of £={h_q : q Î Θ } at q with respect to the covariance inner product S_q as defined in (1) and (2).

                 Here ₣ will stand for the family of densities {f_q(x): q Î Θ}, our curved exponential family.

Procedure for Finding Curvature of Various Distributions from Curved Exponential Family:-

                First the pmf/pdf of the random variables is arranged as the known form of curved exponential family.

 i.e. of the form  

.

Then finding the values of η_i (θ)’s and T_i (x)’s the matrix

 is constructed, , ,and  are computed and the matrix    M_q=  is constructed. ()³ is then computed. If there are some negative values in the above result, then the absolute values are taken. Finally i.e. the statistical curvature for various values of θ is calculated.

                Using this method the values of the statistical curvature corresponding to the possible values of the involved parameters are found. From these values it can be easily concluded that for which values of the involved parameters the curvature has small values and it can be said that for these values of the parameters, the given curved exponential family enjoys the good statistical properties of exponential families.

The Area of Work:-

The details work of analysis to find these values of involved parameters for which the curvature has small values is done on the following distributions which are some particular examples of curved exponential family.

The examples are given below:

1. A set of independently and identically distributed random variables which follow N (θ, θ²), θ is the unknown parameter involved in the distribution.
2. A random variable, say, X that follows gamma(a,1/a), a is the unknown parameter involved in the distribution.
3. X and Y, say, are two random variables such that

      where p is the unknown parameter involved in both the distributions.

4. Let a random variable, say, X = (X₁,X₂) has a Bivariate normal distribution with zero mean, standard deviation equal to 1 and correlation parameter r, r is the involved unknown parameter.

Curvature of some curved distributions:-

1. On Normal Distribution:-

Let X₁,X₂,…,Xn are iid N(θ,θ²).

Here

f(x|θ) =

         =               

       =k(θ)exp(η₁ (θ) T₁ + η₂ (θ) T₂ ) h(x)

Here,

η₁(θ)=1/θ² , h₂=1/θ.

T₁=,   T_2.

Now, the work is to find V, V and cov.

It is known that the moment generating function of X will be

M_X(t)= exp(θt+θ²t²)  (1.0)

Now,

    V

Therefore from (1.0) it can be computed that

E(X⁴)= 10θ⁴

\V(X_i²)=E(X_i⁴)-[E(x_i²)]² = 6θ⁴

\

Now,

Cov(X_i², X_i) = E(X_i³)- E(X_i²) E(X_i) = 2θ³

\= 2nθ³.

Hence the dispersion matrix (variance-covariance matrix) is given by

S_θ = .

Also,

h_θ =

\ =         &        =  

\=     

 =    (1.1)

Similarly,

= .     (1.2)

=  (1.3)

=  (1.4)

So, M_θ is given by

   (1.5)

\ =      (1.6)

and                                 (1.7)

\Curvature= γ_θ    (1.8)

So, for all possible values of θ, the value of the curvature is very small.

2. On Gamma Distribution:-

                Let X be a random variable which follows Gamma(α,1/α).

 i.e X ~ G(α,1/α).

  Then the pdf of X is given by

f(x|θ)=      (2.1)

\ f(x|θ) =  

 = k(θ) h(x)

 = k(θ)  (2.2)

Here η₁(θ) = α-1 and η₂(θ) = 1/α

T₁= log x   and     T₂= x

Now to find the required variances and covariance a simulation procedure is to be followed. After computing the required expressions, using an R program the values of curvature are found.

The curvature of the mentioned Gamma distribution is given in the following figure:

           Figure1

So, from Figure1, it may be concluded that for a = 0 to 46 the value of the curvature is small.

3. On Binomial Distribution:-

Let X follows Binomial(n,p) and Y follows Binomial(m,p²)

i.e

Then the joint p.m.f of (X, Y) is

\    and

T₁(x,y) = x    ,   T₂(x,y) = y.

With this information an R program is used to find the values of the curvature.

The curvature of the mentioned distribution is given in the following figure:

             Figure 2

It is known that there is test statistic for single binomial distribution. Here is the joint distribution of two independent random variables which follow Bin(n,p)  and Bin(m,p²) respectively. So, for the values of p for which the value of the curvature is small, any test statistic which is valid in corresponding exponential family can not be referred.

Inference of the Above Binomial Distributions:

ØTest Procedure:

To overcome this problem this paper would like to find out a test procedure to draw an inference about the null hypothesis. Here is the likelihood ratio test.

From Figure 2 it is seen that for p=0.6 to 0.85 the value of the curvature is small.

This paper wants to test

                     H₀ : p = 0.8   vs.  H₁ : p ≠ 0.8

To find LRT firstly the maximum likelihood estimator of p is found.

Solving, taking n=5, m=3; [Using R],

       (3.0) 

Therefore, for LRT

Using R the values of λ will be found.

Let us consider the following table, from which a decision can be made:

             Table 3

With the help of the Table 3 the decision reached is as follows:

               Table 4

ØEstimation Procedure:

1. Bayes Procedure:

Now the Bayesian approach is applied on this distribution. Using Bayesian method the Bayes estimate of the involved parameter, p is found. Then the Bayes risk and risk in case of original distribution are computed.

 For this purpose, let us consider

, 0< p <1

and let

\The posterior distribution of p is

       (3.1)

Now substituting the binomial expansion of (1+p)^m-y in the integration portion of the expression (3.1), it can be written as

From (3.1) and (3.2)

Using this posterior distribution the maximum likelihood estimate of p can be obtained.

The following steps will found the Bayesian estimation from the posterior distribution.

Under SEL, the Bayes estimation with respect to p(p) is

So, under square error loss the Bayes Estimate of p with respect to prior p(p) is

             (3.3)

= C (say).

1. Maximum Likelihood Estimation:

This estimation is done and also a comparison is given in the following table.

The maximum likelihood of p in classical method and Bayesian method and also the Bayes estimates of p with respect to the prior p(p) is given in the following table (taking x=0-5, y=0-3,a=2, b=3):

                                                                                                                                          (3.4)

Hence the Bayes risk is given by

=0.0007318181 with absolute error < 1e-04    (using R software)

Now, the risk in case of the joint distribution of X and Y are to be found. For this purpose

E^f(x,y|p)(p- E^f(x,y|p)(p))² is to be computed.

Hence the risk in case of joint distribution of X and Y can be found in the following way.

The integration can be solved as the earlier computations [e.g. the procedure followed to find (3.4)] using various values of d_i’s.

Finally, using R software

= 0.002524987

4. On Bivariate Normal:-

 Let us suppose X = (X₁,X₂) has a Bivariate normal distribution with zero mean, standard deviation equal to 1 and correlation parameter r.

Then the density of X is

 (4.1)          

Therefore

To compute the statistical curvature of this distribution firstly the moment generating function of bivariate normal distribution and its corresponding calculations are considered.

The moment generating function of X ~ N( 0, 0, 1, 1, r) is given by

      (4.2)

Equating the coefficients from both sides of (4.2), the bivariate moments are

m₁₁ = coefficient of t₁t₂ = r

m₂₀ = coefficients of t₁²/2! = 1

m₀₂ = coefficient of

m₄₀ = coefficient of

m₀₄ = coefficient of

m₂₂ = coefficient of

Also, m_ij = 0 if i+j is odd.

Now V(T₁) = V( x₁² +x₂²)

                   =

Now,

          V(x₂²) = 2

         &

\V(T₁) = 2 + 2 + 4 r² = 4(1+ r²)     (4.3)

   (4.4)

And after calculation,

Cov( T₁,T₂ ) = 4r         (4.5)

\  (4.6)

With this information an R program is used to find the values of the curvature.

The curvature of the mentioned distribution is given in the following figure:

Figure3

 From the above figure it is observed that when the values of r lie between (-1,-0.5) and (0.5, 1) then this curved family performs just like the corresponding exponential family. But there is a good test statistic for bivariate normal distribution, belongs to exponential family, when the value of r₀ (the value for null hypothesis) is equal to 0. So, in this case a good statistic can not be found.

Remark:

The paper hopes, this can be done in other discrete curved family also and plenty of scope are there for doing further research.

References

1. Defining the Curvature of a Statistical Problem (With Applications to Second Order Efficiency). Bradley Efron. The Annals of Statistics, Vol. 3, No. 6. (Nov., 1975), pp, 1189-1242
2. Paremeterizing And Scoring Mixed Ancestral Graphs. Thomas Richardson & Peter Spirtes. Technical Report No. CMU-PHIL-102, August 1999.
3. Curved Exponential Family Models for Social Networks. David. R. Hunter, Pann State University, March 2, 2006.
4. Notes on Exponential Family. A. Das Gupta, Professor of Statistics, Purdue University. ( http://www.stat.purdue.edu/~dasgupta/expfamily.pdf)
5. Lazega’s Lawyer Dataset. Lazega &  Pattison (1999) and Lazega ( 2001).
6. Testing Statistical Hypotheses, Third Edition. E. L. Lehmann,  Joseph P. Romano.
7. Statistical Software Used:- R