Logistic Regression Odds Ratio — The Math

In this article, I am going to show you why the odds ratio for an X-variable in a logistic regression is simply the exponential of the regression coefficient. This article just contains “the math” and no interpretation. A discussion of interpretation of the odds ratio in logistic regression can be found here. A more basic discussion, that includes definitions of “odds” and “odds ratio” can be found here.

Before I begin, I want to remind you of a basic property of exponents from algebra. Specifically, when two like terms are multiplied together, the exponents add. For example,

    \[e^a e^b = e^{a+b}\]

In what follows, I will actually be going the other way: e^{a+b} = e^a e^b.

As I expect you know, in logistic regression, the log odds is a linear function of the X‘s:

    \[\log\left( p \over 1-p \right) = \alpha + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_p X_p\]

The odds, then, can be found by exponentiating both sides:

    \[{ p \over 1-p } = e^{\alpha + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_p X_p} .\]

What we want to know is what happens to the odds when we add 1 unit to one of the X‘s. We will assess the effect of this change using the odds ratio.

So let’s suppose that we considering a 1-unit change in X_1. In that case, the new odds are given by

    \[{ p_{\text{new}} \over 1-p_{\text{new}} } = e^{\alpha + \beta_1 (X_1+1) + \beta_2 X_2 + \cdots + \beta_p X_p} .\]

Note that I have used a subscript of “new” on the p to show that this is the “new” p, and thus the new odds that corresponds to adding 1-unit to X_1.

I am now going to rearrange the right hand side of this equation, first by multiplying out \beta_1 (X_1+1) = \beta_1 X_1 + \beta_1 and then by pulling out the \beta_1 term using the exponent formula given above. Thus,

    \[\begin{split} { p_{\text{new}} \over 1-p_{\text{new}} }  &= e^{\alpha + \beta_1 X_1 + \beta_1 + \beta_2 X_2 + \cdots + \beta_p X_p} \\ &= e^{\beta_1} e^{\alpha + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_p X_p} \end{split}\]

We can now use this formula to compute the odds ratio:

    \[{{ p_{\text{new}} \over 1-p_{\text{new}} } \over { p \over 1-p }} = { { e^{\beta_1} e^{\alpha + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_p X_p} } \over { e^{\alpha + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_p X_p} } }\]

Now notice that the e^{\alpha + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_p X_p} in both the top and the bottom divide out, so we are left with

    \[{{ p_{\text{new}} \over 1-p_{\text{new}} } \over { p \over 1-p }} = e^{\beta_1}\]

Thus, the odds ratio corresponding to a 1-unit change in an X-variable is just the exponential of the X‘s regression coefficient.

Questions or Comments?

Any questions or comments? Please feel free to comment below. I am always wanting to improve this material.

I have had to implement a very simple “captcha” field because of spam comments, so be a bit careful about that. Enter your answer as a number (not a word). Also, save your comment before you submit it in case you make a mistake. You can use cntrl-A and then cntrl-C to save a copy on the clipboard. Particularly if your comment is long, I would hate for you to lose it.

This entry was posted in TheMath and tagged , , , . Bookmark the permalink.

3 Responses to Logistic Regression Odds Ratio — The Math

  1. Kom says:

    In this case, x is continuous variable and the effect of change by x+1. So how about x is dummy variable (0,1), how does the x change? and how will it affect to the probability outcome?

  2. miku says:

    Hello. Why is it (p(new)/1-p(new)) over (p/1-p)?

    • StatsProf says:

      Thank you for your question!

      I am not sure if you are asking why it is the ratio or why it is p_new, so I will try to answer both.

      Odds have the form p/(1-p). An odds ratio is the ratio of two odds. If I use p1 and p2 for the probabilities corresponding to the two odds, then an odds ratio is p1/(1-p1) over p2/(1-p2).

      Now what we are thinking about in the above discussion is the effect of changing one of the X’s by one unit. That is, changing X1 to X1+1. We are thinking about this (changing from X1 to X1+1) because this is the standard way to interpret regression coefficients in regular linear least-squares regression. So this is the way we are used to thinking about the effects of the X’s.

      So, in what I wrote above, p is the probability corresponding to the “base case” of X1. The p_new corresponds to the “new” case where we have changed X1 to X1+1. So that is why I am calling it p_new. It is the new p after changing from X1 to X1+1.

      What is shown in this article is that the odds ratio for the two probabilities (p and p_new) that correspond to X1 and X1+1 is exp(beta_1).

      Please note that the above article was intended to explain the formula given in the article Logistic Regression Output Part 3. If you arrived at the present page without first reading that page (for example via a search), reading that page first might make things clearer.

      I hope this helps!

      StatsProf

Leave a Reply

Your email address will not be published. Required fields are marked *


ENTER answer below as a NUMBER.
CAREFUL! Error will delete your comment. Save it first (cntrl-A then cntrl-C saves to clipboard). *
Time limit is exhausted. Please reload the CAPTCHA.