## 5.3 Probabilities

A logistic regression model allows us to establish a relationship between a binary outcome variable and a group of predictor variables. It models the logit transformed probability as a linear relationship with the predictor variables. More formally, let $$Y$$ be the binary outcome variable indicating failure/success with $$\{0,1\}$$ and $$p$$ be the probability of $$y$$ to be $$1, p=P(Y=1)$$. Let $$x_{1}, \cdots, x_{k}$$ be a set of predictor variables. Then the logistic regression of $$Y$$ on $$x_{1}, \cdots, x_{k}$$ estimates parameter values for $$\beta_{0}, \beta_{1}, \cdots, \beta_{k}$$ via maximum likelihood method of the following equation $\operatorname{logit}(\pi)=\log \left(\frac{\pi}{1-\pi}\right)=\beta_{0}+\beta_{1} x_{1}+\cdots+\beta_{k} x_{k} .$ Exponentiate and take the multiplicative inverse of both sides, $\frac{1-\pi}{\pi}=\frac{1}{\exp \left(\beta_{0}+\beta_{1} x_{1}+\cdots+\beta_{k} x_{k}\right)} .$ Partial out the fraction on the left-hand side of the equation and add one to both sides, $\frac{1}{\pi}=1+\frac{1}{\exp \left(\beta_{0}+\beta_{1} x_{1}+\cdots+\beta_{k} x_{k}\right)} .$ Change 1 to a common denominator, $\frac{1}{\pi}=\frac{\exp \left(\beta_{0}+\beta_{1} x_{1}+\cdots+\beta_{k} x_{k}\right)+1}{\exp \left(\beta_{0}+\beta_{1} x_{1}+\cdots+\beta_{k} x_{k}\right)}$ Finally, take the multiplicative inverse again to obtain the formula for the probability $$P(Y=1)$$, $\pi=\frac{\exp \left(\beta_{0}+\beta_{1} x_{1}+\cdots+\beta_{k} x_{k}\right)}{1+\exp \left(\beta_{0}+\beta_{1} x_{1}+\cdots+\beta_{k} x_{k}\right)} .$

• Probability: if the probability of success is $$\pi=0.8$$, the probability of failure is $$1-\pi=0.2$$

• Odds: It is the ratio between the probability of success and the probability of failure. $$\operatorname{odds}= \dfrac{\pi}{1-\pi}=\dfrac{0.8}{0.2}=4.$$ That is, the odds of success are 4 to 1.

• Odds Ratio: It is the ratio between Odds. $$\operatorname{OR}= \dfrac{\operatorname{odds}_1}{\operatorname{odds}_2}=\dfrac{\frac{\pi_1}{1-\pi_1}}{\frac{\pi_2}{1-\pi_2}}$$ 