Mathematic for Artificial Intelligence : Calculus

 A simplified guide on how to prep up on Mathematics for Artificial Intelligence, Machine Learning and Data Science: Calculus (Important Pointers only)

Module - II : Calculus 

I. Limits and Continuity.

 -> Limits, functions, derivatives, integrals, and infinite series. 

1. Limits

The limit of $f(x)$ as $x$ approaches $a$ is $L$ if, as $x$ gets arbitrarily close to $a$, $f(x)$ gets arbitrarily close to $L$. This is written as: ${\lim }_{x\to a}f(x)=L$

$$$\mathrm{<b>\epsilon -\delta \; Definition\; :</b>}$

$\mathrm{<b> </b>}$

$\epsilon > 0$

• $\lim_{{x \to a^+}} f(x)$
• $\lim_{{x \to a^-}} f(x)$

$\lim_{{x \to a}} f(x) = \infty$

$f$

$\lim_{{x \to a}} f(x)$

• $f$
• Extreme Value Theorem: If $f$ is continuous on $[a, b]$, then $f$ attains its maximum and minimum values, each at least once, on $[a, b]$.

II. Derivatives and Differentiation Rules.

1. Derivatives

The derivative of a function $f$ at a point $x$, denoted by $f'(x)$ or $\frac{d}{dx}f(x)$, is defined as: $f'(x) = \lim_{{h \to 0}} \frac{f(x + h) - f(x)}{h}$

The derivative at a point is the slope of the tangent line to the function's graph at that point.

2. Differentiation Rules

(i). Power Rule

If $f(x) = x^n$, then $f'(x) = nx^{n-1}$.

(ii). Constant Rule

If $f(x) = c$, where $c$ is a constant, then $f^{\mathrm{\prime }}(x)=0$.

(iii). Constant Multiple Rule

If $f(x) = c \cdot g(x)$, where $c$ is a constant, then $f'(x) = c \cdot g'(x)$.

(iv). Sum Rule

If $f(x) = g(x) + h(x)$, then $f'(x) = g'(x) + h'(x)$.

(v). Difference Rule

If $f(x) = g(x) - h(x)$, then $f'(x) = g'(x) - h'(x)$.

(vi). Product Rule

If $f(x) = g(x) \cdot h(x)$, then $f'(x) = g'(x) \cdot h(x) + g(x) \cdot h'(x)$.

(vii). Quotient Rule

If $f(x) = \frac{g(x)}{h(x)}$, then $f'(x) = \frac{g'(x) \cdot h(x) - g(x) \cdot h'(x)}{h(x)^2}$

(viii). Chain Rule

If $f(x) = g(h(x))$, then $f'(x) = g'(h(x)) \cdot h'(x)$

3. Higher-Order Derivatives

The second derivative of $f$, denoted by $f''(x)$ or $\frac{d^2}{dx^2}f(x)$, is the derivative of the derivative of $f$. Higher-order derivatives are found as per above rules.

Notation

• First derivative: $f'(x)$ or $\frac{d}{dx}f(x)$
• Second derivative: $f''(x)$ or $\frac{d^2}{dx^2}f(x)$
• nth derivative: $f^{(n)}(x)$ or $\frac{d^n}{dx^n}f(x)$

III. Partial Derivatives.

If $f$ is a function of two variables, $x$ and $y$, the partial derivative of $f$ with respect to $x$ is denoted by $\frac{\partial f}{\partial x}$ and is defined as: $\frac{\partial f}{\partial x} = \lim_{{h \to 0}} \frac{f(x+h, y) - f(x, y)}{h}$

Similarly, the partial derivative of $f$ with respect to $y$ is denoted by $\frac{\partial f}{\partial y}$ and is defined as: $\frac{\partial f}{\partial y} = \lim_{{h \to 0}} \frac{f(x, y+h) - f(x, y)}{h}$

 Notations:

• $\frac{\partial f}{\partial x}$ or $f_x$ for the partial derivative with respect to $x$
• $\frac{\mathrm{\partial }f}{\mathrm{\partial }y}$ or $f_y$ for the partial derivative with respect to $y$

Higher-Order Partial Derivatives

• The second partial derivative with respect to $x$: $\frac{\partial^2 f}{\partial x^2}$
• The mixed partial derivative with respect to $x$ and then $y$: $\frac{\partial^2 f}{\partial x \partial y}$

If $f$ is a function of $n$ variables $x_1,x_2,\dots ,x_n$, the notation extends similarly.

 Eg: 

Let $f(x, y) = e^{xy}$.

• First partial derivatives: $\frac{\partial f}{\partial x} = y e^{xy}$ and  $\frac{\partial f}{\partial y} = x e^{xy}$

• Second mixed partial derivatives:

   $\frac{{\mathrm{\partial }}^{2}f}{\mathrm{\partial }x\mathrm{\partial }y}=\frac{\mathrm{\partial }}{\mathrm{\partial }y}\left(y{e}^{xy}\right)=e^{xy}+xy{e}^{xy}$ 

  $\frac{\partial^2 f}{\partial y \partial x} = \frac{\partial}{\partial x} \left( x e^{xy} \right) = e^{xy} + xy e^{xy}$

By Clairaut's theorem, if the mixed partial derivatives are continuous, they are equal: $\frac{\partial^2 f}{\partial x \partial y} = \frac{\partial^2 f}{\partial y \partial x}$

 

IV. Gradient and Directional Derivatives.

1. Gradient

The gradient of a scalar function $f(x_1, x_2, \ldots, x_n)$ is a vector that points in the direction of the greatest rate of increase of the function. It is denoted by $\mathrm{\nabla }f$or $\text{grad }f$ and is defined as: $\nabla f = \left( \frac{\partial f}{\partial x_1}, \frac{\partial f}{\partial x_2}, \ldots, \frac{\partial f}{\partial x_n} \right)$ 

Eg:

Let $f(x,y,z)=x^2+y^2+z^2$.

The gradient of $f$ is: $\nabla f = \left( \frac{\partial f}{\partial x}, \frac{\partial f}{\partial y}, \frac{\partial f}{\partial z} \right) = (2x, 2y, 2z)$

Properties

• The gradient vector points in the direction of the steepest ascent.
• The magnitude of the gradient vector represents the rate of the steepest ascent.

 2. Directional Derivative

 The directional derivative of a function $f$ at a point $\mathbf{a} = (a_1, a_2, \ldots, a_n)$ in the direction of a unit vector $\mathbf{u} = (u_1, u_2, \ldots, u_n)$ is the rate at which $f$ changes at $\mathbf{a}$ in the direction of $\mathbf{u}$. It is denoted by $D_{\mathbf{u}} f(\mathbf{a})$ and is defined as: $D_{\mathbf{u}} f(\mathbf{a}) = \nabla f(\mathbf{a}) \cdot \mathbf{u} = \sum_{i=1}^{n} \frac{\partial f}{\partial x_i} (\mathbf{a}) u_i$

Properties

• The directional derivative is the rate of change of the function in the direction of $\mathbf{u}$.
• It reduces to the partial derivative when $\mathbf{u}$ is aligned with one of the coordinate axes.

3. Relationship Between Gradient and Directional Derivative

The directional derivative in the direction of a unit vector $\mathbf{u}$ can be expressed as the dot product of the gradient and the unit vector: 

$D_{\mathbf{u}} f(\mathbf{a}) = \nabla f(\mathbf{a}) \cdot \mathbf{u}$

This relationship shows that the gradient vector provides all the information needed to compute the directional derivative in any direction.

 

V. Definite and Indefinite Integrals.

1. Definite Integral

The definite integral of $f(x)$ from $a$ to $b$ is written as: $\int_{a}^{b} f(x) \, dx$

It is calculated as the difference in the values of an antiderivative $F(x$ evaluated at the upper and lower limits:

 $\int_{a}^{b} f(x) \, dx = F(b) - F(a)$

It represents the area under the curve of a function between two points on the x-axis. It is a number, not a function, and it accounts for the actual area bounded by the function and the x-axis between two specified limits.

2. Indefinite Integral

The indefinite integral of a function $f(x)$ with respect to $x$ is written as: $\int f(x) \, dx$

The result is a function $F(x)$ plus an arbitrary constant $C$, because the derivative of a constant is zero: 

$\int f(x) \, dx = F(x) + C$

Here, $F'(x) = f(x)$

Basic Rules of Indefinite Integrals

• Power Rule: $\int x^n \, dx = \frac{x^{n+1}}{n+1} + C \quad (n \neq -1)$
• Constant Multiple Rule: $\int k\cdot f(x)dx=k\int f(x)dx$
• Sum Rule: $\int (f(x)+g(x))dx=\int f(x)dx+\int g(x)dx$

3. Fundamental Theorem of Calculus

This theorem connects the concept of differentiation and integration and has two parts:

• First Part: If $F$ is an antiderivative of $f$ on an interval $[a, b]$, then: $\int_{a}^{b} f(x) \, dx = F(b) - F(a)$
• Second Part: If $f$ is continuous on $[a, b]$ and $F$ is defined by: $F(x)={\int }_a^{x}f(t)d\mathrm{t<br>}$
  Then $F$ is continuous on $[a, b]$, differentiable on $(a, b)$, and $F'(x) = f(x)$.

 

VI. Techniques of Integration.

1. Substitution

Substitution, or $u$-substitution, is used to simplify integrals by making a substitution that transforms the integral into a simpler form.

Steps:

• Choose a substitution $u = g(x)$.
• Compute $du=g^{\mathrm{\prime }}(x)dx$.
• Rewrite the integral in terms of $u$ and $du$.
• Integrate with respect to $u$.
• Substitute back to the original variable.

2. Integration by Parts

Integration by parts is based on the product rule for differentiation and is useful for integrating products of functions.

$\int u \, dv = uv - \int v \, du$

Steps:

1. Identify parts of the integrand to set $u$ and $dv$.
2. Differentiate $u$ to find $du$, and integrate $dv$ to find $v$.
3. Apply the above formula.

3. Trigonometric Integrals

Integrating products of trigonometric functions often involves using trigonometric identities to simplify the integrand.

Eg:

Evaluate $\int {\sin }^2(x)dx$.

• Use the identity
  $\sin^2(x) = \frac{1 - \cos(2x)}{2}$.
• The integral becomes
  $\int \frac{1-\cos (2x)}{2}dx=\frac{1}{2}\int (1-\cos (2x))dx$
• Integrate:
  $\frac{1}{2} \left( x - \frac{\sin(2x)}{2} \right) + C = \frac{x}{2} - \frac{\sin(2x)}{4} + C$

4. Trigonometric Substitution

Trigonometric substitution is used to simplify integrals involving square roots of quadratic expressions.

 Rules:

 

 

 

 

5. Partial Fraction Decomposition

Partial fraction decomposition is used to integrate rational functions by expressing them as a sum of simpler fractions.

Steps

• Factor the denominator.
• Decompose the fraction into partial fractions.
• Integrate each partial fraction.

6. Integration by Partial Fractions

Integration by partial fractions is particularly useful for rational functions where the degree of the numerator is less than the degree of the denominator.

Eg:

Evaluate $\int \frac{2x+3}{x^2-x-6} \, dx$.

• Factor the denominator:
  $x^2 - x - 6 = (x-3)(x+2)$.
• Decompose:
  $\frac{2x+3}{(x-3)(x+2)} = \frac{A}{x-3} + \frac{B}{x+2}$
• Find $A$ and $B$:
  $2x + 3 = A(x + 2) + B(x - 3)$.
  Solving gives $A=1$, $B = 1$.
• The integral becomes
  $\int (\frac{1}{x-3}+\frac{1}{x+2})dx=\int \frac{1}{x-3}dx+\int \frac{1}{x+2}dx$
• Integrate:
  $\ln \mathrm{\mid }x-3\mathrm{\mid }+\ln \mathrm{\mid }x+2\mathrm{\mid }+C=\ln \mathrm{\mid }(x-3)(x+2)\mathrm{\mid }+C$

 

VII. Multiple Integrals.

The concept of integration to functions of more than one variable, enabling the calculation of volumes, areas, and other quantities in higher dimensions.

1. Double Integrals

The double integral of $f(x, y)$ over a region $R$ in the $xy$-plane is written as:

 $\iint_R f(x, y) \, dA$

If $R$ is a rectangular region $[a, b] \times [c, d]$, the double integral can be expressed as an iterated integral: 

$\iint_R f(x, y) \, dA = \int_{a}^{b} \int_{c}^{d} f(x, y) \, dy \, dx$

Eg:

Evaluate $\iint_R (x + y) \, dA$, where $R$ is the rectangle $[0, 1] \times [0, 2]$.

$\iint_R (x + y) \, dA = \int_{0}^{1} \int_{0}^{2} (x + y) \, dy \, dx$

First, integrate with respect to $y$:

$\int_{0}^{2} (x + y) \, dy = \left[ xy + \frac{y^2}{2} \right]_{0}^{2} = 2x + 2$

Next, integrate with respect to $x$:

$\int_{0}^{1} (2x + 2) \, dx = \left[ x^2 + 2x \right]_{0}^{1} = 1 + 2 = 3$

So, $$∬R(x+y) dA=3\iint_R (x + y) \, dA = 3

2. Triple Integrals

To integrate functions of three variables over a three-dimensional region.

The triple integral of $f(x, y, z)$ over a region $Q$ in three-dimensional space is written as:
${\iiint }_{Q}f(x,y,z)dV$

If $Q$ is a rectangular box $[a, b] \times [c, d] \times [e, f]$, the triple integral can be expressed as an iterated integral: ${ }_{}$

$\iiint_Q f(x, y, z) \, dV = \int_{a}^{b} \int_{c}^{d} \int_{e}^{f} f(x, y, z) \, dz \, dy \, dx$

 

VIII. Line and Surface Integrals.

1. Line Integrals

Line integrals, also known as path integrals, integrate a function along a curve.

For a scalar field $f(x, y, z)$ and a curve $C$ parameterized by $\mathbf{r}(t) = \langle x(t), y(t), z(t) \rangle$ with $t$ in $[a, b]$, the line integral of $f$ along $C$ is:

$\int_C f \, ds = \int_a^b f(\mathbf{r}(t)) \|\mathbf{r}'(t)\| \, dt$

For a vector field $\mathbf{F} = \langle P, Q, R \rangle$, the line integral along $C$ is:

${<br>\int }_C\mathbf{F}\cdot d\mathbf{r}={\int }_a^b\mathbf{F}(\mathbf{r}(t))\cdot {\mathbf{r}}^{\mathrm{\prime }}(t)dt$

2. Surface Integrals

Surface integrals integrate a function over a surface in three-dimensional space.

For a scalar field $f(x, y, z)$ and a surface $S$ parameterized by $\mathbf{r}(u, v) = \langle x(u, v), y(u, v), z(u, v) \rangle$ with $(u, v)$ in a region $D$, the surface integral is:

$\iint_S f \, dS = \iint_D f(\mathbf{r}(u, v)) \|\mathbf{r}_u \times \mathbf{r}_v\| \, du \, dv$

For a vector field $\mathbf{F} = \langle P, Q, R \rangle$, the surface integral over $S$ is:
${}_{}$

$\iint_S \mathbf{F} \cdot d\mathbf{S} = \iint_D \mathbf{F}(\mathbf{r}(u, v)) \cdot (\mathbf{r}_u \times \mathbf{r}_v) \, du \, dv$