How to calculate cost function J(θ0, θ1) - Machine Learning

What is cost function: The cost function “J( θ₀,θ₁)” is used to measure how good a fit (measure the accuracy of hypothesis function) a line is to the data. If the line is a good fit, then your predictions will be far better. The idea is to minimize the value of J by calculating it from given values of  θ₀ and θ₁.

Line as good fit: The line we're trying to make as good a fit as possible is defined by equation “h_θ(x)= θ₀ + θ₁x”.

Formula:

So how to calculate cost function. Let see the steps

Step 1.  Pick the value of  θ₀  and  θ₁

Step 2. Find value of h_θ(x)=   by using formula “h_θ(x)= θ₀ + θ₁x”.

Step 3. Find  “h_θ(x)-Y”    for all values of X

Step 4. Find  Square of (h_θ(x) -Y) for all values of Y

Step 5. Substitute all values in equation   (1/2m Σ( h(x)-y)²)  which is equal to “J(θ₀, θ₁)”

Example:

Let’s consider we have m sample values.

X	Y
4	1
3	2
2	2
1	4

Total sample test data is 4 i.e. m=4

Example 1: Calculate value of J(θ₀, θ₁)

Step 1.    θ₀=0 and θ₁=1

		Step 2	Step 3	Step 4
X	Y	hθ(x) i.e. (θ0 + θ1x)	hθ(x)-Y	(hθ(x) - y )2
4	1	0+1*4 => 4	4-1 => 3	9
3	2	0+1*3 => 3	3-2 => 1	1
2	2	0+1*2 => 2	2-2 => 0	0
1	4	0+1*1 => 1	1-4 => -3	9

Step 5:  Substitute value in {1/2m Σ( h(x)-y)²}

m=6 ;   h(x)-y)² is calculated, we have to du summation

J(θ₀, θ₁) = (1/(2*4))* (9+1+0+9) = (1/8 )*19 = 2.375

So  J(θ₀, θ₁) = 2.375 for θ₀=0 and θ₁=1

You can pick other values of θ₀ and θ₁ but that we will discuss in other post