CHAPTER 06.04: NONLINEAR REGRESSION: Power Model: Derivation: Part 2 of 2

In the previous segment, I was talking about the power model, and here we have the sum of the square of the residuals.  We want to take the derivative with respect to and b, and put those equal to 0 to be able to figure out what the values of a and b are.  So let's go ahead and go about doing that.  So we've got del Sr by del a is equal to summation, i is equal to 1 to n, so when we're taking the derivative with respect to a, you get yi minus a xi raised to the power b times minus xi raised to the power b, that's what you're going to get when you're going to take the derivative with respect to a there.  Now, let's go ahead and see that what I'm going to get when I take the derivative with respect to b.  Of course, this is equal to 0.  When I take the derivative with respect to b, I get summation of, i is equal to 1 to n, 2 times, again, a xi raised to the power b, times the derivative of this whole quantity with respect to b, and that turns out to be minus a, because you have a constant here, xi raised to the power b times log of xi equal to 0. So keep in mind that the formulas which I have used there are as follows, from your differential calculus class, if I have u as a function of x, d by dx of u squared is 2 u du by dx, that's the formula which I used to get this 2, then the expression u, and then take the derivative of u, and the other one which I am using here, to get this expression here, is simply this formula, d by dx of a raised to the power x is nothing but log of x times x raised to the power . . . a raised to the power x, like this.  So this is the formula which you are getting for d by dx of a raised to the power x. So let's go ahead and see that, we're going to simplify this a little bit further, so if I take this one here, let me see what I get there, I get . . . this should be log of a, it should be log of a, yeah.  So let's go ahead and see if I simplify this, I get 2 here, and now a minus here, so I'm going to get 2 summation . . . -2, because I have a minus here, -2 yi xi raised to the power b, and then plus 2 a summation, xi, i equal to 1 to n, and this will be, this is xi raised to the power b, so I get minus and minus and plus, then I have a 2, then an a, that's what I've written there, then I have xi raised to the power b here and xi raised to the power b, bases are the same, powers are added, I get 2 b is equal to 0, so that's equation number 1.  And from here, what do I get?  Again, I get 2, then a minus, then yi, and then multiply this whole quantity here.  So I get -2, summation, i is equal to 1 to n, yi, so a is there, so I'm going to take a outside, because a is a constant now, so I'm going to get yi xi raised to the power b, log of xi, that's what I'm going to get.  And then plus 2 a squared, because I'm going to get 2, then a, then a, then minus, minus becomes plus, so 2 a squared, all I'm doing is expanding the summation here. 2 a squared is outside the summation, and what is inside the summation? I'll get xi raised to the power b, xi raised to the power b, so that'll be xi raised to the power 2 b, and then log of xi, and that'll be equal to 0, and that'll be the second equation which I will get.  So all I'm doing is simply expanding these two summations here.  Now, you can very well see that I can just get rid of this 2, because I can divide it by 2 throughout.  I can also get rid of this 2 here, because I can divide by 2 throughout, because the right-hand side is 0, and I can also get rid of one of these as here, and the reason why I am able to get rid of these as here is because I'll have to assume that a is not equal to 0, because if a is equal to 0, then I have y is equal to 0 as my regression curve. So I'm going to assume that a is not equal to 0, that's what I am looking for as one of my solutions, so I can rid of these here.  Now, the good thing about this is that what this is going to do, if you look at this particular equation 1 here, that I can get a in terms of b, because a is only here.  There's no a in this part, there's no a in this summation, but as is only here, so I can write a in terms of b, substitute it back in here, and I'll be able to get one single nonlinear equation.  So rather than having to solve two simultaneous nonlinear equations, I'll have to solve only one.  So let me go ahead and write down the expression for a.  So from equation 1, so from equation 1, the expression for a is turning out to be summation, yi xi raised to the power b, divided by summation, xi raised to the power 2 b, i is equal to 1 to n. So that's what we are getting for the value of a.  So that means that a is explicitly known in terms of b. So keep in mind that this doesn't mean that we know the equation for a explicitly.  We only know a explicitly only after we have calculated b.  So what I'm going to do is I'm going to substitute this formula for a, which is in terms of b, in the second equation, so that I can, rather than having to solve two simultaneous nonlinear equations, I'll have to solve only one simultaneous nonlinear equation.  So this one becomes minus yi xi raised to the power b, log of xi, so that is from the previous equation, i is equal to 1 to n, then plus a, and a is this, so I'm going to substitute this value of a wherever there's an a in the second equation, i is equal to 1 to n, yi xi raised to the power b, divided by summation, i is equal to 1 to n, xi raised to the power 2 b, and then what is left?  The other summation is left, i is equal to 1 to n, xi raised to the power 2 b times log of xi equal to 0. Now, this might seem to be too complicated, or too lengthy, and things like that, but you've got to understand that now you have a single nonlinear equation.  The reason why it's nonlinear is because you have xi raised to the power 2 b, and xi raised to the power b, that's what makes it nonlinear, but you can see that this is simply, this whole equation here, everything is known in this particular equation, except for the value of b.  All the xi and yi values are known, because they are simply the values of the data which is given to me.  So f of b is equal to 0.  So this is some function, this left-hand side is f of b, so this is some function of b, so I can use my bisection method, Newton-Raphson method, secant method, and things like that to find the root of this equation, which will give me the value of b, and once I have the value of b, I can go back here . . . I can go back here, so once I have the value of b, I can go back here, and be able to find the value of a, because I know the value of a explicitly in terms of the b constant.  So that's how you derive the constants of the power model, a and b.  And that's the end of this segment.