CHAPTER 09.01: ADEQUACY OF SOLUTIONS: Ill-conditioned and well-conditioned system of equations

In this segment we'll talk about how to differentiate between ill-conditioned and well-conditioned systems of equations. So let’s suppose somebody gives you a system of equations which is in the matrix form and turns out to be ax equals c. What A is the coefficient matrix and c is the right hand side vector and of course x is our solution vector or what we call as the unknown vector. So whenever we are solving or setting up simultaneous linear equations we write them in the form of a times x equals c where a is the coefficient matrix x is the solution vector and c is the right hand side vector. What you would like to see is that if you wanted to find whether this particular system of equations is well-conditioned or ill-conditioned is to say the following: that hey if I make a small change in the elements of the a matrix then how much change is it making in the solution vector? Or if I make a small change in my c vector, then how much change does it make in my solution vector? Because you would like that A if I make a small change in the coefficient matrix, you would like the solution vector to change in a small amount or if you change the right hand side vector you would like the solution vector to change in a small amount.

Because we are going to as we go through the process of setting up simultaneous linear equations for real life problems those might be set up through a program where we’re going to have round-off errors in the calculation of the a matrix and the calculation of the c matrix I suppose. We don’t want such a round off errors or the lack of our use of precision when we use only single precision rather than double precision or quad precision to affect adversely what the solution vector is. If it does we want to have a mechanism of knowing whether it is doing so.

So, let's look at some examples right here to see that if from a simple example if a system of equations is well-conditioned and ill-conditioned. Let’s suppose somebody says is this particular system of equations 1 2 2 3.99 xy is equal to 4 7.999 well-conditioned or ill-conditioned. We want to be able to make the difference between saying that hey if someone gave me a system of equations like this one is it well-conditioned or ill-conditioned. I can see that if I wanted to solve the set of equations it does have a simple solution for example. What is the solution to this one? It's two comma one. So x y is two comma one. So if we take x equal two and y equal one. In fact, if you plug x equal to two and y equal to one in here you will get 4 and 7.999. So that's itself a solution for that. And we want to see that whether if a small change in the caution matrix is it going to result in a very different value x and y I'm going to get. Or if I make a small change in my right hand side vector is it going to make a make change in my x and y. And what I'm doing now here islet’s suppose I make a small change in my caution matrix. So What I'm doing is as follows. I'm taking 1.001 so I'm making a small change in the caution matrix. Changing by the thousands I get 3.998 so what I'm doing is that I’m making a change of about a magnitude of point zero zero one for all of these elements which are here in the caution matrix. And I'm not changing the right hand side.

And I want to see that if hey does it make a big difference in my solution? And what I find out is that hey that if I solve these two equations, two unknowns by hand or by using Matlab or a new kind of calculator the answer that I should get is as follows 3.994 0.001388. And you can see that just by watching it or just by looking at it you can see that this value of 2 has changed to almost 4. This value of 1 has to changed to almost a value of 001. So very different with very small change being made in the caution of the a matrix. So we can very well see that it is not a well-conditioned system of equations. In order to complete the argument lets go and change the right hand side a little bit. So let’s suppose I have 1 2 2 3.999 here. And what I do is I keep the caution matrix the same but I change the right hand side a little bit. So let me make this to be again one thousandths off of a difference there and 7.9998 here and one thousandths of a difference right here. And let’s go and see what I get for values of x and y. So I get x and y here and if I calculate the value I get minus 3.999 and 4.00. So again you're finding out that from the original set of equations you had right here where the solution was 2 and 1 I made a very small change. I changed 4 to 4.001. I changed 7.999 to 7.998. Small change, small relative change in the right hand side vector. But when I saw these two questions these two unknowns by calculator, Matlab, whatever we find out that the solution which I get is different. This was 2 now it is minus 3.999. This was 1 and now it’s 4. So if somebody had to ask me just by intuition if this is a well-conditioned system of equations or an ill-conditioned system of equations I would tell them this to be an ill-conditioned system of equations.

Now in the later segments we'll talk about how do we do this quantitatively without having to do this. But as an illustration to illustrate the point that what does it mean that a particular system of equations well-conditioned and ill-conditioned, this is a very good example to follow. Let’s say if this system of equations is 1 2 2 3 x y is it well conditioned system of equations or ill conditioned. We just want to illustrate the fact whether it is well-conditioned or ill-conditioned. So if you look at the, I didn't put the right hand side here it should be four and seven. So if we have this system of equations is it well-conditioned or ill-conditioned. The solution to this set of equations if you would either solve it or plug in these values of x equals 2 and y equal to 1 you are going to get four and 7 or if you solve these two questions two unknowns you will get x y to be two and one. In order to find out whether this previous system of equations is well-conditioned or ill-conditioned I'm going to conduct two experiments. I'm going to make a small change in my caution matrix.

So let me make a small change in my caution matrix. I'm going to make one-thousandth of a difference to one. I'm going to make 2 to be 2.001. I'm going to make this to be 2.001 and I'm going to make this to be 3.001. I'm going to say x y is equal to 4 and 7. So I have not made any changes in the right hand side but I made a very small change to the caution matrix right here by changing the cautions by a thousandth. And so what I get is x y when I solve these two equations two unknowns by hand or by a calculator or by Matlab I get 2.003 and 0.997. And you can very well see that 2.003 is very close to 2 and 0.997 is very close to 1. So a small change in the caution matrix does not result in a large change in my solution vector. A small change in the caution matrix resulted in a small change in my solution vector. Let's compare for the sake of completion of the experiment lets go and change the right hand side vector a little bit. So we have 2 1 2 2 3 x y equal to, we'll change the right hand side a little bit.

So let’s suppose we make this to be 4.001 and 7.001. So we are changing it by a thousandth here the right hand side vector. And when I saw this set of equations I found out that hey my x and y turned out to be equal to 1.999 and 1.001. So again this number is very close to two and this number is very close to one. So a small change in the right hand side resulted in a small change in my solution vector it did not result in a large change. So in this case if I'm conducting this experiment in finding out this particular system of equations is well-conditioned. Again, we want to figure out what we mean by well-conditioned and ill conditioned systems of equations quantitatively not by just conducting these simply experiments right here. Just for illustration purposes we'll do that in the later segments. But this is a good example of getting started on at least understanding the concept of ill-conditioned and well-conditioned equations. That is the end of this segment