{VERSION 5 0 "IBM INTEL NT" "5.0" }
{USTYLETAB {CSTYLE "Maple Input" -1 0 "Courier" 0 1 255 0 0 1 0 1 0 0
1 0 0 0 0 1 }{CSTYLE "2D Math" -1 2 "Times" 0 1 0 0 0 0 0 0 2 0 0 0 0
0 0 1 }{CSTYLE "2D Comment" 2 18 "" 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 }
{CSTYLE "" -1 256 "" 1 12 0 0 0 0 0 0 0 0 0 0 0 0 0 1 }{CSTYLE "" -1
257 "" 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 1 }{CSTYLE "" -1 258 "" 1 12 0 0
0 0 0 0 0 0 0 0 0 0 0 0 }{PSTYLE "Normal" -1 0 1 {CSTYLE "" -1 -1 "Tim
es" 1 12 0 0 0 1 2 2 2 2 2 2 1 1 1 1 }1 1 0 0 0 0 1 0 1 0 2 2 0 1 }
{PSTYLE "Heading 1" -1 3 1 {CSTYLE "" -1 -1 "Times" 1 18 0 0 0 1 2 1
2 2 2 2 1 1 1 1 }1 1 0 0 6 6 1 0 1 0 2 2 0 1 }{PSTYLE "Heading 2" -1
4 1 {CSTYLE "" -1 -1 "Times" 1 14 0 0 0 1 2 1 2 2 2 2 1 1 1 1 }1 1 0
0 4 4 1 0 1 0 2 2 0 1 }{PSTYLE "Heading 1" -1 256 1 {CSTYLE "" -1 -1 "
Times" 1 18 0 0 0 1 2 1 2 2 2 2 1 1 1 1 }3 1 0 0 6 6 1 0 1 0 2 2 0 1 }
{PSTYLE "Normal" -1 257 1 {CSTYLE "" -1 -1 "Times" 1 12 0 0 0 1 2 2 2
2 2 2 1 1 1 1 }3 1 0 0 0 0 1 0 1 0 2 2 0 1 }{PSTYLE "Heading 2" -1
258 1 {CSTYLE "" -1 -1 "Times" 1 12 0 0 0 1 2 1 2 2 2 2 1 1 1 1 }3 1
0 0 4 4 1 0 1 0 2 2 0 1 }{PSTYLE "Heading 2" -1 259 1 {CSTYLE "" -1
-1 "Times" 1 14 0 0 0 1 2 1 2 2 2 2 1 1 1 1 }3 1 0 0 4 4 1 0 1 0 2 2
0 1 }}
{SECT 0 {PARA 256 "" 0 "" {TEXT -1 46 "Construction of a Non-linear Le
ast Squares Fit" }}{PARA 257 "" 0 "" {TEXT -1 0 "" }}{PARA 258 "" 0 "
" {TEXT -1 9 "Jim Herod" }}{PARA 258 "" 0 "" {TEXT -1 21 "School of Ma
thematics" }}{PARA 258 "" 0 "" {TEXT -1 12 "Georgia Tech" }}{PARA 259
"" 0 "" {TEXT 256 19 "herod@math.gatech.e" }{TEXT 258 2 "du" }}{PARA
0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}{SECT 1
{PARA 3 "" 0 "" {TEXT -1 11 "The Problem" }}{PARA 0 "" 0 "" {TEXT -1
0 "" }}{PARA 0 "" 0 "" {TEXT -1 314 " We model the cooling of a p
itcher of water due to radiation of heat to the surrounding medium. Th
e model is based on the following episode: between seven and eight o'c
lock on the morning of March 9, 1996, the temperature on my porch was \+
about 15 digrees farenheit. A plot of this data can be made as follows
." }}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 101 "minute:=[seq(3*n,n=0..
20)];\ntemp:=[112,104,99,94,88,85,82,78,75,72,69,65,63,60,58,57,55,53,
51,50,48];" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 65 "plot([seq([mi
nute[i],temp[i]],i=1..21)],style=POINT,color=BLACK);" }}}{EXCHG {PARA
0 "> " 0 "" {MPLTEXT 1 0 0 "" }}}}{PARA 0 "" 0 "" {TEXT -1 0 "" }}
{SECT 1 {PARA 3 "" 0 "" {TEXT -1 17 " Typical Analysis" }}{PARA 0 ""
0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 341 "Using the simple mo
del for cooling called Newton's Model for Cooling, it is supposed that
the rate of cooling is proportioned to the difference between the tem
perature of the object and the temperature of the surrounding medium. \+
Thus, the temperature of the pitcher of water is modeled as though its
temperature, T(t), satisfied the equation" }}{PARA 0 "" 0 "" {TEXT
-1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 59 " \+
T '(t) = k ( T(t) - 15)" }}{PARA 0 "" 0 "" {TEXT -1 52 " \+
T(0) = 112." }}{PARA 0 "" 0 "" {TEXT
-1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 41 "Solutions for this equation are
standard." }}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 49 "dsolve(\{diff(
T(t),t)=k*(T(t)-15),T(0)=112\},T(t));" }}}{PARA 0 "" 0 "" {TEXT -1 0 "
" }}{PARA 0 "" 0 "" {TEXT -1 38 "We take the solution to have the form
" }{XPPEDIT 18 0 "T(t) = 15+A*exp(k*t)" "6#/-%\"TG6#%\"tG,&\"#:\"\"\"
*&%\"AGF*-%$expG6#*&%\"kGF*F'F*F*F*" }{TEXT -1 27 ", and wish to find \+
A and k." }}{PARA 0 "" 0 "" {TEXT -1 72 " Typically, an approximat
ion for A and k is found as follows: assume" }}{PARA 0 "" 0 "" {TEXT
-1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 38 " T(t) \+
- 15 = " }{XPPEDIT 18 0 "A*exp(k*t)" "6#*&%\"AG\"\"\"-%$expG6#*&%\"kGF
%%\"tGF%F%" }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT
-1 267 "so that ln( T(t) - 15) = A + k t. Thus, a linear, leas
t squares fit for the logarithm of the data will give an approximation
for A and k. With this in mind, we replace the data with the logarith
m of the data and make a linear, least squares fit for this data." }}
{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 57 "replace:=value->ln(value-15)
;\nlogtemp:=map(replace,temp);" }}}{PARA 0 "" 0 "" {TEXT -1 111 "The f
ollowing is a plot of the logarithm of the temperature. The graph shou
ld be approximately a straight line." }}{EXCHG {PARA 0 "> " 0 ""
{MPLTEXT 1 0 67 "plot([seq([minute[i],logtemp[i]],i=1..21)],style=POIN
T,color=BLUE);" }}}{PARA 0 "" 0 "" {TEXT -1 60 "We now get the linear,
least squares fit for this near-line." }}{EXCHG {PARA 0 "> " 0 ""
{MPLTEXT 1 0 12 "with(stats):" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1
0 50 "fit[leastsquare[[x,y],y=k*x+b]]([minute,logtemp]):" }}}{EXCHG
{PARA 0 "> " 0 "" {MPLTEXT 1 0 14 "ans:=evalf(%);" }}{PARA 0 "> " 0 "
" {MPLTEXT 1 0 0 "" }}}{PARA 0 "" 0 "" {TEXT -1 50 "We find the number
s A and k from the coefficients." }}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT
1 0 45 "k:=coeff(rhs(ans),x);\nA:=exp(op(2,rhs(ans)));" }}}{PARA 0 ""
0 "" {TEXT -1 14 "We graph this " }{TEXT 257 8 "standard" }{TEXT -1
73 " approximation for the data with the data to see how well the fits
looks." }}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 22 "sol:=t->A*exp(k*t
)+15;" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 92 "J:=plot(sol(t),t=0
..60):\nK:=plot([seq([minute[i],temp[i]],i=1..21)],style=POINT,color=B
LUE):" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 22 "plots[display](\{J
,K\});" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 0 "" }}}{PARA 0 "" 0
"" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 116 "This is not a true le
ast squares fit! Rather, A and k were found as least square fits for t
he logarithm of the data." }}}{PARA 0 "" 0 "" {TEXT -1 0 "" }}{SECT 1
{PARA 3 "" 0 "" {TEXT -1 34 "The Error for this Fit of the Data" }}
{PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 133 "A least \+
squares process should choose A and k in order to minimize the followi
ng function, expressed here as a function of AA and KK." }}{EXCHG
{PARA 0 "> " 0 "" {MPLTEXT 1 0 54 "Sum((Temp[ii]-(AA*exp(KK*Minute[ii]
)+15))^2,ii=1..21);" }}}{PARA 0 "" 0 "" {TEXT -1 72 "We define the err
or as a function, ERR, which will depend on aa and kk. " }}{EXCHG
{PARA 0 "> " 0 "" {MPLTEXT 1 0 68 "ERR:=(aa,kk)->sum((temp[ii]-(aa*exp
(kk*minute[ii])+15))^2,ii=1..21);" }}}{PARA 0 "" 0 "" {TEXT -1 118 "We
record the error here for the A and k we have already found. Remember
the value of the error for comparisons later." }}{EXCHG {PARA 0 "> "
0 "" {MPLTEXT 1 0 9 "ERR(A,k);" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT
1 0 0 "" }}}{PARA 0 "" 0 "" {TEXT -1 314 "The next step will be to fin
d a, perhaps, better choice for A and k to minimize this error functi
on. We think of this as a novice in differential calculus would think \+
of the problem; to find the minimum of a fucnction F, take the derivat
ive, set it equal to zero, and look among the solutions x for the equa
tion " }}{PARA 0 "" 0 "" {TEXT -1 56 " \+
F '(x) = 0." }}}{PARA 0 "" 0 "" {TEXT -1 0 "" }}{SECT 1
{PARA 3 "" 0 "" {TEXT -1 37 " Newton's Method For Solving F(x) = 0" }}
{PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 295 " Stu
dents in beginning differential calculus know that if we want to find \+
the minimum value of a function - such as the function ERR above - you
take the derivative, set it equal to zero, and solve. The problem of \+
finding a zero of a function is a familiar one. Usually, we think of s
olving " }}{PARA 0 "" 0 "" {TEXT -1 45 " \+
F(x) = 0. " }}{PARA 0 "" 0 "" {TEXT -1 80 "This could be a job fo
r Newton's Method! A review of this method can be seen in " }}{PARA 0
"" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 46 " www.math.gatech.
edu/~herod/twoyears/newton.ms" }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}
{PARA 0 "" 0 "" {TEXT -1 0 "" }}{SECT 1 {PARA 4 "" 0 "" {TEXT -1 55 "A
brief recollection of the geometry of Newton's method" }}{PARA 0 ""
0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 280 "There, we define a \+
function f for which we will approximate a zero and graph three iterat
es of Newton's method. First, we define the function f for which we wa
nt to find a zero and we define from this f a function that computes t
he point where the tangent line crosses the x-axis." }}{EXCHG {PARA 0
"> " 0 "" {MPLTEXT 1 0 70 "f:=x->exp(x)-2;\nH:=evalf@unapply(a-f(a)/D(
f)(a),a):\nplot(f(x),x=0..3);" }}}{PARA 0 "" 0 "" {TEXT -1 176 "We gen
erate a sequence of iterates which we call x[p], The number x[0] is th
e first guess for a zero. The number x[p] is the p-th approximation, a
s computed by Newton's method." }}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1
0 8 "x[0]:=3;" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 28 "x[1]:=H(x[
0]);x[2]:=H(x[1]);" }}}{PARA 0 "" 0 "" {TEXT -1 175 "One could continu
e this process to compute as many interates as desired , We draw a gra
ph to illustrate what has been done. Different iterates are drawn with
different colors." }}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 34 "p:=plo
t(f(x),x=-0..3,color=BLACK):" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1
0 167 "q1:=x0->plot([[x0,0],[x0,f(x0)],[H(x0),0]],color=RED);\nq2:=x0-
>plot([[x0,0],[x0,f(x0)],[H(x0),0]],color=BLUE);\nq3:=x0->plot([[x0,0]
,[x0,f(x0)],[H(x0),0]],color=GREEN);" }}}{EXCHG {PARA 0 "> " 0 ""
{MPLTEXT 1 0 47 "plots[display](\{p,q1(x[0]),q2(x[1]),q3(x[2])\});" }}
}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 7 "x:='x':" }}}}{PARA 0 "" 0 "
" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 63 "In the context of the p
resent problem, we seek a solution x for" }}{PARA 0 "" 0 "" {TEXT -1
0 "" }}{PARA 0 "" 0 "" {TEXT -1 39 " F '(x)
= 0," }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 113
"where F is multidimensional. This will change some of the calculation
s, but not the structure of Newton's method." }}}{PARA 0 "" 0 ""
{TEXT -1 0 "" }}{SECT 1 {PARA 3 "" 0 "" {TEXT -1 57 "Newton's Method f
or Solving F '(x) = 0 in Multidimensions" }}{PARA 0 "" 0 "" {TEXT -1
0 "" }}{PARA 0 "" 0 "" {TEXT -1 39 "We next consider the problem of so
lving" }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 42 "
F '(x) = 0" }}{PARA 0 "" 0 "" {TEXT
-1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 71 "where x may be a multidimension
vector and F is a real-valued function." }}{PARA 0 "" 0 "" {TEXT -1
131 " As with solving F(x) = 0, we start with an initial guess, sa
y a, and make an improved guess, x, as a solution for the equation" }}
{PARA 0 "" 0 "" {TEXT -1 55 " 0 = F ''(a) ( x \+
- a) + F '(a)." }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 ""
{TEXT -1 100 "We construct the scheme as follows. Choose a, solve F ''
(a) U = F '(a) for U. Define x to be a - U." }}{PARA 0 "" 0 "" {TEXT
-1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 153 "In two dimensions, recall that
for a function g(x,y), the derivative g ' is given by the gradient, a
nd the second derivative g'' is given by the Hessian." }}{PARA 0 "" 0
"" {TEXT -1 0 "" }}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 0 "" }}}
{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 56 "with(linalg):\ngrad(g(x,y),[
x,y]);\nhessian(g(x,y),[x,y]);" }}}{PARA 0 "" 0 "" {TEXT -1 0 "" }}
{PARA 0 "" 0 "" {TEXT -1 54 " We now begin the process. Here again
is the data." }}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 101 "minute:=[s
eq(3*n,n=0..20)];\ntemp:=[112,104,99,94,88,85,82,78,75,72,69,65,63,60,
58,57,55,53,51,50,48];" }}}{PARA 0 "" 0 "" {TEXT -1 0 "" }}{SECT 1
{PARA 4 "" 0 "" {TEXT -1 19 "The first iteration" }}{PARA 0 "" 0 ""
{TEXT -1 121 "The initial guess is the vector z = [A, k] that we compu
ted earlier and the function to be minimized is the ERR function." }}
{PARA 0 "" 0 "" {TEXT -1 0 "" }}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0
9 "z:=[A,k];" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 68 "ERR:=(aa,kk
)->sum((temp[ii]-(aa*exp(kk*minute[ii])+15))^2,ii=1..21);" }}}{EXCHG
{PARA 0 "> " 0 "" {MPLTEXT 1 0 9 "ERR(A,k);" }}}{EXCHG {PARA 0 "> " 0
"" {MPLTEXT 1 0 56 "F2:=subs(\{aa=z[1],kk=z[2]\},hessian(ERR(aa,kk),[a
a,kk])):" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 53 "F1:=subs(\{aa=z
[1],kk=z[2]\},grad(ERR(aa,kk),[aa,kk])):" }}}{PARA 0 "" 0 "" {TEXT -1
40 "The next expression solves the equation " }}{PARA 0 "" 0 "" {TEXT
-1 35 " F2(U) = F1" }}{PARA 0 "" 0 "" {TEXT
-1 39 "and assigns the name S to the solution." }}{EXCHG {PARA 0 "> "
0 "" {MPLTEXT 1 0 19 "U:=linsolve(F2,F1);" }}}{PARA 0 "" 0 "" {TEXT
-1 23 "We define the new guess" }}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1
0 17 "znew:=evalm(z-U);" }}}{PARA 0 "" 0 "" {TEXT -1 93 "and compute h
ow much the new guess differs from the previous in terms of the Euclid
ean norm. " }}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 10 "norm(U,2);" }}
}{PARA 0 "" 0 "" {TEXT -1 46 "Finally, we see how much smaller the err
or is." }}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 48 "evalf(subs(\{aa=zn
ew[1],kk=znew[2]\},ERR(aa,kk)));" }}}{PARA 0 "" 0 "" {TEXT -1 61 "Thes
e steps are repeated for the second and third iterations." }}}{SECT 1
{PARA 4 "" 0 "" {TEXT -1 13 "2nd iteration" }}{EXCHG {PARA 0 "> " 0 "
" {MPLTEXT 1 0 8 "z:=znew;" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0
56 "F2:=subs(\{aa=z[1],kk=z[2]\},hessian(ERR(aa,kk),[aa,kk])):" }}}
{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 53 "F1:=subs(\{aa=z[1],kk=z[2]\}
,grad(ERR(aa,kk),[aa,kk])):" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0
19 "U:=linsolve(F2,F1);" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 17 "
znew:=evalm(z-U);" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 10 "norm(U
,2);" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 48 "evalf(subs(\{aa=zne
w[1],kk=znew[2]\},ERR(aa,kk)));" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT
1 0 0 "" }}}}{SECT 1 {PARA 4 "" 0 "" {TEXT -1 13 "3rd iteration" }}
{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 8 "z:=znew;" }}}{EXCHG {PARA 0 "
> " 0 "" {MPLTEXT 1 0 56 "F2:=subs(\{aa=z[1],kk=z[2]\},hessian(ERR(aa,
kk),[aa,kk])):" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 53 "F1:=subs(
\{aa=z[1],kk=z[2]\},grad(ERR(aa,kk),[aa,kk])):" }}}{EXCHG {PARA 0 "> \+
" 0 "" {MPLTEXT 1 0 19 "U:=linsolve(F2,F1);" }}}{EXCHG {PARA 0 "> " 0
"" {MPLTEXT 1 0 17 "znew:=evalm(z-U);" }}}{EXCHG {PARA 0 "> " 0 ""
{MPLTEXT 1 0 10 "norm(U,2);" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0
52 "F0:=evalf(subs(\{aa=znew[1],kk=znew[2]\},ERR(aa,kk)));" }}}{PARA
0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 98 "Having only the
se three iterations, we compare the graph using the computed A and k w
ith the data." }}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 46 "A:=znew[1];
k:=znew[2];\nsol:=t->A*exp(k*t)+15;" }}}{EXCHG {PARA 0 "> " 0 ""
{MPLTEXT 1 0 92 "J:=plot(sol(t),t=0..60):\nK:=plot([seq([minute[i],tem
p[i]],i=1..21)],style=POINT,color=BLUE):" }}}{EXCHG {PARA 0 "> " 0 ""
{MPLTEXT 1 0 22 "plots[display](\{J,K\});" }}}{PARA 0 "" 0 "" {TEXT
-1 144 "Here is the error as computed in this scheme for comparison wi
th the quicker, but less accurate scheme given at the beginning of thi
s worksheet." }}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 9 "ERR(A,k);" }}
}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 0 "" }}}{EXCHG {PARA 0 "> " 0
"" {MPLTEXT 1 0 0 "" }}}}{PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0
"" {TEXT -1 0 "" }}}{MARK "0 0" 46 }{VIEWOPTS 1 1 0 1 1 1803 1 1 1 1 }
{PAGENUMBERS 0 1 2 33 1 1 }