class: center, middle, inverse, title-slide # Regression as a Conditional Average ### Carlisle Rainey ### 2020-10-22 (updated: 2021-10-20) --- class: center, middle # The Regression Equation --- ## Observed Values Let's start by describing a scatterplot using a line. Indeed, we can think of the regression equation as an equation for a scatterplot. First, let's agree that we won't encounter a scatterplot where all the **observed values** `\((x_i, y_i)\)` fall exactly along a line. As such, we need a notation that allows us to distinguish the regression line from the observed values. --- <!-- --> --- ## Fitted Values and Coefficients We commonly refer to the values along the line as the **fitted values** (or "predicted values" or "predictions" and the observations themselves as the "observed values" or "observations." We use `\(y_i\)` to denote the `\(i\)`th observation of `\(y\)` and use `\(\hat{y}_i\)` to denote the fitted value (usually *given* `\(x_i\)`). We write the equation for the line as `\(\hat{y} = \alpha + \beta x\)` and the fitted values as `\(\hat{y}_i = \alpha + \beta x_i\)` (note the presense and absense of the subscript). We refer to the intercept `\(\alpha\)` and the slope `\(\beta\)` as **coefficients**. --- <!-- --> --- ## Residuals We refer to the difference between the observed value `\(y_i\)` and the fitted value `\(\hat{y}_i = \alpha + \beta x_i\)` as the **residual** `\(r_i = y_i - \hat{y}_i\)`. Thus, for *any* `\(\alpha\)` and `\(\beta\)`, we can write `\(y_i = \alpha + \beta x_i + r_i\)` for the observations `\(i = \{1, 2, ..., n\}\)`. Notice that we can break each `\(y_i\)` into two pieces components 1. the linear function of `\(x_i\)`: `\(\alpha + \beta x_i\)` (i.e., the regression line) 1. the residual `\(r_i\)`. In short, we can describe any scatterplot using the model `\(y_i = \alpha + \beta x_i + r_i\)`. --- <!-- --> --- class: inverse, center, middle ## But how do we choose `\(\alpha\)` and `\(\beta\)` that *best* fits the observed values? --- class: center, middle # An Exercise in Line Fitting --- class: inverse, center, middle # The Regression Line as a Conditional Average --- ## The Conditional Average <!-- --> Have a look at the scatterplot above. What's the portfolio share of a party in a coalition government with a seat share of 25%? --- ## The Conditional Average <!-- --> Your eyes probably immediately begin examining a vertical strip above 25%. --- ## The Conditional Average <!-- --> You probably estimate the average is a little more than 25%; call it 27%. You can see that the SD is about 10% because you'd need to go out about 10 percentage points above and below the average to grab about 2/3rds of the data. --- Now you're informed by the data and ready to answer the question. - Q: What's the portfolio share of a party in a coalition government with a seat share of 25%? - A: It's about 27% give or take 10 percentage points or so. --- ## The Conditional Average <!-- --> If we break the data into many small windows, we can visually create an average (and an SD) for each. Freedman, Pisani, and Purves (2008) refer to this as a "graph of averages." Fox (2008) calls this "naive nonparametric regression." It's a conceptual tool to help us understand regression. --- ## The Conditional Average <!-- --> For some datasets, these averages will fall roughly along a line. In that case, we can described the average value of `\(y\)` for each value of `\(x\)`--that is, the *conditional* average of `\(y\)`--with a line. --- class: inverse, center, middle # The Best Line --- ## The Best Point Before we talk about a good *line*, let's talk about a good *point*. Suppose you have a dataset `\(y = \{y_1, y_2, ... , y_n\}\)` and you want to predict these observations with a single point `\(\theta\)`. Use calculus to find the `\(\theta\)` that minimizes the r.m.s. of the residuals `\(r_i = y_i - \theta\)` or that minimizes `\(f(\theta) = \sqrt{\dfrac{\sum_{i = 1}^n(y_i - \theta)^2}{n}}\)`. -- **Hint 1**: Realize that the `\(\theta\)` that minimizes `\(f(\theta) = \sqrt{\dfrac{\displaystyle \sum_{i = 1}^n (y_i - \theta)^2}{n}}\)` also minimizes `\(g(\theta) = \dfrac{\displaystyle \sum_{i = 1}^n(y_i - \theta)^2}{n}\)`. We know this because the square root function is monotonically increase (for positive values, which this must always be) and preserves the order of observations. In other words, the `\(\theta\)` that produces the smallest RMS of the deviations also produces the smallest MS of the deviations. --- ## The Best Point Before we talk about a good *line*, let's talk about a good *point*. Suppose you have a dataset `\(y = \{y_1, y_2, ... , y_n\}\)` and you want to predict these observations with a single point `\(\theta\)`. Use calculus to find the `\(\theta\)` that minimizes the r.m.s. of the residuals `\(r_i = y_i - \theta\)` or that minimizes `\(f(\theta) = \sqrt{\dfrac{\sum_{i = 1}^n(y_i - \theta)^2}{n}}\)`. **Hint 2**: Realize that the `\(\theta\)` that minimizes `\(g(\theta) = \dfrac{\displaystyle \sum_{i = 1}^n(y_i - \theta)^2}{n}\)` also minimizes `\(h(\theta) = \displaystyle \sum_{i = 1}^n(y_i - \theta)^2\)`. Removing a constant just shifts the curve up or down, but it does not change the `\(\theta\)` that minimizes the curve. So work with `\(h(\theta) = \displaystyle \sum_{i = 1}^n(y_i - \theta)^2\)`. --- ## The Best Point Before we talk about a good *line*, let's talk about a good *point*. Suppose you have a dataset `\(y = \{y_1, y_2, ... , y_n\}\)` and you want to predict these observations with a single point `\(\theta\)`. Use calculus to find the `\(\theta\)` that minimizes the r.m.s. of the residuals `\(r_i = y_i - \theta\)` or that minimizes `\(f(\theta) = \sqrt{\dfrac{\sum_{i = 1}^n(y_i - \theta)^2}{n}}\)`. **Hint 3**: To make things easier, expand `\(h(\theta) = \displaystyle \sum_{i = 1}^n(y_i - \theta)^2\)` to `\(h(\theta) = \displaystyle \sum_{i = 1}^n(y_i^2 - 2\theta y_i + \theta^2)\)`. --- ## The Best Point Before we talk about a good *line*, let's talk about a good *point*. Suppose you have a dataset `\(y = \{y_1, y_2, ... , y_n\}\)` and you want to predict these observations with a single point `\(\theta\)`. Use calculus to find the `\(\theta\)` that minimizes the r.m.s. of the residuals `\(r_i = y_i - \theta\)` or that minimizes `\(f(\theta) = \sqrt{\dfrac{\sum_{i = 1}^n(y_i - \theta)^2}{n}}\)`. **Hint 4**: Distribute the summation operator to obtain `\(h(\theta) = \displaystyle \sum_{i = 1}^n y_i^2 - 2 \theta \sum_{i = 1} y_i + n\theta^2\)`. --- ## The Best Point Before we talk about a good *line*, let's talk about a good *point*. Suppose you have a dataset `\(y = \{y_1, y_2, ... , y_n\}\)` and you want to predict these observations with a single point `\(\theta\)`. Use calculus to find the `\(\theta\)` that minimizes the r.m.s. of the residuals `\(r_i = y_i - \theta\)` or that minimizes `\(f(\theta) = \sqrt{\dfrac{\sum_{i = 1}^n(y_i - \theta)^2}{n}}\)`. **Hint 5**: Now take the derivative of `\(h(\theta)\)` w.r.t. `\(\theta\)`, set that derivative equal to zero, and solve for `\(\theta\)`. The result should be familiar. --- ## The Best Line So far, we have to results: 1. The average is the point that minimizes the RMS of the deviations. 1. We want a line that captures the conditional average. Just as the average minimizes the RMS of the deviations, perhaps the regression line should minimize the RMS of the residuals... -- that's exactly what we do. **We want the pair of coefficients `\((\hat{\alpha}, \hat{\beta})\)` that minimizes the RMS of the residuals or ** `\(\DeclareMathOperator*{\argmin}{arg\,min}\)` `\begin{equation} (\hat{\alpha}, \hat{\beta}) = \displaystyle \argmin_{( \alpha, \, \beta ) \, \in \, \mathbb{R}^2} \sqrt{\frac{r_i^2}{n}} \end{equation}` --- ## The Best Line Let's explore three methods to find the coefficients that minimize the RMS of the residuals. 1. grid search 1. numerical optimization, to get additional intuition and preview more advanced methods 1. analytical optimization --- # Grid Search  --- # Numerical Optimization Remember that we simply need to minimize the function `\(f(\alpha, \beta) = \sqrt{\frac{\sum_{i = 1}^n r_i^2}{n}} = \sqrt{\frac{\sum_{i = 1}^n (y_i - \hat{y}_i)^2}{n}} = \sqrt{\frac{ \sum_{i = 1}^n [y_i - (\alpha + \beta x_i)]^2}{n}}\)`. --- # Numerical Optimization <div id="htmlwidget-5acadfce43fd656f1250" style="width:504px;height:360px;" class="plotly html-widget"></div> <script type="application/json" data-for="htmlwidget-5acadfce43fd656f1250">{"x":{"visdat":{"38df3abc7c76":["function () ","plotlyVisDat"]},"cur_data":"38df3abc7c76","attrs":{"38df3abc7c76":{"x":{},"y":{},"z":{},"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"type":"surface","showscale":false,"inherit":true}},"layout":{"margin":{"b":40,"l":60,"t":25,"r":10},"scene":{"xaxis":{"title":"Slope"},"yaxis":{"title":"Intercept"},"zaxis":{"title":"RMS of Residuals"}},"hovermode":"closest","showlegend":false},"source":"A","config":{"showSendToCloud":false},"data":[{"colorbar":{"title":"rms","ticklen":2},"colorscale":[["0","rgba(68,1,84,1)"],["0.0416666666666667","rgba(70,19,97,1)"],["0.0833333333333333","rgba(72,32,111,1)"],["0.125","rgba(71,45,122,1)"],["0.166666666666667","rgba(68,58,128,1)"],["0.208333333333333","rgba(64,70,135,1)"],["0.25","rgba(60,82,138,1)"],["0.291666666666667","rgba(56,93,140,1)"],["0.333333333333333","rgba(49,104,142,1)"],["0.375","rgba(46,114,142,1)"],["0.416666666666667","rgba(42,123,142,1)"],["0.458333333333333","rgba(38,133,141,1)"],["0.5","rgba(37,144,140,1)"],["0.541666666666667","rgba(33,154,138,1)"],["0.583333333333333","rgba(39,164,133,1)"],["0.625","rgba(47,174,127,1)"],["0.666666666666667","rgba(53,183,121,1)"],["0.708333333333333","rgba(79,191,110,1)"],["0.75","rgba(98,199,98,1)"],["0.791666666666667","rgba(119,207,85,1)"],["0.833333333333333","rgba(147,214,70,1)"],["0.875","rgba(172,220,52,1)"],["0.916666666666667","rgba(199,225,42,1)"],["0.958333333333333","rgba(226,228,40,1)"],["1","rgba(253,231,37,1)"]],"showscale":false,"x":[-3,-2.9,-2.8,-2.7,-2.6,-2.5,-2.4,-2.3,-2.2,-2.1,-2,-1.9,-1.8,-1.7,-1.6,-1.5,-1.4,-1.3,-1.2,-1.1,-1,-0.9,-0.8,-0.7,-0.6,-0.5,-0.4,-0.3,-0.2,-0.0999999999999996,0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1,1.1,1.2,1.3,1.4,1.5,1.6,1.7,1.8,1.9,2,2.1,2.2,2.3,2.4,2.5,2.6,2.7,2.8,2.9,3],"y":[-3,-2.9,-2.8,-2.7,-2.6,-2.5,-2.4,-2.3,-2.2,-2.1,-2,-1.9,-1.8,-1.7,-1.6,-1.5,-1.4,-1.3,-1.2,-1.1,-1,-0.9,-0.8,-0.7,-0.6,-0.5,-0.4,-0.3,-0.2,-0.0999999999999996,0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1,1.1,1.2,1.3,1.4,1.5,1.6,1.7,1.8,1.9,2,2.1,2.2,2.3,2.4,2.5,2.6,2.7,2.8,2.9,3],"z":[[5.96790800246836,5.89940361078668,5.83180932472933,5.76515715631734,5.69948015823146,5.63481242909745,5.57118911437658,5.50864640226482,5.44722151396675,5.38695268767596,5.32787915556428,5.27004111305911,5.21347967967368,5.15823685065037,5.10435543868611,5.05187900503184,5.00085177929905,4.95131856736724,4.90332464686896,4.8569156498368,4.81213743222974,4.76903593021731,4.72765700328828,4.68804626446644,4.65024889815748,4.61430946641552,4.58027170470147,4.54817830850176,4.51807071247927,4.48998886412873,4.46397099419755,4.4400533863991,4.41827014917729,4.39865299246805,4.38123101253349,4.36603048800878,4.35307469029182,4.34238371131528,4.33397431156823,4.32785979097872,4.32404988493512,4.32255068731929,4.32336460196009,4.32649032340516,4.33192284736877,4.33965351066103,4.34967005985852,4.36195674745482,4.37649445374991,4.3932608323136,4.41223047650253,4.43337510423095,4.45666375799789,4.4820630170591,4.50953721859859,4.53904868479762,4.57055795281058,4.6040240048285,4.63940449563024,4.67665597527882,4.7157341049033],[5.86593461341745,5.79687048392556,5.72871939668247,5.66151432431681,5.59528935471285,5.5300796993788,5.46592169720716,5.40285281296081,5.3409116297723,5.28013783490124,5.22057219795628,5.1622565407574,5.10523369799181,5.04954746780636,4.99524255148318,4.94236448136584,4.89095953624493,4.84107464347573,4.79275726719205,4.7460552820991,4.70101683247989,4.65769017623304,4.61612351397753,4.57636480351106,4.53846156019122,4.50246064412026,4.46840803534887,4.43634859866635,4.40632583990419,4.37838165603753,4.35255608171158,4.32888703513535,4.30741006655757,4.28815811275657,4.27116126112308,4.25644652697964,4.24403764775426,4.23395489750116,4.22621492503547,4.22083061862367,4.21781099975208,4.21716114799377,4.2188821584238,4.22297113241201,4.22942120197359,4.23822158720167,4.24935768566742,4.26281119207258,4.27856024589789,4.29657960432439,4.31684083732709,4.33931254155924,4.36396056946575,4.39074826998433,4.41963673720917,4.45058506349529,4.48355059366189,4.51848917719678,4.55535541565781,4.59410290279631,4.63468445527848],[5.76440675049785,5.69476822927722,5.6260452526784,5.55827178043546,5.49148296775882,5.42571517730015,5.3610059863038,5.29739418820095,5.23491978784634,5.173623989544,5.1135491769597,5.05473888397662,4.99723775551906,4.94109149735007,4.88634681384623,4.83305133276952,4.7812535160965,4.73100255603241,4.68234825543607,4.63534089201487,4.59003106581918,4.54646952977639,4.50470700325617,4.46479396895144,4.42678045369151,4.39071579417129,4.35664838897807,4.32462543871549,4.29469267645372,4.26689409116106,4.24127164718144,4.21786500319595,4.19671123442931,4.17784456211396,4.16129609438997,4.1470935828825,4.13526119914875,4.12581933501519,4.11878443053107,4.11416883284889,4.11198068881472,4.11222387342727,4.11489795562237,4.11999820208519,4.12751561901166,4.13743703096315,4.14974519521201,4.16441894928754,4.18143338882556,4.20076007231848,4.22236724897127,4.24622010559806,4.27228102834834,4.30050987502473,4.33086425383955,4.36329980464157,4.39777047891147,4.43422881515886,4.47262620673639,4.51291315949966,4.55503953717036],[5.66334837484105,5.5931204435714,5.52381007799669,5.45545225066257,5.38808321583295,5.32174052563841,5.25646304121923,5.19229093803354,5.12926570443243,5.06743013253741,5.00682830039434,4.94750554432343,4.88950842034172,4.83288465350396,4.7776830739968,4.72395353883133,4.67174683801665,4.6211145841669,4.57210908459996,4.52478319513355,4.47919015497718,4.43538340235948,4.39341637082207,4.35334226645332,4.31521382672636,4.27908306204166,4.2450009815468,4.21301730530683,4.18318016541199,4.15553579912119,4.13012823762886,4.10699899448986,4.08618675811841,4.06772709307098,4.05165215500852,4.03799042429111,4.02676646307532,4.01800070055116,4.01170925057027,4.00790376538566,4.00659132855774,4.00777438930331,4.01145073969967,4.01761353523687,4.0262513582736,4.03734832303103,4.05088421989113,4.06683469598218,4.08517146836286,4.10586256557711,4.12887259295911,4.15416301682666,4.18169246261031,4.21141702201686,4.24329056450472,4.27726504863754,4.31329082925993,4.3513169568832,4.39129146615671,4.4331616508111,4.47687432297398],[5.56278507384366,5.49195236147772,5.42203870363387,5.35308011024183,5.2851139656236,5.21817904949539,5.15231555279328,5.08756508739874,5.0239706887562,4.96157681029372,4.90042930847976,4.84057541727948,4.78206371071458,4.72494405218689,4.66926752920191,4.61508637213024,4.56245385567736,4.51142418180203,4.46205234293717,4.41439396452945,4.36850512613148,4.32444216055667,4.28226143094545,4.24201908599265,4.20377079404775,4.16757145731802,4.13347490796966,4.10153358852244,4.07179821955125,4.04431745832393,4.01913755259343,3.99630199429793,3.97585117837584,3.95782207224574,3.94224790170767,3.92915785906872,3.91857683916552,3.91052520863981,3.90501861331932,3.90206782787451,3.90167865108037,3.9038518490408,3.90858314766646,3.91586327457755,3.92567804947733,3.93800852095461,3.95283114666869,3.97011801298614,3.98983708940603,4.01195251255084,4.03642489412672,4.06321164707044,4.09226732409454,4.12354396300255,4.15699143345013,4.19255778025071,4.2301895588382,4.26983215907448,4.31143011419988,4.35492739234451,4.40026766862603],[5.46274418896529,5.39129098811393,5.32075774330715,5.25118152489584,5.18260087715153,5.1150558448854,5.04858799475068,4.98324043020053,4.91905779897224,4.85608629186859,4.79437363151001,4.73396904964143,4.67492325149915,4.61728836568086,4.56111787792163,4.50646654716761,4.45339030236349,4.4019461184378,4.35219187008961,4.30418616215871,4.25798813560618,4.21365724844907,4.17125303138659,4.13083481832623,4.09246145256626,4.05619097000932,4.02208026146043,3.99018471678619,3.96055785445921,3.93325094075664,3.90831260359345,3.88578844661567,3.86572066971946,3.84814770256298,3.83310385786574,3.82061901131605,3.81071831471103,3.80342194852458,3.79874491943891,3.79669690749833,3.797282166478,3.80049947984423,3.80634217336549,3.81479818407019,3.82585018389764,3.83947575510955,3.85564761337425,3.87433387345011,3.89549835161144,3.91910089840155,3.94509775497342,3.97344192618198,4.00408356371157,4.03697035283016,4.07204789682956,4.1092600938026,4.14854950108625,4.18985768343119,4.23312554170696,4.27829361969189,4.32530238720263],[5.3632549541956,5.29116524308489,5.21999574286757,5.14978460534822,5.08057156306329,5.0123979623862,4.94530679121723,4.87934270011701,4.81455201562005,4.75098274434217,4.68868456637498,4.62770881634626,4.56810845042177,4.50993799743863,4.45325349229826,4.39811238971818,4.34457345645329,4.29269664016041,4.24254291320316,4.19417408988901,4.14765261590484,4.10304132908117,4.06040319107279,4.01980099009889,3.98129701553726,3.94495270590813,3.91082827260131,3.87898230257676,3.84947134417567,3.82234948108563,3.79766790036747,3.77547446123099,3.75581327189376,3.73872428232669,3.72424290093774,3.71239964323766,3.70321982024252,3.69672327378761,3.69292416506065,3.69183082153216,3.69344564610619,3.6977650907902,3.70477969555446,3.71447419139217,3.72682766497788,3.7418137808248,3.75940105552684,3.77955317758888,3.80222936553651,3.82738475647198,3.85497081701016,3.88493576857146,3.9172250193028,3.95178159540451,3.98854656531721,4.0274594510224,4.06845862158577,4.1114816649818,4.15646573514175,4.20334787203407,4.25206529339088],[5.26434864600234,5.19160611700022,5.11978334321693,5.04891957639344,4.97905576352548,4.91023458744118,4.84250050192162,4.77589976009951,4.71048043472488,4.64629242873642,4.5833874744277,4.5218191193532,4.46164269698455,4.40291528001198,4.34569561409567,4.29004402981808,4.23602233058151,4.18369365424771,4.13312230644098,4.08437356364564,4.03751344453673,3.99260844839839,3.94972526001868,3.90893042110461,3.87028996903907,3.83386904469256,3.79973147199368,3.76793931302863,3.73855240354794,3.71162787486589,3.6872196691937,3.66537805639422,3.64614916092358,3.62957450827382,3.61569060049647,3.60452853032486,3.5961136429948,3.59046525407791,3.58759643050165,3.58751384046885,3.59021767626462,3.59570165202431,3.60395307652275,3.6149529990285,3.62867642434436,3.64509259141657,3.6641653084171,3.68585333604239,3.71011080996541,3.73688769293775,3.76613024695602,3.79778151615446,3.8317818116187,3.8680691900817,3.90657991940094,3.94724892476698,3.9900102106989,4.03479725499174,4.08154337185044,4.13018204244119,4.18064721198897],[5.16605874559736,5.0926468414619,5.02015345788759,4.94861896185488,4.87808553825356,4.80859723905084,4.74020002703433,4.67294181273266,4.60687248294047,4.54204391909226,4.47851000354491,4.41632661164692,4.35555158729978,4.29624469956246,4.2384675777228,4.18228362217297,4.12775788839234,4.07495694137669,4.02394867797329,3.97480211480419,3.92758713980234,3.88237422586098,3.83923410571933,3.79823740798379,3.7594542551142,3.72295382528244,3.68880388121598,3.65707027044367,3.62781640272104,3.60110271177326,3.57698610978986,3.55551944426107,3.53675096768311,3.52072383130178,3.50747561433935,3.49703790000539,3.48943590899202,3.48468820009077,3.4828064460634,3.48379529100515,3.4876522932366,3.49436795535436,3.50392584058172,3.51630277211522,3.5314691098853,3.54938909714939,3.57002126770302,3.59331890328993,3.61923053004774,3.64770044254165,3.67866924408915,3.71207439261342,3.74785074211677,3.78593107096182,3.82624658940886,3.86872742020575,3.91330304739369,3.95990272981921,4.00845587708266,4.05889238677344,4.11114294281971],[5.06842111437009,4.9943230735469,4.9211414665028,4.84891778685757,4.77769547732734,4.70751998874614,4.6384388337904,4.57050163386747,4.50376015741908,4.43826834766961,4.37408233762195,4.31126044987619,4.24986317862552,4.18995315098002,4.13159506459403,4.07485559844127,4.01980329351058,3.966508400204,3.91504268932783,3.86547922380002,3.81789208857453,3.77235607682657,3.72894633116927,3.68773793959266,3.64880548693425,3.61222256399739,3.57806123790835,3.54639148890708,3.51728062044571,3.4907926511515,3.46698769881055,3.44592136794705,3.42764415370739,3.4122008755076,3.39963015417636,3.38996394605541,3.38322714666425,3.37943727509423,3.37860424830531,3.38073025203683,3.38580971222652,3.39382936780531,3.40476844266217,3.4185989116184,3.4352858525731,3.45478787471232,3.47705761091611,3.50204226130982,3.52968417430942,3.55992145148467,3.59268856305568,3.62791696176766,3.66553568416021,3.7054719297577,3.74765161035449,3.79199986325988,3.83844152402161,3.88690155570008,3.93730543317052,3.98957948215654,4.04365117373084],[4.97147418333391,4.89667309582817,4.82278542538016,4.74985379891962,4.67792293255917,4.60703970189262,4.53725320724628,4.4686148321956,4.40117829340852,4.33499967961039,4.27013747718666,4.20665257965593,4.14460827796521,4.08407022829305,4.02510639380892,3.9677869566464,3.91218419622405,3.85837233001651,3.80642731296539,3.75642659195421,3.7084488121845,3.66257347290733,3.61888053081285,3.57744995047424,3.53836120259083,3.5016927123681,3.46752126218481,3.43592135468138,3.40696454448933,3.38071874891409,3.35724754987045,3.33660950111981,3.31885745623623,3.30403793359983,3.29219053496582,3.28334743369787,3.27753294754183,3.27476320885558,3.27504594256721,3.27838035892387,3.2847571644841,3.29415869099858,3.3065591380366,3.32192492166593,3.34021511837741,3.36138199092113,3.38537158089636,3.41212435186536,3.44157586643742,3.47365748114236,3.50829704388727,3.54541958024717,3.58494795664482,3.62680351049015,3.67090663944965,3.71717734408885,3.76553572008808,3.81590239800642,3.8681989301176,3.92234812514326,3.97827433275618],[4.87525915740403,4.79973803297484,4.72512629656801,4.65146770942805,4.5788082712847,4.50719630352601,4.43668252772856,4.36732013770289,4.29916486291448,4.2322750208194,4.16671155531376,4.10253805814383,4.03982076976836,3.97862855581986,3.9190328549935,3.86110759392326,3.80492906440911,3.75057575826946,3.69812815514171,3.64766845878092,3.5992802778498,3.55304824789248,3.50905759217274,3.46739362036076,3.42814116568321,3.39138396310196,3.35720397332639,3.32568065993292,3.29689022947158,3.27090484705968,3.24779184244493,3.22761292369224,3.21042341732652,3.19627155477313,3.18519782512565,3.17723441352987,3.1724047427546,3.17072313284922,3.17219459026689,3.1768147336368,3.18456985873841,3.19543714044237,3.20938496473263,3.226373379692,3.24635465076115,3.26927390284935,3.2950698300924,3.32367545324728,3.35501890484269,3.38902422316294,3.42561213778188,3.46470083150795,3.50620666606879,3.55004486147971,3.59613012165186,3.64437720127667,3.69470141128055,3.74701906211482,3.80124784579816,3.8573071589541,3.91511837009456],[4.77982023526842,4.70356208594043,4.62820819660385,4.5538034571033,4.48039515453054,4.40803307105965,4.33676957773529,4.26665972221657,4.19776130812603,4.13013496326873,4.06384419357209,3.99895541916191,3.93553798854226,3.87366416640293,3.81340909015659,3.75485068993478,3.69806956647905,3.64314882118935,3.59017383258083,3.53923197360147,3.49041226473002,3.44380495855857,3.39950105271456,3.35759172953208,3.31816772286107,3.28131861480287,3.24713206794344,3.21569300174651,3.18708272504752,3.16137803989196,3.13865033508448,3.11896469052484,3.10237901545823,3.08894324492542,3.07869861876495,3.07167706635968,3.06790071788753,3.06738155918523,3.07012124262942,3.07611106094355,3.08533208489363,3.09775545982715,3.11334285033045,3.13204701729053,3.15381250763791,3.17857643421519,3.20626932165141,3.23681599380901,3.2701364791997,3.30614691255658,3.3447604132766,3.38588792446487,3.42943899958169,3.47532252699899,3.52344738593916,3.57372303016005,3.6260599982779,3.68037035173445,3.73656804310329,3.79456921870659,3.85429246040961],[4.68520484551577,4.60819278467609,4.53207866624544,4.45690849505948,4.3827308415678,4.30959695632038,4.23756088124079,4.16667955553562,4.0970129136744,4.02862397241429,3.96157890334238,3.89594708687165,3.83180114306575,3.76921693409869,3.70827353260147,3.64905314963881,3.59164101563411,3.53612520726875,3.48259641327814,3.43114763221784,3.3818737957485,3.33487131185997,3.2902375237911,3.24807008225796,3.20846623101375,3.17152200872891,3.13733137365949,3.10598526146607,3.07757059069987,3.05216923466525,3.02985698232331,3.01070251430287,2.99476642259892,2.98210030384999,2.97274595593342,2.96673470583651,2.9640868933132,2.96481152982907,2.9689061459911,2.97635683344588,2.98713847959821,3.00121518598594,3.01854085426865,3.03905991799852,3.06270819398151,3.0894138243003,3.11909827900213,3.15167738995767,3.18706238826081,3.22516092047559,3.26587802271598,3.3091170356389,3.35478044763765,3.40277065758892,3.45299065223821,3.50534459657546,3.55973833828233,3.61607982950229,3.67427947081188,3.73425038340039,3.79590861615796],[4.59146389953583,4.51368126018204,4.43678896233484,4.36083410302136,4.28586652286931,4.21193893944697,4.13910707851433,4.0674298008921,3.99696922216859,3.92779082191768,3.85996353849599,3.79355984482864,3.72865579989278,3.6653310698833,3.60366891232111,3.54375611567831,3.48568288649228,3.42954267548356,3.37543193395167,3.32344979178541,3.27369764888212,3.22627867272405,3.18129719640461,3.13885801361413,3.09906557004001,3.06202305431967,3.02783139605939,2.99658818337468,2.96838651770933,2.94331382905328,2.92145067972644,2.90286958919133,2.88763391544524,2.87579682999467,2.8674004228892,2.86247497158717,2.86103840252813,2.86309596739587,2.86864014757799,2.87765079084903,2.89009547452188,2.90593007996229,2.92509955512755,2.94753883522935,2.97317388710498,3.00192284057081,3.0336971698768,3.06840289015251,3.10594173707011,3.14621230241251,3.18911110336872,3.23453356875569,3.28237493062343,3.3325310145538,3.38489892622228,3.43937763534604,3.49586846095449,3.55427546401149,3.61450575485112,3.67646972374718,3.7400812033173],[4.49865206149246,4.4200825365178,4.34239437279583,4.26563572614307,4.18985768343119,4.11511441720235,4.04146333967363,3.96896525371351,3.89768449780693,3.82768908137844,3.75905080611621,3.69184536813532,3.6261524349483,3.56205569029188,3.49964283891942,3.43900556255137,3.38023941733696,3.32344366249494,3.26872100936085,3.21617727998571,3.16592096483086,3.11806267012304,3.07271444720691,3.02998899887992,2.98999876130285,2.95285486467789,2.91866598142218,2.88753707688237,2.8595680844543,2.83485253388759,2.8134761680434,2.79551558883284,2.78103697686073,2.77009493085091,2.76273147178389,2.75897525259732,2.75884100732603,2.76232926403841,2.76942633450002,2.78010458103315,2.79432294854062,2.81202773812777,2.83315358906983,2.857624628694,2.88535574544592,2.91625393905213,2.9502197030554,2.98714839864971,3.02693158409879,3.06945827045654,3.11461608121022,3.16229230030884,3.21237479940493,3.26475284073865,3.31931775676424,3.37596351130009,3.43458714970066,3.4950891473821,3.55737366710434,3.62134873585335,3.6869263521158],[4.40682803537117,4.32745584311083,4.24895455551302,4.17137334166526,4.09476449827048,4.01918362818086,3.94468982022386,3.87134582781215,3.7992182431731,3.72837766327393,3.65889884264935,3.5908608273618,3.52434706324763,3.45944547044219,3.39624847496524,3.33485298693127,3.27536031379529,3.21787599604552,3.16250955202774,3.10937411827644,3.05858597200539,3.01026392346068,2.9645285678552,2.92150138976013,2.88130371726257,2.84405552896772,2.80987412397648,2.77887267310565,2.75115867846151,2.72683237746929,2.70598513587114,2.68869788118641,2.67503963279744,2.66506618637845,2.65881900822875,2.65632438893657,2.65759289583638,2.66261915052467,2.67138194226911,2.68384467177633,2.69995610389557,2.71965139378261,2.74285333994445,2.76947381016416,2.79941528286282,2.83257244685227,2.86883380616713,2.90808324298809,2.95020149970962,2.99506755009459,3.04255983842066,3.09255737394007,3.14494067541056,3.19959256665479,3.25639882897248,3.31524871979073,3.37603536930744,3.43865606822717,3.50301246019712,3.56901065241294,3.63656125725634],[4.31605486870008,4.23586494731027,4.15653390147056,4.07811185431399,4.00065226163416,3.92421211719509,3.84885216174419,3.77463709317666,3.70163577454717,3.62992143573792,3.55957186356054,3.49066957388895,3.42330195809489,3.35756139460152,3.29354531481514,3.23135621109202,3.17110157282689,3.11289373532207,3.05684962496189,3.00309038356355,2.95174085483026,2.90292891685213,2.85678464685711,2.81343930816727,2.77302415477452,2.73566905622467,2.70150095454497,2.67064217551315,2.64320862813586,2.61930793800081,2.59903757115904,2.58248301417812,2.56971608173758,2.5607934245001,2.55575530620308,2.55462470969953,2.55740681738937,2.56408889312595,2.57464057182242,2.58901454153821,2.60714758280357,2.62896191314,2.65436677250205,2.68326017843689,2.71553077821029,2.75105972847367,2.78972254029892,2.83139083739318,2.87593398677652,2.92322057301848,2.97311969835403,3.0255021009682,3.08024109205516,3.13721331876036,3.19629936483265,3.25738420390531,3.32035752202591,3.38511392663471,3.45155305891689,3.51957962557189,3.58910336476438],[4.22640027201154,4.14537850659779,4.0652019220042,3.9859215208494,3.90759185100927,3.83027124075799,3.75402204062276,3.6789108694266,3.60500886113875,3.53239190812579,3.46114089518128,3.39134191729469,3.32308647249868,3.25647161931551,3.19160008633775,3.12858031938047,3.06752644952224,3.00855816334373,2.95180045496404,2.89738323830825,2.84544079772069,2.79611105591982,2.74953464076028,2.70585373670575,2.66521071364876,2.6277465349329,2.59359895811892,2.5629005558662,2.53577659957713,2.51234286407462,2.49270342607587,2.47694854086113,2.46515268853225,2.45737288205951,2.45364732290221,2.45399447613251,2.45841261642398,2.46687987066584,2.47935475476988,2.49577717428934,2.51606983357128,2.5401399786451,2.56788138643801,2.5991765077652,2.63389867349187,2.67191428117107,2.71308489171084,2.75726918042714,2.80432470249738,2.85410944792605,2.90648317463639,2.96130851958187,3.01845189656086,3.0777841957414,3.13918130398027,3.20252446719193,3.26770051667326,3.33460198080226,3.40312710225082,3.47317977907533,3.54466944601065],[4.13793695242233,4.05607043915751,3.9750336593031,3.89487840346496,3.81566022635179,3.73743871496887,3.66027776723753,3.58424587863591,3.5094164334977,3.43586799642956,3.36368459788876,3.29295600627449,3.22377797691535,3.15625246607898,3.09048779561007,3.02659875107355,2.96470659343976,2.90493896155674,2.84742964014136,2.79231816610155,2.73974924508173,2.68987195069042,2.6428386814742,2.59880385590784,2.55792233397863,2.52034756568451,2.48622948197407,2.45571216192416,2.42893133031945,2.40601176069291,2.38706467817229,2.3721852716352,2.36145043313387,2.35491684212699,2.35261950143208,2.35457081093428,2.36076023536675,2.37115458670962,2.38569890373015,2.40431787508317,2.42691772205603,2.45338843537154,2.48360624898553,2.51743623259561,2.55473489238519,2.59535268423302,2.63913636266502,2.68593110967611,2.73558240802779,2.78793764211044,2.84284742493886,2.90016666187767,2.95975537027165,3.0214792795975,3.08521023954968,3.15082646418482,3.21821263943087,3.28725991941279,3.35786583456858,3.42993413175066,3.50337456366477],[4.05074295883095,3.96802031058123,3.88611011833837,3.80506485076385,3.72494096398657,3.64579920618948,3.56770493714356,3.49072846053445,3.41494536586632,3.34043687539797,3.26729018992577,3.19559882523634,3.12546292867449,3.05698956249203,2.9902929374648,2.92549457673716,2.86272338607578,2.80211560287078,2.74381459259158,2.68797045839146,2.63473942769524,2.58428297957451,2.53676667929575,2.49235869245417,2.45122796135295,2.41354204131297,2.37946461454081,2.34915272354066,2.3227537934928,2.30040254126356,2.28221789461885,2.26830006506749,2.25872792780624,2.25355685939328,2.25281716639082,2.25651320666462,2.26462326193972,2.27710017021324,2.29387267559729,2.31484740717796,2.33991136267705,2.36893475044203,2.40177403569081,2.43827504297304,2.47827598379435,2.52161030266743,2.56810926270383,2.61760421986439,2.66992856056459,2.72491929883899,2.7824183459451,2.84227347708107,2.90433902724616,2.9684763519105,3.03455408890969,3.10244825664912,3.17204222100713,3.24322655985032,3.31589885026446,3.38996339978057,3.46533093924931],[3.96490203512623,3.88131373329911,3.79851871713757,3.71657000372556,3.63552482328766,3.55544496355785,3.47639713453987,3.39845335192131,3.32169133625226,3.24619492351915,3.1720544808735,3.0993673189504,3.02823808937242,2.95877915263611,2.89111089760013,2.82536198926268,2.76166951652773,2.70017900641174,2.64104426597034,2.58442700863518,2.53049621835543,2.47942720386368,2.43140029764895,2.38659916105649,2.34520866954386,2.30741237143512,2.27338953987213,2.24331187046031,2.21733991446875,2.19561937604049,2.17827743701132,2.16541929905982,2.15712514446633,2.15344770938071,2.15441063522839,2.1600077160356,2.17020309682792,2.18493240848078,2.20410475653445,2.22760542420015,2.25529910966906,2.28703349827287,2.32264297092135,2.36195226846745,2.40477996221415,2.45094161799395,2.50025257983328,2.55253033500311,2.60759645253234,2.66527811070579,2.7254092455181,2.78783136213504,2.85239405620119,2.91895529256616,2.98738148687913,3.05754743156934,3.12933610282508,3.20263837992874,3.27735270313415,3.35338469145562,3.4306467374297],[3.88050397643284,3.79604277382918,3.71235375070458,3.62949032234555,3.54751034242744,3.46647649051011,3.38645668640378,3.30752453029732,3.22975976632135,3.15324876560315,3.07808502277104,3.00436965718885,2.93221190684893,2.86172959873855,2.79304957455573,2.72630804487599,2.66165083833045,2.59923350525012,2.53922122794459,2.48178848294412,2.42711839507341,2.37540172041138,2.32683539661983,2.28162060662867,2.23996031710881,2.20205627808461,2.16810550515455,2.13829631040955,2.11280399960355,2.09178640648857,2.07537948337166,2.0636932014303,2.05680802683655,2.05477222311596,2.05760018432117,2.06527193079946,2.07773380820975,2.09490033352601,2.11665704284776,2.14286412697275,2.17336059969263,2.2079687330046,2.24649851002745,2.28875188370981,2.33452667896831,2.38362002937642,2.43583129015005,2.49096441237198,2.5488297968545,2.60924566933874,2.67203903275992,2.73704625869243,2.80411338070554,2.87309614901442,2.94385990004808,3.01627928758939,3.09023791484901,3.16562789980124,3.24234939968376,3.32031011492289,3.39942478794411],[3.79764498174953,3.71230636108047,3.6277168618333,3.54393012639086,3.46100445706905,3.37900324991897,3.29799546301599,3.21805611902359,3.13926684056904,3.06171641524936,2.9855013847792,2.91072664976113,2.83750607765706,2.76596309661786,2.6962312517592,2.62845469317507,2.56278855647226,2.49939918706402,2.43846414929714,2.38017195146034,2.32472140903585,2.27232056293577,2.22318506918607,2.17753598430955,2.13559688941384,2.09759032825411,2.06373358171453,2.03423386251616,2.00928308570956,1.98905244516821,1.97368709291537,1.96330126365055,1.95797419874802,1.95774719360649,1.96262201791002,1.97256084729883,1.98748771251734,2.00729133872453,2.03182913396516,2.0609320077918,2.09440966667265,2.13205604123413,2.17365454340704,2.21898291683041,2.26781751829734,2.31993694039459,2.37512494798032,2.433172749648,2.493880658881,2.55705921946518,2.62252987837374,2.69012528966107,2.75968932765722,2.83107687926278,2.90415347518126,2.97879480973741,3.0548861893088,3.13232194078781,3.21100480408421,3.29084532650244,3.37176127180945],[3.71642799432964,3.63021068667228,3.54471750950936,3.46000214086853,3.37612313273782,3.29314439404044,3.21113571693764,3.13017334746579,3.05034060029004,2.97172851558994,2.89443655362541,2.81857331917329,2.74425730356105,2.67161762622212,2.60079475032793,2.53194113792677,2.46522179905146,2.40081467654442,2.33891079430228,2.27971408213756,2.22344077701406,2.17031829036238,2.12058342775616,2.07447985448562,2.03225472302545,1.99415442026235,1.96041945625023,1.93127860161009,1.906942481993,1.88759694379447,1.87339659837815,1.86445901199447,1.86086001622905,1.86263055719394,1.86975538110877,1.88217368427512,1.89978166386834,1.92243672534702,1.94996296261123,1.98215744787041,2.01879685474808,2.05964398218074,2.10445383078981,2.15297898701277,2.20497417470389,2.26019992560475,2.31842539157427,2.37943037067465,2.44300664785863,2.50895876292817,2.57710431815762,2.64727392977366,2.7193109148922,2.79307079116238,2.86842065203781,2.94523846736213,3.02341234736882,3.1028397984451,3.1834269910597,3.2650880539428,3.34774440371313],[3.63696301878367,3.54986958550227,3.46347342246886,3.37782803325187,3.29299199760587,3.20902950599872,3.12601094758871,3.04401355425999,2.96312210220784,2.88342967083761,2.80503845619641,2.72806063254005,2.65261925064433,2.57884915475158,2.50689789122673,2.43692657072615,2.3691106316897,2.30364043619208,2.24072160995813,2.180575017606,2.12343624379954,2.06955443406438,2.01919034011894,1.97261341973122,1.93009786740926,1.89191750661911,1.85833956152419,1.8296174463532,1.80598285542086,1.78763758859232,1.77474567818102,1.76742646135456,1.76574923768081,1.76973004846628,1.77933091877776,1.79446164422442,1.81498393014006,1.84071745271803,1.87144725141127,1.90693179974767,1.94691113381247,1.99111452132644,2.03926729761233,2.09109664654343,2.14633624096022,2.20472976424172,2.26603340827257,2.33001748523085,2.39646730742473,2.4651834880247,2.53598180271541,2.60869273349273,2.683160795005,2.75924372354563,2.83681159050086,2.91574588640267,2.99593860887505,3.07729137753028,3.15971459095538,3.24312663495744,3.32745314684436],[3.55936740113974,3.47140488157919,3.38411102173191,3.29753892516267,3.21174695742341,3.12679933383549,3.04276677213841,2.95972721465766,2.87776662374501,2.79697985267972,2.71747159172858,2.63935738531321,2.56276471080803,2.48783410188833,2.41472028899292,2.34359331575033,2.27463957258841,2.20806266685091,2.14408402267642,2.08294307455865,2.02489688809268,1.97021901395834,1.91919736309701,1.87213089127712,1.82932491089736,1.79108491847405,1.75770894653829,1.72947861978583,1.70664930546725,1.68943996857166,1.67802352872746,1.67251861529805,1.67298358452021,1.67941347786352,1.69174028301781,1.70983646608111,1.73352135732338,1.7625696729071,1.79672129374798,1.83569141278238,1.8791802783444,1.9268819564452,1.97849175627032,2.03371216871641,2.09225733138654,2.15385614568297,2.2182542351705,2.28521495868713,2.35451968837871,2.42596754275867,2.49937473662396,2.57457367920004,2.65141192295448,2.72975104006588,2.80946548231088,2.89044146314224,2.97257588757754,3.05577534563046,3.13995517778814,3.22503861590432,3.31095599934968],[3.48376605499364,3.39494668043004,3.30676579152501,3.21927585600194,3.13253476821472,3.04660649101769,2.96156177517001,2.87747896346527,2.79444488623809,2.71255585366629,2.63191874804225,2.55265221550711,2.47488795107382,2.39877206241181,2.32446648598005,2.25215041271259,2.18202165858144,2.11429788713784,2.04921755618885,1.9870404197272,1.92804737147889,1.87253937307169,1.82083517670896,1.77326754263923,1.73017768301837,1.69190775512492,1.65879139402424,1.6311425216473,1.60924297822391,1.59332984657917,1.58358360595108,1.58011837343728,1.5829753981383,1.59212064497266,1.60744679077884,1.62877936572828,1.65588624586934,1.68848935332631,1.72627730487698,1.76891785291753,1.81606921878198,1.86738974090693,1.92254557331718,1.9812164239473,2.04309949722366,2.10791190342305,2.1753918335853,2.24529879229172,2.31741314911772,2.3915352267405,2.46748409878925,2.54509622898915,2.62422404757137,2.70473453205465,2.78650783704728,2.86943600084807,2.9534217443501,3.03837736909638,3.12422375544295,3.21088945794014,3.29830989267961],[3.41029161346781,3.32063358523752,3.23158257297039,3.14319016994589,3.0555135350646,2.96861608859776,2.88256829927757,2.79744857298906,2.71334425332228,2.63035274356037,2.54858275794098,2.46815570671556,2.38920721393334,2.31188875805685,2.23636941227252,2.16283764223454,2.09150309232025,2.02259825563647,1.95637987676786,1.89312987952246,1.83315554693519,1.77678861370107,1.72438287404401,1.67630988030698,1.63295233651608,1.59469490870569,1.56191240803394,1.53495566349289,1.514135861608,1.49970861367331,1.49185939164669,1.49069210870755,1.49622240374935,1.50837661079718,1.52699656794589,1.55184955360919,1.58264195431423,1.61903492058512,1.6606602881362,1.70713535231132,1.75807554719063,1.81310456073227,1.87186181826188,1.93400754527618,1.99922577607025,2.06722573083401,2.13774197244212,2.2105337046196,2.28538350781172,2.36209574240679,2.44049478893416,2.52042324488038,2.60174015840396,2.68431934955034,2.76804784801291,2.85282446137287,2.93855847754063,3.02516849852111,3.11258139860839,3.20073139789666,3.28955924099263],[3.33908448308592,3.24861280936728,3.15871574934896,3.0694437908496,2.98085309688102,2.89300625391394,2.80597312624073,2.71983183031326,2.6346698436778,2.55058526326512,2.46768822690087,2.38610250936232,2.30596729924455,2.2274391540927,2.15069411706667,2.07592995671462,2.00336845966617,1.93325766135432,1.86587383956826,1.80152301822241,1.74054163557933,1.68329592917739,1.63017949477509,1.5816084171454,1.53801338914367,1.49982838524452,1.46747578691615,1.44134839132848,1.42178943067713,1.40907245561936,1.40338347759448,1.4048078819487,1.41332416139582,1.4288055099226,1.45102900907776,1.47969090866477,1.51442569483788,1.55482641641605,1.60046405307498,1.65090435373621,1.70572130935425,1.76450706577933,1.82687853596926,1.89248122452646,1.96099086820546,2.03211347743801,2.10558428593709,2.18116601504452,2.25864675971813,2.33783771575043,2.41857089728318,2.50069693989199,2.58408304533449,2.66861109665406,2.75417595392618,2.84068392912432,2.92805143144467,3.01620377050112,3.1050741030103,3.19460250816673,3.2847351773294],[3.27029277203354,3.17904015330041,3.08832928584013,2.99820934213394,2.90873524761893,2.81996847894125,2.73197798375165,2.64484124007192,2.55864547498607,2.47348906368072,2.38948313022495,2.30675337023946,2.22544211171457,2.14571062216555,2.06774165590278,1.9917422114915,1.91794643265095,1.84661853126194,1.77805553385756,1.71258954888875,1.65058912041866,1.59245908139767,1.538638166499,1.48959353143362,1.44581131907672,1.40778260106163,1.37598449603669,1.35085706675977,1.33277766198879,1.32203546767013,1.31880978888514,1.32315559959543,1.33499895963339,1.35414316218918,1.3802844580684,1.41303455961511,1.45194633101508,1.49653921753813,1.54632180063243,1.60080998016398,1.65954032011099,1.7220788470786,1.78802601687924,1.85701871786973,1.92873015219859,2.00286831356469,2.07917362501619,2.15741615003792,2.23739266243678,2.31892376058537,2.40185113795358,2.48603507022133,2.57135214466083,2.65769323537035,2.74496171463553,2.833071883362,2.92194760010815,3.01152108730111,3.10173189375561,3.19252599397753,3.28385500649826],[3.20407206188902,3.11207980950694,3.02059658076078,2.92967006105921,2.83935373450185,2.74970772857636,2.66079979621834,2.57270645789964,2.48551432936555,2.39932166339665,2.31424013607219,2.23039690870714,2.14793699473084,2.06702595443219,1.98785292696901,1.91063398436807,1.83561575096347,1.76307916672739,1.69334317582441,1.62676798409425,1.56375734595782,1.50475911693533,1.45026306681338,1.40079474796486,1.35690415611748,1.31914815386743,1.28806630839209,1.26415101066992,1.24781438553206,1.23935616499201,1.23893771702482,1.24656713803534,1.26209848767542,1.28524532893746,1.31560576105402,1.35269415666716,1.3959743709841,1.44489010219027,1.49888971254405,1.5574444880423,1.62006058414378,1.68628563987815,1.75571131517178,1.82797297167797,1.90274752611799,1.97975026483421,2.05873118062414,2.13947120500301,2.22177856682434,2.30548540723271,2.3904447130034,2.47652758694283,2.56362084791255,2.65162493856405,2.7404521119293,2.83002486574601,2.92027459388731,3.01114042626156,3.10256823126094,3.19450975778641,3.28692189678361],[3.14058498914996,3.04790395493666,2.95570008000862,2.86401944955117,2.77291396401905,2.68244222608732,2.59267058083759,2.50367433688673,2.41553920059883,2.32836296012329,2.24225746035519,2.15735091326445,2.07379058909221,1.99174593054733,1.91141212099753,1.8330141136229,1.75681108409105,1.68310119411449,1.61222643387152,1.54457713258954,1.48059547580119,1.42077704352492,1.3656690125546,1.31586332736204,1.27198299546053,1.23465994718594,1.2045038935718,1.18206348140617,1.16778359233144,1.16196512983952,1.16473490864119,1.17603224949482,1.1956154466618,1.22308656306115,1.25792892692123,1.29954977958893,1.34732107312031,1.400613622138,1.4588224576565,1.52138338606921,1.58778205426685,1.65755732820808,1.73030076597704,1.80565367254775,1.88330285127741,1.96297582106201,2.04443599032069,2.12747807578902,2.2119239153749,2.29761873509333,2.38442787609445,2.47223395752521,2.56093443594036,2.65043951631227,2.74067036946694,2.83155761349094,2.92304002075972,3.01506341681355,3.10757974183316,3.20054624967306,3.29392482317686],[3.08000060125054,2.98669208818546,2.89383060242917,2.80146059788131,2.7096323380502,2.61840282105969,2.52783687354609,2.43800844649587,2.34900215221509,2.26091508841819,2.17385900254891,2.08796285620432,2.00337585460512,1.9202710071083,1.83884927785468,1.75934436441323,1.68202809682791,1.60721636516197,1.53527534024694,1.46662752473022,1.40175683419851,1.34121144796343,1.28560261125827,1.23559702168628,1.19190013253037,1.1552280518806,1.12626718756154,1.10562367182677,1.09376854481633,1.0909883388724,1.09735203170244,1.11270274889901,1.1366764417957,1.16874259752063,1.20825708571694,1.2545162857098,1.3068041310912,1.36442770529001,1.42674065228719,1.49315599145395,1.56315090146177,1.63626607453271,1.71210180555731,1.79031240807813,1.87060002257149,1.95270847039961,2.03641751522037,2.1215377022863,2.20790582864128,2.29538103025926,2.38384143767995,2.47318133704869,2.56330877029121,2.65414351114965,2.74561535975156,2.83766270540204,2.93023131434287,3.02327330578654,3.11674628537637,3.21061261028691,3.3048387644875],[3.02249345188152,2.92863006780885,2.83518632218087,2.74220511429924,2.64973513700828,2.55783183647864,2.46655855618939,2.37598790369785,2.28620338682409,2.19730137517213,2.10939345328005,2.02260924257459,1.93709977945266,1.85304154377687,1.77064123141035,1.69014134846106,1.6118266610232,1.53603144257941,1.46314729265375,1.39363101551573,1.32801160191273,1.26689471631317,1.21096227423881,1.16096383967677,1.11769604295821,1.08196665930695,1.05454220364064,1.03608236480826,1.02707062759777,1.02775555531531,1.03811795519049,1.05787347839725,1.08650987280544,1.12334817003061,1.16761231516224,1.21849330155152,1.27519933082375,1.33698943179845,1.40319212802387,1.47321266474177,1.54653256123593,1.62270464668416,1.70134587247208,1.78212939630802,1.86477682112166,1.94905105115284,2.03474996188903,2.12170092492199,2.20975614542017,2.29878873096891,2.38868939735518,2.47936371765912,2.57072982863852,2.6627165187566,2.75526163306984,2.84831074043442,2.9418160176125,3.03573531271896,3.13003135707876,3.22467110008195,3.31962514617405],[2.96824240299928,2.87390881063611,2.77997135610077,2.686471594793,2.59345686652362,2.5009812888959,2.40910694994977,2.31790534431093,2.22745910716091,2.13786411237079,2.04923201513441,1.9616933350401,1.87540119164664,1.79053581883148,1.70730999159713,1.6259754906813,1.54683069053636,1.4702292590185,1.39658976187102,1.32640561362347,1.26025423654941,1.19880341575657,1.14281167237586,1.09311820376728,1.05061710260984,1.01621119123201,0.990744202140025,0.974916900542279,0.969201656958887,0.973776535320471,0.988498674571509,1.01292572211319,1.04637823343541,1.08802403295897,1.13696314152447,1.19229781266011,1.2531811373313,1.31884489302262,1.38861108332778,1.46189248223993,1.53818676321132,1.61706753334776,1.69817440102491,1.78120329698358,1.86589765905691,1.9520407175474,2.03944890886039,2.12796633919354,2.21746017543261,2.30781682949894,2.39893880860005,2.49074211758189,2.58315411564722,2.67611174543853,2.76956006668261,2.86345103884314,2.95774250751156,3.05239735775499,3.14738280457248,3.24266979623387,3.3382325108201],[2.91742910669524,2.82272261530653,2.72839391095445,2.63448357438487,2.5410379914661,2.44811038240608,2.3557620454568,2.26406386508534,2.17309814678389,2.08296085559137,1.99376435325148,1.90564074963427,1.81874600674119,1.73326495591626,1.64941740556409,1.56746551737132,1.48772259461182,1.41056332324674,1.33643528061005,1.26587109518688,1.19949989578991,1.13805552165541,1.08237735598115,1.03339784744846,0.992109611626397,0.95950604576097,0.936494646125765,0.923792505161518,0.92182588538344,0.930662846219474,0.950001949315739,0.979221156876256,1.01746962826634,1.06377385510964,1.11713257278856,1.17658639952855,1.24125981403005,1.31038021058634,1.38328112351274,1.45939612067356,1.53824816184106,1.61943748807349,1.70262977424575,1.78754538921912,1.87395007641316,1.96164707816143,2.05047058736028,2.14028035546748,2.23095727667976,2.32239978055905,2.41452088645162,2.50724579598441,2.60050992150614,2.69425726726722,2.7884390960881,2.88301282740174,2.97794112321472,3.07319112710285,3.168733828215,3.26454352773009,3.36059738957448],[2.87023614978899,2.77526708758047,2.6806639586601,2.58646691668599,2.4927219987321,2.39948219722815,2.30680876248264,2.21477279162412,2.12345717413146,2.03295898193304,1.9433924138151,1.85489242974517,1.76761924004515,1.68176384478607,1.59755484459266,1.51526675319342,1.43523001124739,1.35784278804177,1.28358439102729,1.21302957125568,1.14686206345514,1.08588417634868,1.03101711736338,0.983284367895752,0.943769039543044,0.913538095457355,0.893534388652,0.884452121686389,0.886626983238953,0.899977365970533,0.924018999323667,0.957947265230693,1.00075712079146,1.05136419316366,1.10870127560065,1.17178085424296,1.23972667238477,1.31178278463855,1.3873088790626,1.46576866448215,1.54671575592597,1.62977958064319,1.71465254011807,1.80107889476979,1.88884542433653,1.9777737203454,2.06771389237988,2.15853945819681,2.25014320531926,2.34243384032427,2.43533327250053,2.52877440670377,2.62269934454025,2.71705791320386,2.8118064576986,2.90690684532673,3.00232564177493,3.09803342640313,3.19400422087432,3.2902150104233,3.38664534114021],[2.82684485811511,2.73173666102448,2.63699042821898,2.54264662347166,2.44875176062932,2.35535953062765,2.26253217697602,2.17034218158916,2.0788743392804,1.98822831984957,1.898521842167,1.80989461530734,1.72251323706143,1.63657727748652,1.55232680789791,1.47005164938238,1.3901025808138,1.31290461135549,1.23897209486407,1.16892479877244,1.10350284028566,1.04357647149208,0.990144022445818,0.944308486946145,0.907222052119464,0.879991582451324,0.863549951374278,0.858517240276427,0.865092587637407,0.883016714744698,0.911620453835834,0.949939569228056,0.996854311581446,1.05121446520292,1.11192858816796,1.17801465123254,1.24861997895751,1.32302123383761,1.400613622138,1.48089562490131,1.5634529686515,1.64794372431783,1.73408530697075,1.82164353955026,1.91042364896337,2.00026294567786,2.09102491261706,2.18259444664338,2.27487402988646,2.36778064549594,2.46124328714545,2.55520094158011,2.64960094829112,2.74439766041995,2.83955134692998,2.93502728866108,3.03079503076644,3.1268277617822,3.22310179565976,3.31959613786354,3.41629212039191],[2.78743277613676,2.69232172971513,2.59757793167233,2.50324308166944,2.40936521530895,2.31599990404913,2.22321172414378,2.13107606274041,2.03968134766193,1.94913181052868,1.85955092149412,1.77108566829627,1.68391189177441,1.59824093117217,1.51432786703834,1.43248165911599,1.35307742517511,1.27657092765476,1.20351491396997,1.13457611598472,1.0705502219783,1.01236979043855,0.961096981270573,0.917890092670262,0.883932710695241,0.86032078427285,0.847919021290752,0.847219854277201,0.858251883945673,0.880574307546564,0.91335969142826,0.955531653447828,1.00591031188449,1.0633298502207,1.12671433166761,1.19511505722253,1.2677203220012,1.34384881204309,1.42293515117325,1.50451296413798,1.58819837365301,1.67367526386957,1.76068273794676,1.84900473728937,1.93846158933659,2.02890318846123,2.12020351923181,2.21225626403854,2.3049712780407,2.39827175428628,2.49209193695221,2.58637526995543,2.68107289193186,2.77614240749041,2.8715468795549,2.96725399928895,3.0632353992325,3.15946608241423,3.2559239457857,3.352589380696,3.44944493656159],[2.75217086032907,2.65720543818891,2.56262307744175,2.46846781087648,2.37479043909765,2.28164982543902,2.18911448358057,2.09726453231788,2.006194111948,1.91601438175999,1.82685724877654,1.73888001417686,1.65227116418583,1.56725757177826,1.48411340287479,1.40317101198995,1.32483402174959,1.24959252617863,1.17803980313815,1.11088886666222,1.04898540045626,0.993310961716756,0.944967175600805,0.905129415751835,0.874960316560446,0.855483402440896,0.847436220696371,0.851143022320247,0.866452965105862,0.892769307848919,0.929157324317757,0.974489419968031,1.02758256404336,1.08730042868667,1.15261377682141,1.22262619586479,1.29657668925178,1.37382946809544,1.45385821306746,1.53622914957372,1.62058517094792,1.70663194409122,1.79412621877305,1.88286621542402,1.97268382704413,2.06343833873895,2.15501138606599,2.24730291023938,2.34022790912908,2.43371382108969,2.52769841145245,2.62212805857295,2.71695635810586,2.81214298145527,2.90765273793905,3.00345480084864,3.09952206590581,3.19583061712184,3.29235928015605,3.38908924726481,3.48600376107402],[2.72122045059579,2.6265602065284,2.53231046601642,2.43851882791052,2.34524025461185,2.25253848571553,2.16048777071462,2.06917500110193,1.97870234319439,1.88919049879591,1.80078275161623,1.71364999235442,1.6279969514941,1.5440698986289,1.46216607457212,1.38264507644131,1.30594225951425,1.23258385636149,1.16320279433156,1.09855292021756,1.03951733814126,0.987103882108704,0.9424180876086,0.90660334746218,0.880742531848785,0.865728124289136,0.862126999323744,0.870080881723499,0.889279775165566,0.919019203442614,0.958318357370908,1.00605757879549,1.06109831289548,1.1223668773656,1.18890083067731,1.25986624804499,1.33455638350005,1.41238044397341,1.49284838355953,1.57555516376739,1.66016621211608,1.74640477256266,1.83404127745463,1.92288460006969,2.0127749409616,2.10357808049541,2.19518074848485,2.28748689532777,2.38041468531726,2.47389406641672,2.56786479971778,2.66227485572159,2.7570791039042,2.85223823741338,2.9477178868984,3.04348788703332,3.13952166679835,3.23579574047468,3.33228928093763,3.42898376047914,3.52586264726909],[2.69473011085904,2.60054410261192,2.50680850662528,2.41357580120206,2.32090659170235,2.22887116421445,2.13755138556385,2.04704303452204,1.95745866991986,1.86893116612242,1.78161807438153,1.69570699775115,1.61142219218971,1.5290326159833,1.44886162210199,1.37129838447336,1.29681090314607,1.22595994134302,1.15941236712269,1.09795095350216,1.04247568583423,0.993989343411092,0.953558636935387,0.922243640982866,0.900995334390025,0.890534587678576,0.891241331576982,0.90308934794425,0.925650922124334,0.958169551221959,0.999674020942901,1.04909837832656,1.10538077903801,1.16752983941565,1.23465994718594,1.30600323293798,1.3809068410663,1.4588224576565,1.53929276689726,1.62193757015536,1.70644095729701,1.79254009718054,1.88001576037523,1.96868446392979,2.05839203694966,2.14900838458975,2.24042324043835,2.33254272320049,2.4252865428264,2.51858572895069,2.61238077877888,2.70662014190994,2.80125897621977,2.89625812232113,2.99158325479509,3.08720417686126,3.18309423185819,3.27922981020668,3.37558993472362,3.47215591047791,3.56891102802044],[2.67283245515564,2.57929720545618,2.48626523420467,2.39379522005445,2.30195488580821,2.21082270592579,2.12048998530476,2.03106339598707,1.94266807747985,1.85545142755072,1.76958773185755,1.68528379849848,1.60278576928491,1.52238725788249,1.4444388888782,1.36935913640084,1.29764601690337,1.2298885827285,1.16677618533576,1.10910207258278,1.05775618558728,1.01370059134287,0.977921075358144,0.951351864648601,0.934778642840705,0.928736696729408,0.933430550492426,0.948700866097071,0.974050344732794,1.00871939349594,1.05178683940975,1.10226870421703,1.1591967205945,1.22167007730311,1.28888266282021,1.36013207419694,1.43481706150993,1.51242875559173,1.59253931866108,1.67479020244991,1.75888116794824,1.84456057198014,1.9316170542953,2.01987256657077,2.10917659903715,2.19940143372388,2.29043825662414,2.3821939778591,2.47458863023511,2.56755323793483,2.66102806643793,2.75496118141638,2.84930725823872,2.94402659508921,3.03908429190987,3.13444956476796,3.23009517116986,3.32599692657037,3.42213329610293,3.51848504857375,3.61503496217917],[2.65564109578641,2.56293812876001,2.47080433747273,2.37930584337035,2.28851884818948,2.19853149641772,2.1094461270587,2.02138199882783,1.93447858799404,1.8488995726187,1.76483762761673,1.68252015562194,1.60221605793236,1.52424358897078,1.44897920645561,1.37686708253301,1.30842851741514,1.24426982901918,1.18508633847059,1.13165889377137,1.08483827092985,1.04551245630853,1.0145533591283,0.992744045935679,0.980695142724263,0.9787671761891,0.987019456448255,1.00520128811578,1.03278838677236,1.06905288647198,1.11314702366592,1.16418148006164,1.2212865105047,1.28365219703897,1.35054995656776,1.42134000415362,1.49546970200992,1.57246680468996,1.65193040836292,1.73352135732338,1.81695308649357,1.90198337025873,1.98840714215247,2.076050381177,2.16476497740688,2.25442445640337,2.3449204361453,2.43615969798248,2.52806176670909,2.62055690999423,2.71358448196122,2.80709154873206,2.90103174494613,2.99536431964584,3.09005333766743,3.18506700900497,3.28037712375789,3.37595857443427,3.4717889507455,3.56784819474438,3.66411830635461],[2.64324786186602,2.55156088756725,2.46052162491344,2.3702047106464,2.28069596265571,2.19209438158529,2.10451454770013,2.01808948889418,1.93297410455725,1.84934923489118,1.76742646135456,1.68745370306862,1.60972162304658,1.53457075672887,1.46239909433475,1.39366955007807,1.328916294775,1.26874828954455,1.21384757112494,1.1649590677915,1.12286835084676,1.08836438614879,1.06218677300216,1.04496145596115,1.03713458535918,1.03891859731735,1.05026451694588,1.07086846029718,1.10021041758518,1.13761446310544,1.1823156813518,1.23352100674211,1.29045642226996,1.35239841817472,1.41869136361126,1.48875413983938,1.56207957306681,1.63822962257929,1.71682847133894,1.79755492369599,1.88013494393552,1.96433477572208,2.04995483238407,2.13682440048744,2.2247971193397,2.31374716119241,2.40356602356366,2.49415984518501,2.58544716384556,2.67735704394105,2.7698275116773,2.86280424551665,2.95623947809575,3.05009107332151,3.14432174868592,3.23889841812968,3.33379163516058,3.42897511953213,3.5244253537384,3.62012123799436,3.71604379434386],[2.63572043703846,2.54523228947918,2.45548215073793,2.36655398792797,2.27854404455191,2.19156294642617,2.1057381920204,2.02121708841938,1.93817019472071,1.85679532767855,1.77732216383921,1.70001742910455,1.62519058711415,1.55319980351197,1.4844577511047,1.41943650791432,1.35867037399996,1.30275491569611,1.25234003192246,1.2081145395784,1.17078005019129,1.14101319497678,1.11941783811779,1.10647253986779,1.10248204822875,1.10754316026593,1.12153334191946,1.14412509028612,1.17482228758541,1.21300972485491,1.25800547605238,1.30910769723616,1.36563104393103,1.42693143960505,1.49242035583763,1.56157090895793,1.63391826260951,1.70905648650771,1.78663349886295,1.86634521041646,1.94792957512063,2.03116095246409,2.11584498487011,2.20181406689693,2.28892340858544,2.37704765529755,2.4660780078377,2.55591978080806,2.64649033837479,2.73771735142551,2.82953732639834,2.92189436268094,3.01473910178839,3.10802783723651,3.20172175902626,3.29578631094423,3.39019064251446,3.48490714048133,3.57991102724118,3.67518001574728,3.77069401215921],[2.63310055229923,2.54399001279195,2.4557181947145,2.36837887984978,2.28207913648611,2.19694147665609,2.11310636706213,2.03073513417806,1.9500132953013,1.87115432781973,1.79440385231006,1.72004414241375,1.64839877411945,1.57983707551924,1.51477782260365,1.45369134224515,1.39709884588929,1.34556748320248,1.29969939544439,1.26011316364253,1.22741673927475,1.20217241630247,1.18485660954614,1.17581965107403,1.1752525339341,1.18316743751051,1.19939645315961,1.22360880943156,1.25534265648276,1.29404476423803,1.33911109267249,1.38992272633596,1.44587407883669,1.50639255126893,1.57095040825138,1.63907042309104,1.71032702092861,1.78434446931785,1.86079333937218,1.93938611898676,2.01987256657077,2.10203516903306,2.18568490833389,2.27065743457378,2.35680967662895,2.444016881799,2.53217005455502,2.62117375460915,2.71094421162047,2.80140771491498,2.89249923973229,2.98416127555441,3.07634282634189,3.16899855661877,3.26208806112259,3.3555752390887,3.44942775715797,3.54361658740312,3.63811560910112,3.73290126468031,3.82795226178678],[2.63540284361768,2.54784149748298,2.46122823899866,2.37566676800274,2.2912749160109,2.20818678759042,2.12655520432389,2.04655446670378,1.96838343148212,1.89226887135034,1.81846903455342,1.74727724696963,1.67902529043666,1.614086142086,1.55287546900008,1.49585105220084,1.44350909886932,1.39637626256918,1.35499624162387,1.31991021018968,1.29163118230829,1.27061373330342,1.25722209947463,1.25170106477368,1.25415457727654,1.26453622006325,1.28265349958592,1.3081850588308,1.34070738653007,1.3797262208295,1.42470789728723,1.47510702687661,1.5303884764624,1.59004309282391,1.65359764912302,1.72062004327897,1.79072093903028,1.86355295863295,1.93880833961941,2.01621574389397,2.09553670310832,2.17656201892393,2.2591083131117,2.34301483531763,2.42814057802811,2.51436171121317,2.6015693269569,2.68966747221866,2.77857144217299,2.86820630488942,2.95850562878551,3.04941038623923,3.14086800930123,3.23283157618828,3.32525910992635,3.41811297301542,3.51135934424895,3.6049677658188,3.6989107505779,3.79316344083522,3.88770331134425],[2.6426144461709,2.556763724074,2.47197701655383,2.38836764339161,2.30606366556295,2.22520993599688,2.14597038397292,2.06853052153419,1.99310013500279,1.91991608612923,1.84924509122166,1.78138626834189,1.71667313914637,1.65547464380953,1.59819458324884,1.54526876482881,1.49715903804011,1.45434343083565,1.41730181840159,1.38649705159228,1.3623522685795,1.34522612721183,1.33538869358811,1.33300134755843,1.33810396540113,1.35061165841698,1.37032167080634,1.39692917183052,1.43004920920252,1.46924142684937,1.51403434571347,1.56394681163703,1.61850525462825,1.6772563575342,1.73977542186298,1.80567110741357,1.87458736200204,1.9462033314655,2.02023192654632,2.09641757926866,2.17453358132182,2.25437927730139,2.33577729151466,2.41857089728318,2.5026215884178,2.58780687927914,2.67401833839525,2.76115984754663,2.84914607090148,2.93790111524588,3.02735736122986,3.11745444589367,3.20813837791512,3.29936076861587,3.39107816352132,3.48325146102614,3.57584540638579,3.66882815078602,3.76217086661805,3.85584741130394,3.9498340330815],[2.65469535114811,2.57070389553494,2.48789662760124,2.4063957988537,2.32633872251051,2.24787965935967,2.17119185565764,2.09646969574692,2.02393090205577,1.9538186731136,1.88640359375149,1.82198507942014,1.76089202983622,1.70348227224467,1.65014028584329,1.60127264201762,1.55730060848418,1.51864949751294,1.48573463559527,1.45894431645574,1.43862073863462,1.42504061020756,1.4183976353096,1.41878925953186,1.42620968797511,1.44055030637086,1.46160743576936,1.4890961713091,1.52266822338897,1.56193137801557,1.60646840639358,1.65585381355114,1.70966750070385,1.76750504216369,1.82898474164909,1.89375190991225,1.96148092200954,2.03187561590726,2.10466853200917,2.17961940272625,2.25651320666462,2.335158016701,2.41538280073244,2.49703527912306,2.57997990230472,2.66409598314464,2.7492759989711,2.83542406522016,2.922454574624,3.01029099122524,3.09886478616976,3.18811450140582,3.27798492754088,3.36842638279305,3.45939408096381,3.55084757748323,3.64275028373201,3.73506904096039,3.82777374616785,3.92083702325957,4.01423393365086],[2.67157950228644,2.58958097568499,2.5088886428501,2.42963265017879,2.3519582069857,2.27602724061027,2.20202011630062,2.13013736298512,2.06060131461129,1.99365753587981,1.92957585053373,1.86865073129131,1.81120074761142,1.75756671018222,1.70810811523549,1.6631975005771,1.62321240708618,1.58852482231507,1.55948828264393,1.53642321522955,1.51960155596422,1.50923208337269,1.50544813064884,1.50829926296696,1.51774808820807,1.5336726677658,1.55587417371857,1.58408870870196,1.61800173968684,1.65726347399025,1.70150369227187,1.75034493954975,1.80341343013742,1.86034743867714,1.92080326561186,1.98445906427363,2.05101691128337,2.12020351923181,2.1917699595431,2.26549070802512,2.34116226279614,2.41860152429732,2.49764407510877,2.57814245504753,2.65996449433279,2.74299174328026,2.82711801952029,2.9122480816182,2.99829642988839,3.0851862300474,3.17284835229588,3.26122051680951,3.35024653598088,3.43987564374709,3.53006190272027,3.620763680445,3.71194318682577,3.8035660655252,3.89560103288407,3.98801955862807,4.08079558328919],[2.69317656309422,2.61328798146589,2.53482704501671,2.45793047546553,2.38274973429781,2.30945239621222,2.23822350189708,2.1692668158893,2.10280588561869,2.0390847620113,1.97836820150869,1.92094112736825,1.86710709089995,1.81718545090009,1.77150699604519,1.73040778752036,1.69422111460848,1.66326764510684,1.63784411142616,1.61821117515836,1.60458140381892,1.59710849836683,1.59587895206047,1.60090715024442,1.6121345408426,1.62943298476858,1.65261184974752,1.6814279645587,1.71559729367765,1.75480715411926,1.79872794558327,1.84702363176444,1.899360516166,1.95541413630066,2.01487431829346,2.07744857760489,2.14286412697275,2.21086877510646,2.28123098672949,2.35373934174727,2.42820159026448,2.50444345854264,2.58230732313393,2.66165083833045,2.7423455761919,2.82427571836273,2.90733682386603,2.99143468614668,3.07648428495183,3.16240883341001,3.24913891726773,3.33661172115932,3.42477033563593,3.51356313817269,3.60294324129593,3.69286800117119,3.78329858035923,3.87419955890533,3.96553858842608,4.05728608436358,4.14941495206788],[2.71937424719296,2.64169488242905,2.56556181126228,2.49111681038103,2.41851576768036,2.34792974215032,2.27954592619443,2.21356842794281,2.15021876578658,2.0897359393704,2.03237591302688,1.97841032331703,1.9281242083857,1.88181256200465,1.83977554991746,1.80231230080406,1.76971330592986,1.74225162957041,1.72017333321276,1.70368772134211,1.69295818241545,1.68809447690672,1.68914727555608,1.69610556100559,1.70889719691666,1.72739259729373,1.75141107131956,1.7807291473709,1.8150900395991,1.85421342013464,1.89780477472341,1.94556380479246,1.99719154666599,2.05239607036011,2.11089677274498,2.17242738445747,2.23673786937502,2.30359541843872,2.37278473684559,2.4441078051025,2.51738326810172,2.5924455778423,2.66914398817854,2.74734147589723,2.82691364225028,2.90774763284166,2.98974110114101,3.07280123135347,3.15684382935915,3.24179248541891,3.3275778088685,3.41413673270852,3.50141188453818,3.58935101943802,3.67790650999945,3.76703488858968,3.85669643703097,3.94685481908861,4.03747675145239,4.12853170922473,4.21999166227016],[2.75004107713429,2.67465193304904,2.60092291312142,2.52899921649593,2.4590392583434,2.39121540458218,2.32571454571453,2.26273842615208,2.20250362592715,2.14524107252758,2.09119494459419,2.04062082056744,1.99378292960151,1.95095038568576,1.91239233538635,1.87837202957281,1.84913994022046,1.8249261781112,1.80593261020244,1.79232520197358,1.78422719031705,1.7817136985789,1.78480832044809,1.79348202605373,1.8076545064848,1.82719781410001,1.85194192373543,1.88168167497213,1.91618448040969,1.95519819967184,1.99845866533112,2.04569647652678,2.09664281915426,2.15103420431851,2.20861612445105,2.26914570317162,2.33239346150242,2.39814434434133,2.46619815370406,2.53636952559566,2.60848757087463,2.68239528117025,2.75794878151888,2.8350164935179,2.91347825726494,2.99322444741329,3.07415510825814,3.15617912461677,3.23921343904716,3.32318232132105,3.40801669270821,3.49365350525991,3.58003517466684,3.6671090642206,3.75482701677799,3.84314493129819,3.93202238040426,4.02142226544752,4.11131050567506,4.20165575828281,4.29242916635186],[2.78502942334348,2.71199325356967,2.64072450776788,2.57137014924815,2.50408924820242,2.43905339865315,2.37644692766323,2.31646681791873,2.25932225170228,2.2052336736853,2.15443126464366,2.1071527217059,2.06364025719385,2.02413676159237,1.98888112971795,1.95810282331308,1.93201583461037,1.91081231514341,1.89465622797969,1.88367745135433,1.8779667886465,1.87757230960029,1.88249735625214,1.89270040281467,1.90809678301006,1.92856212029892,1.95393714596206,1.98403348986072,2.01863998954925,2.05752908264402,2.10046291194214,2.14719886432602,2.19749436437991,2.25111083689809,2.30781682949894,2.36739034319164,2.42962045504139,2.49430833580656,2.56126777057624,2.63032528617871,2.7013199791268,2.77410312500241,2.84853763651925,2.92449742437657,3.00186670318715,3.08053927457876,3.16041781110358,3.24141315775377,3.32344366249494,3.40643454308112,3.49031729428928,3.5750291374104,3.66051251218349,3.74671461021124,3.83358694813211,3.92108497834559,4.00916773482052,4.09779751140103,4.18693957001567,4.27656187626012,4.36663485993505],[2.82417867444416,2.75354048184972,2.68476911208118,2.6180116854562,2.55342616599446,2.49118148379141,2.43145741998193,2.37444418447459,2.32034160958795,2.269357879486,2.22170771821704,2.17760997085915,2.1372845353825,2.10094863934966,2.06881250604911,2.04107451730337,2.01791605008365,1.99949623285096,1.98594692406046,1.97736824723803,1.97382501382093,1.97534432066543,1.98191452519658,1.99348568722858,2.00997143889787,2.03125212250601,2.05717893854307,2.08757878538396,2.12225945284388,2.16101485075535,2.2036300018799,2.2498855937992,2.29956195506561,2.35244238721912,2.40831583999798,2.46697895921007,2.52823756502137,2.59190763438537,2.65781586743423,2.72579991657223,2.79570835123859,2.8674004228892,2.94074568522768,3.01562351515987,3.09192257102037,3.16954021668525,3.24838193339164,3.32836073543497,3.40939660133361,3.49141592841431,3.57435101594474,3.6581395797844,3.74272429991646,3.82805240105007,3.91407526565154,4.00074807819553,4.08802949906005,4.1758813662729,4.26426842320992,4.3531580703192,4.44252013897351],[2.86731840136648,2.79910633886894,2.73285158356208,2.66869991929765,2.60680661744226,2.54733629939634,2.49046254814316,2.43636721066123,2.38523933110845,2.33727365709455,2.29266866983209,2.25162410519854,2.2143379580797,2.18100299726389,2.15180286168495,2.12690785814455,2.10647063082268,2.09062191703809,2.07946663389473,2.07308054907738,2.07150777079026,2.07475924529231,2.0828123793101,2.09561181731944,2.1130713121829,2.13507654472898,2.16148868452586,2.19214844725781,2.22688039559746,2.26549724735604,2.30780399064983,2.353601652944,2.40269062141222,2.45487345981493,2.50995720783945,2.56775518043809,2.62808830676371,2.6907860616611,2.75568704883394,2.82263929551627,2.89150031542002,2.96213699135532,3.03442532236947,3.10825007335615,3.18350435840757,3.26008918305642,3.33791296515697,3.41689104954262,3.49694522775607,3.57800327101687,3.65999848208833,3.74286926974085,3.82655874799432,3.91101436117708,3.99618753499067,4.08203335315824,4.16851025880683,4.25557977944948,4.34320627425647,4.43135670221008,4.52000040970172],[2.9142713974127,2.84849797976771,2.78476277332625,2.72320889148568,2.66398754306953,2.60725767667728,2.55318537430316,2.50194294869378,2.4537077009052,2.40866030075734,2.3669827644931,2.32885602182066,2.29445708901547,2.26395589539556,2.23751184561317,2.21527023683844,2.19735868378041,2.18388373045548,2.17492784043773,2.1705469527458,2.17076876636758,2.17559187349098,2.18498580383557,2.19889197686606,2.21722549346901,2.23987764216081,2.26671895364529,2.2976026152879,2.33236805430463,2.37084451276411,2.41285446446044,2.4582167580646,2.50674940761788,2.55827198623515,2.61260760898351,2.66958451476657,2.72903727428638,2.79080766220236,2.85474523738466,2.92070767680255,2.9885609072422,3.05817907573433,3.12944439513186,3.20224689635031,3.27648411382509,3.35206072604765,3.42888816879088,3.50688423490668,3.58597267139615,3.66608278179353,3.74714903972686,3.82911071776319,3.91191153425686,3.99549931983661,4.07982570433809,4.16484582436622,4.2505180512149,4.33680373854601,4.42366698900693,4.51107443882193,4.59899505930976],[2.96485650143039,2.90152003586371,2.84029276008002,2.78131397138249,2.72472968418004,2.67069210061667,2.61935884153751,2.57089190460921,2.52545632106945,2.48321849100599,2.44434418986331,2.40899625631983,2.37733199351866,2.34950034089205,2.32563890066715,2.30587092889883,2.29030242221092,2.27901944476778,2.27208584214463,2.26954147759377,2.27140110129474,2.27765392617215,2.28826393841529,2.30317092207598,2.32229213079249,2.34552450108397,2.37274727470703,2.40382488423639,2.43860995620836,2.47694629795702,2.51867175459753,2.56362084791255,2.61162713577116,2.66252525635734,2.71615264394054,2.77235092108298,2.8309669856913,2.89185382039247,2.95487105690975,3.01988533017521,3.08677045661835,3.1554074691387,3.22568453831752,3.2974968059401,3.37074615324703,3.44534092276426,3.52119560923822,3.59823053221482,3.67637150018975,3.75554947402532,3.83570023545849,3.91676406498282,3.99868543213707,4.08141270023378,4.16489784677537,4.24909620019582,4.33396619310338,4.41946913185367,4.50556898203233,4.59223216925019,4.67942739453672],[3.01889113810083,2.95797728686845,2.8992336152086,2.84279466044973,2.78880035273366,2.73739535568637,2.68872818619868,2.64295009131561,2.60021366643419,2.56067120830399,2.52447280891733,2.49176421213743,2.46268447325572,2.43736348145711,2.41591942470494,2.39845629365903,2.38506153348682,2.37580395749406,2.37073203259465,2.36987263292699,2.37323033484367,2.38078729586415,2.39250372498646,2.40831891575846,2.42815278059803,2.45190779839941,2.47947126980285,2.51071776663641,2.54551166362082,2.58370965001015,2.6251631342607,2.66972047364652,2.71722898065903,2.76753667712991,2.82049378392954,2.8759539480164,2.93377521924176,2.993820796735,3.05595956924971,3.12006647602496,3.18602271503841,3.25371582450998,3.3230396616039,3.39389429984473,3.46618586409391,3.53982631923805,3.61473322615879,3.69082947617771,3.76804301304379,3.84630654967517,3.9255572852781,4.00573662712939,4.08678992020043,4.16866618689693,4.25131787845883,4.33470063898646,4.41877308260512,4.50349658393137,4.58883508173996,4.67475489553552,4.76122455459299]],"type":"surface","frame":null}],"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.2,"selected":{"opacity":1},"debounce":0},"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot","plotly_sunburstclick"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}</script> Hill-climbing algorithms, such as Newton-Raphson, find the optimum *numerically* by investigating the shape of `\(f\)` at its current location, taking a step uphill, and then repeating. When no step leads uphill, the algorithm has found the optimum. Under meaningful restrictions (e.g., no local optima), these algorithms find the *global* optimum. --- # Analytical Optimization Remember that we simply need to minimize the function `\(f(\alpha, \beta) = \displaystyle \sqrt{\frac{\sum_{i = 1}^n [y_i - (\alpha + \beta x_i)]^2}{n}}\)`. This is equivalent to minimizing `\(h(\alpha, \beta) = \sum_{i = 1}^n(y_i - \alpha - \beta x_i)^2\)`. We sometimes refer to this quantity as the SSR or "sum of squared residuals." To minimize `\(h(\alpha, \beta)\)`, remember that we need to solve for `\(\frac{\partial h}{\partial \alpha} = 0\)` and `\(\frac{\partial h}{\partial \beta} = 0\)` (i.e., the first-order conditions). --- # Analytical Optimization Using the chain rule, we have the partial derivatives `\(\frac{\partial h}{\partial \alpha} = \sum_{i = 1}^n [2 \times (y_i - \alpha - \beta x_i) \times (-1)] = -2 \sum_{i = 1}^n(y_i - \alpha + \beta x_i)\)` and `\(\frac{\partial h}{\partial \beta} = \sum_{i = 1}^n 2 \times (y_i - \alpha - \beta x_i) \times (-x_i) = -2 \sum_{i = 1}^n(y_i - \alpha - \beta x_i)x_i\)` and the two first-order conditions `\(-2 \sum_{i = 1}^n(y_i - \hat{\alpha} + \hat{\beta} x_i) = 0\)` and `\(-2 \sum_{i = 1}^n(y_i - \hat{\alpha} + \hat{\beta} x_i)x_i = 0\)` --- class: center # Analytical Optimization ### 1st Order Condition `\(-2 \sum_{i = 1}^n(y_i - \hat{\alpha} - \hat{\beta} x_i) = 0\)` `\(\sum_{i = 1}^n(y_i - \hat{\alpha} - \hat{\beta} x_i) = 0\)` (divide both sizes by -2) `\(\sum_{i = 1}^n y_i - \sum_{i = 1}^n \hat{\alpha} - \sum_{i = 1}^n \hat{\beta} x_i = 0\)` (distribute the sum) `\(\sum_{i = 1}^n y_i - n \hat{\alpha} - \hat{\beta}\sum_{i = 1}^n x_i = 0\)` (move constant `\(\beta\)` in front and realize that `\(\sum_{i = 1}^n \hat{\alpha} = n\hat{\alpha}\)`) `\(\sum_{i = 1}^n y_i = n \hat{\alpha} + \hat{\beta}\sum_{i = 1}^n x_i\)` (rearrange) `\(\frac{\sum_{i = 1}^n y_i}{n} = \hat{\alpha} + \hat{\beta} \frac{\sum_{i = 1}^n x_i}{n}\)` (divide both sides by `\(n\)`) `\(\overline{y} = \hat{\alpha} + \hat{\beta} \overline{x}\)` (recognize the average of `\(y\)` and of `\(x\)`) **The 1st first-order condition implies that the regression line `\(\hat{y} = \hat{\alpha} + \hat{\beta}x\)` equals `\(\overline{y}\)` when `\(x = \overline{x}\)`. Thus, the regression line must go through the point `\((\overline{x}, \overline{y})\)` or "the point of averages."** --- --- class: center # Analytical Optimization ### 1st First-Order Condition `\(-2 \sum_{i = 1}^n(y_i - \hat{\alpha} - \hat{\beta} x_i) = 0\)` ... `\(\overline{y} = \hat{\alpha} + \hat{\beta} \overline{x}\)` **The 1st first-order condition implies that the regression line `\(\hat{y} = \hat{\alpha} + \hat{\beta}x\)` equals `\(\overline{y}\)` when `\(x = \overline{x}\)`. Thus, the regression line must go through the point `\((\overline{x}, \overline{y})\)` or "the point of averages."** --- class: center # Analytical Optimization ### 2nd First-Order Condition [Do some similar, but a bit more complicated math.] `\(\hat{\beta} = r \times \dfrac{\text{SD of } y}{\text{SD of }x} = \dfrac{\sum_{i = 1}^n (x_i - \overline{x})(y_i - \overline{y})}{\sum_{i = 1}^n (x_i - \overline{x})^2} = \dfrac{\sum_{i = 1}^n x_i y_i - n \overline{x} \overline{y}}{\sum_{i = 1}^n x_i^2 - n \overline{x}^2}\)` --- class: center # The Best Line `\(\hat{\beta} = r \times \dfrac{\text{SD of } y}{\text{SD of }x}\)` `\(\hat{\alpha} = \overline{y} - \hat{\beta}\overline{x}\)` --- # The RMS of the Residuals Sometimes the RMS of the residuals is called the "RMS error (of the regression)," the "standard error of the regression," or denoted as `\(\hat{\sigma}\)`. Here's how we can think of a regression model as a **description** of a scatterplot: The points/observations `\(y_i\)` are about `\(\hat{\alpha} + \hat{\beta} x_i\)`, give or take the RMS-of-the-residuals or so. <!-- -->