</p> <p class="has-line-data" data-line-end="2" data-line-start="1">The goal of a regression task is to build models based on features to predict a target quantity, that is, a numeric value. After a regression model is applied to a test set, the next step is to evaluate the model performance by checking the error between the regression output and the true value. A certain set of metrics is often used to evaluate the regression model such as mean squared error (MSE), root mean squared error (RMSE), mean absolute error (MAE) and R squared, etc. What are the differences among those metrics? How does one choose a metric correctly? How does the metric translate back to the business world? In this blog, we will go through various regression metrics and address the questions above.</p> <p class="has-line-data" data-line-end="4" data-line-start="3">After we know which metric to use, we need to figure out how to compute them. Although open source packages provide abundant APIs to compute various kinds of model metrics, when the dataset is large enough, it is not that easy to scale up the computation. If the dataset resides in Oracle Database, we can leverage Oracle Machine Learning tools to compute such metrics without moving the data out of the database.</p> <p class="has-line-data" data-line-end="7" data-line-start="5">The newly released Oracle Machine Learning for Python <a href="https://blogs.oracle.com/machinelearning/introducing-oracle-machine-learning-for-python-v2">OML4Py</a> API brings benefits that are similar to those in <a href="https://www.oracle.com/database/technologies/datawarehouse-bigdata/oml4r.html">OML4R</a>: transparency layer, in-database algorithms, and embedded Python execution. OML4Py also introduced automated machine learning.<br />In this blog, we will demonstrate how to compute regression metrics in a scalable way using OML4Py.</p> <p class="has-line-data" data-line-end="15" data-line-start="12">We use the dataset customer insurance lifetime value for our demonstration, an Oracle-produced dataset. The use case involves an insurance company targeting customers likely to buy insurance based on their lifetime value, demographic, and financial features for each customer. The following is a glimpse into this dataset with a subset of the columns.<br /><img alt src="https://i2.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/2fa0224ca07997e7b1b9e665e62695f7/data11.png?w=1440&ssl=1" style="width: 769px; height: 195px;" data-recalc-dims="1" data-lazy-src="https://i2.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/2fa0224ca07997e7b1b9e665e62695f7/data11.png?w=1440&is-pending-load=1#038;ssl=1" srcset="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class=" jetpack-lazy-image"><noscript><img alt="" src="https://i2.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/2fa0224ca07997e7b1b9e665e62695f7/data11.png?w=1440&ssl=1" style="width: 769px; height: 195px;" data-recalc-dims="1"/></noscript></p> <p class="has-line-data" data-line-end="15" data-line-start="12"><img alt src="https://i2.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/d515a55f1ca481a4371ee0fd17f7b0e6/data12.PNG?w=1440&ssl=1" style="width: 818px; height: 185px;" data-recalc-dims="1" data-lazy-src="https://i2.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/d515a55f1ca481a4371ee0fd17f7b0e6/data12.PNG?w=1440&is-pending-load=1#038;ssl=1" srcset="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class=" jetpack-lazy-image"><noscript><img alt="" src="https://i2.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/d515a55f1ca481a4371ee0fd17f7b0e6/data12.PNG?w=1440&ssl=1" style="width: 818px; height: 185px;" data-recalc-dims="1"/></noscript></p> <p class="has-line-data" data-line-end="17" data-line-start="16">Based on the column names, we can see that the dataset contains user demographic features such as state, region, gender, marital status, and some financial features like income, credit card limits.</p> <p class="has-line-data" data-line-end="19" data-line-start="18">The main business problem here is to predict the income of each customer, so that we can use it as a guideline for marketing various products to each individual. The column ‘INCOME’ contains this information. This is a typical regression problem and we can use all features columns provided in this dataset to build a model.</p> <p class="has-line-data" data-line-end="22" data-line-start="21">In this blog, we show how to build a regression model using the in-database Generalized Linear Model algorithm provided in Oracle Machine Learning using OML4Py.</p> <pre> <code class="has-line-data" data-line-end="29" data-line-start="23">%python <span class="hljs-keyword">import</span> pandas <span class="hljs-keyword">as</span> pd <span class="hljs-keyword">import</span> numpy <span class="hljs-keyword">as</span> np <span class="hljs-keyword">import</span> matplotlib.pyplot <span class="hljs-keyword">as</span> plt <span class="hljs-keyword">import</span> oml </code></pre> <p class="has-line-data" data-line-end="30" data-line-start="29">First, we load the data by creating proxy object of the underlying table.</p> <pre> <code class="has-line-data" data-line-end="34" data-line-start="31">CUST_DF = oml.sync(table =<span class="hljs-string">'CUSTOMER_INSURANCE_LTV'</span>) CUST_DF = CUST_DF.drop([<span class="hljs-string">'FIRST_NAME'</span>, <span class="hljs-string">'STATE'</span>, <span class="hljs-string">'PROFESSION'</span>, <span class="hljs-string">'LAST_NAME'</span>]) </code></pre> <p class="has-line-data" data-line-end="35" data-line-start="34">We split the entire dataset into train and test datasets.</p> <pre> <code class="has-line-data" data-line-end="43" data-line-start="36">target = <span class="hljs-string">'INCOME'</span> TRAIN, TEST = CUST_DF.split(ratio = (<span class="hljs-number">0.8</span>,<span class="hljs-number">0.2</span>)) TRAIN_X = TRAIN.drop(target) TRAIN_Y = TRAIN[target] TEST_X = TEST TEST_Y = TEST[target] </code></pre> <p class="has-line-data" data-line-end="44" data-line-start="43">Now, we are ready to fit the regression model. Note that we also leverage the auto-generation of features provided in generalized linear models to boost the performance.</p> <pre> <code class="has-line-data" data-line-end="57" data-line-start="45">settings = {'GLMS_FTR_GENERATION': GLMS_FTR_GENERATION_ENABLE', 'GLMS_FTR_SELECTION':'GLMS_FTR_SELECTION_ENABLE'} try: oml.drop(model = 'GLM_REGRESSION_MODEL') except: print('No such model') glm_mod = oml.glm("regression", **settings) glm_mod.fit(TRAIN_X, TRAIN_Y, model_name = 'GLM_REGRESSION_MODEL', case_id = 'CUSTOMER_ID') </code></pre> <p class="has-line-data" data-line-end="58" data-line-start="57">We can obtain the predictions along with other columns using the following code:</p> <pre> <code class="has-line-data" data-line-end="62" data-line-start="59">target = <span class="hljs-string">'INCOME'</span> RES_DF = glm_mod.predict(TEST.drop(target), supplemental_cols = TEST) </code></pre> <p class="has-line-data" data-line-end="63" data-line-start="62">The result looks like</p> <p class="has-line-data" data-line-end="66" data-line-start="64"><img alt src="https://i2.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/9762b4fab77ec6f56195b560ad06a711/pred_res.png?w=1440&ssl=1" style="width: 472px; height: 278px;" data-recalc-dims="1" data-lazy-src="https://i2.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/9762b4fab77ec6f56195b560ad06a711/pred_res.png?w=1440&is-pending-load=1#038;ssl=1" srcset="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class=" jetpack-lazy-image"><noscript><img alt="" src="https://i2.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/9762b4fab77ec6f56195b560ad06a711/pred_res.png?w=1440&ssl=1" style="width: 472px; height: 278px;" data-recalc-dims="1"/></noscript><br />Next, we will discuss several popular metrics for evaluating the performance of the regression.</p> <p class="has-line-data" data-line-end="69" data-line-start="68">Mean squared error is one of the most popular metrics for evaluation of regression models. This metric takes the square of the regression error, sums them up and then takes the mean. Why take the square of the regression error? It is because the error can be positive or negative, to avoid summing them up and having them cancel each other, the square is taken first.</p> <p class="has-line-data" data-line-end="71" data-line-start="70">The following code computes the mean squared error using the OML4Py transparency layer.</p> <p class="has-line-data" data-line-end="73" data-line-start="72">We first add a column of the error</p> <pre> <code class="has-line-data" data-line-end="77" data-line-start="75">RES_DF = RES_DF.concat({<span class="hljs-string">"ERROR"</span>: RES_DF[target] - RES_DF[<span class="hljs-string">'PREDICTION'</span>]}) </code></pre> <p class="has-line-data" data-line-end="78" data-line-start="77">Then we add a column of the square of the error</p> <pre> <code class="has-line-data" data-line-end="82" data-line-start="79">RES_DF = RES_DF.concat({<span class="hljs-string">'ERROR_SQ'</span>: RES_DF[<span class="hljs-string">'ERROR'</span>]*RES_DF[<span class="hljs-string">'ERROR'</span>]}) print(<span class="hljs-string">'Mean squared error (MSE) '</span>) </code></pre> <p class="has-line-data" data-line-end="83" data-line-start="82">Using the mean() function provided by OML4Py</p> <pre> <code class="has-line-data" data-line-end="86" data-line-start="84">print(RES_DF[<span class="hljs-string">'ERROR_SQ'</span>].mean()) </code></pre> <p class="has-line-data" data-line-end="88" data-line-start="86">We obtain the MSE<br />1264942.64</p> <p class="has-line-data" data-line-end="90" data-line-start="89">One drawback of MSE is that the scale of the original data is changed after the square is taken. In our example, we want to get a sense of how far away the income prediction is from the true value in the unit of US dollar. However, mean squared error gives us a unit of dollar^2.</p> <p class="has-line-data" data-line-end="92" data-line-start="91">To resolve this problem, we can take the square root of the MSE. This leads to root mean squared error (RMSE).</p> <pre> <code class="has-line-data" data-line-end="97" data-line-start="93">print(<span class="hljs-string">'RMSE'</span>) <span class="hljs-keyword">import</span> math print(math.sqrt(RES_DF[<span class="hljs-string">'ERROR_SQ'</span>].mean())) </code></pre> <p class="has-line-data" data-line-end="100" data-line-start="97">We obtain root mean squared error as<br />1124.7</p> <p class="has-line-data" data-line-end="100" data-line-start="97">Now the number makes more sense since the unit remains the same. We know on average, our prediction error is around $1124.00. It may be considered acceptable since the income in our dataset has mean value $64895.38 and standard deviation $6552.95.</p> <p class="has-line-data" data-line-end="103" data-line-start="102">Besides MSE and RSME, there is another popular metric called mean absolute error (MAE). Instead of taking square, this metric takes the absolute value of the regression error to avoid the cancellation. One benefit that comes with this metric is that MAE keeps the original unit.</p> <p class="has-line-data" data-line-end="105" data-line-start="104">Let us first compute MAE using the following code.</p> <pre> <code class="has-line-data" data-line-end="109" data-line-start="106">print(<span class="hljs-string">'MAE'</span>) print(abs(RES_DF[<span class="hljs-string">'ERROR'</span>]).mean()) </code></pre> <p class="has-line-data" data-line-end="112" data-line-start="109">We obtain the following number<br />476.9</p> <p class="has-line-data" data-line-end="112" data-line-start="109">This indicates that on average, our prediction error is $476.9. This is only one half of the RMSE value. Which is closer to reality?</p> <p class="has-line-data" data-line-end="114" data-line-start="113">Let us get more sense of the data by visualization. Using matplotlib, we can obtain the following residual plot, which plots the original target value (INCOME) in the x-axis and the residual error, i.e. the difference between the prediction and the target value on the y-axis.</p> <pre> <code class="has-line-data" data-line-end="134" data-line-start="116"><span class="hljs-keyword">import</span> numpy <span class="hljs-keyword">as</span> np <span class="hljs-keyword">import</span> matplotlib.pyplot <span class="hljs-keyword">as</span> plt plt.style.use(<span class="hljs-string">'seaborn'</span>) plt.figure(figsize=[<span class="hljs-number">9</span>,<span class="hljs-number">7</span>]) target = <span class="hljs-string">'INCOME'</span> x = np.matrix(RES_DF[[<span class="hljs-string">'PREDICTION'</span>]].pull()) y = np.matrix(RES_DF[[target]].pull()) plt.plot(x, y-x, <span class="hljs-string">'.'</span>) plt.hlines(y=<span class="hljs-number">0</span>, xmin= min(x), xmax= max(x), colors=<span class="hljs-string">'black'</span>, linestyles=<span class="hljs-string">'solid'</span>, alpha=<span class="hljs-number">0.8</span>) plt.xlabel(<span class="hljs-string">'PREDICTION'</span>) plt.ylabel(<span class="hljs-string">'RESIDUAL'</span>) plt.title(<span class="hljs-string">'Prediction vs. residuals'</span>) plt.grid(<span class="hljs-keyword">True</span>) plt.show() </code></pre> <p class="has-line-data" data-line-end="135" data-line-start="134"><img alt src="https://i1.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/8e07a7b046ea70d81a1a04a59a2e69f5/prediction_residual.png?w=1440&ssl=1" style="width: 648px; height: 504px;" data-recalc-dims="1" data-lazy-src="https://i1.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/8e07a7b046ea70d81a1a04a59a2e69f5/prediction_residual.png?w=1440&is-pending-load=1#038;ssl=1" srcset="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class=" jetpack-lazy-image"><noscript><img alt="" src="https://i1.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/8e07a7b046ea70d81a1a04a59a2e69f5/prediction_residual.png?w=1440&ssl=1" style="width: 648px; height: 504px;" data-recalc-dims="1"/></noscript></p> <p class="has-line-data" data-line-end="137" data-line-start="136">The plot indicates that the majority of the residual error lies within the boundary of $5000. The points beyond that can be viewed as outliers. There are only 16 such data points in this dataset. Let us do an experiment by removing the outliers and see how the error metrics change.</p> <pre> <code class="has-line-data" data-line-end="141" data-line-start="139">RES_DF = RES_DF [RES_DF [<span class="hljs-string">'ERROR'</span>] < <span class="hljs-number">5000</span>][[<span class="hljs-string">'INCOME'</span>,<span class="hljs-string">'PREDICTION'</span>, <span class="hljs-string">'CUSTOMER_ID'</span>]] </code></pre> <p class="has-line-data" data-line-end="142" data-line-start="141">The metrics are obtained for the data without outliers. For better comparison, we list the numbers in the following table:</p> <p class="has-line-data" data-line-end="144" data-line-start="143"><img alt src="https://i1.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/1eb4c2c377ae26e86d3ee900ad2c6179/table.PNG?w=1440&ssl=1" style="width: 454px; height: 168px;" data-recalc-dims="1" data-lazy-src="https://i1.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/1eb4c2c377ae26e86d3ee900ad2c6179/table.PNG?w=1440&is-pending-load=1#038;ssl=1" srcset="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class=" jetpack-lazy-image"><noscript><img alt="" src="https://i1.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/1eb4c2c377ae26e86d3ee900ad2c6179/table.PNG?w=1440&ssl=1" style="width: 454px; height: 168px;" data-recalc-dims="1"/></noscript></p> <p class="has-line-data" data-line-end="144" data-line-start="143">With only 16 data points removed, the MSE changes more than one half from the original value. The RMSE drops 30%. But MAE changes a little, around 11%.</p> <p class="has-line-data" data-line-end="149" data-line-start="148">This illustrates the impact of the outliers to MSE and RMSE. For both MSE and RMSE, they require the error to be squared first and that amplifies the impact of large errors. Especially for MSE, as a sum of squared errors, its value is very sensitive to large prediction errors. While MAE takes only the absolute value so it does not have this impact. Even if there are only a few outliers, the large absolute value can be averaged out.</p> <p class="has-line-data" data-line-end="153" data-line-start="152">How do we choose between MSE, RMSE, and MAE? This depends on the business requirement. In practice, if the business goal has an emphasis on outliers, we should use MSE and RMSE.</p> <p class="has-line-data" data-line-end="155" data-line-start="154">Think about the use case for managing a cloud service. We want to build a regression model to predict the size of memory needed for each customer’s application and allocate those resources for the coming weeks. If we only focus on the MAE, we may neglect the impact of some large regression errors. Then the reality is that some customers will actually experience poor cloud performance. Sometimes, one such event can damage the reputation of a business. In this case, we need to pay attention to any large regression errors and keep improving the model. Therefore, this case requires the use of MSE or RMSE, which is sensitive to the impact of the large errors.</p> <p class="has-line-data" data-line-end="157" data-line-start="156">As for the comparison of MSE and RMSE in this case, MSE is more sensitive to large errors than RMSE. So if the cloud service management wants to highlight the outliers as much as possible, then MSE is preferred. However, the drawback is the unit. Since the quantity to predict is size of memory, which is often in the unit of MB, it is more convenient to use RMSE since it keeps the unit MB.</p> <p class="has-line-data" data-line-end="159" data-line-start="158">If the business goal does not care about outliers and only focuses on the average, MAE is a better choice. For example, consider the use case of predicting customer income, where our goal is to choose customers with high income, such as targeting customers with a yearly income above $100,000. In this case, even if the model produces some large error such as predicting a customer with a yearly income around $100,000 to be $200,000, that is still a useful prediction since it lies above our threshold. So in this case, we can use MAE.</p> <p class="has-line-data" data-line-end="162" data-line-start="161">Another set of metrics focus on how much the regression model explains the variation of the target value using the features.</p> <p class="has-line-data" data-line-end="164" data-line-start="163">Think about an extreme situation. Instead of building a regression model, we just use the mean value of the income to predict all the income of the customers. What would happen? The mean income can predict some of the people with income in the middle well but definitely not the people with lower and higher income. This variation of the income requires explanation, but the constant mean income cannot provide that. A decent regression model should provide a good estimate of high income or low income by leveraging certain features and that can provide a good description of variation of the income. Therefore, R squared is used to measure how the model explains such variation.</p> <p class="has-line-data" data-line-end="166" data-line-start="165">The benefit of R squared and Adjusted R squared is that the scale does not depend on the actual dataset. For any dataset, no matter where the quantity rests, whether in the range of hundreds or millions, the measure is always within [0, 1]. This is a nice property to evaluate he model performance.</p> <p class="has-line-data" data-line-end="168" data-line-start="167">In OML4Py, the R squared can be computed using the score function as follows</p> <pre> <code class="has-line-data" data-line-end="173" data-line-start="170">print(<span class="hljs-string">'R squared'</span>) Rsq = glm_mod.score(TEST_X, TEST_Y) </code></pre> <p class="has-line-data" data-line-end="174" data-line-start="173">In this use case, we obtain</p> <pre> <code class="has-line-data" data-line-end="178" data-line-start="175">R squared <span class="hljs-number">0.97023</span> </code></pre> <p class="has-line-data" data-line-end="179" data-line-start="178">This means that our regression model can explain 97% of the variation of the income. The model did a good job!</p> <p class="has-line-data" data-line-end="181" data-line-start="180">Besides the R squared, people noticed that if we add more features, the R squared usually increases. But a high number of features might lead to overfitting, which means that the model tends to fit the noise of the data and will not generalize well on a new dataset. To evaluate the model more fairly, the adjusted R squared was created. This metric will decrease as the number of features increase. We can obtain the adjusted R squared metric based on the R squared we obtain.</p> <pre> <code class="has-line-data" data-line-end="188" data-line-start="183">k = len(glm_mod.coef) n = TEST_X.shape[<span class="hljs-number">0</span>] print(<span class="hljs-string">'Adjusted R^2'</span>) print(<span class="hljs-number">1</span> - ( <span class="hljs-number">1</span> - Rsq)*(n-<span class="hljs-number">1</span>)/(n - k -<span class="hljs-number">1</span>)) </code></pre> <p class="has-line-data" data-line-end="189" data-line-start="188">In this case, we obtain</p> <pre> <code class="has-line-data" data-line-end="193" data-line-start="190">Adjusted R^<span class="hljs-number">2</span> <span class="hljs-number">0.969</span> </code></pre> <p class="has-line-data" data-line-end="194" data-line-start="193">The value drops a little but not too much. This is because we are using a reasonable number of features and there is no overfitting issue.</p> <p class="has-line-data" data-line-end="197" data-line-start="196">We provide the following function to compute all the metrics given the input as the test dataset, regression model, and target column name.</p> <pre> <code class="has-line-data" data-line-end="230" data-line-start="198"><span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">regr_stats</span><span class="hljs-params">(TEST, model, target)</span>:</span> RES_DF = model.predict(TEST.drop(target), supplemental_cols = TEST) RES_DF = RES_DF.concat({<span class="hljs-string">"ERROR"</span>: RES_DF[target] - RES_DF[<span class="hljs-string">'PREDICTION'</span>]}) RES_DF = RES_DF.concat({<span class="hljs-string">'ERROR_SQ'</span>: RES_DF[<span class="hljs-string">'ERROR'</span>]*RES_DF[<span class="hljs-string">'ERROR'</span>]}) mse = RES_DF[<span class="hljs-string">'ERROR_SQ'</span>].mean() print(<span class="hljs-string">"MSE"</span>) print(mse) <span class="hljs-keyword">import</span> math rmse = math.sqrt(RES_DF[<span class="hljs-string">'ERROR_SQ'</span>].mean()) print(<span class="hljs-string">"RMSE"</span>) print(rmse) mae = abs(RES_DF[<span class="hljs-string">'ERROR'</span>]).mean() print(<span class="hljs-string">"MAE"</span>) print(mae) Rsq = model.score(TEST.drop(target), TEST[target]) print(<span class="hljs-string">'R squared'</span>) print(Rsq) k = len(model.coef) n = TEST_X.shape[<span class="hljs-number">0</span>] adjR2 = <span class="hljs-number">1</span> - ( <span class="hljs-number">1</span> - Rsq)*(n-<span class="hljs-number">1</span>)/(n - k -<span class="hljs-number">1</span>) print(<span class="hljs-string">'Adjusted R^2'</span>) print(adjR2) print(<span class="hljs-string">"%table Measure t Value n"</span> + <span class="hljs-string">"Mean Squared Error (MSE) t"</span>+ str(np.round(mse, <span class="hljs-number">4</span>)) + <span class="hljs-string">"n"</span>+ <span class="hljs-string">"Root Mean Squared Error (RMSE) t"</span> + str(np.round(rmse,<span class="hljs-number">4</span>)) + <span class="hljs-string">" n"</span>+ <span class="hljs-string">"Mean Absolute Error (MAE) t"</span> + str(np.round(mae,<span class="hljs-number">4</span>)) + <span class="hljs-string">"n"</span> + <span class="hljs-string">"R squared t"</span> + str(np.round(Rsq,<span class="hljs-number">4</span>)) + <span class="hljs-string">"n"</span> + <span class="hljs-string">"Adjusted R squared t"</span> + str(np.round(adjR2,<span class="hljs-number">4</span>)) + <span class="hljs-string">"n"</span>) <span class="hljs-keyword">return</span> RES_DF </code></pre> <p class="has-line-data" data-line-end="231" data-line-start="230">Applying the function as follows, we print out the metrics in the notebook.</p> <pre> <code class="has-line-data" data-line-end="235" data-line-start="233">_ = regr_stats(TEST, glm_mod, target) </code></pre> <p class="has-line-data" data-line-end="236" data-line-start="235"><img alt src="https://i2.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/f9504f450271a55d001cd82d16c20c77/regr_output.png?w=1440&ssl=1" style="width: 791px; height: 198px;" data-recalc-dims="1" data-lazy-src="https://i2.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/f9504f450271a55d001cd82d16c20c77/regr_output.png?w=1440&is-pending-load=1#038;ssl=1" srcset="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class=" jetpack-lazy-image"><noscript><img alt="" src="https://i2.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/f9504f450271a55d001cd82d16c20c77/regr_output.png?w=1440&ssl=1" style="width: 791px; height: 198px;" data-recalc-dims="1"/></noscript></p> <p class="has-line-data" data-line-end="238" data-line-start="237">In this blog, we discussed popular metrics for regression and presented how to compute those metrics using OML4Py. We also discussed the pros and cons of using those metrics and the interpretation of each metric and how to explain the impact of the predictions generated by the model.</p> </p></div> <p><br /> <br /><a href="https://blogs.oracle.com/machinelearning/metrics-for-regression-using-oml4py"> Source link </a></p> <div class="post-views post-1831 entry-meta"> <span class="post-views-icon dashicons dashicons-chart-bar"></span> <span class="post-views-label">Post Views:</span> <span class="post-views-count">50</span> </div><!-- AddThis Advanced Settings above via filter on the_content --><!-- AddThis Advanced Settings below via filter on the_content --><!-- AddThis Advanced Settings generic via filter on the_content --><!-- AddThis Share Buttons above via filter on the_content --><!-- AddThis Share Buttons below via filter on the_content --><div class="at-below-post addthis_tool" data-url="https://machinelearningmastery.in/2021/08/16/metrics-for-regression-using-oml4py/"></div><!-- AddThis Share Buttons generic via filter on the_content --><div class="sharedaddy sd-sharing-enabled"><div class="robots-nocontent sd-block sd-social sd-social-official sd-sharing"><h3 class="sd-title">Share this:</h3><div class="sd-content"><ul><li class="share-twitter"><a href="https://twitter.com/share" class="twitter-share-button" data-url="https://machinelearningmastery.in/2021/08/16/metrics-for-regression-using-oml4py/" data-text="Metrics for Regression Using OML4Py" data-via="sitworld" >Tweet</a></li><li class="share-facebook"><div class="fb-share-button" data-href="https://machinelearningmastery.in/2021/08/16/metrics-for-regression-using-oml4py/" data-layout="button_count"></div></li><li class="share-linkedin"><div class="linkedin_button"><script type="in/share" data-url="https://machinelearningmastery.in/2021/08/16/metrics-for-regression-using-oml4py/" data-counter="right"></script></div></li><li class="share-reddit"><div class="reddit_button"><iframe src="https://www.reddit.com/static/button/button1.html?newwindow=true&width=120&url=https%3A%2F%2Fmachinelearningmastery.in%2F2021%2F08%2F16%2Fmetrics-for-regression-using-oml4py%2F&title=Metrics%20for%20Regression%20Using%20OML4Py" height="22" width="120" scrolling="no" frameborder="0"></iframe></div></li><li class="share-telegram"><a rel="nofollow noopener noreferrer" data-shared="" class="share-telegram sd-button" href="https://machinelearningmastery.in/2021/08/16/metrics-for-regression-using-oml4py/?share=telegram" target="_blank" title="Click to share on Telegram"><span>Telegram</span></a></li><li class="share-jetpack-whatsapp"><a rel="nofollow noopener noreferrer" data-shared="" class="share-jetpack-whatsapp sd-button" href="https://machinelearningmastery.in/2021/08/16/metrics-for-regression-using-oml4py/?share=jetpack-whatsapp" target="_blank" title="Click to share on WhatsApp"><span>WhatsApp</span></a></li><li class="share-print"><a rel="nofollow noopener noreferrer" data-shared="" class="share-print sd-button" href="https://machinelearningmastery.in/2021/08/16/metrics-for-regression-using-oml4py/#print" target="_blank" title="Click to print"><span>Print</span></a></li><li class="share-tumblr"><a class="tumblr-share-button" target="_blank" href="https://www.tumblr.com/share" data-title="Metrics for Regression Using OML4Py" data-content="https://machinelearningmastery.in/2021/08/16/metrics-for-regression-using-oml4py/" title="Share on Tumblr">Share on Tumblr</a></li><li class="share-pinterest"><div class="pinterest_button"><a href="https://www.pinterest.com/pin/create/button/?url=https%3A%2F%2Fmachinelearningmastery.in%2F2021%2F08%2F16%2Fmetrics-for-regression-using-oml4py%2F&media=https%3A%2F%2Fi1.wp.com%2Fmachinelearningmastery.in%2Fwp-content%2Fuploads%2F2020%2F11%2Fwp-1605237118213.jpg%3Ffit%3D96%252C96%26ssl%3D1&description=Metrics%20for%20Regression%20Using%20OML4Py" data-pin-do="buttonPin" data-pin-config="beside"><img src="https://i2.wp.com/assets.pinterest.com/images/pidgets/pinit_fg_en_rect_gray_20.png?w=1440" data-recalc-dims="1" data-lazy-src="https://i2.wp.com/assets.pinterest.com/images/pidgets/pinit_fg_en_rect_gray_20.png?w=1440&is-pending-load=1" srcset="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class=" jetpack-lazy-image"><noscript><img src="https://i2.wp.com/assets.pinterest.com/images/pidgets/pinit_fg_en_rect_gray_20.png?w=1440" data-recalc-dims="1" /></noscript></a></div></li><li class="share-skype"><div class="skype-share" data-href="https://machinelearningmastery.in/2021/08/16/metrics-for-regression-using-oml4py/" data-lang="en-US" data-style="small" data-source="jetpack" ></div></li><li class="share-email"><a rel="nofollow noopener noreferrer" data-shared="" class="share-email sd-button" href="https://machinelearningmastery.in/2021/08/16/metrics-for-regression-using-oml4py/?share=email" target="_blank" title="Click to email this to a friend"><span>Email</span></a></li><li class="share-end"></li></ul></div></div></div><div class='sharedaddy sd-block sd-like jetpack-likes-widget-wrapper jetpack-likes-widget-unloaded' id='like-post-wrapper-170785677-1831-6148f2b9afb6a' data-src='https://widgets.wp.com/likes/#blog_id=170785677&post_id=1831&origin=machinelearningmastery.in&obj_id=170785677-1831-6148f2b9afb6a' data-name='like-post-frame-170785677-1831-6148f2b9afb6a'><h3 class="sd-title">Like this:</h3><div class='likes-widget-placeholder post-likes-widget-placeholder' style='height: 55px;'><span class='button'><span>Like</span></span> <span class="loading">Loading...</span></div><span class='sd-text-color'></span><a class='sd-link-color'></a></div> <div id='jp-relatedposts' class='jp-relatedposts' > <h3 class="jp-relatedposts-headline"><em>Related</em></h3> </div> </div> </div><!-- .entry-content --> <div class="screen-reader-text" itemprop="datePublished" itemtype="https://schema.org/Date">2021-08-16</div> </article><!-- .entry --> <div id="loop-nav-wrap" class="loop-nav"><div class="prev">Previous Post: <a href="https://machinelearningmastery.in/2021/08/16/kdnuggets-top-blogs-rewards-for-july-2021/" rel="prev">KDnuggets Top Blogs Rewards for July 2021</a></div><div class="next">Next Post: <a href="https://machinelearningmastery.in/2021/08/16/top-stories-aug-9-15-the-difference-between-data-scientists-and-ml-engineers/" rel="next">Top Stories, Aug 9-15: The Difference Between Data Scientists and ML Engineers</a></div></div><!-- .loop-nav --> <section id="comments-template"> <div id="respond" class="comment-respond"> <h3 id="reply-title" class="comment-reply-title">Leave a Reply <small><a rel="nofollow" id="cancel-comment-reply-link" href="/2021/08/16/metrics-for-regression-using-oml4py/#respond" style="display:none;">Cancel reply</a></small></h3><form action="https://machinelearningmastery.in/wp-comments-post.php" method="post" id="commentform" class="comment-form" novalidate><p class="comment-notes"><span id="email-notes">Your email address will not be published.</span></p><p class="comment-form-comment"><label for="comment">Comment</label> <textarea id="comment" name="comment" cols="45" rows="8" maxlength="65525" required="required"></textarea></p><p class="comment-form-author"><label for="author">Name</label> <input id="author" name="author" type="text" value="" size="30" maxlength="245" /></p> <p class="comment-form-email"><label for="email">Email</label> <input id="email" name="email" type="email" value="" size="30" maxlength="100" aria-describedby="email-notes" /></p> <p class="comment-form-url"><label for="url">Website</label> <input id="url" name="url" type="url" value="" size="30" maxlength="200" /></p> <p class="comment-form-cookies-consent"><input id="wp-comment-cookies-consent" name="wp-comment-cookies-consent" type="checkbox" value="yes" /> <label for="wp-comment-cookies-consent">Save my name, email, and website in this browser for the next time I comment.</label></p> <p class="form-submit"><input name="submit" type="submit" id="submit" class="submit" value="Post Comment" /> <input type='hidden' name='comment_post_ID' value='1831' id='comment_post_ID' /> <input type='hidden' name='comment_parent' id='comment_parent' value='0' /> </p><p style="display: none;"><input type="hidden" id="akismet_comment_nonce" name="akismet_comment_nonce" value="550e69250c" /></p><input type="hidden" id="ak_js" name="ak_js" value="126"/><textarea name="ak_hp_textarea" cols="45" rows="8" maxlength="100" style="display: none !important;"></textarea></form> </div><!-- #respond --> </section><!-- #comments-template --> </div><!-- #content-wrap --> </main><!-- #content --> <aside id="sidebar-primary" class="sidebar sidebar-primary hgrid-span-3 layout-narrow-right " role="complementary" itemscope="itemscope" itemtype="https://schema.org/WPSideBar"> <div class=" sidebar-wrap"> <section id="tag_cloud-3" class="widget widget_tag_cloud"><h3 class="widget-title"><span>Categories</span></h3><div class="tagcloud"><a href="https://machinelearningmastery.in/category/articles/" class="tag-cloud-link tag-link-404 tag-link-position-1" style="font-size: 11.529411764706pt;" aria-label="Articles (7 items)">Articles</a> <a href="https://machinelearningmastery.in/category/automation-anywhere/" class="tag-cloud-link tag-link-158 tag-link-position-2" style="font-size: 9.0588235294118pt;" aria-label="Automation Anywhere (2 items)">Automation Anywhere</a> <a href="https://machinelearningmastery.in/category/certification/" class="tag-cloud-link tag-link-12 tag-link-position-3" style="font-size: 10.352941176471pt;" aria-label="Certification (4 items)">Certification</a> <a href="https://machinelearningmastery.in/category/cloud/" class="tag-cloud-link tag-link-289 tag-link-position-4" style="font-size: 10.352941176471pt;" aria-label="Cloud (4 items)">Cloud</a> <a href="https://machinelearningmastery.in/category/code/" class="tag-cloud-link tag-link-511 tag-link-position-5" style="font-size: 8pt;" aria-label="Code (1 item)">Code</a> <a href="https://machinelearningmastery.in/category/database-2/" class="tag-cloud-link tag-link-593 tag-link-position-6" style="font-size: 8pt;" aria-label="Database (1 item)">Database</a> <a href="https://machinelearningmastery.in/category/data-science/" class="tag-cloud-link tag-link-9 tag-link-position-7" style="font-size: 12.117647058824pt;" aria-label="Data Science (9 items)">Data Science</a> <a href="https://machinelearningmastery.in/category/data-science-topics/" class="tag-cloud-link tag-link-530 tag-link-position-8" style="font-size: 9.0588235294118pt;" aria-label="data science topics (2 items)">data science topics</a> <a href="https://machinelearningmastery.in/category/data-science-update/" class="tag-cloud-link tag-link-13 tag-link-position-9" style="font-size: 22pt;" aria-label="Data Science Update (478 items)">Data Science Update</a> <a href="https://machinelearningmastery.in/category/deep-learning/" class="tag-cloud-link tag-link-290 tag-link-position-10" style="font-size: 11.235294117647pt;" aria-label="Deep Learning (6 items)">Deep Learning</a> <a href="https://machinelearningmastery.in/category/financial-assistance/" class="tag-cloud-link tag-link-8 tag-link-position-11" style="font-size: 8pt;" aria-label="Financial assistance (1 item)">Financial assistance</a> <a href="https://machinelearningmastery.in/category/google-cloud/" class="tag-cloud-link tag-link-583 tag-link-position-12" style="font-size: 9.0588235294118pt;" aria-label="Google Cloud (2 items)">Google Cloud</a> <a href="https://machinelearningmastery.in/category/interview-tips/" class="tag-cloud-link tag-link-181 tag-link-position-13" style="font-size: 8pt;" aria-label="Interview tips (1 item)">Interview tips</a> <a href="https://machinelearningmastery.in/category/machine-learning/" class="tag-cloud-link tag-link-11 tag-link-position-14" style="font-size: 15.647058823529pt;" aria-label="Machine Learning (39 items)">Machine Learning</a> <a href="https://machinelearningmastery.in/category/open-data-source/" class="tag-cloud-link tag-link-207 tag-link-position-15" style="font-size: 9.7647058823529pt;" aria-label="Open Data Source (3 items)">Open Data Source</a> <a href="https://machinelearningmastery.in/category/power-bi/" class="tag-cloud-link tag-link-341 tag-link-position-16" style="font-size: 9.0588235294118pt;" aria-label="Power BI (2 items)">Power BI</a> <a href="https://machinelearningmastery.in/category/project-management/" class="tag-cloud-link tag-link-409 tag-link-position-17" style="font-size: 9.7647058823529pt;" aria-label="Project Management (3 items)">Project Management</a> <a href="https://machinelearningmastery.in/category/python/" class="tag-cloud-link tag-link-2 tag-link-position-18" style="font-size: 12.117647058824pt;" aria-label="Python (9 items)">Python</a> <a href="https://machinelearningmastery.in/category/quiz-of-the-day/" class="tag-cloud-link tag-link-429 tag-link-position-19" style="font-size: 8pt;" aria-label="Quiz of the Day (1 item)">Quiz of the Day</a> <a href="https://machinelearningmastery.in/category/robotic-process-automation/" class="tag-cloud-link tag-link-159 tag-link-position-20" style="font-size: 9.0588235294118pt;" aria-label="Robotic Process Automation (2 items)">Robotic Process Automation</a> <a href="https://machinelearningmastery.in/category/r-programming/" class="tag-cloud-link tag-link-157 tag-link-position-21" style="font-size: 9.0588235294118pt;" aria-label="R Programming (2 items)">R Programming</a> <a href="https://machinelearningmastery.in/category/sas/" class="tag-cloud-link tag-link-156 tag-link-position-22" style="font-size: 10.823529411765pt;" aria-label="SAS (5 items)">SAS</a> <a href="https://machinelearningmastery.in/category/statistics/" class="tag-cloud-link tag-link-337 tag-link-position-23" style="font-size: 10.352941176471pt;" aria-label="Statistics (4 items)">Statistics</a> <a href="https://machinelearningmastery.in/category/tableau/" class="tag-cloud-link tag-link-340 tag-link-position-24" style="font-size: 9.0588235294118pt;" aria-label="Tableau (2 items)">Tableau</a> <a href="https://machinelearningmastery.in/category/visualization/" class="tag-cloud-link tag-link-10 tag-link-position-25" style="font-size: 12.352941176471pt;" aria-label="visualization (10 items)">visualization</a></div> </section><section id="newsletterwidget-2" class="widget widget_newsletterwidget"><div class="tnp tnp-widget"><form method="post" action="https://machinelearningmastery.in/?na=s"> <input type="hidden" name="nr" value="widget"><input type="hidden" name="nlang" value=""><div class="tnp-field tnp-field-email"><label for="tnp-email">Email</label> <input class="tnp-email" type="email" name="ne" value="" required></div> <div class="tnp-field tnp-field-button"><input class="tnp-submit" type="submit" value="Subscribe" > </div> </form> </div></section> <section id="recent-posts-2" class="widget widget_recent_entries"> <h3 class="widget-title"><span>Recent Posts</span></h3> <ul> <li> <a href="https://machinelearningmastery.in/2021/09/20/how-to-label-time-series-efficiently-and-boost-your-ai/">How to label time series efficiently – and boost your AI</a> </li> <li> <a href="https://machinelearningmastery.in/2021/09/20/how-to-be-a-data-scientist-without-a-stem-degree/">How to be a Data Scientist without a STEM degree</a> </li> <li> <a href="https://machinelearningmastery.in/2021/09/17/arcelik-hosts-global-aws-deepracer-league-using-new-live-feature-to-educate-over-200-employees-on-machine-learning/">Arçelik hosts global AWS DeepRacer League using new LIVE feature to educate over 200 employees on machine learning</a> </li> <li> <a href="https://machinelearningmastery.in/2021/09/17/paradoxes-in-data-science-kdnuggets/">Paradoxes in Data Science – KDnuggets</a> </li> <li> <a href="https://machinelearningmastery.in/2021/09/17/what-2-years-of-self-teaching-data-science-taught-me/">What 2 years of self-teaching data science taught me</a> </li> <li> <a href="https://machinelearningmastery.in/2021/09/17/introducing-tensorflow-similarity-kdnuggets/">Introducing TensorFlow Similarity – KDnuggets</a> </li> <li> <a href="https://machinelearningmastery.in/2021/09/16/what-is-the-real-difference-between-data-engineers-and-data-scientists/">What Is The Real Difference Between Data Engineers and Data Scientists?</a> </li> <li> <a href="https://machinelearningmastery.in/2021/09/16/adventures-in-mlops-with-github-actions-iterative-ai-label-studio-and-nbdev/">Adventures in MLOps with Github Actions, Iterative.ai, Label Studio and NBDEV</a> </li> <li> <a href="https://machinelearningmastery.in/2021/09/16/the-machine-deep-learning-compendium-open-book/">The Machine & Deep Learning Compendium Open Book</a> </li> <li> <a href="https://machinelearningmastery.in/2021/09/16/easy-sql-in-native-python/">Easy SQL in Native Python</a> </li> </ul> </section> </div><!-- .sidebar-wrap --> </aside><!-- #sidebar-primary --> </div><!-- .main-content-grid --> </div><!-- #main --> <footer id="footer" class="site-footer footer hgrid-stretch inline-nav" role="contentinfo" itemscope="itemscope" itemtype="https://schema.org/WPFooter"> <div class="hgrid"> <div class="hgrid-span-6 footer-column"> <section id="hootkit-ticker-9" class="widget widget_hootkit-ticker"> <div class="ticker-widget ticker-usercontent ticker-simple ticker-userstyle ticker-withbg ticker-style1" style="background:#f1f1f1;color:#ff4530;" ><i class="fa-weixin fab ticker-icon"></i> <div class="ticker-msg-box" data-speed='0.03'> <div class="ticker-msgs"> <div class="ticker-msg"><div class="ticker-msg-inner">Subscribe for the latest news, updates, tips and more delivered right to your inbox.</div></div> </div> </div> </div></section> </div> <div class="hgrid-span-3 footer-column"> <section id="media_image-13" class="widget widget_media_image"><img width="220" height="49" src="https://i2.wp.com/machinelearningmastery.in/wp-content/uploads/2019/12/Machine-Learning-Mastery-banner.gif?fit=220%2C49&ssl=1" class="image wp-image-127 attachment-full size-full jetpack-lazy-image" alt="" loading="lazy" style="max-width: 100%; height: auto;" data-lazy-src="https://i2.wp.com/machinelearningmastery.in/wp-content/uploads/2019/12/Machine-Learning-Mastery-banner.gif?fit=220%2C49&ssl=1&is-pending-load=1" srcset="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" /></section> </div> <div class="hgrid-span-3 footer-column"> <section id="hootkit-social-icons-8" class="widget widget_hootkit-social-icons"> <div class="social-icons-widget social-icons-small"><a href="https://github.com/machinelearningmasteryindia" class=" social-icons-icon fa-github-block" target="_blank"> <i class="fa-github fab"></i> </a><a href="mailto:machinelearningmasteryindia@gmail.com" class=" social-icons-icon fa-envelope-block"> <i class="fa-envelope fas"></i> </a><a href="https://www.linkedin.com/in/machine-learning-b065081a9/" class=" social-icons-icon fa-linkedin-block" target="_blank"> <i class="fa-linkedin-in fab"></i> </a><a href="https://twitter.com/sitworld" class=" social-icons-icon fa-twitter-block" target="_blank"> <i class="fa-twitter fab"></i> </a></div></section> </div> </div> </footer><!-- #footer --> <div id="post-footer" class=" post-footer hgrid-stretch linkstyle"> <div class="hgrid"> <div class="hgrid-span-12"> <p class="credit small"> <a class="privacy-policy-link" href="https://machinelearningmastery.in/privacy-policy/">Privacy Policy</a> Designed using <a class="theme-link" href="https://wphoot.com/themes/unos/" title="Unos WordPress Theme">Unos</a>. Powered by <a class="wp-link" href="https://wordpress.org">WordPress</a>. </p><!-- .credit --> </div> </div> </div> </div><!-- #page-wrapper --> <!--googleoff: all--><div id="cookie-law-info-bar" data-nosnippet="true"><span>This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. <a role='button' tabindex='0' class="cli_settings_button" style="margin:5px 20px 5px 20px;" >Cookie settings</a><a role='button' tabindex='0' data-cli_action="accept" id="cookie_action_close_header" class="medium cli-plugin-button cli-plugin-main-button cookie_action_close_header cli_action_button" style="display:inline-block; margin:5px; ">ACCEPT</a></span></div><div id="cookie-law-info-again" style="display:none;" data-nosnippet="true"><span id="cookie_hdr_showagain">Privacy & Cookies Policy</span></div><div class="cli-modal" data-nosnippet="true" id="cliSettingsPopup" tabindex="-1" role="dialog" aria-labelledby="cliSettingsPopup" aria-hidden="true"> <div class="cli-modal-dialog" role="document"> <div class="cli-modal-content cli-bar-popup"> <button type="button" class="cli-modal-close" id="cliModalClose"> <svg class="" viewBox="0 0 24 24"><path d="M19 6.41l-1.41-1.41-5.59 5.59-5.59-5.59-1.41 1.41 5.59 5.59-5.59 5.59 1.41 1.41 5.59-5.59 5.59 5.59 1.41-1.41-5.59-5.59z"></path><path d="M0 0h24v24h-24z" fill="none"></path></svg> <span class="wt-cli-sr-only">Close</span> </button> <div class="cli-modal-body"> <div class="cli-container-fluid cli-tab-container"> <div class="cli-row"> <div class="cli-col-12 cli-align-items-stretch cli-px-0"> <div class="cli-privacy-overview"> <h4>Privacy Overview</h4> <div class="cli-privacy-content"> <div class="cli-privacy-content-text">This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.</div> </div> <a class="cli-privacy-readmore" aria-label="Show more" tabindex="0" role="button" data-readmore-text="Show more" data-readless-text="Show less"></a> </div> </div> <div class="cli-col-12 cli-align-items-stretch cli-px-0 cli-tab-section-container"> <div class="cli-tab-section"> <div class="cli-tab-header"> <a role="button" tabindex="0" class="cli-nav-link cli-settings-mobile" data-target="necessary" data-toggle="cli-toggle-tab"> Necessary </a> <div class="wt-cli-necessary-checkbox"> <input type="checkbox" class="cli-user-preference-checkbox" id="wt-cli-checkbox-necessary" data-id="checkbox-necessary" checked="checked" /> <label class="form-check-label" for="wt-cli-checkbox-necessary">Necessary</label> </div> <span class="cli-necessary-caption">Always Enabled</span> </div> <div class="cli-tab-content"> <div class="cli-tab-pane cli-fade" data-id="necessary"> <div class="wt-cli-cookie-description"> Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information. </div> </div> </div> </div> <div class="cli-tab-section"> <div class="cli-tab-header"> <a role="button" tabindex="0" class="cli-nav-link cli-settings-mobile" data-target="non-necessary" data-toggle="cli-toggle-tab"> Non-necessary </a> <div class="cli-switch"> <input type="checkbox" id="wt-cli-checkbox-non-necessary" class="cli-user-preference-checkbox" data-id="checkbox-non-necessary" checked='checked' /> <label for="wt-cli-checkbox-non-necessary" class="cli-slider" data-cli-enable="Enabled" data-cli-disable="Disabled"><span class="wt-cli-sr-only">Non-necessary</span></label> </div> </div> <div class="cli-tab-content"> <div class="cli-tab-pane cli-fade" data-id="non-necessary"> <div class="wt-cli-cookie-description"> Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website. </div> </div> </div> </div> </div> </div> </div> </div> <div class="cli-modal-footer"> <div class="wt-cli-element cli-container-fluid cli-tab-container"> <div class="cli-row"> <div class="cli-col-12 cli-align-items-stretch cli-px-0"> <div class="cli-tab-footer wt-cli-privacy-overview-actions"> <a id="wt-cli-privacy-save-btn" role="button" tabindex="0" data-cli-action="accept" class="wt-cli-privacy-btn cli_setting_save_button wt-cli-privacy-accept-btn cli-btn">SAVE & ACCEPT</a> </div> </div> </div> </div> </div> </div> </div> </div> <div class="cli-modal-backdrop cli-fade cli-settings-overlay"></div> <div class="cli-modal-backdrop cli-fade cli-popupbar-overlay"></div> <!--googleon: all--> <div id="fb-root"></div> <script async defer crossorigin="anonymous" src="https://connect.facebook.net/en_US/sdk.js#xfbml=1&version=v8.0&appId=683648729088349&autoLogAppEvents=1"> </script> <!--Start of Tawk.to Script (0.5.5)--> <script type="text/javascript"> var Tawk_API=Tawk_API||{}; var Tawk_LoadStart=new Date(); (function(){ var s1=document.createElement("script"),s0=document.getElementsByTagName("script")[0]; s1.async=true; s1.src='https://embed.tawk.to/5ec04a92967ae56c521a742a/default'; s1.charset='UTF-8'; s1.setAttribute('crossorigin','*'); s0.parentNode.insertBefore(s1,s0); })(); </script> <!--End of Tawk.to Script (0.5.5)--> <script type="text/javascript"> window.WPCOM_sharing_counts = {"https:\/\/machinelearningmastery.in\/2021\/08\/16\/metrics-for-regression-using-oml4py\/":1831}; </script> <script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script> <div id="fb-root"></div> <script>(function(d, s, id) { var js, fjs = d.getElementsByTagName(s)[0]; if (d.getElementById(id)) return; js = d.createElement(s); js.id = id; js.src = 'https://connect.facebook.net/en_US/sdk.js#xfbml=1&appId=249643311490&version=v2.3'; fjs.parentNode.insertBefore(js, fjs); }(document, 'script', 'facebook-jssdk'));</script> <script> document.body.addEventListener( 'is.post-load', function() { if ( 'undefined' !== typeof FB ) { FB.XFBML.parse(); } } ); </script> <script type="text/javascript"> ( function () { var currentScript = document.currentScript; // Helper function to load an external script. function loadScript( url, cb ) { var script = document.createElement( 'script' ); var prev = currentScript || document.getElementsByTagName( 'script' )[ 0 ]; script.setAttribute( 'async', true ); script.setAttribute( 'src', url ); prev.parentNode.insertBefore( script, prev ); script.addEventListener( 'load', cb ); } function init() { loadScript( 'https://platform.linkedin.com/in.js?async=true', function () { if ( typeof IN !== 'undefined' ) { IN.init(); } } ); } if ( document.readyState === 'loading' ) { document.addEventListener( 'DOMContentLoaded', init ); } else { init(); } document.body.addEventListener( 'is.post-load', function() { if ( typeof IN !== 'undefined' ) { IN.parse(); } } ); } )(); </script> <script id="tumblr-js" type="text/javascript" src="https://assets.tumblr.com/share-button.js"></script> <script type="text/javascript"> ( function () { // Pinterest shared resources var s = document.createElement( 'script' ); s.type = 'text/javascript'; s.async = true; s.setAttribute( 'data-pin-hover', true ); s.src = window.location.protocol + '//assets.pinterest.com/js/pinit.js'; var x = document.getElementsByTagName( 'script' )[ 0 ]; x.parentNode.insertBefore(s, x); // if 'Pin it' button has 'counts' make container wider function init() { var shares = document.querySelectorAll( 'li.share-pinterest' ); for ( var i = 0; i < shares.length; i++ ) { var share = shares[ i ]; if ( share.querySelector( 'a span:visible' ) ) { share.style.width = '80px'; } } } if ( document.readyState !== 'complete' ) { document.addEventListener( 'load', init ); } else { init(); } } )(); </script> <script> (function(r, d, s) { r.loadSkypeWebSdkAsync = r.loadSkypeWebSdkAsync || function(p) { var js, sjs = d.getElementsByTagName(s)[0]; if (d.getElementById(p.id)) { return; } js = d.createElement(s); js.id = p.id; js.src = p.scriptToLoad; js.onload = p.callback sjs.parentNode.insertBefore(js, sjs); }; var p = { scriptToLoad: 'https://swx.cdn.skype.com/shared/v/latest/skypewebsdk.js', id: 'skype_web_sdk' }; r.loadSkypeWebSdkAsync(p); })(window, document, 'script'); </script> <div id="sharing_email" style="display: none;"> <form action="/2021/08/16/metrics-for-regression-using-oml4py/" method="post"> <label for="target_email">Send to Email Address</label> <input type="email" name="target_email" id="target_email" value="" /> <label for="source_name">Your Name</label> <input type="text" name="source_name" id="source_name" value="" /> <label for="source_email">Your Email Address</label> <input type="email" name="source_email" id="source_email" value="" /> <input type="text" id="jetpack-source_f_name" name="source_f_name" class="input" value="" size="25" autocomplete="off" title="This field is for validation and should not be changed" /> <img style="float: right; display: none" class="loading" src="https://machinelearningmastery.in/wp-content/plugins/jetpack/modules/sharedaddy/images/loading.gif" alt="loading" width="16" height="16" /> <input type="submit" value="Send Email" class="sharing_send" /> <a rel="nofollow" href="#cancel" class="sharing_cancel" role="button">Cancel</a> <div class="errors errors-1" style="display: none;"> Post was not sent - check your email addresses! </div> <div class="errors errors-2" style="display: none;"> Email check failed, please try again </div> <div class="errors errors-3" style="display: none;"> Sorry, your blog cannot share posts by email. </div> </form> </div> <script data-cfasync="false" type="text/javascript">if (window.addthis_product === undefined) { window.addthis_product = "wpp"; } if (window.wp_product_version === undefined) { window.wp_product_version = "wpp-6.2.6"; } if (window.addthis_share === undefined) { window.addthis_share = {}; } if (window.addthis_config === undefined) { window.addthis_config = {"data_track_clickback":true,"ui_atversion":"300"}; } if (window.addthis_plugin_info === undefined) { window.addthis_plugin_info = {"info_status":"enabled","cms_name":"WordPress","plugin_name":"Share Buttons by AddThis","plugin_version":"6.2.6","plugin_mode":"AddThis","anonymous_profile_id":"wp-2f16336e765908d13c2d341ff0393457","page_info":{"template":"posts","post_type":["post","page","e-landing-page"]},"sharing_enabled_on_post_via_metabox":false}; } (function() { var first_load_interval_id = setInterval(function () { if (typeof window.addthis !== 'undefined') { window.clearInterval(first_load_interval_id); if (typeof window.addthis_layers !== 'undefined' && Object.getOwnPropertyNames(window.addthis_layers).length > 0) { window.addthis.layers(window.addthis_layers); } if (Array.isArray(window.addthis_layers_tools)) { for (i = 0; i < window.addthis_layers_tools.length; i++) { window.addthis.layers(window.addthis_layers_tools[i]); } } } },1000) }()); </script><script src='https://machinelearningmastery.in/wp-content/plugins/jetpack/_inc/build/photon/photon.min.js?ver=20191001' id='jetpack-photon-js'></script> <script src='https://machinelearningmastery.in/wp-includes/js/comment-reply.min.js?ver=5.6.5' id='comment-reply-js'></script> <script id='hoverIntent-js-extra'> var hootData = {"stickySidebar":"disable","contentblockhover":"enable","contentblockhovertext":"disable"}; </script> <script src='https://machinelearningmastery.in/wp-includes/js/hoverIntent.min.js?ver=1.8.1' id='hoverIntent-js'></script> <script src='https://machinelearningmastery.in/wp-content/themes/unos/js/jquery.superfish.min.js?ver=1.7.5' id='jquery-superfish-js'></script> <script src='https://machinelearningmastery.in/wp-content/themes/unos/js/jquery.fitvids.min.js?ver=1.1' id='jquery-fitvids-js'></script> <script src='https://machinelearningmastery.in/wp-content/themes/unos/js/jquery.parallax.min.js?ver=1.4.2' id='jquery-parallax-js'></script> <script id='ap-frontend-js-js-extra'> var ap_form_required_message = ["This field is required","accesspress-anonymous-post"]; var ap_captcha_error_message = ["Sum is not correct.","accesspress-anonymous-post"]; </script> <script src='https://machinelearningmastery.in/wp-content/plugins/accesspress-anonymous-post/js/frontend.js?ver=2.8.1' id='ap-frontend-js-js'></script> <script src='https://machinelearningmastery.in/wp-content/plugins/hootkit/assets/jquery.lightSlider.min.js?ver=1.1.2' id='jquery-lightSlider-js'></script> <script src='https://machinelearningmastery.in/wp-content/plugins/hootkit/assets/widgets.min.js?ver=2.0.7' id='hootkit-widgets-js'></script> <script id='hootkit-miscmods-js-extra'> var hootkitMiscmodsData = {"ajaxurl":"https:\/\/machinelearningmastery.in\/wp-admin\/admin-ajax.php"}; </script> <script src='https://machinelearningmastery.in/wp-content/plugins/hootkit/assets/miscmods.min.js?ver=2.0.7' id='hootkit-miscmods-js'></script> <script src='https://machinelearningmastery.in/wp-content/plugins/page-links-to/dist/new-tab.js?ver=3.3.5' id='page-links-to-js'></script> <script src='https://s7.addthis.com/js/300/addthis_widget.js?ver=5.6.5#pubid=ra-5e0c443d44eeeb15' id='addthis_widget-js'></script> <script src='https://machinelearningmastery.in/wp-content/plugins/jetpack/vendor/automattic/jetpack-lazy-images/src/js/intersectionobserver-polyfill.min.js?ver=1.1.2' id='jetpack-lazy-images-polyfill-intersectionobserver-js'></script> <script id='jetpack-lazy-images-js-extra'> var jetpackLazyImagesL10n = {"loading_warning":"Images are still loading. Please cancel your print and try again."}; </script> <script src='https://machinelearningmastery.in/wp-content/plugins/jetpack/vendor/automattic/jetpack-lazy-images/src/js/lazy-images.min.js?ver=1.1.2' id='jetpack-lazy-images-js'></script> <script src='https://machinelearningmastery.in/wp-content/plugins/jetpack/_inc/build/postmessage.min.js?ver=9.8.1' id='postmessage-js'></script> <script src='https://machinelearningmastery.in/wp-content/plugins/jetpack/_inc/build/jquery.jetpack-resize.min.js?ver=9.8.1' id='jetpack_resize-js'></script> <script src='https://machinelearningmastery.in/wp-content/plugins/jetpack/_inc/build/likes/queuehandler.min.js?ver=9.8.1' id='jetpack_likes_queuehandler-js'></script> <script src='https://machinelearningmastery.in/wp-content/themes/unos/js/hoot.theme.min.js?ver=2.9.11' id='hoot-theme-js'></script> <script src='https://machinelearningmastery.in/wp-content/plugins/youtube-embed-plus/scripts/fitvids.min.js?ver=13.4.3' id='__ytprefsfitvids__-js'></script> <script id='wpgdprc.js-js-extra'> var wpgdprcData = {"ajaxURL":"https:\/\/machinelearningmastery.in\/wp-admin\/admin-ajax.php","ajaxSecurity":"dfe2f68189","isMultisite":"","path":"\/","blogId":""}; </script> <script src='https://machinelearningmastery.in/wp-content/plugins/wp-gdpr-compliance/dist/js/front.min.js?ver=1629244814' id='wpgdprc.js-js'></script> <script src='https://machinelearningmastery.in/wp-includes/js/wp-embed.min.js?ver=5.6.5' id='wp-embed-js'></script> <script id='jetpack_related-posts-js-extra'> var related_posts_js_options = {"post_heading":"h4"}; </script> <script src='https://machinelearningmastery.in/wp-content/plugins/jetpack/_inc/build/related-posts/related-posts.min.js?ver=20210604' id='jetpack_related-posts-js'></script> <script defer src='https://machinelearningmastery.in/wp-content/plugins/akismet/_inc/form.js?ver=4.1.12' id='akismet-form-js'></script> <script id='sharing-js-js-extra'> var sharing_js_options = {"lang":"en","counts":"1","is_stats_active":"1"}; </script> <script src='https://machinelearningmastery.in/wp-content/plugins/jetpack/_inc/build/sharedaddy/sharing.min.js?ver=9.8.1' id='sharing-js-js'></script> <script id='sharing-js-js-after'> var windowOpen; ( function () { function matches( el, sel ) { return !! ( el.matches && el.matches( sel ) || el.msMatchesSelector && el.msMatchesSelector( sel ) ); } document.body.addEventListener( 'click', function ( event ) { if ( ! event.target ) { return; } var el; if ( matches( event.target, 'a.share-facebook' ) ) { el = event.target; } else if ( event.target.parentNode && matches( event.target.parentNode, 'a.share-facebook' ) ) { el = event.target.parentNode; } if ( el ) { event.preventDefault(); // If there's another sharing window open, close it. if ( typeof windowOpen !== 'undefined' ) { windowOpen.close(); } windowOpen = window.open( el.getAttribute( 'href' ), 'wpcomfacebook', 'menubar=1,resizable=1,width=600,height=400' ); return false; } } ); } )(); var windowOpen; ( function () { function matches( el, sel ) { return !! ( el.matches && el.matches( sel ) || el.msMatchesSelector && el.msMatchesSelector( sel ) ); } document.body.addEventListener( 'click', function ( event ) { if ( ! event.target ) { return; } var el; if ( matches( event.target, 'a.share-telegram' ) ) { el = event.target; } else if ( event.target.parentNode && matches( event.target.parentNode, 'a.share-telegram' ) ) { el = event.target.parentNode; } if ( el ) { event.preventDefault(); // If there's another sharing window open, close it. if ( typeof windowOpen !== 'undefined' ) { windowOpen.close(); } windowOpen = window.open( el.getAttribute( 'href' ), 'wpcomtelegram', 'menubar=1,resizable=1,width=450,height=450' ); return false; } } ); } )(); </script> <iframe src='https://widgets.wp.com/likes/master.html?ver=202138#ver=202138' scrolling='no' id='likes-master' name='likes-master' style='display:none;'></iframe> <div id='likes-other-gravatars'><div class="likes-text"><span>%d</span> bloggers like this:</div><ul class="wpl-avatars sd-like-gravatars"></ul></div> <script>!function(){window.advanced_ads_ready_queue=window.advanced_ads_ready_queue||[],advanced_ads_ready_queue.push=window.advanced_ads_ready;for(var d=0,a=advanced_ads_ready_queue.length;d<a;d++)advanced_ads_ready(advanced_ads_ready_queue[d])}();</script><script src='https://stats.wp.com/e-202138.js' defer></script> <script> _stq = window._stq || []; _stq.push([ 'view', {v:'ext',j:'1:9.8.1',blog:'170785677',post:'1831',tz:'-5.5',srv:'machinelearningmastery.in'} ]); _stq.push([ 'clickTrackerInit', '170785677', '1831' ]); </script> </body> </html> <!-- Page generated by LiteSpeed Cache 4.4.1 on 2021-09-20 15:14:41 -->