</p> <p class="has-line-data" data-line-end="1" data-line-start="0">In part I, we discussed popular metrics such as accuracy, confusion matrix, precision, recall and the F1 score. The common characteristic for those metrics is that they rely on a given threshold for producing the ultimate prediction. In most cases, a classification model originally produces a probably score. In order to arrive at a prediction, one needs to come up with a threshold: a case is predicted as positive when the probably score is greater than the threshold and vice versa. Therefore, if we increase the threshold, we will have a higher recall and lower precision. How are precision and recall are affected when one adjusts the threshold? The precision-recall curve can provide a fuller picture.</p> <p class="has-line-data" data-line-end="4" data-line-start="3">The precision-recall curve plots the recall on the x-axis and precision on the y-axis. We plot all the possible pairs of recall and precision by varying the threshold from the lowest possible value to the highest. Then we connect all points and form a curve. The idea is to get an overview of how precision and recall behave when we vary the threshold to a wide range.</p> <p class="has-line-data" data-line-end="6" data-line-start="5">In this case, we need to compute pairs of precision and recall for each threshold we can choose. Looks like we need to write a loop in Python. Actually, we have a better choice here because all the data is stored in database. We can leverage a SQL window function to compute it efficiently.</p> <p class="has-line-data" data-line-end="8" data-line-start="7">SQL window functions allow the user to compute quantities from multiple rows that are related to the current row. In this particular use case, SQL window functions allow us to go through each row and then compute the precision and recall using all rows with prediction score less than the score of the current row. The query can be written in a concise way and runs fast.</p> <p class="has-line-data" data-line-end="10" data-line-start="9">We first generate the following view based on our positive probability and counts.</p> <pre> <code class="has-line-data" data-line-end="26" data-line-start="11">CREATE OR REPLACE VIEW PR_SUMMARY_V AS WITH pos_prob_and_counts AS ( SELECT PREDICTION_PROBABILITY(GLM_METRIC_MDL, <span class="hljs-number">1</span> USING *) pos_prob, BUY_INSURANCE FROM CUST_TEST_TBL ) SELECT BUY_INSURANCE, pos_prob, SUM(BUY_INSURANCE) OVER (ORDER BY pos_prob DESC) pos_acc, SUM(BUY_INSURANCE) OVER (ORDER BY pos_prob DESC) /COUNT(<span class="hljs-number">1</span>) OVER (ORDER BY pos_prob DESC) precision, SUM(BUY_INSURANCE) OVER (ORDER BY pos_prob DESC) /SUM(BUY_INSURANCE) OVER () recall FROM pos_prob_and_counts order by pos_prob desc </code></pre> <p class="has-line-data" data-line-end="27" data-line-start="26">We can then generate the plot using native Python with data retrieved from this view using OML4Py:</p> <pre> <code class="has-line-data" data-line-end="41" data-line-start="28">PR_DF = oml.sync(schema = <span class="hljs-string">'JIE'</span>, view = <span class="hljs-string">'PR_SUMMARY_V'</span>) fig, ax = plt.subplots(nrows=<span class="hljs-number">1</span>, ncols=<span class="hljs-number">1</span>,figsize=[<span class="hljs-number">10</span>,<span class="hljs-number">8</span>]) pr_df = PR_DF.pull() pr_df = pr_df.sort_values(<span class="hljs-string">'POS_PROB'</span>, ascending = <span class="hljs-keyword">False</span>) x = pr_df[<span class="hljs-string">'RECALL'</span>].values y = pr_df[<span class="hljs-string">'PRECISION'</span>].values ax.set_title(<span class="hljs-string">'Precision-Recall curve'</span>) ax.plot(x[<span class="hljs-number">1</span>:], y[<span class="hljs-number">1</span>:], color=<span class="hljs-string">'blue'</span>, lw=<span class="hljs-number">2</span>, label=<span class="hljs-string">'P-R curve'</span>) ax.legend(loc=<span class="hljs-string">"upper right"</span>) ax.set_xlabel(<span class="hljs-string">'Recall'</span>, size=<span class="hljs-number">13</span>) ax.set_ylabel(<span class="hljs-string">'Precision'</span>, size=<span class="hljs-number">13</span>) plt.show() </code></pre> <p class="has-line-data" data-line-end="43" data-line-start="42"><img alt src="https://i0.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/8fef199564bf40031e56e802c920a108/prcurve.png?w=1440&ssl=1" style="width: 720px; height: 576px;" data-recalc-dims="1" data-lazy-src="https://i0.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/8fef199564bf40031e56e802c920a108/prcurve.png?w=1440&is-pending-load=1#038;ssl=1" srcset="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class=" jetpack-lazy-image"><noscript><img alt="" src="https://i0.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/8fef199564bf40031e56e802c920a108/prcurve.png?w=1440&ssl=1" style="width: 720px; height: 576px;" data-recalc-dims="1"/></noscript></p> <p class="has-line-data" data-line-end="45" data-line-start="44">In the precision- recall curve, we plot the points of (recall, precision) using the threshold from the highest possible value to the lowest. In the curve, the highest threshold corresponds to the leftmost point and the lowest one corresponds to the rightmost point. From the leftmost, we see a spike with precision equal to 1 and then falls quickly. This is because we start from the highest threshold and it usually achieves high precision since most of the negative cases tend to have low probability score. At the leftmost point, it has lowest recall since we only predict the cases with high probability as positive cases and leave majority of the other positive cases out.</p> <p class="has-line-data" data-line-end="47" data-line-start="46">As we increase the threshold, we tend to predict more cases as positive and that naturally leads to lower precision and higher recall so the curve tends to be decreasing. In our case, the precision drops quickly and settle down after the point with recall = 0.1. This indicates that our model is not that good at precision, which matches with what we observed from the confusion matrix. The bright side is that our model can maintain the precision for a wide range of thresholds.</p> <p class="has-line-data" data-line-end="49" data-line-start="48">In the precision-recall curve, we also plot a reference straight line which is the ratio of the positive examples in the dataset. This compares the behavior that a random guess would have. We can see that the precision we obtained here is way above this reference line, which indicates the predictive power of the model.</p> <p class="has-line-data" data-line-end="52" data-line-start="51">Besides precision and recall, lift is a widely used metric particularly designed for marketing campaigns. The idea is that if we target all customers randomly, how many customers who wants to buy insurance can we reach? With a random approach, we reach only the ratio of the positive cases in the original dataset. Then we ask, after using the model, how much benefit can we get compared to randomly selecting customers?</p> <p class="has-line-data" data-line-end="54" data-line-start="53">To that end, lift provides two types of plots. One is called a lift chart, which looks a bit of like AUC but is different. It plots the top percentage of the customers targeted on the x-axis, and the recall obtained by doing that. For instance, the point at 10% means that if we rank the customers based on the probability score, and target the top 10%, how much recall we can achieve. The straight line is the scenario that we target customers randomly. Thus, the lift curve should be above the straight line.</p> <p class="has-line-data" data-line-end="56" data-line-start="55">Another plot is called waterfall analysis. The idea is to segment customers based on the probability score. It generates a series of user segments such as customer probability score between [0, 10%], [10%, 20%], …, etc. Then for each segment, we compute the percentage of customers who want to buy insurance in that segment and plot it as histogram. This provides a detailed view of how much benefit can we get when we target each customer segment.</p> <p class="has-line-data" data-line-end="58" data-line-start="57">The following python code generates the lift chart and waterfall analysis .</p> <pre> <code class="has-line-data" data-line-end="82" data-line-start="60">fig, axes = plt.subplots(nrows=<span class="hljs-number">1</span>, ncols=<span class="hljs-number">2</span>,figsize=[<span class="hljs-number">20</span>,<span class="hljs-number">7</span>]) lift_df = LIFT_DF.pull() x = lift_df[<span class="hljs-string">'COVERED'</span>].values[<span class="hljs-number">0</span>] y = lift_df[<span class="hljs-string">'POS_RATE'</span>].pull() x = [i*<span class="hljs-number">100</span> <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> x] y = [i*<span class="hljs-number">100</span> <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> y] axes[<span class="hljs-number">0</span>].set_title(<span class="hljs-string">'Lift Chart'</span>) axes[<span class="hljs-number">0</span>].plot(x, y, color=<span class="hljs-string">'blue'</span>, lw=<span class="hljs-number">2</span>, label=<span class="hljs-string">'Lift curve'</span>) axes[<span class="hljs-number">0</span>].plot([<span class="hljs-number">0</span>, <span class="hljs-number">100</span>], [<span class="hljs-number">0</span>, <span class="hljs-number">100</span>], lw=<span class="hljs-number">2</span>, linestyle=<span class="hljs-string">'--'</span>, color=<span class="hljs-string">'grey'</span>, label=<span class="hljs-string">'Random guess'</span>) axes[<span class="hljs-number">0</span>].legend(loc=<span class="hljs-string">"lower right"</span>) axes[<span class="hljs-number">0</span>].set_xlabel(<span class="hljs-string">'Top Customer Targeted %'</span>, size=<span class="hljs-number">13</span>) axes[<span class="hljs-number">0</span>].set_ylabel(<span class="hljs-string">'Recall %'</span>, size=<span class="hljs-number">13</span>) axes[<span class="hljs-number">1</span>].set_title(<span class="hljs-string">'Waterfall Analysis'</span>) axes[<span class="hljs-number">1</span>].bar(decile_df[<span class="hljs-string">'DECILE'</span>],decile_df[<span class="hljs-string">'POS_RATE'</span>]*<span class="hljs-number">100</span>,color=<span class="hljs-string">'blue'</span>, alpha=<span class="hljs-number">0.6</span>, label=<span class="hljs-string">'Model'</span>) axes[<span class="hljs-number">1</span>].axhline(LIFT_DF[<span class="hljs-string">'BUY_INSURANCE'</span>].sum()*<span class="hljs-number">100</span>/LIFT_DF.shape[<span class="hljs-number">0</span>], color=<span class="hljs-string">'grey'</span>, linestyle=<span class="hljs-string">'--'</span>, label=<span class="hljs-string">'Avg TARGET'</span>) axes[<span class="hljs-number">1</span>].legend(loc=<span class="hljs-string">"upper right"</span>) axes[<span class="hljs-number">1</span>].set_xlabel(<span class="hljs-string">'Decile'</span>, size=<span class="hljs-number">13</span>) axes[<span class="hljs-number">1</span>].set_ylabel(<span class="hljs-string">'Actual Customers Targeted %'</span>, size=<span class="hljs-number">13</span>) plt.show() </code></pre> <p class="has-line-data" data-line-end="84" data-line-start="83"><img alt src="https://i0.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/96ebd7142a2cf809224f5d8e44d1291d/liftcurve.png?w=1440&ssl=1" style="width: 593px; height: 481px;" data-recalc-dims="1" data-lazy-src="https://i0.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/96ebd7142a2cf809224f5d8e44d1291d/liftcurve.png?w=1440&is-pending-load=1#038;ssl=1" srcset="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class=" jetpack-lazy-image"><noscript><img alt="" src="https://i0.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/96ebd7142a2cf809224f5d8e44d1291d/liftcurve.png?w=1440&ssl=1" style="width: 593px; height: 481px;" data-recalc-dims="1"/></noscript></p> <p class="has-line-data" data-line-end="84" data-line-start="83"><img alt src="https://i0.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/d2c0beac4679bc96e13d91f567acb5bc/waterfall.png?w=1440&ssl=1" style="width: 591px; height: 457px;" data-recalc-dims="1" data-lazy-src="https://i0.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/d2c0beac4679bc96e13d91f567acb5bc/waterfall.png?w=1440&is-pending-load=1#038;ssl=1" srcset="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class=" jetpack-lazy-image"><noscript><img alt="" src="https://i0.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/d2c0beac4679bc96e13d91f567acb5bc/waterfall.png?w=1440&ssl=1" style="width: 591px; height: 457px;" data-recalc-dims="1"/></noscript></p> <p class="has-line-data" data-line-end="84" data-line-start="83">In this use case, the lift chart and waterfall chart probably fit better for stakeholders to understand the impact of the model. From the charts, we can see the benefit of applying our model in targeting customers. In the case where a company has few resources to target customers, we can probably only focus on the customer segments with percentage of actual customers above the dash line. This is because the dashed line is the percentage of actual customers, which is the ratio that can be achieved by just targeting customers randomly. Targeting customer segments above the dashed line will lead to reaching more actual customers.</p> <p class="has-line-data" data-line-end="91" data-line-start="90">ROC and AUC are widely used metric by data scientists. Although it may not appear easy to explain to stakeholders, it is still a valuable metric for checking model performance for the following reasons:</p> <ol> <li class="has-line-data" data-line-end="94" data-line-start="92"> <p class="has-line-data" data-line-end="93" data-line-start="92">ROC and AUC do not require a particular threshold. This is convenient because we do not need to come up with a threshold, which is required in precision and recall.</p> </li> <li class="has-line-data" data-line-end="96" data-line-start="94"> <p class="has-line-data" data-line-end="95" data-line-start="94">ROC and AUC measure how correctly the data is ranked. This is useful in our use case because we actually do not care about how accurate the probability score is. What we need to do is to prioritize the customers to target.</p> </li> </ol> <p class="has-line-data" data-line-end="97" data-line-start="96">For scalability, we implemented the AUC computation by calling a SQL query using oml.cursor. This is an efficient and fast computation using Oracle SQL window functions to speed up the computation of AUC score. Based on an experiment on expanded dataset with 11 million rows, the open source approach, which requires pulling the data into the memory, takes 27 minutes. While using the following in-DB approach, it only takes 52 seconds! Notice that we invoke SQL from within our Python function using the database cursor object, which gives us access to the database using the execute function.</p> <pre> <code class="has-line-data" data-line-end="133" data-line-start="99"><span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">auc_score</span><span class="hljs-params">(table_name, prob, target)</span>:</span> <span class="hljs-keyword">import</span> oml cr = oml.cursor() query_template = <span class="hljs-string">""" WITH pos_prob_and_counts AS ( SELECT <PROB1> pos_prob, DECODE(<TARGET>, 1, 1, 0) pos_cnt FROM <TABLE> ), tpf_fpf AS ( SELECT pos_cnt, SUM(pos_cnt) OVER (ORDER BY pos_prob DESC) /SUM(pos_cnt) OVER () tpf, SUM(1 - pos_cnt) OVER (ORDER BY pos_prob DESC) / SUM(1 - pos_cnt) OVER () fpf FROM pos_prob_and_counts ), trapezoid_areas AS ( SELECT 0.5 * (fpf - LAG(fpf, 1, 0) OVER (ORDER BY fpf, tpf))*(tpf + LAG(tpf, 1, 0) OVER (ORDER BY fpf, tpf)) area FROM tpf_fpf WHERE pos_cnt = 1 OR (tpf = 1 AND fpf = 1) ) SELECT SUM(area) auc FROM trapezoid_areas"""</span> query = query_template.replace(<span class="hljs-string">'<PROB1>'</span>, prob) query = query.replace(<span class="hljs-string">'<TARGET>'</span>, target) query = query.replace(<span class="hljs-string">'<TABLE>'</span>, table_name) _ = cr.execute(query) auc = cr.fetchall() cr.close() <span class="hljs-keyword">return</span> auc[<span class="hljs-number">0</span>][<span class="hljs-number">0</span>] </code></pre> <p class="has-line-data" data-line-end="135" data-line-start="134">To show the ROC curve, we can prepare a view by adapting the SQL query used in the function auc_score.</p> <pre> <code class="has-line-data" data-line-end="149" data-line-start="137">create <span class="hljs-keyword">or</span> replace view roc_data_v <span class="hljs-keyword">as</span> WITH pos_prob_and_counts AS ( SELECT PREDICTION_PROBABILITY(GLM_METRIC_MDL, <span class="hljs-number">1</span> USING *) pos_prob, DECODE(BUY_INSURANCE, <span class="hljs-number">1</span>, <span class="hljs-number">1</span>, <span class="hljs-number">0</span>) pos_cnt FROM CUST_TEST_TBL ) SELECT pos_cnt, SUM(pos_cnt) OVER (ORDER BY pos_prob DESC) / SUM(pos_cnt) OVER () tpf, SUM(<span class="hljs-number">1</span> - pos_cnt) OVER (ORDER BY pos_prob DESC) /SUM(<span class="hljs-number">1</span> - pos_cnt) OVER () fpf FROM pos_prob_and_counts </code></pre> <p class="has-line-data" data-line-end="150" data-line-start="149">Then we can generate the plot using the following Python code.</p> <pre> <code class="has-line-data" data-line-end="160" data-line-start="151">fig, ax = plt.subplots(nrows=<span class="hljs-number">1</span>, ncols=<span class="hljs-number">1</span>,figsize=[<span class="hljs-number">10</span>,<span class="hljs-number">8</span>]) x = ROC_DF[<span class="hljs-string">'FPF'</span>].pull() y = ROC_DF[<span class="hljs-string">'TPF'</span>].pull() ax.set_title(<span class="hljs-string">'ROC curve'</span>) ax.plot(x, y, color=<span class="hljs-string">'blue'</span>, lw=<span class="hljs-number">2</span>, label=<span class="hljs-string">'ROC curve'</span>) ax.plot([<span class="hljs-number">0</span>, <span class="hljs-number">1</span>], [<span class="hljs-number">0</span>, <span class="hljs-number">1</span>], lw=<span class="hljs-number">2</span>, linestyle=<span class="hljs-string">'--'</span>, color=<span class="hljs-string">'grey'</span>, label=<span class="hljs-string">'Random guess'</span>) ax.legend(loc=<span class="hljs-string">"lower right"</span>) plt.show() </code></pre> <p class="has-line-data" data-line-end="162" data-line-start="161"><img alt src="https://i0.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/5536a5c50cf32f038fcb0391b1474368/roccurve.png?w=1440&ssl=1" style="width: 720px; height: 576px;" data-recalc-dims="1" data-lazy-src="https://i0.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/5536a5c50cf32f038fcb0391b1474368/roccurve.png?w=1440&is-pending-load=1#038;ssl=1" srcset="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class=" jetpack-lazy-image"><noscript><img alt="" src="https://i0.wp.com/cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/00b40098-051d-415c-be23-4ceb933d5311/Image/5536a5c50cf32f038fcb0391b1474368/roccurve.png?w=1440&ssl=1" style="width: 720px; height: 576px;" data-recalc-dims="1"/></noscript></p> <p class="has-line-data" data-line-end="162" data-line-start="161"><a id="Conclusion_163"/>Conclusion</p> <p class="has-line-data" data-line-end="165" data-line-start="164">In this blog, we discussed popular metrics for classification and presented a way of computing those metrics using Oracle SQL and OML4Py. We discussed the pros and cons of using those metrics and focused on how to interpret each metric and explain the impact of the predictions generated by the model.</p> </p></div> <p><br /> <br /><a href="https://blogs.oracle.com/machinelearning/metrics-for-classification-using-oml4py-part-ii"> Source link </a></p> <div class="post-views post-1516 entry-meta"> <span class="post-views-icon dashicons dashicons-chart-bar"></span> <span class="post-views-label">Post Views:</span> <span class="post-views-count">50</span> </div><!-- AddThis Advanced Settings above via filter on the_content --><!-- AddThis Advanced Settings below via filter on the_content --><!-- AddThis Advanced Settings generic via filter on the_content --><!-- AddThis Share Buttons above via filter on the_content --><!-- AddThis Share Buttons below via filter on the_content --><div class="at-below-post addthis_tool" data-url="https://machinelearningmastery.in/2021/06/11/metrics-for-classification-using-oml4py-part-ii/"></div><!-- AddThis Share Buttons generic via filter on the_content --><div class="sharedaddy sd-sharing-enabled"><div class="robots-nocontent sd-block sd-social sd-social-official sd-sharing"><h3 class="sd-title">Share this:</h3><div class="sd-content"><ul><li class="share-twitter"><a href="https://twitter.com/share" class="twitter-share-button" data-url="https://machinelearningmastery.in/2021/06/11/metrics-for-classification-using-oml4py-part-ii/" data-text="Metrics for Classification Using OML4Py Part II" data-via="sitworld" >Tweet</a></li><li class="share-facebook"><div class="fb-share-button" data-href="https://machinelearningmastery.in/2021/06/11/metrics-for-classification-using-oml4py-part-ii/" data-layout="button_count"></div></li><li class="share-linkedin"><div class="linkedin_button"><script type="in/share" data-url="https://machinelearningmastery.in/2021/06/11/metrics-for-classification-using-oml4py-part-ii/" data-counter="right"></script></div></li><li class="share-reddit"><div class="reddit_button"><iframe src="https://www.reddit.com/static/button/button1.html?newwindow=true&width=120&url=https%3A%2F%2Fmachinelearningmastery.in%2F2021%2F06%2F11%2Fmetrics-for-classification-using-oml4py-part-ii%2F&title=Metrics%20for%20Classification%20Using%20OML4Py%20Part%20II" height="22" width="120" scrolling="no" frameborder="0"></iframe></div></li><li class="share-telegram"><a rel="nofollow noopener noreferrer" data-shared="" class="share-telegram sd-button" href="https://machinelearningmastery.in/2021/06/11/metrics-for-classification-using-oml4py-part-ii/?share=telegram" target="_blank" title="Click to share on Telegram"><span>Telegram</span></a></li><li class="share-jetpack-whatsapp"><a rel="nofollow noopener noreferrer" data-shared="" class="share-jetpack-whatsapp sd-button" href="https://machinelearningmastery.in/2021/06/11/metrics-for-classification-using-oml4py-part-ii/?share=jetpack-whatsapp" target="_blank" title="Click to share on WhatsApp"><span>WhatsApp</span></a></li><li class="share-print"><a rel="nofollow noopener noreferrer" data-shared="" class="share-print sd-button" href="https://machinelearningmastery.in/2021/06/11/metrics-for-classification-using-oml4py-part-ii/#print" target="_blank" title="Click to print"><span>Print</span></a></li><li class="share-tumblr"><a class="tumblr-share-button" target="_blank" href="https://www.tumblr.com/share" data-title="Metrics for Classification Using OML4Py Part II" data-content="https://machinelearningmastery.in/2021/06/11/metrics-for-classification-using-oml4py-part-ii/" title="Share on Tumblr">Share on Tumblr</a></li><li class="share-pinterest"><div class="pinterest_button"><a href="https://www.pinterest.com/pin/create/button/?url=https%3A%2F%2Fmachinelearningmastery.in%2F2021%2F06%2F11%2Fmetrics-for-classification-using-oml4py-part-ii%2F&media=https%3A%2F%2Fi0.wp.com%2Fmachinelearningmastery.in%2Fwp-content%2Fuploads%2F2021%2F07%2Fprcurve.png%3Ffit%3D720%252C576%26ssl%3D1&description=Metrics%20for%20Classification%20Using%20OML4Py%20Part%20II" data-pin-do="buttonPin" data-pin-config="beside"><img src="https://i2.wp.com/assets.pinterest.com/images/pidgets/pinit_fg_en_rect_gray_20.png?w=1440" data-recalc-dims="1" data-lazy-src="https://i2.wp.com/assets.pinterest.com/images/pidgets/pinit_fg_en_rect_gray_20.png?w=1440&is-pending-load=1" srcset="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class=" jetpack-lazy-image"><noscript><img src="https://i2.wp.com/assets.pinterest.com/images/pidgets/pinit_fg_en_rect_gray_20.png?w=1440" data-recalc-dims="1" /></noscript></a></div></li><li class="share-skype"><div class="skype-share" data-href="https://machinelearningmastery.in/2021/06/11/metrics-for-classification-using-oml4py-part-ii/" data-lang="en-US" data-style="small" data-source="jetpack" ></div></li><li class="share-email"><a rel="nofollow noopener noreferrer" data-shared="" class="share-email sd-button" href="https://machinelearningmastery.in/2021/06/11/metrics-for-classification-using-oml4py-part-ii/?share=email" target="_blank" title="Click to email this to a friend"><span>Email</span></a></li><li class="share-end"></li></ul></div></div></div><div class='sharedaddy sd-block sd-like jetpack-likes-widget-wrapper jetpack-likes-widget-unloaded' id='like-post-wrapper-170785677-1516-6145dee5c422b' data-src='https://widgets.wp.com/likes/#blog_id=170785677&post_id=1516&origin=machinelearningmastery.in&obj_id=170785677-1516-6145dee5c422b' data-name='like-post-frame-170785677-1516-6145dee5c422b'><h3 class="sd-title">Like this:</h3><div class='likes-widget-placeholder post-likes-widget-placeholder' style='height: 55px;'><span class='button'><span>Like</span></span> <span class="loading">Loading...</span></div><span class='sd-text-color'></span><a class='sd-link-color'></a></div> <div id='jp-relatedposts' class='jp-relatedposts' > <h3 class="jp-relatedposts-headline"><em>Related</em></h3> </div> </div> </div><!-- .entry-content --> <div class="screen-reader-text" itemprop="datePublished" itemtype="https://schema.org/Date">2021-06-11</div> </article><!-- .entry --> <div id="loop-nav-wrap" class="loop-nav"><div class="prev">Previous Post: <a href="https://machinelearningmastery.in/2021/06/02/edelweiss-improves-cross-sell-using-machine-learning-on-amazon-sagemaker/" rel="prev">Edelweiss improves cross-sell using machine learning on Amazon SageMaker</a></div><div class="next">Next Post: <a href="https://machinelearningmastery.in/2021/06/16/implementing-automation-in-procurement/" rel="next">Implementing automation in procurement</a></div></div><!-- .loop-nav --> <section id="comments-template"> <div id="respond" class="comment-respond"> <h3 id="reply-title" class="comment-reply-title">Leave a Reply <small><a rel="nofollow" id="cancel-comment-reply-link" href="/2021/06/11/metrics-for-classification-using-oml4py-part-ii/#respond" style="display:none;">Cancel reply</a></small></h3><form action="https://machinelearningmastery.in/wp-comments-post.php" method="post" id="commentform" class="comment-form" novalidate><p class="comment-notes"><span id="email-notes">Your email address will not be published.</span></p><p class="comment-form-comment"><label for="comment">Comment</label> <textarea id="comment" name="comment" cols="45" rows="8" maxlength="65525" required="required"></textarea></p><p class="comment-form-author"><label for="author">Name</label> <input id="author" name="author" type="text" value="" size="30" maxlength="245" /></p> <p class="comment-form-email"><label for="email">Email</label> <input id="email" name="email" type="email" value="" size="30" maxlength="100" aria-describedby="email-notes" /></p> <p class="comment-form-url"><label for="url">Website</label> <input id="url" name="url" type="url" value="" size="30" maxlength="200" /></p> <p class="comment-form-cookies-consent"><input id="wp-comment-cookies-consent" name="wp-comment-cookies-consent" type="checkbox" value="yes" /> <label for="wp-comment-cookies-consent">Save my name, email, and website in this browser for the next time I comment.</label></p> <p class="form-submit"><input name="submit" type="submit" id="submit" class="submit" value="Post Comment" /> <input type='hidden' name='comment_post_ID' value='1516' id='comment_post_ID' /> <input type='hidden' name='comment_parent' id='comment_parent' value='0' /> </p><p style="display: none;"><input type="hidden" id="akismet_comment_nonce" name="akismet_comment_nonce" value="16eba0d5cd" /></p><input type="hidden" id="ak_js" name="ak_js" value="241"/><textarea name="ak_hp_textarea" cols="45" rows="8" maxlength="100" style="display: none !important;"></textarea></form> </div><!-- #respond --> </section><!-- #comments-template --> </div><!-- #content-wrap --> </main><!-- #content --> <aside id="sidebar-primary" class="sidebar sidebar-primary hgrid-span-3 layout-narrow-right " role="complementary" itemscope="itemscope" itemtype="https://schema.org/WPSideBar"> <div class=" sidebar-wrap"> <section id="tag_cloud-3" class="widget widget_tag_cloud"><h3 class="widget-title"><span>Categories</span></h3><div class="tagcloud"><a href="https://machinelearningmastery.in/category/articles/" class="tag-cloud-link tag-link-404 tag-link-position-1" style="font-size: 11.529411764706pt;" aria-label="Articles (7 items)">Articles</a> <a href="https://machinelearningmastery.in/category/automation-anywhere/" class="tag-cloud-link tag-link-158 tag-link-position-2" style="font-size: 9.0588235294118pt;" aria-label="Automation Anywhere (2 items)">Automation Anywhere</a> <a href="https://machinelearningmastery.in/category/certification/" class="tag-cloud-link tag-link-12 tag-link-position-3" style="font-size: 10.352941176471pt;" aria-label="Certification (4 items)">Certification</a> <a href="https://machinelearningmastery.in/category/cloud/" class="tag-cloud-link tag-link-289 tag-link-position-4" style="font-size: 10.352941176471pt;" aria-label="Cloud (4 items)">Cloud</a> <a href="https://machinelearningmastery.in/category/code/" class="tag-cloud-link tag-link-511 tag-link-position-5" style="font-size: 8pt;" aria-label="Code (1 item)">Code</a> <a href="https://machinelearningmastery.in/category/database-2/" class="tag-cloud-link tag-link-593 tag-link-position-6" style="font-size: 8pt;" aria-label="Database (1 item)">Database</a> <a href="https://machinelearningmastery.in/category/data-science/" class="tag-cloud-link tag-link-9 tag-link-position-7" style="font-size: 12.117647058824pt;" aria-label="Data Science (9 items)">Data Science</a> <a href="https://machinelearningmastery.in/category/data-science-topics/" class="tag-cloud-link tag-link-530 tag-link-position-8" style="font-size: 9.0588235294118pt;" aria-label="data science topics (2 items)">data science topics</a> <a href="https://machinelearningmastery.in/category/data-science-update/" class="tag-cloud-link tag-link-13 tag-link-position-9" style="font-size: 22pt;" aria-label="Data Science Update (475 items)">Data Science Update</a> <a href="https://machinelearningmastery.in/category/deep-learning/" class="tag-cloud-link tag-link-290 tag-link-position-10" style="font-size: 11.235294117647pt;" aria-label="Deep Learning (6 items)">Deep Learning</a> <a href="https://machinelearningmastery.in/category/financial-assistance/" class="tag-cloud-link tag-link-8 tag-link-position-11" style="font-size: 8pt;" aria-label="Financial assistance (1 item)">Financial assistance</a> <a href="https://machinelearningmastery.in/category/google-cloud/" class="tag-cloud-link tag-link-583 tag-link-position-12" style="font-size: 9.0588235294118pt;" aria-label="Google Cloud (2 items)">Google Cloud</a> <a href="https://machinelearningmastery.in/category/interview-tips/" class="tag-cloud-link tag-link-181 tag-link-position-13" style="font-size: 8pt;" aria-label="Interview tips (1 item)">Interview tips</a> <a href="https://machinelearningmastery.in/category/machine-learning/" class="tag-cloud-link tag-link-11 tag-link-position-14" style="font-size: 15.647058823529pt;" aria-label="Machine Learning (39 items)">Machine Learning</a> <a href="https://machinelearningmastery.in/category/open-data-source/" class="tag-cloud-link tag-link-207 tag-link-position-15" style="font-size: 9.7647058823529pt;" aria-label="Open Data Source (3 items)">Open Data Source</a> <a href="https://machinelearningmastery.in/category/power-bi/" class="tag-cloud-link tag-link-341 tag-link-position-16" style="font-size: 9.0588235294118pt;" aria-label="Power BI (2 items)">Power BI</a> <a href="https://machinelearningmastery.in/category/project-management/" class="tag-cloud-link tag-link-409 tag-link-position-17" style="font-size: 9.7647058823529pt;" aria-label="Project Management (3 items)">Project Management</a> <a href="https://machinelearningmastery.in/category/python/" class="tag-cloud-link tag-link-2 tag-link-position-18" style="font-size: 12.117647058824pt;" aria-label="Python (9 items)">Python</a> <a href="https://machinelearningmastery.in/category/quiz-of-the-day/" class="tag-cloud-link tag-link-429 tag-link-position-19" style="font-size: 8pt;" aria-label="Quiz of the Day (1 item)">Quiz of the Day</a> <a href="https://machinelearningmastery.in/category/robotic-process-automation/" class="tag-cloud-link tag-link-159 tag-link-position-20" style="font-size: 9.0588235294118pt;" aria-label="Robotic Process Automation (2 items)">Robotic Process Automation</a> <a href="https://machinelearningmastery.in/category/r-programming/" class="tag-cloud-link tag-link-157 tag-link-position-21" style="font-size: 9.0588235294118pt;" aria-label="R Programming (2 items)">R Programming</a> <a href="https://machinelearningmastery.in/category/sas/" class="tag-cloud-link tag-link-156 tag-link-position-22" style="font-size: 10.823529411765pt;" aria-label="SAS (5 items)">SAS</a> <a href="https://machinelearningmastery.in/category/statistics/" class="tag-cloud-link tag-link-337 tag-link-position-23" style="font-size: 10.352941176471pt;" aria-label="Statistics (4 items)">Statistics</a> <a href="https://machinelearningmastery.in/category/tableau/" class="tag-cloud-link tag-link-340 tag-link-position-24" style="font-size: 9.0588235294118pt;" aria-label="Tableau (2 items)">Tableau</a> <a href="https://machinelearningmastery.in/category/visualization/" class="tag-cloud-link tag-link-10 tag-link-position-25" style="font-size: 12.352941176471pt;" aria-label="visualization (10 items)">visualization</a></div> </section><section id="newsletterwidget-2" class="widget widget_newsletterwidget"><div class="tnp tnp-widget"><form method="post" action="https://machinelearningmastery.in/?na=s"> <input type="hidden" name="nr" value="widget"><input type="hidden" name="nlang" value=""><div class="tnp-field tnp-field-email"><label for="tnp-email">Email</label> <input class="tnp-email" type="email" name="ne" value="" required></div> <div class="tnp-field tnp-field-button"><input class="tnp-submit" type="submit" value="Subscribe" > </div> </form> </div></section> <section id="recent-posts-2" class="widget widget_recent_entries"> <h3 class="widget-title"><span>Recent Posts</span></h3> <ul> <li> <a href="https://machinelearningmastery.in/2021/09/17/paradoxes-in-data-science-kdnuggets/">Paradoxes in Data Science – KDnuggets</a> </li> <li> <a href="https://machinelearningmastery.in/2021/09/17/what-2-years-of-self-teaching-data-science-taught-me/">What 2 years of self-teaching data science taught me</a> </li> <li> <a href="https://machinelearningmastery.in/2021/09/17/introducing-tensorflow-similarity-kdnuggets/">Introducing TensorFlow Similarity – KDnuggets</a> </li> <li> <a href="https://machinelearningmastery.in/2021/09/16/what-is-the-real-difference-between-data-engineers-and-data-scientists/">What Is The Real Difference Between Data Engineers and Data Scientists?</a> </li> <li> <a href="https://machinelearningmastery.in/2021/09/16/adventures-in-mlops-with-github-actions-iterative-ai-label-studio-and-nbdev/">Adventures in MLOps with Github Actions, Iterative.ai, Label Studio and NBDEV</a> </li> <li> <a href="https://machinelearningmastery.in/2021/09/16/the-machine-deep-learning-compendium-open-book/">The Machine & Deep Learning Compendium Open Book</a> </li> <li> <a href="https://machinelearningmastery.in/2021/09/16/easy-sql-in-native-python/">Easy SQL in Native Python</a> </li> <li> <a href="https://machinelearningmastery.in/2021/09/15/launch-amazon-sagemaker-studio-from-external-applications-using-presigned-urls/">Launch Amazon SageMaker Studio from external applications using presigned URLs</a> </li> <li> <a href="https://machinelearningmastery.in/2021/09/15/kdnuggets-top-blogs-rewards-for-august-2021/">KDnuggets Top Blogs Rewards for August 2021</a> </li> <li> <a href="https://machinelearningmastery.in/2021/09/15/datacated-expo-oct-5-live-streamedexplore-new-ai-data-science-tech/">DATAcated Expo, Oct 5, Live-streamed,Explore new AI / Data Science Tech</a> </li> </ul> </section> </div><!-- .sidebar-wrap --> </aside><!-- #sidebar-primary --> </div><!-- .main-content-grid --> </div><!-- #main --> <footer id="footer" class="site-footer footer hgrid-stretch inline-nav" role="contentinfo" itemscope="itemscope" itemtype="https://schema.org/WPFooter"> <div class="hgrid"> <div class="hgrid-span-6 footer-column"> <section id="hootkit-ticker-9" class="widget widget_hootkit-ticker"> <div class="ticker-widget ticker-usercontent ticker-simple ticker-userstyle ticker-withbg ticker-style1" style="background:#f1f1f1;color:#ff4530;" ><i class="fa-weixin fab ticker-icon"></i> <div class="ticker-msg-box" data-speed='0.03'> <div class="ticker-msgs"> <div class="ticker-msg"><div class="ticker-msg-inner">Subscribe for the latest news, updates, tips and more delivered right to your inbox.</div></div> </div> </div> </div></section> </div> <div class="hgrid-span-3 footer-column"> <section id="media_image-13" class="widget widget_media_image"><img width="220" height="49" src="https://i2.wp.com/machinelearningmastery.in/wp-content/uploads/2019/12/Machine-Learning-Mastery-banner.gif?fit=220%2C49&ssl=1" class="image wp-image-127 attachment-full size-full jetpack-lazy-image" alt="" loading="lazy" style="max-width: 100%; height: auto;" data-lazy-src="https://i2.wp.com/machinelearningmastery.in/wp-content/uploads/2019/12/Machine-Learning-Mastery-banner.gif?fit=220%2C49&ssl=1&is-pending-load=1" srcset="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" /></section> </div> <div class="hgrid-span-3 footer-column"> <section id="hootkit-social-icons-8" class="widget widget_hootkit-social-icons"> <div class="social-icons-widget social-icons-small"><a href="https://github.com/machinelearningmasteryindia" class=" social-icons-icon fa-github-block" target="_blank"> <i class="fa-github fab"></i> </a><a href="mailto:machinelearningmasteryindia@gmail.com" class=" social-icons-icon fa-envelope-block"> <i class="fa-envelope fas"></i> </a><a href="https://www.linkedin.com/in/machine-learning-b065081a9/" class=" social-icons-icon fa-linkedin-block" target="_blank"> <i class="fa-linkedin-in fab"></i> </a><a href="https://twitter.com/sitworld" class=" social-icons-icon fa-twitter-block" target="_blank"> <i class="fa-twitter fab"></i> </a></div></section> </div> </div> </footer><!-- #footer --> <div id="post-footer" class=" post-footer hgrid-stretch linkstyle"> <div class="hgrid"> <div class="hgrid-span-12"> <p class="credit small"> <a class="privacy-policy-link" href="https://machinelearningmastery.in/privacy-policy/">Privacy Policy</a> Designed using <a class="theme-link" href="https://wphoot.com/themes/unos/" title="Unos WordPress Theme">Unos</a>. Powered by <a class="wp-link" href="https://wordpress.org">WordPress</a>. </p><!-- .credit --> </div> </div> </div> </div><!-- #page-wrapper --> <!--googleoff: all--><div id="cookie-law-info-bar" data-nosnippet="true"><span>This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. <a role='button' tabindex='0' class="cli_settings_button" style="margin:5px 20px 5px 20px;" >Cookie settings</a><a role='button' tabindex='0' data-cli_action="accept" id="cookie_action_close_header" class="medium cli-plugin-button cli-plugin-main-button cookie_action_close_header cli_action_button" style="display:inline-block; margin:5px; ">ACCEPT</a></span></div><div id="cookie-law-info-again" style="display:none;" data-nosnippet="true"><span id="cookie_hdr_showagain">Privacy & Cookies Policy</span></div><div class="cli-modal" data-nosnippet="true" id="cliSettingsPopup" tabindex="-1" role="dialog" aria-labelledby="cliSettingsPopup" aria-hidden="true"> <div class="cli-modal-dialog" role="document"> <div class="cli-modal-content cli-bar-popup"> <button type="button" class="cli-modal-close" id="cliModalClose"> <svg class="" viewBox="0 0 24 24"><path d="M19 6.41l-1.41-1.41-5.59 5.59-5.59-5.59-1.41 1.41 5.59 5.59-5.59 5.59 1.41 1.41 5.59-5.59 5.59 5.59 1.41-1.41-5.59-5.59z"></path><path d="M0 0h24v24h-24z" fill="none"></path></svg> <span class="wt-cli-sr-only">Close</span> </button> <div class="cli-modal-body"> <div class="cli-container-fluid cli-tab-container"> <div class="cli-row"> <div class="cli-col-12 cli-align-items-stretch cli-px-0"> <div class="cli-privacy-overview"> <h4>Privacy Overview</h4> <div class="cli-privacy-content"> <div class="cli-privacy-content-text">This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.</div> </div> <a class="cli-privacy-readmore" aria-label="Show more" tabindex="0" role="button" data-readmore-text="Show more" data-readless-text="Show less"></a> </div> </div> <div class="cli-col-12 cli-align-items-stretch cli-px-0 cli-tab-section-container"> <div class="cli-tab-section"> <div class="cli-tab-header"> <a role="button" tabindex="0" class="cli-nav-link cli-settings-mobile" data-target="necessary" data-toggle="cli-toggle-tab"> Necessary </a> <div class="wt-cli-necessary-checkbox"> <input type="checkbox" class="cli-user-preference-checkbox" id="wt-cli-checkbox-necessary" data-id="checkbox-necessary" checked="checked" /> <label class="form-check-label" for="wt-cli-checkbox-necessary">Necessary</label> </div> <span class="cli-necessary-caption">Always Enabled</span> </div> <div class="cli-tab-content"> <div class="cli-tab-pane cli-fade" data-id="necessary"> <div class="wt-cli-cookie-description"> Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information. </div> </div> </div> </div> <div class="cli-tab-section"> <div class="cli-tab-header"> <a role="button" tabindex="0" class="cli-nav-link cli-settings-mobile" data-target="non-necessary" data-toggle="cli-toggle-tab"> Non-necessary </a> <div class="cli-switch"> <input type="checkbox" id="wt-cli-checkbox-non-necessary" class="cli-user-preference-checkbox" data-id="checkbox-non-necessary" checked='checked' /> <label for="wt-cli-checkbox-non-necessary" class="cli-slider" data-cli-enable="Enabled" data-cli-disable="Disabled"><span class="wt-cli-sr-only">Non-necessary</span></label> </div> </div> <div class="cli-tab-content"> <div class="cli-tab-pane cli-fade" data-id="non-necessary"> <div class="wt-cli-cookie-description"> Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website. </div> </div> </div> </div> </div> </div> </div> </div> <div class="cli-modal-footer"> <div class="wt-cli-element cli-container-fluid cli-tab-container"> <div class="cli-row"> <div class="cli-col-12 cli-align-items-stretch cli-px-0"> <div class="cli-tab-footer wt-cli-privacy-overview-actions"> <a id="wt-cli-privacy-save-btn" role="button" tabindex="0" data-cli-action="accept" class="wt-cli-privacy-btn cli_setting_save_button wt-cli-privacy-accept-btn cli-btn">SAVE & ACCEPT</a> </div> </div> </div> </div> </div> </div> </div> </div> <div class="cli-modal-backdrop cli-fade cli-settings-overlay"></div> <div class="cli-modal-backdrop cli-fade cli-popupbar-overlay"></div> <!--googleon: all--> <div id="fb-root"></div> <script async defer crossorigin="anonymous" src="https://connect.facebook.net/en_US/sdk.js#xfbml=1&version=v8.0&appId=683648729088349&autoLogAppEvents=1"> </script> <!--Start of Tawk.to Script (0.5.5)--> <script type="text/javascript"> var Tawk_API=Tawk_API||{}; var Tawk_LoadStart=new Date(); (function(){ var s1=document.createElement("script"),s0=document.getElementsByTagName("script")[0]; s1.async=true; s1.src='https://embed.tawk.to/5ec04a92967ae56c521a742a/default'; s1.charset='UTF-8'; s1.setAttribute('crossorigin','*'); s0.parentNode.insertBefore(s1,s0); })(); </script> <!--End of Tawk.to Script (0.5.5)--> <script type="text/javascript"> window.WPCOM_sharing_counts = {"https:\/\/machinelearningmastery.in\/2021\/06\/11\/metrics-for-classification-using-oml4py-part-ii\/":1516}; </script> <script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script> <div id="fb-root"></div> <script>(function(d, s, id) { var js, fjs = d.getElementsByTagName(s)[0]; if (d.getElementById(id)) return; js = d.createElement(s); js.id = id; js.src = 'https://connect.facebook.net/en_US/sdk.js#xfbml=1&appId=249643311490&version=v2.3'; fjs.parentNode.insertBefore(js, fjs); }(document, 'script', 'facebook-jssdk'));</script> <script> document.body.addEventListener( 'is.post-load', function() { if ( 'undefined' !== typeof FB ) { FB.XFBML.parse(); } } ); </script> <script type="text/javascript"> ( function () { var currentScript = document.currentScript; // Helper function to load an external script. function loadScript( url, cb ) { var script = document.createElement( 'script' ); var prev = currentScript || document.getElementsByTagName( 'script' )[ 0 ]; script.setAttribute( 'async', true ); script.setAttribute( 'src', url ); prev.parentNode.insertBefore( script, prev ); script.addEventListener( 'load', cb ); } function init() { loadScript( 'https://platform.linkedin.com/in.js?async=true', function () { if ( typeof IN !== 'undefined' ) { IN.init(); } } ); } if ( document.readyState === 'loading' ) { document.addEventListener( 'DOMContentLoaded', init ); } else { init(); } document.body.addEventListener( 'is.post-load', function() { if ( typeof IN !== 'undefined' ) { IN.parse(); } } ); } )(); </script> <script id="tumblr-js" type="text/javascript" src="https://assets.tumblr.com/share-button.js"></script> <script type="text/javascript"> ( function () { // Pinterest shared resources var s = document.createElement( 'script' ); s.type = 'text/javascript'; s.async = true; s.setAttribute( 'data-pin-hover', true ); s.src = window.location.protocol + '//assets.pinterest.com/js/pinit.js'; var x = document.getElementsByTagName( 'script' )[ 0 ]; x.parentNode.insertBefore(s, x); // if 'Pin it' button has 'counts' make container wider function init() { var shares = document.querySelectorAll( 'li.share-pinterest' ); for ( var i = 0; i < shares.length; i++ ) { var share = shares[ i ]; if ( share.querySelector( 'a span:visible' ) ) { share.style.width = '80px'; } } } if ( document.readyState !== 'complete' ) { document.addEventListener( 'load', init ); } else { init(); } } )(); </script> <script> (function(r, d, s) { r.loadSkypeWebSdkAsync = r.loadSkypeWebSdkAsync || function(p) { var js, sjs = d.getElementsByTagName(s)[0]; if (d.getElementById(p.id)) { return; } js = d.createElement(s); js.id = p.id; js.src = p.scriptToLoad; js.onload = p.callback sjs.parentNode.insertBefore(js, sjs); }; var p = { scriptToLoad: 'https://swx.cdn.skype.com/shared/v/latest/skypewebsdk.js', id: 'skype_web_sdk' }; r.loadSkypeWebSdkAsync(p); })(window, document, 'script'); </script> <div id="sharing_email" style="display: none;"> <form action="/2021/06/11/metrics-for-classification-using-oml4py-part-ii/" method="post"> <label for="target_email">Send to Email Address</label> <input type="email" name="target_email" id="target_email" value="" /> <label for="source_name">Your Name</label> <input type="text" name="source_name" id="source_name" value="" /> <label for="source_email">Your Email Address</label> <input type="email" name="source_email" id="source_email" value="" /> <input type="text" id="jetpack-source_f_name" name="source_f_name" class="input" value="" size="25" autocomplete="off" title="This field is for validation and should not be changed" /> <img style="float: right; display: none" class="loading" src="https://machinelearningmastery.in/wp-content/plugins/jetpack/modules/sharedaddy/images/loading.gif" alt="loading" width="16" height="16" /> <input type="submit" value="Send Email" class="sharing_send" /> <a rel="nofollow" href="#cancel" class="sharing_cancel" role="button">Cancel</a> <div class="errors errors-1" style="display: none;"> Post was not sent - check your email addresses! </div> <div class="errors errors-2" style="display: none;"> Email check failed, please try again </div> <div class="errors errors-3" style="display: none;"> Sorry, your blog cannot share posts by email. </div> </form> </div> <script data-cfasync="false" type="text/javascript">if (window.addthis_product === undefined) { window.addthis_product = "wpp"; } if (window.wp_product_version === undefined) { window.wp_product_version = "wpp-6.2.6"; } if (window.addthis_share === undefined) { window.addthis_share = {}; } if (window.addthis_config === undefined) { window.addthis_config = {"data_track_clickback":true,"ui_atversion":"300"}; } if (window.addthis_plugin_info === undefined) { window.addthis_plugin_info = {"info_status":"enabled","cms_name":"WordPress","plugin_name":"Share Buttons by AddThis","plugin_version":"6.2.6","plugin_mode":"AddThis","anonymous_profile_id":"wp-2f16336e765908d13c2d341ff0393457","page_info":{"template":"posts","post_type":["post","page","e-landing-page"]},"sharing_enabled_on_post_via_metabox":false}; } (function() { var first_load_interval_id = setInterval(function () { if (typeof window.addthis !== 'undefined') { window.clearInterval(first_load_interval_id); if (typeof window.addthis_layers !== 'undefined' && Object.getOwnPropertyNames(window.addthis_layers).length > 0) { window.addthis.layers(window.addthis_layers); } if (Array.isArray(window.addthis_layers_tools)) { for (i = 0; i < window.addthis_layers_tools.length; i++) { window.addthis.layers(window.addthis_layers_tools[i]); } } } },1000) }()); </script><script src='https://machinelearningmastery.in/wp-content/plugins/jetpack/_inc/build/photon/photon.min.js?ver=20191001' id='jetpack-photon-js'></script> <script src='https://machinelearningmastery.in/wp-includes/js/comment-reply.min.js?ver=5.6.5' id='comment-reply-js'></script> <script id='hoverIntent-js-extra'> var hootData = {"stickySidebar":"disable","contentblockhover":"enable","contentblockhovertext":"disable"}; </script> <script src='https://machinelearningmastery.in/wp-includes/js/hoverIntent.min.js?ver=1.8.1' id='hoverIntent-js'></script> <script src='https://machinelearningmastery.in/wp-content/themes/unos/js/jquery.superfish.min.js?ver=1.7.5' id='jquery-superfish-js'></script> <script src='https://machinelearningmastery.in/wp-content/themes/unos/js/jquery.fitvids.min.js?ver=1.1' id='jquery-fitvids-js'></script> <script src='https://machinelearningmastery.in/wp-content/themes/unos/js/jquery.parallax.min.js?ver=1.4.2' id='jquery-parallax-js'></script> <script id='ap-frontend-js-js-extra'> var ap_form_required_message = ["This field is required","accesspress-anonymous-post"]; var ap_captcha_error_message = ["Sum is not correct.","accesspress-anonymous-post"]; </script> <script src='https://machinelearningmastery.in/wp-content/plugins/accesspress-anonymous-post/js/frontend.js?ver=2.8.1' id='ap-frontend-js-js'></script> <script src='https://machinelearningmastery.in/wp-content/plugins/hootkit/assets/jquery.lightSlider.min.js?ver=1.1.2' id='jquery-lightSlider-js'></script> <script src='https://machinelearningmastery.in/wp-content/plugins/hootkit/assets/widgets.min.js?ver=2.0.7' id='hootkit-widgets-js'></script> <script id='hootkit-miscmods-js-extra'> var hootkitMiscmodsData = {"ajaxurl":"https:\/\/machinelearningmastery.in\/wp-admin\/admin-ajax.php"}; </script> <script src='https://machinelearningmastery.in/wp-content/plugins/hootkit/assets/miscmods.min.js?ver=2.0.7' id='hootkit-miscmods-js'></script> <script src='https://machinelearningmastery.in/wp-content/plugins/page-links-to/dist/new-tab.js?ver=3.3.5' id='page-links-to-js'></script> <script src='https://s7.addthis.com/js/300/addthis_widget.js?ver=5.6.5#pubid=ra-5e0c443d44eeeb15' id='addthis_widget-js'></script> <script src='https://machinelearningmastery.in/wp-content/plugins/jetpack/vendor/automattic/jetpack-lazy-images/src/js/intersectionobserver-polyfill.min.js?ver=1.1.2' id='jetpack-lazy-images-polyfill-intersectionobserver-js'></script> <script id='jetpack-lazy-images-js-extra'> var jetpackLazyImagesL10n = {"loading_warning":"Images are still loading. Please cancel your print and try again."}; </script> <script src='https://machinelearningmastery.in/wp-content/plugins/jetpack/vendor/automattic/jetpack-lazy-images/src/js/lazy-images.min.js?ver=1.1.2' id='jetpack-lazy-images-js'></script> <script src='https://machinelearningmastery.in/wp-content/plugins/jetpack/_inc/build/postmessage.min.js?ver=9.8.1' id='postmessage-js'></script> <script src='https://machinelearningmastery.in/wp-content/plugins/jetpack/_inc/build/jquery.jetpack-resize.min.js?ver=9.8.1' id='jetpack_resize-js'></script> <script src='https://machinelearningmastery.in/wp-content/plugins/jetpack/_inc/build/likes/queuehandler.min.js?ver=9.8.1' id='jetpack_likes_queuehandler-js'></script> <script src='https://machinelearningmastery.in/wp-content/themes/unos/js/hoot.theme.min.js?ver=2.9.11' id='hoot-theme-js'></script> <script src='https://machinelearningmastery.in/wp-content/plugins/youtube-embed-plus/scripts/fitvids.min.js?ver=13.4.3' id='__ytprefsfitvids__-js'></script> <script id='wpgdprc.js-js-extra'> var wpgdprcData = {"ajaxURL":"https:\/\/machinelearningmastery.in\/wp-admin\/admin-ajax.php","ajaxSecurity":"5f20633e8d","isMultisite":"","path":"\/","blogId":""}; </script> <script src='https://machinelearningmastery.in/wp-content/plugins/wp-gdpr-compliance/dist/js/front.min.js?ver=1629244814' id='wpgdprc.js-js'></script> <script src='https://machinelearningmastery.in/wp-includes/js/wp-embed.min.js?ver=5.6.5' id='wp-embed-js'></script> <script id='jetpack_related-posts-js-extra'> var related_posts_js_options = {"post_heading":"h4"}; </script> <script src='https://machinelearningmastery.in/wp-content/plugins/jetpack/_inc/build/related-posts/related-posts.min.js?ver=20210604' id='jetpack_related-posts-js'></script> <script defer src='https://machinelearningmastery.in/wp-content/plugins/akismet/_inc/form.js?ver=4.1.12' id='akismet-form-js'></script> <script id='sharing-js-js-extra'> var sharing_js_options = {"lang":"en","counts":"1","is_stats_active":"1"}; </script> <script src='https://machinelearningmastery.in/wp-content/plugins/jetpack/_inc/build/sharedaddy/sharing.min.js?ver=9.8.1' id='sharing-js-js'></script> <script id='sharing-js-js-after'> var windowOpen; ( function () { function matches( el, sel ) { return !! ( el.matches && el.matches( sel ) || el.msMatchesSelector && el.msMatchesSelector( sel ) ); } document.body.addEventListener( 'click', function ( event ) { if ( ! event.target ) { return; } var el; if ( matches( event.target, 'a.share-facebook' ) ) { el = event.target; } else if ( event.target.parentNode && matches( event.target.parentNode, 'a.share-facebook' ) ) { el = event.target.parentNode; } if ( el ) { event.preventDefault(); // If there's another sharing window open, close it. if ( typeof windowOpen !== 'undefined' ) { windowOpen.close(); } windowOpen = window.open( el.getAttribute( 'href' ), 'wpcomfacebook', 'menubar=1,resizable=1,width=600,height=400' ); return false; } } ); } )(); var windowOpen; ( function () { function matches( el, sel ) { return !! ( el.matches && el.matches( sel ) || el.msMatchesSelector && el.msMatchesSelector( sel ) ); } document.body.addEventListener( 'click', function ( event ) { if ( ! event.target ) { return; } var el; if ( matches( event.target, 'a.share-telegram' ) ) { el = event.target; } else if ( event.target.parentNode && matches( event.target.parentNode, 'a.share-telegram' ) ) { el = event.target.parentNode; } if ( el ) { event.preventDefault(); // If there's another sharing window open, close it. if ( typeof windowOpen !== 'undefined' ) { windowOpen.close(); } windowOpen = window.open( el.getAttribute( 'href' ), 'wpcomtelegram', 'menubar=1,resizable=1,width=450,height=450' ); return false; } } ); } )(); </script> <iframe src='https://widgets.wp.com/likes/master.html?ver=202137#ver=202137' scrolling='no' id='likes-master' name='likes-master' style='display:none;'></iframe> <div id='likes-other-gravatars'><div class="likes-text"><span>%d</span> bloggers like this:</div><ul class="wpl-avatars sd-like-gravatars"></ul></div> <script>!function(){window.advanced_ads_ready_queue=window.advanced_ads_ready_queue||[],advanced_ads_ready_queue.push=window.advanced_ads_ready;for(var d=0,a=advanced_ads_ready_queue.length;d<a;d++)advanced_ads_ready(advanced_ads_ready_queue[d])}();</script><script src='https://stats.wp.com/e-202137.js' defer></script> <script> _stq = window._stq || []; _stq.push([ 'view', {v:'ext',j:'1:9.8.1',blog:'170785677',post:'1516',tz:'-5.5',srv:'machinelearningmastery.in'} ]); _stq.push([ 'clickTrackerInit', '170785677', '1516' ]); </script> </body> </html> <!-- Page generated by LiteSpeed Cache 4.4.1 on 2021-09-18 07:13:17 -->