{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Assignment 11\n", "## Chapter 9" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Student ID: *Double click here to fill the Student ID*\n", "\n", "#### Name: *Double click here to fill the name*\n", "\n", "部分習題需要寫數學式。\n", "\n", "若不想用打的，可用手寫後，進行拍照或掃描，並使用以下指令引入圖片。\n", "\n", "請確認圖片清晰程度為可辨識。\n", "\n", "`![title](test.png)`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2\n", "We have seen that in $p = 2$ dimensions, a linear decision boundary takes the form $β_0+β_1X_1+β_2X_2 = 0$.We now investigate a non-linear decision boundary." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(a) Sketch the curve \n", "\n", "$$(1 + X_1)^2 + (2 − X_2)^2 = 4.$$" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# coding your answer here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> Ans: *double click here to answer the question.*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(b) On your sketch, indicate the set of points for which \n", "\n", "$$(1 + X_1)^2 + (2 − X_2)^2 > 4,$$\n", "\n", "as well as the set of points for which \n", "\n", "$$(1 + X_1)^2 + (2 − X_2)^2 ≤ 4$$." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# coding your answer here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> Ans: *double click here to answer the question.*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(c) Suppose that a classifier assigns an observation to the blue class if\n", "\n", "$$(1 + X_1)^2 + (2 − X_2)^2 > 4,$$\n", "\n", "and to the red class otherwise. To what class is the observation $(0, 0)$ classified? $(−1, 1)$? $(2, 2)$? $(3, 8)$?" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# coding your answer here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> Ans: *double click here to answer the question.*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Argue that while the decision boundary in (c) is not linear in terms of $X_1$ and $X_2$, it is linear in terms of $X_1, X^2_1, X_2,$ and $X^2_2$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> Ans: *double click here to answer the question.*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3\n", "Here we explore the maximal margin classifier on a toy data set." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(a) We are given $n = 7$ observations in $p = 2$ dimensions. For each observation, there is an associated class label." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Obs.X1X2Y
134Red
222Red
344Red
414Red
521Blue
643Blue
741Blue
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Sketch the observations." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# coding your answer here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(b) Sketch the optimal separating hyperplane, and provide the equation for this hyperplane (of the form (9.1)).\n", "\n", "$$β_0 + β_1X_1 + β_2X_2 = 0$$
(9.1)
" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# coding your answer here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> Ans: *double click here to answer the question.*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(c)Describe the classification rule for the maximal margin classifier. It should be something along the lines of “Classify to Red if $β_0 +β_1X_1 +β_2X_2>0$, and classify to Blue otherwise.” Provide\n", "the values for $β_0, β_1,$ and $β_2$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> Ans: *double click here to answer the question.*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(d)On your sketch, indicate the margin for the maximal margin hyperplane." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# coding your answer here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(e) Indicate the support vectors for the maximal margin classifier." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# coding your answer here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(f) Argue that a slight movement of the seventh observation would $not$ affect the maximal margin hyperplane." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> Ans: *double click here to answer the question.*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(g) Sketch a hyperplane that is not the optimal separating hyperplane, and provide the equation for this hyperplane." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# coding your answer here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> Ans: *double click here to answer the question.*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(h) Draw an additional observation on the plot so that the two classes are no longer separable by a hyperplane." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# coding your answer here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5\n", "We have seen that we can fit an **SVM** with a non-linear kernel in order to perform classification using a non-linear decision boundary.We will now see that we can also obtain a non-linear decision boundary by performing logistic regression using non linear transformations of the features." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(a) Generate a data set with $n = 500$ and $p = 2$, such that the observations belong to two classes with a quadratic decision boundary between them. For instance, you can do this as follows:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "```Python\n", "> np.random.seed(1)\n", "> x1 = np.random.rand(500) - 0.5\n", "> x2 = np.random.rand(500) - 0.5\n", "> y = 1 * (x1**2 - x2**2 > 0)\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# coding your answer here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(b) Plot the observations, colored according to their class labels. Your plot should display $X_1$ on the $x$-axis, and $X_2$ on the $y$-axis." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# coding your answer here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> Ans: *double click here to answer the question.*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(c) Fit a logistic regression model to the data, using $X_1$ and $X_2$ as predictors." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# coding your answer here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> Ans: *double click here to answer the question.*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(d) Apply this model to the $training$ $data$ in order to obtain a predicted class label for each training observation. Plot the observations, colored according to the $predicted$ class labels. The decision boundary should be linear." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# coding your answer here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> Ans: *double click here to answer the question.*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(e) Now fit a logistic regression model to the data using non-linear functions of $X_1$ and $X_2$ as predictors (e.g. $X^2_1,$ $X_1×X_2,$ $\\operatorname{log}(X_2),$ and so forth)." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# coding your answer here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> Ans: *double click here to answer the question.*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(f) Apply this model to the $training$ $data$ in order to obtain a predicted class label for each training observation. Plot the observations, colored according to the $predicted$ class labels. The decision boundary should be obviously non-linear. If it is not, then repeat (a)-(e) until you come up with an example in which the predicted class labels are obviously non-linear." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# coding your answer here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> Ans: *double click here to answer the question.*" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.3" } }, "nbformat": 4, "nbformat_minor": 4 }