蒜蓉鸡蛋面
最近学会了手工鸡蛋面，然后就开始各种黑暗尝试。我第一次的黑暗尝试是在面中加葱，所以揉出来的面是绿色的。
叶某人的碎碎念
Believe in Mathematics
Factor Analysis
This article is my notes on the topic of factor analysis. These notes come out of lecture 13 and 14 of Andrew Ng's online course. Roughly speaking, factor analysis models some \(n\) dimensional observed data with the assumption that these data are actually from some \(d\) dimensional plane in \(\R\), up to some Gaussian distributed errors. Let's make it more precise.
Suppose we have a set of observed data \(\{x^{(1)}, \dots, x^{(m)}\}\) implicitly labeled by some latent random variable \(z \in \R^d\) where
$$z \sim \mathcal{N}(0, I).$$
Factor analysis model tries to model \(P(x)\) using the assumption that
$$\begin{equation}
x|z \sim \mathcal{N}(\mu+\Lambda z, \Psi),
\label{cond-xz}
\end{equation}$$
for some \(\mu \in \R^n, \Lambda \in \R^{n \times d}\) and diagonal matrix \(\Psi \in \R^{n \times n}\). These \(\mu, \Lambda\) and \(\Psi\) are parameters of the model.
Expectation-Maximization algorithm
In this article, I will collect my notes on Expectation-Maximization algorithm (EM) based on lecture 12 and 13 of Andrew Ng's online course. Given a set of unlabeled data points EM tries iteratively to determine the distribution of data, assuming that all data points are implicitly labeled (unobserved latent variables). For simplicity, we shall limit ourselves to the case where there are only finitely many implicit labels.
Description of the problem
Given a set of unlabeled data \(\{x^{(1)}, \dots, x^{(m)}\}\), our goal is to determine \(P(x)\), the distribution of \(x\), with the following assumptions.
Assumptions 1 and 2 will gives us a set of parameters \(\theta = (\phi_1, \dots, \phi_j, a_1,\dots, a_j)\) and
$$\begin{equation}
P(x; \theta) = \sum_{j=1}^k P(x|z=j; \theta)P(z=j; \theta).
\label{px}
\end{equation}$$
We want to find this set of parameters so that the likelihood function
$$L(\theta) = \prod_{i=1}^m P(x^{(i)}) = \prod_{i=1}^m \sum_{j=1}^k P(x^{(i)}|z=j; \theta)P(z=j; \theta).$$
is maximized. Or equivalently, the log likelihood function below is maximized:
$$\begin{equation}
l(\theta) = \sum_{i=1}^m \log\left(\sum_{j=1}^k P(x^{(i)}, z=j; \theta)\right),
\label{log-likelihood}
\end{equation}$$
where
$$P(x^{(i)}, z=j; \theta) = P(x^{(i)}|z=j; \theta)P(z=j; \theta).$$
Digit recognition, Softmax
在这篇博客中，我将使用softmax模型来识别手写数字。文章的第一部分是关于softmax模型的理论推导，而第二部分则是模型的实现。softmax的本质是一个线性模型，所以推导所需要的理论在我之前的一篇博客Generalized Linear Model已经详细介绍过了。softmax是逻辑回归(logistic regression)的推广：逻辑回归使用Bernoulli分布（二项分布），而softmax使用多项分布。
Support Vector Machine
This article is my notes on support vector machine for Lecture 7 and 8 of Machine Learning by Andrew Ng.
Intuition
In a binary classification problem, we can use logistic regression
$$h_\theta(x) = \frac{1}{1+e^{-\theta^T x}} = g(\theta^T x),$$
where \(g\) is the sigmoid function with a figure of it below.
Then given input \(x\), the model predicts \(1\) if and only if \(\theta^x \ge 0\), in which case \(h_\theta(x) = g(\theta^T x) \ge 0.5\); and it predicts \(0\) if and only if \(\theta^T x < 0\). Moreover, based on the shape of sigmoid function, if \(\theta^T x >> 0\), we are very confident that \(y=1\). Likewise, if \(\theta^T x << 0\), we are very confident that \(y=0\). Therefore, we hope that for the training set \(\{(x^{(i)}, y^{(i)})\}_{i=1}^m\), we can find such a \(\theta\) that \(\theta^T x^{(i)} >> 0\) if \(y^{(i)}=1\) and \(\theta^T x^{(i)} << 0\) if \(y^{(i)}=0\).
Generative Model
This article is my notes on generative model for Lecture 5 and 6 of Machine Learning by Andrew Ng. What we do in logistic regression using generalized linear model is that, we approximate \(P(y|x)\) using given data. This kind of learning algorithms is discriminative, in which we predict \(y\) based on the input features \(x\). On the contrary, generative model is to model \(P(x|y)\), the probability of the features \(x\) given class \(y\). In other words, we want to study how the features structure looks like given a class \(y\). If we also learn what \(P(y)\) is, we can easily recover \(P(y|x)\), for example, in the binary classification problem,
$$\begin{equation}
P(y=1|x) = \frac{P(x|y=1)P(y=1)}{P(x)},
\label{eqn:bayes}
\end{equation}$$
where \(P(x) = P(x|y=0)P(y=0) + P(x|y=1)P(y=1)\).
In this article, we are going to see a simple example of generative model on Gaussian discriminant analysis and Naive Bayes.
Generalized Linear Model (Examples)
This article is a companion article to my another post Generalized Linear Model. In this article, I will implement some of the learning algorithms in Generalized Linear Model. To be more specific, I will do some examples on linear regression and logistic regression. With some effort, google search gives me some very good example data sets to work with. The datasets collected by Larry Winner is one of the excellent sets, which will be used in the article.
The implementations here use Python. Required 3rd party libraries are:
- Requests: use to get online data
- Matplotlib: use for plotting
- NumPy: use for matrix calculation
Generalized Linear Model
This article on Generalized Linear Model (GLM) is based on the first four lectures of Machine Learning by Andrew Ng. But the structure of the article is quite different from the lecture. I will talk about exponential family of distributions first. Then I will discuss the general idea of GLM. Finally, I will try to derive some well known learning algorithms from GLM.
Exponential Family
\(\eta\) is called the natural parameter and \(T(y)\) is called the sufficient statistic.
简单的python爬虫
意志力
今天早上读完了鲍迈斯特的《意志力》。有趣的是，能看完这本书也是一种意志力的运用吧。
自我损耗造成的影响是双重的：一方面意志力减弱了，另外一方面渴望变强了。
“自我损耗”是作者以及书中提到的研究者在做实验的时候常用的手段。这也有警醒的意味：意志力减弱的时候，渴望还会变强！
面临一个让他们产生内心冲突（一方面非常想要，一方面真不该要）的新诱惑，如果他们已经挡住了之前的诱惑，特别是如果上个诱惑过去没多久新诱惑就来了，那么他们更容易屈服。
1.你的意志力是有限的，使用就会消耗。 2.你从同一账户提取意志力用于各种不同任务。
我们可以把意志力的运用分为四大类，控制思维,控制情绪,控制冲动,控制表现。
