Generalized Linear Model (Examples)

This article is a companion article to my another post Generalized Linear Model. In this article, I will implement some of the learning algorithms in Generalized Linear Model. To be more specific, I will do some examples on linear regression and logistic regression. With some effort, google search gives me some very good example data sets to work with. The datasets collected by Larry Winner is one of the excellent sets, which will be used in the article.

The implementations here use Python. Required 3rd party libraries are:

Read More

Generalized Linear Model

This article on Generalized Linear Model (GLM) is based on the first four lectures of Machine Learning by Andrew Ng. But the structure of the article is quite different from the lecture. I will talk about exponential family of distributions first. Then I will discuss the general idea of GLM. Finally, I will try to derive some well known learning algorithms from GLM.

Exponential Family

A family of distributions is an exponential family if it can be parametrized by vector $\eta$ in the form $$P(y; \eta) = b(y)\exp(\eta^{T} T(y)-a(\eta)),$$ where $T(y)$ and $b(y)$ are (vector-valued) functions in terms of $y$, and $a(\eta)$ is a function in terms of $\eta$.

\(\eta\) is called the natural parameter and \(T(y)\) is called the sufficient statistic.

Read More


一个网络爬虫大致可以分成三个部分:获取网页,提取信息以及保存信息。Python有很多爬虫框架,其中最出名要数Scrapy了。这也是我唯一用过的Python爬虫框架,用起来很省心。让我苦恼的是,Scrapy在我的Raspberry Pi Zero W安装起来很麻烦,而且我觉得我爬取的网页比较容易处理,没有必要用这么重量级的框架。抱着学习的心态,我开始自己造轮子了。


Read More






1.你的意志力是有限的,使用就会消耗。 2.你从同一账户提取意志力用于各种不同任务。


Read More

Pelican Signals


(1) Pelican有一个叫做Pelican的类,含有程序的主体框架。当Pelican的一个实例pelican初始化完成之后(基本设置,加载插件),发出第一个signal。


Read More

Lambda Calculus

This post is my note for What is seminar on Lambda Calculus.

Lambda calculus was created by Alonzo Church in the 1930s, and was used by him to solve Entscheidungsproblem in 1936, which is related to Hilbert's tenth problem. In the same year, Alan Turing independently solved Entscheidungsproblem using his invention Turing machine. Shortly after, Turing realized that these two models are actually equivalent as models of computation.

In this note, I will first give the formal definition of lambda expressions. Then with the help of Python, I am going to show how to do Boolean algebra and basic arithmetic using lambda calculus, which to some extend gives an illustration that Turing machine and lambda calculus are equivalent.


Lambda calculus consists of lambda expressions and operations on them. There are three basic elements in Lambda expression:

  1. variables: x, y, z, ...
  2. symbols in abstraction: λ and .
  3. parentheses for association: ()

Read More

maupassant: a Pelican theme

我现在用的博客生成软件是Hexo。这个软件可以快速将Markdown格式的文章转成html格式,并且包含发布到github page的工具。Hexo是基于Nodejs,所以某天我就在想有没有一个基于Python的博客生成软件。为什么我会这样想?因为



Read More

Cassini Ovals

This post is my note for What is seminar on Cassini Ovals.


An ellipse is a geometric object formed by locus of points which have fixed sum of distances to two fixes foci.


In the above figure, we can express the definition of ellipses in one simple equation:


for some \(c>0\). What if we change the addition in the above equation to multiplication? This comes the definition of Cassini ovals.

Read More

Rademacher functions

This post is my note for What is…? seminar on Rademacher function.

For a real number \(x\) in \([0, 1]\), let \((a_1(x), a_2(x), \cdots, a_n(x), \cdots)\) be its binary representation. That is to say, \[\begin{equation} x = \sum_{n=1}^\infty \frac{a_n(x)}{2^n}. \label{20170623-1} \end{equation}\]

For some \(x\), there might be two possible binary representation. Take \(\frac{1}{2}\) for example, it can be represented as \((1, 0, 0, \cdots)\) or \((0, 1, 1, \cdots)\). In this situation, we always prefer the former representation, in which all terms become \(0\) eventually.

Definition 1. Let \(n \ge 1\) be an integer. The \(n\)-th Rademacher function \(r_n: [0, 1] \to \{-1,1\}\) is defined to be \[r_n(x) = 1 - 2a_n(x).\]

Read More

Lüroth expansion

This post is my note for What is…? seminar on Lüroth expansion.

Definition 1. A Lüroth expansion of a real number \(x \in (0, 1]\) is a (possibly) infinite sequence \((a_1, a_2, \cdots, a_n, \cdots)\) with \(a_n \in \N\) and \(a_n \ge 2\) for all \(n \ge 1\) such that \[\begin{equation} x = \frac{1}{a_1} + \frac{1}{a_1(a_1-1)a_2} + \cdots + \frac{1}{a_1(a_1-1)\cdots a_{n-1}(a_{n-1}-1)a_n} + \cdots \end{equation}\]

By abuse of notation, we will something write the right hand side of Definition 1 as \((a_1, a_2, \cdots, a_n, \cdots)\).

Given a system of representation of real numbers in \((0, 1]\), we need to determine whether these representations are 1 to 1 corresponding the numbers in \((0, 1]\). Aslo we should study the (Lüroth) shift map: \[\begin{equation} T: (a_1, a_2, \cdots, a_n, \cdots) \mapsto (a_2, \cdots, a_n, \cdots). \label{20170620-2} \end{equation}\]

Read More