2017

The Most Beautiful Equation

6 minute read

Published:

"Like a Shakespearean sonnet that captures the very essence of love, or a painting that brings out the beauty of the human form that is far more than just skin deep, Euler's Equation reaches down into the very depths of existence."

-- Keith Devlin

Read more

Mathematics and Finance Hand in Hand

4 minute read

Published:

This week's blogging cause is an effort of putting together my recent random thoughts on the relationship between maths and the financial market. In the recent two decades or so, there has been a coup d’etat in the financial trading world, where human decisions have been largely replaced by sophisticated computer systems. Read more

2017

A Peep into Kalman Filter

3 minute read

Published:

Deep insecurity about my non-mathy background urged me to look into things that pump into my ears all the time. Today I decided to do some quick reading on Kalman filter, and now it's time to write down my understanding. Bear with me, those "mathemagicians". Read more

Hey robot, why you are so smart?

5 minute read

Published:

Are you being misguided here by the catchy name? Lucky you, this is gonna be my last proper Gaussian Process (GP) post. Just to assure you, I did not just use "robot" in the post name to wave hands at you; but I do intend to explain in this post how to use GP to "teach" a robot to use it's arm properly. Read more

Boosting II: Gradient Boosting

3 minute read

Published:

Having briefly introduced AdaBoost in a previous post, today I want to explain briefly about another Boosting method called Gradient Boosting. In a broad sense, it's based on the same idea as used in AdaBoost, that is in every iteration we fit the residuals from the previous iteration. For regression problems, the ultimate goal is to make accurate predictions approximating the true value; for classification problems, the goal is to classify observations with the correct labels. A common way to measure the performance of a learning algorithm is by the use of a loss function. Here in Gradient Boosting, we adopt $latex L(y, F(x)) $ to denote a measure of distance between the true response value $latex y $ and an estimate or approximation $latex F(x) $. We can think of boosting as approximating an optimal $latex F^{*}(x) $ by a sequential additive expansion of the form: Read more

Boosting I: AdaBoost

4 minute read

Published:

Adaptive Boosting (AdaBoost) is one of the most commonly used Machine Learning methods for both classification and regression problems. It is a kind of ensemble learning meta-algorithm that can be applied on top of many other methods to improve performance, such as Bagging, Random Forests, etc. The idea is to combine a set of sequentially developed weak learners (rule of thumb) and come up with a single best final classifier at the end. Let me set the scene in a binary classification setting. Read more

New lights shed on Regression

3 minute read

Published:

The best way to end my weekend is, well, bragging (no...blogging) about what new stuff I found during the weekend. Equipped with a basic understanding of what Gaussian Process (GP) is from a previous masterclass, I decided to do some further reading in this fascinating area. Read more

Not That Model Selection

5 minute read

Published:

Model selection? Model selection! Maybe not the 'model selection' in your mind though. Actually, this blog is meant to be a memo on another masterclass us STOR-i students had today after the previous Gaussian Processes masterclass that I also blogged about. This masterclass was given by Prof. Gerda Claeskens from KU Leuven, who introduced us to the main criteria used in model selection. Read more

Musings about Soulmate

3 minute read

Published:

When can you find your soulmate (if you don't have one yet...), and how? Sounds like a philosophical and psychological question. Yes of course the question can be answered from those mental/spiritual perspectives,  but let's do some maths here. Okay, here is the plan, I'm going to address this question in the two following ways. lol

                                 Source: https://lovelace-media.imgix.net/

1. Drake Equation

Read more

Model-Free Reinforcement Learning

4 minute read

Published:

In one of my previous posts, I wrote briefly about Markov Decision Processes (MDP). Today, let's move into the area of reinforcement learning (RL), which is strongly linked to MDP, in that it also deals with problems that require sequential decision making.  The difference lies in that, instead of waiting till the end of the time horizon before we choose our policy (a set of actions specified for each time point at each state), we base our decisions on all the accumulative experience earned in the past. Real-life applications abound in a wide range of fields, including robotics, economics, and control theory. We term the decision maker in a system as an agent. The idea of RL is to empower the agent with both retro-dictive and pre-dictive powers. Read more

Bagging->Random Forests->Boosting

4 minute read

Published:

Today, I'm going to talk about a set of classification methods in machine learning in the order as the above title suggests. Keen readers may remember that I mentioned briefly in one of my earlier posts about classification methods for image recognition. There seems to be an everlasting discussion in machine learning community about the trade-off between prediction accuracy and model interpretability. The theme of today's post will reside more on the side of model interpretability. Irregardless of the not so self-evident names of Bagging, Random Forests, and Boosting; they are all branches of the unified tree-based methods. Read more

Stochastic Programming Buzz

4 minute read

Published:

Just immersed in another two-day masterclass about stochastic programming given by Prof. Stein Wallace, and with a book on Stochastic Programming written by him rests on my desk at arm's distance, I feel compelled to sort through my scribbled notes and write something on this. Read more

Falling in Love with Gaussian Processes

4 minute read

Published:

Today us STOR-i students had our first masterclass this year in Gaussian Processes given by a great speaker Neil Lawrence who specialises in Machine Learning. Gaussian process models are extremely flexible models in a way that it allows us to place probability distributions over functions. Read more

All Roads Lead to Rome - Image Recognition

6 minute read

Published:

Give you a picture and ask you to identify certain features in it, that's pretty easy. Give you two pictures and ask you to identify the common features in these two, that's fairly simple as well. What if you were given a stream of photos and being asked to identify certain features in each one of them, a bit intimidating?  Not that bad with the help of some smart methodology and technology. Up till now, I've been made aware there are at least two types of methods to approach image recognition type of problems (Forgive me for being ignorant if you know more).  Wavelets transforms can be applied to capture information in images, Classification methods are also being widely used in the setting of image pattern recognition. These two methods are my focus today, I will talk briefly about what they do, how they work, and the difference between these two. Read more

A Gentle Intro to MDP

5 minute read

Published:

Suffering from my 'memoryless property' a lot, with a MDP coursework alarm ringing in my head at the moment, and vaguely remember this dynamic programming research talk from Chris Kirkbride recently; I decided to organize all that I know about MDP in this one blog post. Hopefully after finish writing this post, I'll have a clean and organized storage of MDP in my head; and hopefully after finish reading this post,  you'll get something useful as well. Read more

Time Series Primer/Revisited

3 minute read

Published:

This afternoon we had two soft intro talks to the area of Change Point Detection and  Time Series Analysis (in a discrete setting mainly). In some extent, change point detection is time series as well. Many times,  people are interested in detecting the changes over time and making good predictions for the future. Also, when one abrupt change happens, a natural question to ask is - Is the change intrinsic or is it just an outlier? These are interesting areas I'd like to explore further, but here let me refresh my memory with what I've done with time series so far. Read more

What on earth is Statistical Learning?

4 minute read

Published:

Everyone is talking about stats learning or machine learning, as if they are the sexiest terms on earth. Literally, does it have something to do with statistics? Or machine? Depends on the area you are in, and depends on the people you are talking to. According to the omniscient Wiki, stats learning deals with the problem of finding a predictive function based on data.  As for my understanding of stats learning, it's no more than just a tool, that helps people to better understand their data and of course, to thus make better predictions. Usually, people classify stats learning into two categories, supervised and unsupervised. Read more