||representing things|| [kiran's theoryblog]

{about} {projects} {notes} {contact}

Presentations at ICML 2016

Well it’s been a year since my first post, so let’s do another post today in the hopes that I’ll actually start writing things this summer!

Over the past few days I’ve been attending ICML 2016, and it’s been pretty interesting so far. I’m presenting a poster and a talk for the Multi-View Representation Learning Workshop on Thursday, so I’ve pretty much been preparing for that (poster, presentation and revisions) and attending sessions. I’ve kind of neglected the poster sessions, so I’ll probably spend more time wandering around the posters and engaging people tomorrow.

There were some really crystal clear presentations in the Learning Theory and Continuous Optimization sessions today which I quite enjoyed. On the other side of things, I felt most of the Deep Learning sessions fell kind of flat, with a few exceptions. Perhaps one of the issues with Deep Learning papers is that they’re not really suitable for presentation format unless there’s something truly novel going on in the architecture. Too often the punchline of a deep learning paper seems to be that some slightly modified technique was tried and it improved upon the baseline of some typical dataset/ task by a small portion. This fact isn’t necessarily a bad thing, but it does make for difficult talks since there’s no real concrete punchline you can get at without getting dirty with the details - and talks are not typically the place to get dirty with the details. Of course, there are some exceptions to this rule in my mind. For instance, sequence-to-sequence learning felt like a genuinely new innovation in comparison to previous work in the area.

Anyways, it’s been pretty interesting attending as this conference is my first machine learning conference. I also really liked Fei-Fei Li and Dan Spielman’s talks today. Both were particularly good presenters in terms of clarity and command. I guess one advantage of attending most of the talks is you start to see how you should go about improving your own delivery of talks, which is an important skill to have pretty much anywhere, though particularly in academia. One thing my advisor has reminded me of is the importance of avoiding too many words in presentations/posters, which is something I’m trying to work on. Coming up with the right figures that clearly deliver a point is pretty challenging at first!

So, now to talk about the future. I’ve been sitting on several ideas for blog posts for the past several months (probably more like 9 months). I plan to get to them sometime soon, after organizing all my computer/internet clutter so that I can be more productive. Here’s a list of potential topics:

  • What is meant by ``Machine Learning’’? An overview of all the relevant fields I think which overlap it.

  • Machine Learning courses in colleges.

  • A list of reading I’m doing this summer.

  • Machine Learning as Science++: Various positions on what constitutes truth.

  • Discussion of the current state of NLP/NLU: What I believe and what I’m skeptical of, and what I plan to test and implement with my friend Holden Lee

  • Discussion of my current research and perhaps some fun visualizations.

  • Some technical exercise? I need to practice technical writing skills… This is supposed to be my theory blog after all.

MathJax Tests

First post, woo hoo! This blog will be roughly about my thoughts about research and maybe other things, I haven’t really decided yet. Probably some notes and projects.

In the meantime, here are some tests to ensure MathJax is working.

\begin{equation} x_{t+1} = \prod_{\mathcal{K}} \left(x_{t} - \eta \nabla_t\right) \tag{1} \label{eq:OGD} \end{equation}

Equation \eqref{eq:OGD} is the online gradient descent update. See \eqref{eq:vandermonde} for a matrix. Consider \(x, y \in \mathbb{R}\): Then, \(x + y \in \mathbb{R}\) as well (this math is inline).

Here’s the Vandermonde matrix:

\begin{pmatrix} 1 & a_1 & {a_1}^2 & \cdots & {a_1}^n \\ 1 & a_2 & {a_2}^2 & \cdots & {a_2}^n \\ \tag{2.1} \label{eq:vandermonde} \vdots & \vdots & \vdots & \ddots & \vdots \\ 1 & a_m & {a_m}^2 & \cdots & {a_m}^n \\ \end{pmatrix}

Here’s some matrix multiplication:

\begin{align} \begin{pmatrix} 1 & 0 \\ 0 & e^{i\pi} \\ \tag{2.2} \end{pmatrix} \begin{pmatrix} u \\ v \end{pmatrix} &= \begin{pmatrix} u \\ -v \end{pmatrix} \\ \large\equiv \\
\begin{bmatrix} 1 & 0 \\ 0 & -1 \\ \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} &= \begin{bmatrix} x \\ -y \end{bmatrix} \end{align}

And finally, we have the Cauchy-Schwarz inequality:

Here’s a useful link to some MathJax tricks.