Skip to main content

Histograms and kernel density estimation KDE 2

You can download this whole post as a Jupyter notebook here

Why histograms

As we all know, Histograms are an extremely common way to make sense of discrete data. Whether we mean to or not, when we're using histograms, we're usually doing some form of density estimation. That is, although we only have a few discrete data points, we'd really pretend that we have some sort of continuous distribution, and we'd really like to know what that distribution is. For instance, I was recently grading an exam and trying to figure out what the underlying distribution of grades looked like, whether I should curve the exam, and, if so, how I should curve it.

I'll poke at this in an IPython Notebook; if you're doing this in a different environments, you may wish to uncomment out the commented lines so that your namespace is properly polluted.

In [1]:
from __future__ import division
%pylab inline

grades = array((93.5,93,60.8,94.5,82,87.5,91.5,99.5,86,93.5,92.5,78,76,69,94.5,89.5,92.8,78,65.5,98,98.5,92.3,95.5,76,91,95,61.4,96,90))
junk = hist(grades)
Populating the interactive namespace from numpy and matplotlib

March Madness, Monte Carlo Style!

I’m teaching Thermal Physics this term, so obviously I rearranged the syllabus so that we could all run Monte Carlo simulations for March Madness!

Quick summary for the class is at the end

The basics

We started off writing an energy function and figuring out how to play a simple 8-team bracket (side-note: IPython notebooks were great for this purpose! We were able to record our class sessions and post them online, along with a “Scientific Cookbook”). This version of Thermal Physics doesn’t have a programming prerequisite, so the coding parts of the assignments stopped with running and analyzing 8-team brackets.

We then spent a chunk of class time comparing our simulations with the 2-state paramagnets we’d seen in Schroeder’s text (we covered section 8.2 a bit out of sequence). In that context, it was clear what a Monte Carlo move was. We convinced ourselves that we could do something similar in bracket space, but that we couldn’t just flip a single game (we also had to consider flipping the games that depended on that game). The coding for that part was definitely beyond the scope of this class, but I hacked up an ugly version for us to use.

Read more…

Getting a sensible Python install on my Mac

I recently had to get Python + things I like up and running on my Mac from scratch. That’s a significantly harder task than you might guess, so one of the main reasons I’m posting this is for my own benefit (I’ll have to do this all again in February when I give my current laptop back to NIH and buy one of my own). There’s a decent chance this will be useful for others, though. A significant amount of experience and gnashing of teeth has told me

1) ActiveState is awesome, but not everything works perfectly with it.
2) Ditto for Python 2.7.
3) Never mix macports and fink.

So, here’s my setup for a working python2.6 system that happens to meet my current needs:

Read more…

A simple Python logging example

I’m slowly converting most of my lab to Python. One of the great things is how fantastic the documentation for things like matplotlib is. On the other hand, some of the Python documentation itself is ridiculous.

Somebody wanted to set up some simple logging. He wanted to have a couple of different logging levels, he wanted to be able to switch between them from the command line, and he wanted to be able to spit messages to stdout as well. The official docs have a trivial example that’s very nice, but a bit too trivial and then several examples that were way too much for someone who’s in his first week of Python. He also really wanted to be able to specify numbers on the command line, and didn’t want to remember what the logging module’s internal constants (e.g. logging.CRITICAL == 50) are. Instead, he wanted to be able to give words or small, easy to remember numbers. Here’s the simple example we came up with:

Read more…