Big Gummy Bears

Normally, when confronted with (inevitably weird and annoying) YouTube commercials, I’m hovering over the “Skip Ad” link, waiting for it to be enabled.

But today, I saw an advertisement so odd, that I was forced to watch through the whole thing just to figure out if it was a parody. It wasn’t. Congratulations Vat19, on a successful commercial – if I’m ever in the market for giant gummy bears, I’ll come your way.

Who thinks up these things?

Fiber to the people

My home internet sucks, relatively speaking. Anytime something shows up as “HD”, I know that it’s not going to work out for me. This is not at all surprising, given that I only have one choice (Time Warner) and they continually send me advertisements offering to spend $100 a month in order to get the bandwidth I’m supposed to get for $50. The sad thing is that if you look at the wireless routers visible from my apartment (> 50), everyone in the building (and nearby buildings) has the same problem. If only we could just share one good connection, we’d all be so much happier.

So here’s what I think should happen.

Apartment buildings should have fiber run to the building, run ethernet/wireless to each floor, and charge $40 a month to access it. Why? Because they’d make money off of it, that’s why. After the initial cost to run the fiber, they could contract with Cogent/Level 3/ATT to provide transit. Based on my crude knowledge of the state of connection costs from 5 years ago, it would cost about $5 a month to give every user 10Mb/s of dedicated service.

And everyone in the building would get 10 times the bandwidth of Time Warner to boot. Hurray!

I’ve been daydreaming about starting a company that contracts to do just this (drag fiber to buildings and contract for support). I know, I know, a lot of this types of companies already exist, but still… let me daydream.

Tesla GPU not showing up for cudaDeviceCount/getDevice

I spent a few hours today debugging this — GPU programming is its own special form of hell. It turns out I had to disable ECC and reset the GPU:

nvidia-smi -i 1 -e 0
nvidia-smi -i 1 -r

Naturally, there was no indication of an ECC error having occurred, and just resetting the GPU’s doesn’t help matters.

Sigh.

why couldn’t john kerry be like this when he was running for president?

Sigh. The John Kerry post-2008 is just so much more relaxed and entertaining. I’d vote for him. (Admittedly, I voted for him in 2008 as well).

How to Beat Jet Lag: Forget About It (John Kerry)

I can’t say my past experiments with jet lag remedies have been very scientific. When I’m flying, I usually take an Ambien and listen to one of my own speeches on my iPod. I’m out in seconds. But it doesn’t always work, and sometimes you’ll have some funny moments from being overtired. There was an incident in New Orleans, at Mardi Gras, in 1997. But the video has been destroyed and I gave the beads back.”

a simple editor with xtext

XText is a pretty cool plugin/SDK for Eclipse that makes it relatively easy to make a full-featured editor for your favorite language. I recently (wasted a few days) played around with it and created a simple editor for my current language of choice, IcedCoffeescript. (If you’re really curious/bored, the source is on github.)

Almost all of the information you find on Xtext requires defining a complete grammar for your language, which is a bit too much hassle if you’re just starting out and all you really want is some pretty syntax. Fortunately, it’s pretty straightforward to get a syntax only parser if you want one – simply define a grammar with all of your terminals:

terminal KW_X: "x";
terminal KW_Y: "y";
terminal KW_Z: "z";
terminal ANY: .;

and a single top-level rule:

Script: ANY+;

You then can supply a highlighting configuration to specify how various terms should be highlighted, and a TokenMapper that maps from actual tokens to term classes. (Yes, I know the terminology gets a bit confusing/baroque). Here’s some links to the various parts I used in my project:

too much configuration, or: you shouldn’t have to tweak parameters

Whenever I encounter a paper like this – Towards automatic optimization of MapReduce programs – and there are a lot of them, I find myself sighing inwardly. (Heck, we even had a student of ours who ended up tweaking a bunch of these knobs: http://www.news.cs.nyu.edu/node/146).

This seems to be a common refrain in Java programs, but Hadoop especially – rather then either choosing a sensible constant, or adapting a value at runtime, let’s foist all of the work onto the user. But the way it’s phrased is clever – we’re not avoiding the decision, we’re just making it so the user can configure things however they want. I’ve done this a lot myself – it’s just so easy to add a flag to your command line or to your config file and pride yourself on a job well (not) done.

The key issue here is that, as a user, I don’t know what to put in for these values, I don’t know what’s important to change, and so I’m the absolute worst person to be responsible for these things.

Seriously, why are you giving the user these parameters to tweak?

  • io.sort.record.percent
  • io.sort.factor
  • mapreduce.job.split.metainfo.maxsize
  • mapred.heartbeats.in.second

What inevitably happens is we don’t know what any of these things actually mean when it comes to making things faster, so we end up searching the internet for the magic numbers to plug in, rerunning our jobs a whole bunch and wasting a crap-load of time.

This is not a desirable user experience. I mean, here’s the interface a car exposes to me:

There’s a “go faster” pedal and a “go slower” pedal. These correspond to all sorts of complicated, dynamic magic inside of the engine compartment, but I don’t need to know about them – the system handles it for me. Moreover, it can adjust parameters at runtime, in response to the behavior of the car – unlike most of our lazy computer programs!

If only our programs could be more like cars (though hopefully with better gas mileage).

personal package management

Occasionally I find I need a package that isn’t in my distribution, or I need to rebuild from source for whatever reason. In the past, I’ve always been conflicted about that age old question:

Where do I install this bugger?

The default for most packages, /usr/local, is fine for most purposes, but then there are annoyances – if I want to use this package on other machines, then I’d be better to put it under ~ (/home/power), (our cluster has a shared NFS mount). But then I’m filling up my home directory with random bins and sbins and includes and if/when I need to uninstall something I always get confused and blow away the wrong thing (since all of the binaries end up in /home/power/bin)

Installing into subdirectories (/home/power/my-package) has it’s own annoyances – I have to make sure to update my $PATH everytime, and I start to get confused when there are too many things in my home directory (I don’t know why, I just do!).

Fortunately, I’ve come across a nice solution to all this. I install everything into /home/power/pkg:

[email protected]> ls -l pkg                                                                                                                                                                                
...
drwxrwxr-x  7 power power 4096 Nov 19 19:42 openmpi
drwxrwxr-x  6 power power 4096 Jul 24 18:48 oprofile
drwxr-xr-x  6 power power 4096 Feb 19  2011 paperpile
drwxrwxr-x  4 power power 4096 Nov  6  2011 parallel
drwxrwxr-x  5 power power 4096 May  9  2012 perl5
drwxrwxr-x  6 power power 4096 Apr 15  2012 postgresql
drwxr-xr-x  7 power power 4096 Feb  9  2012 pypy-1.8
drwxr-xr-x  7 power power 4096 Jun  7 12:50 pypy-1.9
drwxr-xr-x  6 power power 4096 Apr 27  2012 python-2.7

And add the following to my bashrc:

for d in /home/power/pkg/*/bin; do export PATH=$PATH:$d done
for d in /home/power/pkg/*/lib; do 
  export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$d 
done

Now every time I install a package under ‘pkg’, my PATH is automatically updated to discover it. And if do need to remove a package, I just blow away the directory.

I won’t claim that this is brilliant or original. But it works for me, so… it’s nice.

bash-isms i wish i knew earlier

I’ve been using bash for far too long (> 10 years now), and I was reminded the other day (after helping a colleague with some coding) of some of the random tricks that you pick up over time. I’ve put them into a (almost completely wrong) timeline. Are there any other common idioms that I’m forgetting to put in here?

Day 1:

“Oh man, look at that, I can hit up-arrow and repeat the previous command!”

You’re too excited by the novelty that you don’t pay attention to the fact it’s slower then retyping the old command.

Day 30:

After again hitting up-arrow 30+times to find an old command, you accidentally come upon reverse i-search (

CTRL+R

) and realize what it means.

You start using quick substitution:

echo foo
^foo^bar

And are feeling pretty much like a master.

Month 6:

HERE documents are now completely under your control, and you have started writing scripts to try to automate everything, even operations that you know you’ll only have to do once.

You spend at least one entire day fiddling with themes, in the name of “productivity”.

Year 1:

You’re feeling more confident, and moreover, the reckless abandon of your Bash youth seems to have passed. After a brief spell with the vi key bindings, you’re back to the emacs bindings, but feeling invigorated by your exploration. You realize that there is a separation between inputrc and bashrc, but you don’t really have time to investigate further. After all, you just added

set completion-query-items 10000
set completion-ignore-case On

to your

.inputrc

and are far too excited about the idea of never being asked:

Display all 3425 possibilities? (y or n)?

every again.

Year 2:

export HISTSIZE=1000000
export HISTFILESIZE=1000000
shopt -s histappend

How could it have taken you so long to search for this? No longer will having multiple terminals open cause you to lose your hard earned history. You anticipate the point in time where you will have accumulated so many commands in your history file that you will never have to type a new one.

Year 3:

You dabble with ZSH after seeing a friends super colorful console. You give up after you realize that zsh is missing most of the awesome TAB-completions you are by now accustomed to. By accident, you try tab-completing an scp command and are floored by the fact it’s actually based on the remote filesystem. You start trying to write your own completion scripts, but realize these are things better left to experts. It is slowly dawning on you that you are not an expert.

You’ve also become more confident in your escapes – you feel not the slightest bit scared about using arithmetic now:

MYPORT=$((PORTBASE + 1))

And you use the subshell escape

result=$(echo foo)

to differentiate yourself from those silly backtick users who don’t know what they’re missing:

ROOT=$(dirname $(dirname $(readlink -f $0)))

Year 5:

You accidentally hit

CTRL+X CTRL+E

again, but this time you noticed the magic keystrokes that got you here. An

$EDITOR

window for modifying the command line? How cool is this? Now your Awk scripts will become even more powerful (and ever more incomprehensible).

Year 10:

Your scripts have started to become Zen-like koans of existential beauty. Your full knowledge of the power of

trap EXIT

allows you to impress your neighbors, whose adulation you accept with a wry smile. You know when to

CTRL- (SIGQUIT)

and when to

CTRL-C (SIGINT)
  • you use force only when required.

You have come to the realization that you are just a beginner, and have so much more to learn.