The Shadow File

Long-form Reading 2011

2011-12-16T11:01:00.001-05:00

Here are some long-form articles I've enjoyed this year.

The Hazards of Nerd Supremacy: The Case of Wikileaks (theatlantic.com)

The Octopus Conspiracy: One Woman's Search for Her Father's Killer (wired.com)

Confessions of a Prep School College Counselor (theatlantic.com)

The Great Rubber Robbery: How Julius Fromm's Condom Empire Fell to the Nazis (berlinbooks.org)

Bursting the Bubble (about David Vetter, the "Bubble Boy", houstonpress.com)

The Stutterer: How He Makes His Voice Heard (slate.com)

The Day The Movies Died (gq.com)

Man vs. Machine on Wall Street: How Computers Beat the Market (theatlantic.com)

Authorities in Awe of Drug Runner's Jungle Build, Kevlar-Coated Supersubs (wired.com)

Kiki Kannibal: The Girl Who Played With Fire (rollingstone.com)

The Battle for Tora Bora (tnr.com)

The Humility Imperative: CEOs Keep Your Arrogance in Check (inc.com)

How to Land Your Kid in Therapy (theatlantic.com)

How One Man Hacked His Way Into the Slot-Machine Industry (wired.com)

Cyber Weapons: The New Arms Race (businessweek.com)

Don't Be Evil (on google's narrowly conceived, yet handicapping motto, tnr.com)

A Few Too Many: Is there any hope for the hung over? (newyorker.com)

The Man With the $16 House (dallasobserver.com)

The Wrestler in Real Life (on Ric Flair's long decline, grantland.com)

Enter the Cyber-dragon (on how China is operating with impunity on America's defense and commercial computer networks, vanityfair.com)

Apple's Supply Chain Secret? Hoard Lasers (business week.com)

50 Years of Stupid Grammar Advice (thechronicle.com)

The Rise and Fall of the Columbia House Record Club -- and How We learned to Steal Music (thephoenix.com)

How a Financial Pro Lost His House (nytimes.com)

When Did the GOP Lose Touch With Reality? (nymag.com)

What is Sony Now? (businessweek.com)

The Pest Who Shames Companies Into Fixing Security Flaws (wired.com)

Thanks to Marco Arment's excellent Instapaper service and iOS app for helping find and keep track of excellent reading material on the web.

Reading List 2011

2011-12-13T15:15:00.000-05:00

I was using up all of my accumulated credits on Audible.com just now, and realized I've listened to several great audiobooks over the last year. Here's a list of what I've listened to in 2011, along with a link to the book on Audible.com. I recommend them all.

Judas Unchained, Peter F. Hamilton, Part 2 of the Commonwealth Saga (link)
The Gun, C. J. Chivers (link)
The Windup Girl, Paolo Bacigalupi (link)
Snow Crash, Neal Stephenson (link)
Embassytown, China Mieville (link)
Pattern Recognition, William Gibson (link)
Spook Country, William Gibson (link)
Zero History, William Gibson (link)

I'm pretty excited about my reading list for 2012 (and likely beyond--I had 12 credits to use), which I will share soon.

Multi-page Javascript Bookmarklet

2011-11-13T19:40:00.002-05:00

[Disclaimer: I don't know crap about javascript, other than what I pieced together from the 'tubes.]
I read something somewhere (I don't remember where) about having a bookmarklet that automatically opens your favorite sites, each in their own tab, when you click on it. I wanted to set up my own, but the article didn't explain how to do it (just suggested it as an idea). It's actually pretty easy. Below is commented code:

javascript:(function(){
  /* save a reference to the current window */
  oldWin=window;
  
  /*
   * Replace the current window's or tab's website with this one.
   * If you just want additional windows (or tabs) opened
   * skip this.
   */
  location.href='https://mail.google.com';
  
  /* Each additional window/tab you want opened */
  window.open('https://voice.google.com');
  window.open('http://www.accuweather.com');
   
  /* restore focus to the original tab, since each
   * new window gets focus
   */
  oldWin.focus();
})();

Paste this entire javascript fragment in as the address for your bookmark. It may be easier to bookmark an arbitrary page, then edit that bookmark's title and address.

On the Mac App Store and Sandbox Restrictions

2011-11-04T07:14:00.012-04:00

Update: My friend, Chad, creator of Pear Note, has an interesting perspective that is much more in-depth than what I have written. While I don't agree with his conclusion, he is a successful full-time Mac developer where I have no Mac development experience. Also Chad has more SELinux background that I do.

There has been a lot of gnashing of teeth lately over Apple's forthcoming Sandbox restrictions on software sold through the Mac App Store. In the near future, applications sold through the Mac App Store will be confined by very restrictive security policies. These security policies will make it impossible for applications to do many of the things we are accustomed to applications being able to do, such as interact with other applications. Some have described this change, pejoratively, as the "iOS-ification" of the Mac. I think these people miss some important points.

I used to work at a company where most of my time involved developing either SELinux security policies (a technical mechanism that the operating system uses to restrict applications' behavior) or applications designed to be confined by SELinux policies. Unfortunately, although SELinux is extremely powerful, it isn't used to confine most Linux desktop applications. It exposes users to problems that are beyond their ability (or desire) to troubleshoot and fix. Having spent time working with under-appreciated access control mechanisms, I am really excited to see Sandboxes coming into the mainstream on an operating system known for its ease of use. This is a Good Thing for users.

SELinux is similar in some ways to Mac OS X's (and iOS's) Sandbox mechanism. These mechanisms enforce a system-wide security policy that cannot be overridden through traditional user-based file ownership and permissions. For example, an application executed by you shouldn't be allowed to change or delete all of your data (without asking you first) simply because you own the data. An unconfined application could be tricked into doing just that.

Apple has implemented a trusted process model in OS X Lion to handle risky behaviors. An application confined by a Sandbox profile might not be allowed to write its data to your hard disk. Instead, it has to send its data to a Lion-provided process whose job it is to write the data. That trusted process is only allowed to access data on the hard disk. It is not allowed to receive connections from other computers on the Internet. An attacker on the Internet would have to compromise not only your application, but also the trusted process in order to gain access to your data on your hard disk. Think of a building that requires entry through one door followed by another, and no single person has both keys. You have to co-opt both key holders to gain access.

Many things that applications normally do on their own will now have to be proxied through a trusted interface provided by Lion. Some things, such as adding cover art to iTunes albums, become impossible where no sanctioned interface is provided. In the near term, applications in the Mac App Store will become simpler and less capable because of this. I suspect that over time, though, Apple will provide more robust set of mechanisms that applications will be able to use to do more things. This way, applications will be able to do more on behalf of the user without introducing significant risk.

I like knowing that applications I install through the Mac App Store will be confined by a security mechanism that mitigates mistakes made by the developer. I'm happy with it as long as I can obtain applications from outside the App Store when I'm willing to assume more risk. You have choices. If you want an application that not only won't but can't eat your data (or syphon it off to the Russian mafia) get it from the Mac App Store. If you want an application that has more features--and more risk--buy it ~~elsewhere~~ from Chad's website.

Fake Reviews on Amazon?

2010-12-19T10:43:00.006-05:00

We all know there are shady marketing/image promotion firms that do things like post fake reviews on Amazon.com and others. However, I've never actually seen a review that I was certain was a fake. That is, before today.

Yesterday I bought a no-name 21 bottle thermo-electric wine cooler from Tuesday Morning. I wanted a feel for whether I got a good deal, so I did some post-purchase comparison shopping on Amazon.com. I couldn't find my exact model there. Again, I'm certain it's a no-name, made-in-china, rebranded by a distributor type affair. I did find a couple of models on Amazon that are extremely similar to my own. So much so that I would not be surprised if they are actually the same unit with minor cosmetic revisions.

As I was reading the customer reviews, they were mostly pretty terrible, complaining of failures within 18 months of purchase. Most also complained that there wasn't a place locally to get the appliance serviced. Unsurprising. Oh, well. Live and learn, I guess :-(.

However, what was interesting to me was that for each of the two items, I found one review that was uncharacteristically positive. Even less likely, both reviews were impossibly similar, following a template, highlighting the same positive features, and using similar wording.

I'm not going to link to the Amazon product listing because I don't want to help boost those items' page ranking, but I have pasted in the product name as found on Amazon, as well as the full review in each case.

From the "NewAir AW-210ED NewAir Thermoelectric Wine Cooler With Dual Temperature Zones"

A four star review:

I ordered this as a present for my wife, and read a lot of reviews. There were so many negative product reviews for wine coolers, I really didn't know which one was actually the truth. I didn't know what to buy or who to believe.

This fridge is really quiet. In an empty room without any other appliances on, you can't hear it.

It's beautiful. It's aesthetically pleasing and fits in well into our dining room. She loves the sleek black finish, we get lots of compliments on it. It gets a lot of attention believe it or not. Some people are really impressed by it. My friends think NewAir should of made room for cans of Bud Light, but I disagree. The unit stores my $[...]bottles of Opus One. It is definitely a higher end unit meant to store wine collections not beer for UFC fights. Same high quality as Cuisinart models we've seen at Bloomingdales, but bigger which means more room for WINE! I would definitely choose this unit over a built in unit. This model requires some room for venting like most refrigerators, which makes sense. We had a built in wine cooler in our last house and it was nothing but problems. When they had to fix because the compressor went I had to contact a laundry list of people because the manufacturer went out of business. Then the guy who came to fix it decided it was ok to damage our granite bar top to access the built in wine cooler.

No assembly except putting on the handle. Let's face it not many of us spoiled Southern Californians are handy.

he temperature holds steady and does not reset when exposed to sunlight, or when the room is warm. It's really easy to read the temp setting, really easy to set, and it works like a dream. The LED lighting is awesome. Also, you can feel the difference in temperature between the 2 zones.

Dollar for dollar, this was competitively priced. I really liked my experience with Air & Water. They were great. And, we're SO happy with it.

I know you will be too.

And from the EdgeStar 21 Bottle Dual Zone Wine Cooler

A five star review:

I ordered this as a present, and read a lot of reviews. There are so many negative product reviews for wine fridges, I really didn't know what to buy. So, let me put your fears to rest:

1) This fridge is really quiet. In an empty room with no AC on, you can't hear it.

2) It's beautiful. Really nice stainless finish, we get lots of compliments on it. On par with Cuisinart models we've seen at Macy's, but bigger.

3) No assembly except putting on the handle. Hallelujah!!!

4) Consistent temperature - the temp holds steady and does not reset when exposed to sunlight, or when the room is warm. It's really easy to read the temp setting, really easy to set, and it works like a dream. You can feel the difference in temperature between the 2 zones.

Dollar for dollar, this was competitively priced. I really liked my interaction with Compact Appliance. They were great. And, we're SO happy with it.

Hope you will be too.

Here are the points addressed by each review, in the same order:

The unit is quiet, and in a quiet room, you can't hear it
The unit is beautiful
Requires little to no assembly
Keeps a consistent temperature
Competitively priced
Interaction with the manufacturer/distributor was great (WTF? you bought it from Amazon.com)
We're so happy and "hope you will be too."

It's kind of amazing how blatant these reviews are. I guess the most Amazon can do is close each user's account. Since the reviewers are disposable, the PR firm just hires more, or even creates new accounts under new names.

Remarks on Apple, Flash and the "I Hate Apple" People

2010-05-01T12:10:00.005-04:00

I've want to put down in words a few loosely related thoughts on controversy regarding Apple's policy of not allowing Adobe Flash Player on its mobile devices, and on the I Hate Apple crowd in general.

First, the "I'm anti-apple" people. I hear this mantra repeated frequently by friends and colleagues and also by noisy people on the Internet. I tend to rail against this sort of thinking, and as a result, come off as an Apple apologist to people who know me. I'll clarify: there are plenty of legitimate reasons to be angry with Apple. This vitriol, however, just seems irrational. Apple, as a company, has a lot of surface area and characterizing them in such broad strokes is problematic. They do things that are frustrating, but they also do things that are great. Most people who revel in waving their Apple-hating flag struggle to articulate just why they they hate them so much. It usually goes something like "grumble grumble, closed, proprietary, grumble grumble, App Store, grumble, no flash on the iPhone." The argument usually doesn't hold together well, and often these people will even admit that they just can't quite put their finger on what they hate so much about Apple. Look, we all are frustrated with Apple's App Store policies, even veteran iPhone and Mac developers. Even John Gruber, often accused of being an apologist is frustrated:
Serious App Store Doubts
Excerpts From the Diary of an App Store Reviewer
The App Store’s Exclusionary Policies

Further, if we're going to jump on the Apple-hating bandwagon, which is very in vogue nowadays, there are a lot of big-company-hating bandwagons we're going to have to jump on and start bitching about and boycotting. That's tiring. I don't have enough hours in the day to hate every company that needs hating. Sigh. Pick your battles.

Moving on to Apple's prohibition of Adobe Flash Player. It's not that complicated. Flash on the iPhone specifically, and cross-platform development frameworks in general, have the effect of diluting all platforms, including the iPhone, down to the lowest common set of platform features. No company should want their mobile device to be reduced to a state of generic similarity to their competitors' devices. Apple is perfectly justified, in my opinion, in wanting to keep this sort of shovel-ware out of the App Store.
This week Apple CEO Steve Jobs penned an open letter regarding his company's position on Adobe Flash Player on the iPhone OS:
Thoughts on Flash
I won't dissect the letter; it's been covered. But I agree with everything in it.

In response, this blustery counterpoint was posted on Linux-Magazine.com:
Apple's Steve Jobs is spreading FUD on Flash

This author's post actually makes me angry. It makes me angry because he is so bigoted against Apple, that he actually comes to Adobe's defense, conveniently forgetting that they've been dragging down the Linux desktop experience for years.

I self-identify as both a a Mac user and a Linux user. For years I was a full-time Linux user. I was the most obnoxious of Linux bigots, so I have that perspective. I have unhappy memories of flash on Linux. Since flash video became prevalent on the Internet, Adobe flash player has been the bane of Linux users' existence. Its performance on Linux has always been abysmal. I remember times when my laptop's fans would spin up to full speed, and the battery would start draining, and it would start scorching my lap. I would have to go hunt down whatever Firefox tab had a flash-based banner ad that was eating my CPU for lunch.

Adobe was slow to update it to the latest release, taking a year or more after it was released on Windows to release a new version for Linux. Also, Adobe still hasn't released a 64-bit version of the Flash Player for Linux or the Mac. Yeah. Seriously. It's 2010. Although major Linux distributions have gotten a lot better at making flash installation easy, you used to have to do weird 32-bit library wrapping voodoo to get Adobe Flash Player to work with 64-bit Linux and Firefox.

Because the Adobe Flash Player is closed and proprietary, Linux distributions couldn't ship with it installed. Users would always have to jump through hoops such as configuring third-party package repositories in order to install it.

Yes, as the author points out, flash is an open specification, so anyone technically can implement it. But to date, there haven't been any open flash players that are worth a damn. Open implementations such as Gnash are turds. They are several versions of the flash specification out of date, their performance is abysmal, and getting them configured and working is black magic. So let's leave flash's "openness" aside, because that's a red herring.

What is material is that much of the Internet depends on your browser being able to play flash. This is just as bad as the Internet of a few years ago requiring Internet Explorer, and being broken on every other browser. An Internet dependent on Adobe Flash Player is bad for Linux users. An Internet that has embraced HTML5, CSS, and Javascript is good for Linux users. Get off your Apple-hating high horse and acknowledge the fact that the two of you are on the same side.

Ubuntu Netbook Remix in VMware

2009-08-31T17:16:00.002-04:00

I needed to install Ubuntu Netbook Remix (UNR) in a VMware Virtual Machine so that I could try to reproduce a bug. It was kind of an asspain. Problem is, UNR is made to be installed from a USB flash drive, and VMware can't boot from USB devices. There isn't a DVD or CD ISO image for UNR 9.04 available. It turns out that you can make a DVD image from the UNR .img.

I found the following guide on the Ubuntu forums (see below for a link). It should work from most linux distros:

Make a directory to hold the contents of the UNR DVD image:


$ mkdir unr

Make a mount point to mount the UNR .img file:


$ mkdir realunr

Mount the UNR .img as a loop device:


$ sudo mount -o loop -t vfat ubuntu-9.04-netbook-remix-i386.img realunr/

Copy all the contents from the UNR image to your unr/ directory. Note that realunr/.diskimage is critical:

$ rsync -a realunr/ unr/

Rename syslinux to isolinux:

unr$ mv syslinux isolinux
unr$ mv isolinux/syslinux.cfg isolinux/isolinux.cfg

Generate a DVD image from the contens of unr with the following incantation:

mkisofs -o ubuntu-9.04-netbook-remix-i386.iso -r -J -l \
-V "Ubuntu-Netbook-Remix 9.04 i386" -b isolinux/isolinux.bin \
-c isolinux/boot.cat -no-emul-boot -boot-load-size 4 \
-boot-info-table unr/

mkisofs will generate a DVD image that you can either boot a VM from or burn to a DVD.

Guide on ubuntuforums.org

Jailbroken iPhone, Cydia Apps, Upgrading iPhone to 3.0.1

2009-08-07T18:16:00.004-04:00

I've been running my iPhone jailbroken for a couple of weeks. At first I felt dirty about having callously cast aside many of the legitimate security mechanisms in the iPhone, such as sandboxing of 3rd party apps. Utlimately, I have given into my baser instincts and am sticking with it.

Today I upgraded my iPhone to 3.0.1. There isn't an official jailbreak for 3.0.1 yet, but the 3.0 jailbreak works. The problem in general with upgrading a jailbroken phone is that there's no real way to back up all of your unauthorized customizations. You've basically had your way with the underlying Unix OS. Not everything you install from Cydia is a self-contained app. There are system modifications like installing OpenSSH and system utilities. Anyway, what it comes down to is that you need to reinstall all of your Cydia packages after upgrading and re-jailbreaking.

Here's how the process went for me:
In iTunes:
-do a backup
-choose to restore, which will also update. Don't choose upgrade.

After this is done, leave itunes, and follow the normal jailbreak process
-Run redsn0w
-Point redsn0w at your 3.0 iPhone OS ipsw file.

Once redsn0w is done, you're jailbroken again and running (mostly) 3.0.1. You still need to reinstall your Cydia packages. I found some of my customizations returned once I had reinstalled the Cydia packages, but not all.

On the topic of Cydia packages, here's a list of packages that I installed and really like:

BigBoss Recommended; This gets you a bunch of command line utilities that you'd expect to be on most unix systems. It actually is a metapackage that brings a ton of other packages with it.
MobileTerminal; A terminal application (not an SSH client)
MusicControls; A paid app that lets you background and control many music applications like Slacker and Pandora
OpenSSH; client and server. Be sure to reset passwords for root and mobile. Also disable root logins. Disable sshd from running automatically.
SBSettings; let's you tweak many hidden settings, replacing BossPrefs. Also required for disabling autostarting of the sshd service
Safari Download Manager; A paid app. Lets you download and save files, even ones that Safari doesn't know how to handle. Great for downloading files and getting them off you phone.
iFile; A paid app that lets you browse the entire filesystem and open files. Also has a built-in web server that will serve up files to your laptop. Useful when you have no Internet, but need to download a file.
SysInfoPlus; Shows you lots of technical information about your iPhone. Mostly just neat.
Status Notifier; shows various status indicators like new mail icon in the iPhones status bar at the top of the screen
BlackDarkness theme; This is a complete system theme that replaces lots of things, including many icons. I think it's kind of neat. Brings winterboard with it.

That's about it. The BigBoss blog is recommending that you stand pat until there's a redsn0w update. Supposedly it's going to make things a bit easier. It's not clear in what way. At the very least, they point out that for now, you'll still end up with a 3.0 kernel. Hmmm. :-/

http://thebigboss.org/2009/07/31/iphone-v301-released/

I'm on twitter

2009-06-13T10:04:00.001-04:00

Feel free to follow me. The stuff I post there is awesome and will rock your world.

GEICO Music

2009-06-11T20:47:00.002-04:00

I hate to admit it, but GEICO commercials have a pretty good track record for featuring good pop music. Whether it's 3 Doors Down, Royksopp, or The Sounds, the songs in GEICO's ads are usually pretty good.

The latest that I like is "Somebody's Watching Me" by Mysto & Pizzi. This song is featured in the ridiculous "Kash" commercials. You can download it for free. While you're there, check out the "making of" video. It's surprisingly entertaining.

I don't like the commercials, and I'm not about to switch insurance providers, but, in general, if you hear a song in a GEICO spot, you can probably add it to your library with no regrets.

Spotlight does Math

2009-05-14T05:58:00.006-04:00

I just discovered that Leopard's Spotlight does math. Try it. Bring up spotlight with ⌘+space. Then start typing something like
7pi*8.3 or
sqrt(2)
and the answer is the first search result.

This isn't quite a powerful as the Google calculator. It doesn't appear to do base conversion, and it doesn't let you copy the result for pasting into an application, but it seems handy nonetheless.

Mac OS X handy bash aliases

2009-05-06T19:34:00.003-04:00

Here are a few handy aliases that I keep in my .bashrc in OS X:

alias cppwd='eval "echo `pwd` | tr -d \\\\n | pbcopy"'

This copies my current working directory to the OS X pasteboard so I can Cmd+V it into another terminal. Often I want several terminals open in the same directory at once.

alias burn='drutil burn -noverify'

This one lets me type

burn mydisc.iso

to burn an ISO disc image to CD or DVD without having to open Disk Utility.
And a variation:

alias vburn='drutil burn'

This does the same as the other but also does the disc verification step.

A Gun for the Time Hole

2009-04-15T08:57:00.003-04:00

An ordinary AK-47 Kalashnikov automatic rifle would not be allowed through the time hole. One made of bacon, however, is an ideal candidate for the time-traveling freedom fighter.

FiOS at Work

2009-03-23T07:19:00.004-04:00

Finally.

Handling HTTP Redirection in Ruby

2009-03-15T12:18:00.003-04:00

I have a Ruby project where I'm dumping a bunch of bookmarks from delicious.com, then fetching each bookmarked page for analysis.

One of the problems I encountered early on is that the some of the web pages bookmarked would redirect to some other location. Simply checking for HTTP response code 200 was insufficient. I needed to check for redirection as well.

A quick Google search for "ruby follow http redirect" yields lots of results. Unfortunately, they're all very similar, and not quite right. In general, the examples you come across (even the one in the official Ruby documentation) don't handle the case when the redirected location is path relative to the original location. So you end up doing a get on a URL that looks like "../../redirected/location/index.html," which clearly won't work.

It turns out that detecting relative redirection is fairly straightforward:


 until( found || attempts>=@@MAX_ATTEMPTS)
     attempts+=1
     http=Net::HTTP.new(url.host,url.port)
     http.open_timeout = 10
     http.read_timeout = 10
     path=url.path
     path="/" if path==""

     req=Net::HTTP::Get.new(path,{'User-Agent'=>@@AGENT})
     if url.instance_of? URI::HTTPS
       http.use_ssl=true
       http.verify_mode = OpenSSL::SSL::VERIFY_NONE
     end
     resp=http.request(req)
     if resp.code=="200"
       break
     end
     if (resp.header['location']!=nil)
       newurl=URI.parse(resp.header['location'])
       if(newurl.relative?)
           puts "url was relative"
           newurl=url+resp.header['location']
       end
       url=newurl

     else
       found=true #resp was 404, etc
     end #end if location
   end #until

The trick here is to ask the redirected url object if it is relative. If it is, then add the redirected path onto the old url object. the URI class overrides the '+' operator (what is this, C++?) so that you can concatenate the new path onto the old URL, by doing:
newurl=url+resp.header['location']

Mounting LVM Disks in Ubuntu

2009-03-12T09:30:00.006-04:00

I always thought LVM (Linux's Logical Volume Manager) was kind of neat for the flexibility it gives you in adding and removing disks and resizing volumes such. However, in practice, I find it's usually more trouble than it's worth. It adds a layer of complexity between me and my data.

Often I need to mount a disk configured with LVM on another Linux machine or in an Ubuntu live CD environment. Out of the box the logical volumes aren't recognized, so I can't mount them.

It's fairly easy to add LVM support and mount the volumes though.

You install the lvm2 package, load the device mapper kernel module, and then activate any lvm volume groups on your disk.

$ sudo apt-get install lvm2
$ sudo modprobe dm-mod
$ sudo vgchange -a y
(assuming your disk with logical volumes is already connected)
$ sudo mount /dev/mapper/<logical volume name> /mnt

And that's all there is to it.

If you want to deactivate the volume groups (recommended before unplugging a USB disk with logical volumes):
$ sudo vgchange -a n
Warning: the above command will deactivate all volume groups, so check the vgchange manpage first, if that's not what you want.

The Rules of the Time Hole

2009-03-09T20:25:00.011-04:00

There was a great video on College Humor addressing why terminators travel back in time naked.

See more at CollegeHumor

That said, I think it's worth discussing the rules of time travel or "the time hole" as my colleague Ross and I put it. So if you've followed the Terminator movies, and, more recently, Terminator: The Sarah Connor Chronicles, the rules of the time hole are fairly easy to infer.

Things can go through the time hole if:

They are made of meat
They are wrappped in meat
They look very much like they are made of meat
If several things meeting the above rules go through all at once, something else may sneak through behind them.

In the original Terminator movie, we saw Kyle Reese, John Connor's soon-to-be father, come through the time hole. As he is 100% human, Kyle clearly qualifies for time travel under rule #1--he is made of meat. Also in the same film, the Terminator, a Cyberdyne Systems Model 101, comes through the time hole, qualifying under rule #2. The 101 consists of a metal skeleton that is wrapped in meat.

In Terminator 2, we see the T-1000, which is constructed of a "mimetic poly-alloy," i.e., liquid metal, travel through time. Clearly, the T-1000 qualifies under rule #3 as it looks, very convincingly, like it is made of meat. Similarly, in Terminator 3, we see the T-X come back in time. The T-X is much like the T-1000, except hot.

Later, in the The Sarah Connor Chronicles, we see Cameron (a terminator wrapped in meat), John, and Sarah (both made of meat) travel back in time. However, just before the time hole closes, Cromartie's severed head (devoid of its meat wrapping) flies through after them.

I checked all terminator-related facts presented above on wikipedia.org.

OS X: How to detect whether you're on battery from the shell

2009-01-10T17:36:00.002-05:00

Sometimes you need to determine programatically whether a computer is on battery or on AC. For example, I have a backup script that takes hourly snapshots of my home directory on my MacBook Pro (which I hopefully will post about at some point). However, I don't want the script to run if I'm on battery. Ideally, launchd, which kicks off the script, could give me this option, but I don't think it can. I'll have to let the script make the decision when it gets executed.

If I were writing the script for Linux, I'd have it look through /proc/acpi/ to check the state of the battery. There's no /proc in OS X. There is, however, a power management command, pmset.

If I run pmset -g while on AC, I get:


Currently drawing from 'AC Power'
-InternalBattery-0     100%; AC attached; not charging

If I run it while on battery, I get:


Currently drawing from 'Battery Power'
-InternalBattery-0     100%; discharging; (no estimate)

I simply have my script grep the output from pmset for "discharging":


checkBattery()
{
     local ret=0;
     if $PMSET -g batt | $GREP -q discharging;
     then
             ret=1;
     fi
     return $ret;
}

See the pmset(1) manpage for more info.

Converting a .dmg to a .iso

2009-01-09T19:07:00.003-05:00

I have an operating system install image that I need to install in VMware Fusion. The problem is the install image is a MacOS disk image file, or .dmg. I need a DVD image, or a .iso, in order to boot the VM and do the install.

It turns out that the Mac OS X command hdiutil to convert the .dmg to a .iso:


hdiutil convert .dmg -format UDTO -o .iso

See the hdiutil(1) manpage for more info.

How to sudoedit non-interactively

2009-01-08T09:31:00.009-05:00

Okay, this one's a bit esoteric, but I think it's pretty cool. How do you use 'sudoedit' non-interactively such as from a script? Just a brief background about sudo: sudo is an authentication mechanism in Unix & Linux that allows unprivileged users to run specific commands (as defined by the system administrator) with root privileges without having the root password. This has several advantages over logging in as root:

Users can have specific, limited set of root privileges without having the entire set of root privileges.
Users use their own password, so the root password doesn't have to be shared. If a user's sudo privileges are revoked, the root password doesn't have to be reset.
Each use of sudo is audited per user, so that each time sudo privileges are invoked, there is an event in the system logs that identifies the specific user and the command they ran.

Sudoedit is a command that is related to sudo. It lets users edit files that normally only root can edit, such as system configuration files. However instead of using "sudo" followed by the editing command, the user runs "sudoedit filename" and sudo invokes the user's default editor, letting them edit the file.

So, what if you want to make changes to a system file via a script, and the only access you have to the file is via sudoedit? It isn't useful to have the script call sudoedit and then bring up vi for you to manually make the changes.

Well, the way sudoedit works, is that sudo makes a temporary copy of the file you want to edit, then calls your default editor, giving it the name of the temp file as the first argument. So, say your default editor is "fooedit". You run "sudoedit /etc/systemfile" and sudo makes a copy of /etc/systemfile to /tmp then runs "fooedit /tmp/tempfile". Your changes are saved to the temp file. When your editor exits, sudo copies the temp file over the original, and then removes the temp file.

From your main script, stage the pre-edited version of the system file, then set your default editor to a custom script that will copy that staged file over the temp file. When it exits, sudo will copy the temp file into place as normal.

Your script would go like this:


#!/bin/sh

export EDITOR="/bin/sh ./editor.sh";
PRE_STAGE=/tmp/stage.tmp;

echo "my new config"> $PRE_STAGE;

sudoedit /etc/config;

/bin/rm $PRE_STAGE;

exit;

Your custom script would look like:


#!/bin/sh
CAT=/bin/cat
PRE_STAGE="/tmp/stage.tmp"

$CAT $PRE_STAGE > "$1";
exit;

This script just cats your pre-staged file over whatever sudo passed in as the first argument ($1).

The only interaction you'll have to do is type your password for sudo, if you haven't already done so in the last few minutes (sudo temporarily remembers your authentication).

NVIDIA's Game-Changing Video Accelaration in Linux

2008-12-21T10:51:00.005-05:00

Recently NVIDIA introduced their Video Decode and Presentation API for Unix, or VDPAU, along with support in their beta 180.xx drivers. This API allows application developers to offload decoding and rendering to the system's NVIDIA video card. Graphics cards have long had some level of hardware assist for video decoding, but this lets applications offload almost all the work to the GPU. I believe NVIDIA calls this capability "Pure Video" in Windows.

Since I've been experimenting with HD recording in MythTV, I decided to give the new drivers a try. You have to download the latest beta Linux drivers from NVIDIA. See the nvnews forum to find out what the latest release is. NVIDIA doesn't advertise beta releases. You also need an NVIDIA GeForce 8xxx or 9xxx GPU.

In order to take advantage of VDPAU in MythTV, you have to build from trunk. See the MythTV VDPAU wiki article for more information.

I gave this a try and the results are amazing. The test system is a circa 2004 Athlon 64. Without VDPAU, the CPU was maxed out between video decoding and X's rendering. With VDPAU, cpu utilization was between 5 and 8 %. This was watching Live TV, which meant recording, encoding, decoding and rendering. This is a complete game-changer for MythTV. One of the biggest problems with MythTV, and other homebrew PVRs, is that the frontend needs to be fairly powerful. Generally this means around $500 for a capable system. It's hard to justify spending this much money in each room you want to watch TV (though I've managed to rationalize it anyway ;-). What this means is it is now theoretically possible to use a very cheap system for a frontend as long as it has an NVIDIA GPU in it. There are a number of low-profile graphics cards on Newegg for around $40-$50. Also, NVIDIA's Ion chipset looks especially interesting.

MythTV, HD-PVR, Trunk

2008-12-21T10:22:00.005-05:00

Until Hauppauge released the HD-PVR this past May, there was no way to record in high-definition from your cable or satellite box. Your HD recording options were limited to over-the-air, unencrypted QAM, or, in rare instances, unencrypted firewire from the cable box. In any case there was no way to record the good stuff in HD. I'll spare the details about the HD-PVR itself, and instead include some links at the end.

Of course, Linux support for the HD-PVR wasn't available straight away. However, this is a big breakthrough for homebrew PVRs so the MythTV/Linux community has been hard at work. It there is a Linux driver for the HD-PVR, but you have to check it out and build it yourself (See links at the end). Also, early work has been done on MythTV to support the HD-PVR, but in order to get that support, you have to run MythTV from trunk, rather than using your distribution's packaged version. This means checking out source and building it yourself. The source code changes daily, and at times things get worse before they get better. Whatever you check out on any given day should be in decent shape, but it may not be. You are forging MythTV from the molten center of the Earth, and it's still hot and a little bit on fire.

What is now in trunk will eventually make it into your distribution, but there's generally a 6-12 month lag, depending on your distribution's release cycle. Well, I'm ready to give the HD-PVR a try now, so I decided to give trunk a go. I set up a spare machine on my work bench to be a frontend and backend. After getting the HD-PVR drivers compiled and working, I checked out MythTV from subversion and built it. There are good documents detailing how to do this. It's actually not that hard. The build system for MythTV is very nice. I recommend starting with a MythTV based distribution like Mythbuntu, and removing all the mythtv packages. This will save you a lot of grunt work like setting up MySQL, the mythtv user, init script, etc.

Anyway, I've got the HD-PVR working with MythTV. I haven't used it for any actual TV watching or recording yet. As soon as I can get a frontend set up, I'll be able to actually try using it day-to-day.

Relevant links:
http://www.geektonic.com/2008/06/hauppauge-hd-pvr-unboxing-first-look.html
http://mythtvnews.com/2008/05/30/hauppauge-hd-pvr-support-coming-for-mythtv-devices-are-shipping-now/
http://mythtvnews.com/2008/06/14/hauppauge-hd-pvr-linux-driver-released-alpha-quality/
http://www.mythtv.org/wiki/index.php/Ubuntu_Installation_Guides
http://www.mythtv.org/wiki/index.php/Hauppauge_HD-PVR

Ditching apt-cacher-ng for Squid

2008-12-21T09:17:00.007-05:00

apt-cacher-ng hasn't exactly worked out as well as I had hoped. I kept having a problem where the Releases file from the apt repository would be reported as corrupted. I could go into the cache directory and manually remove that file, forcing it to be downloaded again, but that would only help with the system I was updating at the moment. The next system that I tried to update would have the same problem.

I've read a number of articles about just using Squid as an apt cache. I avoided this at first because Squid isn't really made for that. For example squid has no way of knowing when a specific version of a package has been made obsolete by a newer version. I assume apt-cacher-ng and apt-proxy know how to do this (maybe not). Also Squid isn't intended to cache arbitrarily sized objects for indefinite periods of time. I decided to give it a try though, since a number of people seem to be having success with this.

I had to tweak Squid's default configuration a bit to make it suitable for apt-caching.
The Squid configuration file is well documented, but below are the directives of interest:

refresh_pattern deb$ 1577846 100% 1577846
refresh_pattern Packages.gz$ 1440 100% 1440
cache_dir ufs /var/spool/squid 15000 2 8
maximum_object_size 409600 KB

The first line says to cache anything ending ('$' is the end-of-line anchor) in "deb" for 3 years. There are some packages that rarely get updated if ever, so I want to make they stay in the cache the entire time I'm using a given distribution release. I figure I'll probably be on a particular distribution release for no longer than three years.

The second line says to cache anything ending in "Packages.gz" for one day.

The "cache_dir" line says to put Squid's cache in /var/spool/squid (I believe this is the default). It also says to let it grow no larger than 15,000 megabytes. My system partition gets very little use, so 15Gb is no problem for me. The second two numbers tell Squid how to structure the cache. "2" says to create two level 1 directories and "8" says to create 8 level 2 directories. The default is 16 and 256. I read an article where the author was having a problem with the hard drive never spinning down because Squid was rescanning the cache every few seconds. The author said reducing the number of L1 and L2 directories helped. If anyone can find this article, please post a link in the comments.

The "maximum_object_size" tells squid to cache objects up to 400Mb. There shouldn't be any debs even close to that large. Squid's default is much smaller than this.

In order to get apt to use the proxy, I created a file called "proxy" in /etc/apt/apt.conf.d/. You can call the file whatever you want. The file contains the line:

Acquire::http::Proxy "http://hoth:3128";

"hoth" is the hostname of the Squid server. Make sure there are no "Acquire" directives in any other apt configuration file:

/etc/apt/ # grep -rn Acquire *

I discovered that on some systems there was an "Acquire" directive explicitly telling apt not to use a proxy.

So far, Squid has been working pretty well. There was one instance where apt failed to completely download a package, but after running apt-get dist-upgrade again, the package successfully downloaded.

End of Semester

2008-12-21T09:13:00.003-05:00

Well the fall semester finally ended. Between TA work, the IDS class, the Java EE class and my actual job, this semester was a lot of work. Java EE wasn't 15 hours a week as anticipated. More like 20-30 hour each week, with a couple weeks (right before projects were due) around 55-60 hours. I did end up with As in both classes, though. Anyway, now I'm getting caught up on all that sitting around doing nothing that I've neglected for the last four months.

FiOS

2008-10-07T17:52:00.004-04:00

We just got our FiOS internet hooked up. I opted for the 20Mbs up, 5Mbs down option. Here's a bandwith test from speedtest.net:Other than being really annoyed at how difficult (i.e. almost impossible) Verizon makes it to use your own router instead of theirs, it's pretty awesome. I'm in the process of wget-ting the internet as we speak. Starting with .com.

Awesome Log Message

2008-10-01T08:20:00.003-04:00

I saw the greatest log message in my MacBook's system log this morning.

SyncServer[8551]: SyncServer: Truth vacuumed. Next vacuum date 2008-10-15 08:13:17 -0400

I'm sure it's something normal, and, given appropriate context, would make perfect sense. But I am just amused that the SyncServer is vacuuming up all the truth. No wonder I have a hard time finding useful log messages when I'm trying to troubleshoot something.

iTunes and Squeezebox Integration

2008-09-13T13:00:00.003-04:00

I've discovered something very cool with Squeezebox. iTunes integration. Okay, I've known all along that Squeezebox (more accurately, Squeezecenter, the backend software) can use your iTunes library, but I've never tried it. My network configuration is orders of magnitude more complex that most people's is. I assumed you needed to be running Squeezecenter on the same system (i.e., your desktop) as where you were running iTunes. Moreover, I assumed it would choke if its view of where the music lives was different that that of iTunes.

On my network, we use NFS homes and shares mounted at /Network/Servers/hoth/<> on our MacBooks. On the file server, the music share is found under /share/mp3s/music (sans the /Network/Servers/hoth). And the iTunes library (I'm using Steph's) on the server is under /home/steph/Music/iTunes (again sans the /Network...). I thought these differences alone constituted a recipe for fail.

Well I gave it a try this morning, and a few minor gotchas notwithstanding, it works great for me. You tell Squeezecenter where the iTunes xml file is located, as well as "where the music really lives", and it rescans your library and imports playlists (including snapshots of smart playlists). Evidently, it doesn't take the iTunes library's view of the where the music lives as gospel.

The gotchas were easy to deal with. One, which I anticipated was read permission on Steph's iTunes library, which is buried deep in her home directory, whose perms are 700. I fixed this with a little posix acl work (I <3 teh acls).

/home # setfacl -m u:squeezecenter:r-X steph
/home # setfacl -R -m u:squeezecenter:r-X steph/Music

And make sure new files and dirs created get the right acl:

/home # sudo setfacl -R -d -m u:squeezecenter:r-X steph/Music

'mkay, with that solved, I told Squeezecenter where to find the iTunes sauce. It was hard to tell if it was actually using the iTunes library or not, since it's the same set of music either way. However, no new playlists showed up, so I guessed it didn't take. Blanking out the path to the music and playlists that Squeezecenter uses in its "normal," non-iTunesey mode of operation, then reconfiguring the iTunes stuff, did the trick. Now I'm listening to Steph's awesome playlists on the downstairs Squeezebox.

Further, this solves the problem of how to get playlists into Squeezecenter. Don't get me wrong--it's totally balls that iTunes won't export (or even import!) m3u playlists. But at least we have a good (better?) workaround.

apt-cacher-ng

2008-09-09T08:34:00.005-04:00

I have several machines on my network running Ubuntu Linux which need to be updated regularly to keep up with security updates and the like. Several months ago I decided to try using a caching proxy to get updates from the Ubuntu package repositories in order to reduce load on the Ubuntu servers as well as to speed up the updating process on my systems. After spending some time using apt-proxy, and being generally dissatisfied with its reliability, I decided to give apt-cacher-ng a try.

Unfortunately apt-cacher-ng's documentation is a bit inaccessible. Aside from not being available on the project's website (it's in PDF form in the documentation directory after you install it), the documentation, although complete, is kind of impenetrable. There also really isn't anything that I could find along the lines of HOW-TOs or FAQs on the 'tubes. I eventually got it worked out, and, at least for the common case, it turned out to be more straight-forward than the documentation leads you to believe. Here's how I set mine up.

I put entries in my acng.conf like the following:
Remap-ubuntu: http://us.archive.ubuntu.com/ubuntu
Remap-medibuntu: http://packages.medibuntu.org/

The way this works is you specify names for each Remap-[whatever] where [whatever] is whatever you want to call it--they just have to be unique. Then the part after the ':' is the literal URL of the repository you're caching including the directory path on that server, if there is one.

The documentation describes a third part that follows a ";". If your second part is the actual URL of the repository, you don't need the third part, as you see above.

In my sources.list, I put
deb http://my-server-name:3142/packages.medibuntu.org hardy free non-free
deb http://my-server-name:3142/us.archive.ubuntu.com/ubuntu hardy main restricted universe multiverse
Here, the URL to the apt repositories is your local server name (port is 3142 unless you change it in acng.conf) followed by a directory that is the exact same as whatever the fully qualified domain name of the real apt repository was, followed by the actual directory path that you appended to the actual apt repository in your acng.conf.

There are lots of other options described in the apt-cacher-ng documentation, but I believe this is the common case, and it's working fine so far.

Update:
The directory structure that gets created under /var/cache looks like:


apt-cacher-ng
|-- canonical-partner
|   `-- dists
|       `-- hardy
|-- ubuntu
|   |-- dists
|   |   |-- hardy
|   |   |-- hardy-backports
|   |   `-- hardy-updates
|   `-- pool
|       |-- main
|       |-- restricted
|       `-- universe
`-- ubuntu-security
    |-- dists
    |   `-- hardy-security
    `-- pool
        |-- main
        |-- multiverse
        `-- universe

TA work

2008-09-07T17:51:00.004-04:00

An instructor I had last fall emailed me over the summer asking me if I'd be interested in being a grader, since I did really will in the class when I took it. When I replied that I'd be interested, he added that I could contribute to a lecture if I liked.

Naturally, I'm really stoked. I've decided to do a presentation on why client-side security is unenforceable. It will include a demo showing how you can trivially decompile a Java program, modify, and recompile it to exploit a flawed architecture where trust is placed in the client.

Enterprise Java class

2008-09-03T20:22:00.002-04:00

I started one of my two classes this semester--Enterprise Computing with Java. It's basically going to deal with a variety of Java EE technologies. The instructor said to expect an average of about 15 hour/week, with around 20-30 hours some weeks, so this is going to be a lot of work. My head is already spinning with all the Java EE acronyms from the first lecture.

Between this class, my Java Security class (starting next week), the TA work I'm doing, and my real job, I guess I'll get some sleep in December, when the semester ends.

Upgrade nfs/ldap server to Ubuntu 8.04

2008-08-29T11:23:00.004-04:00

I finally got around to upgrading my NFS/OpenLDAP/Samba/postfix/Squeezebox and OpenVPN servers from Ubuntu 6.06 to 8.04. I wanted to stay with the Long Term Support (LTS) release so I skipped the intervening releases.

Ubuntu supports an in-place upgrade that, in the past, has worked quite well on my desktops. However the file server is the most critical system on my network, so I had to make sure I had the time to do it right and to roll back, if necessary.

In order to give myself the option of backing, out, I booted the system from an Ubuntu disc and imaged the boot drive onto a spare hard drive.

I performed the upgrade first on the OpenVPN server since its configuration is much simpler, and it is less critical to the infrastructure. That upgrade went swimmingly, other than being prompted whether to replace or keep a number of config files.

I then upgraded the file server. Overall, it went much more smoothly than I anticipated, given how much manual configuration went into getting OpenLDAP, Samba, postfix, and NFS working and integrated.

The upgrade procedure choked on some errant whitespace in slapd.conf and terminated prematurely. As far as I can tell, though it did everything else it needed to do before bailing. It probably skipped upgrading OpenLDAP when it failed, then tried again after it had done everything else.

I also had a problem with a hard-coded device node in the fstab for my backup drive. All hard drives how get /dev/sdX, so the boot drive is now /dev/sda, which is what the USB backup drive used to be. Once I replaced that with a UUID instead of the now incorrect /dev/sda1, the system would again boot properly.

A weird problem that cropped up was that the squeezecenter software could no longer find any music. It turned out that the local squeezecenter user wasn't showing up as a member of the Domain Users ldap group and didn't have access to the mp3s directory. After much investigation, I discovered a directive in ldap.conf that tells libnss which usernames to ignore. After removing squeezecenter from that list, its group memberships then resolved correctly and the squeezecenter software could access the music again.

So now my most important system is running the up-to-date Ubuntu 8.04.

Back to the Blog

2008-08-29T11:14:00.003-04:00

It's been a long time since I posted anything to this blog. I had mostly given up on it. I think the problem is that I tried not to post anything that wasn't a well-researched, detailed, technical article. As such, there was a lot of overhead involved in preparing posts, and I rarely did them.

I'd like to get back into it, and try to post more often, so I'm going to try to make this more of a personal notebook of things that I think are interesting or want to remember for later. I may end up being my only reader, but that's fine.

EMI Selling non-DRMed tracks on ITunes Music store

2007-04-06T08:18:00.001-04:00

Apple and EMI have announced that tracks from EMI's catalog will be available from the ITunes Music Store without DRM. I must admit I never thought I would see this happen. I think this marks the beginning of a big change in online media. I think most people don't really understand what DRM is or what they're giving up by buying DRMed media. I think that for most people, as long as the DRM is sufficiently unrestrictive, then it really doesn't make any difference. I have always been worried that in time, Big Content, would lull consumers into giving up their fair use rights little by little. The reason this announcement is big is that, for the first time, a major label is tacitly admitting that DRMed media isn't necessarily just as good as the unrestricted variety. For the first time, some people will stop consider the possibility that DRM devalues media and is something they don't want.

I think this is big, but it's just a start. Notwithstanding the facts that you still require ITunes to buy the EMI tracks, and they are downloaded in Apple's AAC, as opposed to the near-universal MP3, there is the larger complication that nobody really keeps up with what labels their artists are on and which of the Big Four the labels are associated with. Look at the enormous list of RIAA member labels. The majority of those are affiliated with one or more of the Big Four. Which ones are part of EMI? Who knows? It's not that easy. Check out RIAARadar.com's disputed labels list. This is definitely nontrivial.

Anyway, this is exciting news. I can't wait to see where it leads.

ESR Gives up on Red Hat, Switches to Ubuntu

2007-02-28T08:44:00.001-05:00

I meant to post this last week, when it would have been timely, but was having problems with my blogger account.

Last week, Eric Raymond, author of the famous essay, The Cathedral and the Bazaar, posted a message to several open-source mailing lists expressing his frustration with Red Hat Linux. According to the post, he's switching to my distribution of choice, Ubuntu.

Personally, although Red Hat is among the most popular of distributions, I've been a bit frustrated with it for a long time. For one thing they have a habit of maintaining their own forks of much of the software included in the distro. In other words, they make changes to the upstream projects, such as Firefox or the Linux kernel, that make them unique to Red Hat. It's not that they don't make the changes available--they do, and they're required to do so under those projects' respective licenses. It's just that the software you get with Red Hat exhibits behaviors, sometimes bugs, that are unique to Red Hat. Ubuntu makes changes to the software they package as well, but they don't generally maintain separate forks. They generally get their changes merged into the upstream project.

An example is the Firefox web browser. For the longest time, the version of Firefox that shipped with Fedora Core (Red Hat's unsupported community distribution) was compiled with Javascript 1.4, which broke websites that needed Javascript 1.5. However if you downloaded the exact same version of Firefox directly from the Firefox website, it came with Javascript 1.5. Why was it necessary to build Firefox differently than the way it comes from the upstream project?

Another thing that Red Hat does is include a lot of configuration utilities that ease system administration by automating the generation of config files for various services. However, when editing those same files by hand, you frequently are greeted with "This file was generated by the such-and-such utility. Do not edit." Easing the tedium of hand configuring system services is one thing. But taking away the ability to hand configure services is another. What happens when the utility won't generate the configuration to do what you need. If you hand-tune the config, then your changes will get clobbered the next time the utility is run.

This is in addition to the fact that if you're coming from another distribution, and you expect to edit a config file which is in approximately the same place on every system, on Red Hat, you can't do it--you have to learn some tool that exists only on Red Hat. What good does it do to have a standardized configuration file in a standardized place, if you can't edit it in a standardized way?

In general I became frustrated by too many things done in the "Red Hat" way, that, once learned, are not useful or relevant on any other system.

Perl Program to Check for IP Address Change

2007-02-27T14:48:00.001-05:00

My parents' ISP, BellSouth (now also known as AT&T), has had the habit recently of frequently resetting their DSL connection, which results in service interruption as well as a new IP address. One weekend, I noticed the IP address changing several times per hour. I wanted a way of checking the IP address for changes, and logging it, so that I would be able to observe the frequency over time that this happens.

To solve this problem, I wrote a Perl program that I could run from my own network, that would do the checking, and if the address changes, write the new address to a log. As a bonus I decided to have the program send me an email notification of the change.

I have my parents' computer registered under a dynamic DNS hostname which is a free service from DynDNS.com . There are other, similar services, but I've been using DynDNS.com for about seven years, and I like them a lot. There are lots of clients that will automatically update your IP address with DynDNS. Many residential NAT routers, such as the one my parents use, have this functionality built in. Every time their connection is reset, the built-in dynamic DNS client on their router detects the reset, and updates the new IP address.

The dynamic hostname gives me something to do lookups against, even when the IP address has changed.

The script follows:


#!/usr/bin/perl

use Fcntl;
use Tie::File;
use Socket;
use Mail::Mailer;
my $name = "hostname.tocheck.com";
my $to_address='to_address@gmail.com,another_address@mail.com';
my $from_address='from_address@mail.com';
my $subject="$name IP Address Change";
my $message_body;
my $logpath;
my $logfile = "hostname.log";
my $oldip;
#Uncomment to turn on additional output
#my $DEBUG=1;




##############################################
#Subroutine to set (and create if necessary) #
#the log path.                               #
##############################################
sub setLogPath{
       if ($> != 0){
               print "About to set logpath to $ENV{'HOME'}/log\n" unless (!$DEBUG);
               defined($ENV{'HOME'}) or die "Can't set logdirectory: I don't know where HOME is!\n";
               $logpath="$ENV{'HOME'}/log";
               if(! -d $logpath){
                       mkdir($logpath) or die "Can't create $logpath: $!\n";
               }
       } else {
               $logpath="/var/log";
       }
}





##############################################
#Read the log file and                       #
#get the last IP written.                    #
##############################################
sub getOldIp {
       my @fields = split /\s+/,$iplog[-1]; #split the last line of the file on whitespace.
       my $ip = $fields[-1]; #ip is the last field, whether there's 1 field or many
       return $ip
}



##############################################
#Do DNS resolution to get the IP address as  #
#a number.  Only if resoltion was            #
#successful, convert the number to a         #
#dotted-quad string.                         #
##############################################
sub getNewIp {
       my $ip;
       my $netip = inet_aton($name);
       if (defined($netip)){
               $ip = inet_ntoa($netip);
       }
       return $ip;
}




##############################################
#If getNewIp() was successful (i.e. newip is #
#defined), compare newip to oldip. return 1  #
#if the different, 0 if the same, -1 if      #
#newip is undefined.                         #
##############################################
sub checkIp{
       if (!defined($newip)){
               return -1;
       } elsif ($newip ne $oldip){
               return 1;
       } else {
               return 0;
       }
}




##############################################
#Get the current UTC, and parse out in       #
#defined), compare newip to oldip. return 1  #
#if the different, 0 if the same, -1 if      #
#newip is undefined.                         #
##############################################
sub doTimeStamp{
       ($second, $minute, $hour, $dayOfMonth, $month, $yearOffset, $dayOfWeek, $dayOfYear, $daylightSavings) = gmtime();
       $year=1900+$yearOffset;
       $month=$month+1;
       my $timeStamp=sprintf("%d%02d%02d %02d:%02d:%02d\tUTC",$year,$month,$dayOfMonth,$hour,$minute,$second);
       return $timeStamp;
}




##############################################
#Build and send the email using Perl's slick #
#Mailer class.                               #
##############################################
sub message{
       $message_body="The IP Address for $name has changed.\nOld address:\t$oldip\nNew address:\t$newip\n";
       $mailer = Mail::Mailer->new("sendmail");
       $mailer->open({ From    => $from_address,
                       To      => $to_address,
                       Subject => $subject,
                    })
               or die "Can't open: $!\n";
       print $mailer $message_body;
       $mailer->close();
}


##############################################
#Main program:                               #
#-Set the logpath                            #
#-open the logfile                           #
#-Get the last known IP address              #
#-Get the current IP address                 #
#-Compare the last and current addresses     #
#-If there was a change, log the change and  #
#  Send an email                             #
#-If couldn't resolve hostname, display error#
#-Else do nothing                            #
#-Close the logfile                          #
#-Exit with $check status                    #
##############################################
setLogPath();
print "Opening $logpath/$logfile\n" unless(!$DEBUG);
tie @iplog, Tie::File, "$logpath/$logfile" or die "Couldn't open $logpath/$logfile : $!\n";
$oldip = getOldIp();
print "Old IP is $oldip.\n" unless(!$DEBUG);
$newip = getNewIp();
my $check = checkIp();
if ($check == 1){
       $timeStamp=doTimeStamp();
       $iplog[ $#iplog +1 ] = "$timeStamp\t$newip";  #add a new line to the end of the file
       &message;
} elsif ($check == -1){
               printf (STDERR "Couldn't resolve %s.\n", $name);
}
untie(@iplog);

exit $check;

This past weekend the script logged over 80 IP address changes on Saturday.

Christmas Wish List

2006-12-01T00:55:00.000-05:00

Note: Several links were wrong. Hopefully they are now fixed.

My ~~friends~~ ~~family~~ mom has been asking about my wish list, so here it is. Each item is linked to where it may be found online. The usual caveat applies--people shopping from this list should deconflict with each other. Of course gift cards can be duplicated!

Readers should feel free to post a link to their own wishlists in the comments, or in an email.
Be sure to check back for updates.

Subscription to 2600 magazine
(lifetime subscription would be nice:-)
Harmony 688 programmable universal remote:
Messenger bag from Banana Republic
Death Cab for Cutie, "Plans"
"She Wants Revenge"
Garden State soundtrack
Mac OS X Tiger for Unix Geeks
The Perl Cookbook
Perl Hacks
Wicked Cool Perl Scripts
Transformers Season Two Boxed Set, Part 1
and
Transformers Season Two Boxed Set, Part 2

Gift cards from:

Play an mp3 over SSH

2006-11-28T13:01:00.000-05:00

Secure Shell really is the coolest thing since sliced bread. Well, actually it's probably cooler. If you're not already familiar with SSH, then you probably are unaware of all the crazy things you can use it for. For example, you can burn a CD to a remote computer over SSH.

You can also play an mp3 (or several mp3s) over SSH. Here's how

When you ssh (lowercase refers to the actual Unix command) into a machine, it usually is done like this:

ssh uidzero@machinename.com

This connects you to the machine, and, after authenticating as the user, 'uidzero', executes the user's default shell, usually '/bin/sh'. However, you can have have it execute some other command by specifying the command at the end. For example,

ssh uidzero@machinename.com 'cat /etc/passwd'

would display the contents of the remote machine's passwd file (/etc/passwd is world readable).

So to play an mp3 file, we need to cat the contents of the file back across the ssh connection, like we just did with /etc/passwd.

However, we also need an mp3 playback program that can play from standard in. 'mpg123' is an ideal candidate. By specifying a dash, '-', you can tell it to play from standard input. You can try it like this,

cat file.mp3 | mpg123 -

This reads the mp3 in a stream of bytes and sends them to the standard input of the mpg123 program.

Now to put it together, we ssh into the remote machine, catting the mp3 we want, and piping the output of the ssh command to our local mpg123's standard input.

ssh uidzero@machinename.com 'cat /mp3s/song.mp3' | mpg123 -

Okay, so there's probably not much reason to do this particular thing, but it's easy to see the the flexibility of SSH. Combined with Unix's ability to send and receive bytes to and from anywhere you choose, SSH is a veritable Swiss Army Knife.

MacBook

2006-11-18T11:03:00.001-05:00

update #1: somebody read my blog
udpate #2: I added a link (inline) to the website where the tun/tap kernel extensions can be found for use with OpenVPN

I just got an apple MacBook (thank goodness for 0% credit cards and educational discount). I opted for the black model, and upgraded from 512mb ram to 1gb (this was an option that was cheaper to include on the order than to do myself). I had been holding off ordering it since the MacBook pros had just been updated, and there were rumors the MacBooks might get similar treatment before year end. When a few weeks went by with no sign of updates, I took the dive and placed the order. That was a Monday. On Wednesday, Apple introduced new MacBooks with a whole slew of updates including 2ghz Core 2 Duo, 1gb RAM, 120gb hard drive (up from 80) in the black model. I called Apple customer service to see if there was a chance they would upgrade my order to the new model since it had not yet shipped. They confirmed that the order was already upgraded (allegedly prior to my calling). Woohoo! My timing couldn't be more perfect. Since 1gb of RAM was now part of the base configuration, my order was now $100 less!

Well a week after placing the original order, my MacBook finally shipped, and I received it this past Tuesday. Of course the first thing I wanted to do was get connected to my wireless and surf from the couch. Of course, until I had the new machine configured to connect to OpenVPN, so I had to disable all the firewall rules on my router. I set the WEP key to the wireless and immediately connected. Instantly something was wrong--the connection was very stuttery. It would work then not work, and nothing I could think of seemed to help. I continued troubleshooting the rest of the evening, before finally giving up for the rest of the night. The next day, some googling revealed that others were having similar compatibility problems with the same wireless AP that I have--the ancient D-Link DI-614+. I was prepared to bite the bullet and order a new(er) Linksys wireless router, which would finally bring an end to the reign of 802.11b in our house, and usher in a "new" era of higher-speed 802.11g. I didn't want to spend the money, but the prospect of getting 802.11g, made it easier to swallow. Before placing the order, I did a last bit of troubleshooting. I wanted to see if there were any settings on the D-Link wireless router that would make a difference. The first thing I tried was to turn of "4x-mode." I wasn't exactly sure what this was, but I was certain it had something to do with enhanced performance if you had all D-Link hardware, which I didn't. This instantly did the trick (I may have changed one or more other settings at the same time--I'm not sure)! Oh well, so much for coming into the New Millenium--I'm still on pokey 802.11b, but it works, and is more than sufficient for simple web browsing & email.

The other issue that cropped up was trouble shooting static DHCP assignment. This is so that every time the machine connects to the network, it should be assigned the same IP address, based on it's hardware address. I have the dynamic pool (pool from which non-static addresses are randomly doled out) restricted down to exactly one address. Every time I would lease an address it would be the single dynamic address, not the static address which I had assigned. Incidentally, I was troubleshooting this issue simultaneously while dealing with the flaky wireless issue, multiplying my frustration. The next day, it occurred to me what I had forgotten. The wireless network is on a different subnet from the wired. I was assigning the wireless interface on the MacBook a static address from the wrong subnet. Of course the DHCP server was smart enough not to reply to a request with an address from the wrong subnet, so it could only give out an address from the dynamic pool. Once I fixed this, and then restarted the DHCP server, everything was fine. I was relieved.

OpenVPN was pretty straightforward, but it's a bit disappointing that it's not as well supported for OS X as under Linux and other Unix OSes. There are only a handful of people developing and maintaining the various components required to use OpenVPN. The first thing I discovered was a package called Tunnelblick, which includes the Tunnelblick GUI, as well as OpenVPN, the necessary kernel extensions (like Linux kernel modules--think "drivers") to configure the network tunnel. Unfortunately the kernel extensions that came with Tunnelblick wouldn't load--evidently due to incompatibility with the very latest version of OS X (10.4.8). I search around and found the website for the author of the kernel extensions, where I found a new, "experimental" version of the drivers that allegedly is compatible. The website had an Ominous warning:

This might crash your machine! If it does, don't blame me. I had some crashes testing the new code when I did lots of loading/unloading cycles. It seems to work reasonably well once it is loaded, so don't be to scared.

Wow. Scary. With trepidation, I installed and loaded the "experimental" kernel extensions. So far they've been working flawlessly. I've also been using the release candidate of Tunnelblick 3.0. It's a bit nicer, attempts to configure your DNS, and shows log output from the OpenVPN connection.

So I've had the MacBook for five days, and so far, I love it. One thing is for sure; Apple knows how to get the aesthetics right. This is a beautiful, elegant machine. It may not seem like a big deal, but having used it for a few days, I realize how much difference that last "5%" makes when it comes to the physical design. It did take some initial tweaking to get things just like I want, and there are still some things that I can't configure to my liking. However unlike Windows, when you can't configure something the way you want it, it doesn't feel like an oversight--it feels like Apple has decided that you don't need it that way.

I immediately installed Firefox and Thunderbird, as well as enigmail (Thunderbird really should come with this) for Thunderbird, and gpg (open source PGP), and a slew of other things. I have to say the state of OpenOffice on OS X is just sad. It doesn't integrade with the OS at all. It feels like 1998 Solaris. I also installed MS Office 2004. Not available in native format for Intel Macs, it runs emulated in Rosetta. I am amazed at how fast it is.

For the past few years, I've been using Linux almost exclusively, and I understand how it works on a fairly technical level. It drives me crazy to be using an OS where I don't understand the underlying mechanics of how things work. OS X has Unix underpinnings, which is nice, and I'll definitely be learning about that, but there are a lot of differences from Linux or other Unixes. I plan on installing Linux and probably Windows, for a triple boot. However, I plan on traveling with it over Thanksgiving, so I don't want to break it just yet.

Anyway, I love the MacBook. It is beautiful, and competitively priced to similarly spec'd PCs.

New Car, No Dipstick, WTF?

2006-10-18T10:36:00.000-04:00

Our new car (which we've had since August) has no dipstick. That's right, there's no dipstick to check the oil level. Rather you flip through the options in the trip computer, and, assuming the engine is running and you've just driven at least 6.5 miles, the oil level is displayed on-screen in the instrument cluster. When we test drove the car, the salesman highlighted that messy oil-checking was not necessary--the computer would let you know if anything is amiss. When we picked up the car and received an orientation of every little feature, again, we were shown how there was nothing to do under the hood and that all would be seen to by the computer. I was skeptical, but I decided to reserve judgement. Perhaps they really did get this down, and everything really works as promised.

Well, fast forward to this past weekend. The computer just alerted us that the engine was low on oil by at least a quart. With 2800 miles on it, I thought this very odd but dutifully added the oil anyway. Then the computer still said it needed another quart. Now I was really suspicious. Either it was really burning oil, or the computer was wrong and I had just added oil when it didn't need it. Either explanation is bad. Without a dipstick, there was no way to tell. Afraid to drive it in case of over-full oil, I left it parked the rest of the weekend until the dealer was open again on Monday. Keeping the car two days, they told me the oil level sensor had failed and was being replaced.

So now I am prepared to pass judgement. The "no dipstick" idea is dumb. I am almost certain this is a marketing decision rather than an engineering one. Dipsticks are reliable. Dipsticks don't have integer overflows, bounds-checking violations, or kernel panics. Computers do. No one ever says "I don't know how much oil I have because I can't get a reliable reading from the damn dipstick." Cleverness for its own sake is almost never a good idea, and there is elegance in simplicity.

Server progress: OpenLDAP

2006-10-17T10:29:00.000-04:00

Making good progress on OpenLDAP & Samba. I've got them both configured. I used the smbldap-tools scripts to add a user and set a password. I then was able to log in with that user. Home directory was created. Checking the id showed proper User ID as well as group membership. Removing the user successfully removed home directory as well.

I haven't yet test this from the windows side though.

Todo:

Add a machine to the domain (should create machine account automatically)
Add log on from windows machine
TLS for encrypted LDAP communication
Migrate data from old server
Migrate user profiles from old server

Battlestar Galactica Season 3 Premiere

2006-10-16T10:53:00.000-04:00

I finally got around to watching the premiere episode of the third season of BSG last night. Actually, I didn't watch all of it--it is 2 hours, and it was a school night. Not sure what to say other than this show just gets better and better. The Ron Moore continues to take a bad situation and make it worse. Things are really a mess right now for the human race.

Regarding aesthetics, the show has the same grittiness, the same great cinematography, and great musical score we've come to expect from the series. The story continues to blur the distinction between good and evil in contrast to the original series where the Bad Guy consisted of shiny robots bent on killing humans.

A friend of mine blogged about the parallels he sees in the the show relative to the Iraq war, and about how insurgents are just fighting against an oppressive occupation which is trying to force a way of life on an unreceptive people. I immediately took issue with this assessment--my friend is a Raging Liberal, after all, and it is natural to see things that support your own viewpoint, and to overlook the things that don't. However, after having seen (most of) the season 3 premiere, I now admit that there are a number of parallels that writer-director Ron Moore has intentionally and carefully placed in plain view and spoon-feeds to the audience. However, I feel that the Iraq war analogy really falls apart after just a bit of close scrutiny. The Cylons came out of nowhere after not being heard of for many years and whooped the human race. After reducing humans to mere thousands, the Cylons have subjugated the remaining few.

By contrast, in the Real World, the US, after many years of isolationism despite being attacked over and over again, finally has taken action to bring an entire culture forward about 1,000 years into the 21st century. This is entirely an act of self preservation. You can only get your butt whooped for so long before you have to do something. This is a problem that is not been solved before. There is no right answer. We chose to pick a country with whom we have been at war for 12 years, and turn it into a democracy. No subjugation involved. Just go vote and stop killing people--that's all we ask. Check back in about 10 years to see how it turns out.

So, I agree that there is an intended analogy between the show and the war in Iraq. However, I think it is a flawed one. I think, perhaps, that a better comparison would be the Maquis during WWII, which fought the Nazis in occupied France (who, like Baltar, opted to surrender rather than fight).

Anyway, political undertone or not, this is a great show, and I'm really looking forward to seeing what happens next. I really don't see how things can get much worse.

Server Problems & More OpenVPN

2006-10-10T11:09:00.000-04:00

Since I originally posted about OpenVPN, it has become a critical service for me. Not only do I VPN home from work everyday, but I also VPN out of the wireless sandbox when I'm roaming ( i.e., surfing from the couch) around the house. Naturally, the first service I implemented on the new server was OpenVPN. Well, last Saturday, I was looking forward to making some progress on OpenLDAP and Samba, when the server started acting up. It would lock up after running for only a few minutes. After going through all the obvious troubleshooting, the problem only seemed to worsen. Eventually the machine wouldn't even POST, and I resigned to the reality that the thing had Shit the Bed. I ordered a new motherboard (due today), and accepted the fact that I was now set back 4 more days plus $50.

However, I still had problem of what to do about the OpenVPN server. Well, I immediately scrambled to install OpenVPN on my desktop machine (seriously, I really can't live without OpenVPN). I had to get the hard drive out of the server and hook it up, in order to get the VPN configuration file as well as the Certificate Authority, which I would need to create an SSL cert for the "temporary" server. In addition I had to modify the firewall rules on my router to allow VPN traffic to the my desktop. Anyway, the upside is that I'm getting good at generating SSL certs, and can do it in a matter of minutes. This certainly will come in handy when I finally get back to OpenLDAP.

New Server

2006-10-10T10:48:00.000-04:00

I'm in the process of building a new server to replace my aging Red Hat 9 Samba/NFS/misc server. Actually, not new hardware, but rather a new OS and new services. The plan is for it to be an OpenLDAP/Samba integration project, so that I can have common network-based authentication from Linux & Windows (and possibly Mac OS X) machines. Right now I can authenticate against Samba from Windows and have network shares, roaming profiles, etc. Linux accounts are all local, however. That is ghetto.

So, the "new" server isn't on new hardware, so much as recycled hardware. The hardware that is new consists of a 250Gb hard drive for shared & home directories plus two more of the same (but in USB enclosures) for rotating backups, in addition to a 4U rackmount case. Everything else is recycled. The nice thing is a modern operating system that gets updates for 5 years.

OpenVPN

2006-08-28T12:03:00.000-04:00

I have discovered the thing that shall replace sliced bread as the gold standard by which cool things are measured. That thing is OpenVPN. Basically, it is an open source VPN solution that lets you do virtual private networking using SSL for encryption.

For a while I have been looking for a solution that will let me tunnel my Internet traffic through my home network, when I'm out and about with my laptop, such as at hotels, etc. This would allow me to surf the web, check email, etc without my traffic being intercepted by other people who may also have access to the network. In the past, I have used secure shell (SSH) to tunnel web traffic to my home network. That solution works fairly well, but it's a bit ghetto. For one thing, it only secures your web traffic, and you have to manually configure your web browser to use a proxy server. If you POP your email, that goes out unsecured. And I won't bore you with and explanation of the problems with tunneling TCP over TCP.

Anyway, when I connect to my OpenVPN server at home, it's just as if I've plugged right into my home network--right down to mounting NFS shares, printing to my printer (!), telnetting to my Tivo, streaming my mp3s. It's just like being at home. Plus it's all over SSL, so to anyone sniffing my traffic, all they see is SSL encrypted traffic destined to and originating from my home network. The only limiting factor is that my the upstream on my home DSL line is 384k, so that is a bottleneck.

Also, now I am finally able to lock down my wireless network. When I'm at home, I just connect over the wireless to my OpenVPN server just as if I were out and about and everything going out over the air is now SSL encrypted. By configuring my router's firewall rules to not trust the wireless subnet, except for OpenVPN traffic, anyone who is able to crack the WEP key and get on the wireless is in a sandbox and can't talk to anything since they don't have a certificate for the vpn.

I'll try to post some details later, such as my firewall configuration and how I architected the network to secure the wireless. At any rate, I'm very pleased with OpenVPN. It really does what I need.

Execution on SPARC CPUs

2006-08-24T07:38:00.008-04:00

Execution on SPARC CPUs

As a followon to the x86 execution demo, I'd like to take a look at execution on SPARC CPUs.

If Berkley RISC architecture can be said to be the textbook definition of RISC, then SPARC CPUs are a textbook implementation. This is because the SPARC architecuture is heavily influenced by Berkeley RISC I & II.

In this demo we'll take a look at the same simple adder program from the x86 demo. This is to establish a context in which we can compare the essentials of x86 and SPARC execution. However, it should be noted that the adder program is extremely simplistic. As such, there are a number of characteristics of the SPARC architecture which are worthy of analysis, which will not be exposed during the course of this demo. I'll try to highlight a few of these in the conclusion.

Before we get started, a general introduction of how things work with SPARC is in order.

Addressing & Memory access:
On the SPARC, there are only two addressing modes: Register-indirect and immediate. An operand is either loaded from the effective address already stored in a register, or it is included right in the instruction itself. There is no concept of direct addressing on SPARC. Also, there are only two types of memory accesses: Load and Store. Data is loaded from memory into registers, or stored from registers into memory. In fact, RISC architectures are sometimes referred to as load/store architectures.

Registers:
In contrast to the x86 architecture we looked at previously, which has only six general-purpose registers (only four if you don't count the index registers), SPARC CPUs have many more. The SPARC specification allows for anywhere from 64 to upwards of 528 (on SPARC v9). Like Berkeley RISC, SPARC implements register windowing, which allows a called function to have a unique set of registers from the calling function. These registers are organized into four groups--eight each of global, in, out, and local registers. The eight global registers remain visible to all functions across function calls. The out registers in a function's window become the in registers of the next function's window. The local registers are available only to the current function.

These general purpose registers are referred to by multiple names. Collectively they can be referred to by %r0 through %r32. More commonly they are referred to by synonyms denoting their role within the current window: %g0-%g7, %i0-%i7, %o0-%07, and %l0-%l7.

To put this together, %g0-%g7, are available across all functions. When calling a function %o0-%07, become %i0-%i7 of the called function, and conversely, %i0-%i7 are the %o0-%07 of the calling function. This window overlap facilitates passing of parameters to called functions. %l0-%l7 are the local registers.

That's not quite all there is to registers, however. We mentioned %g0-%g7 as being global registers. %g0 merits special attention. It is hardwired to zero. Anything loaded into %g0 will be discarded, and anything read from it will always be zero. In addition, %o6, %o7, %i6, and %i7 also have a special purpose. When calling a function the return address will be stored in %o7, and the stack pointer will be stored in %o6 (also called %sp). This means that %i7 contains the return address of the caller and %i6 contains the caller's stack pointer--current function's base pointer. We'll discuss this more when we talk about calling functions and the call stack, next.

Aside from the general purpose registers, there are variety of other types of registers. There are control registers, such as Program Counter, next Program Counter, a partial arithmetic register, and Processor Status Register. In addition, there are a variable number of floating point registers as well as various other miscellaneous registers which are beyond the scope of this discussion.

Instruction Format:

SPARC instructions fall into one of several categories. I will briefly discuss arithmetic and memory instructions here, but the reader is encouraged to consult the SPARC v9 Architecture Manual for more detail.

Arithmetic instructions take three arguments. This may consist of a source register, a (13-bit) immediate operand, and a destination register. Alternatively a second source register may be substituted for the immediate operand.

For example:


add %i5, %g1, %g1
add %i5, 512, %g1

The first adds the value found in input register 5 to the value found the the first global register and stores the result in the first global register.

The second example addes the value found in input register 5 to the literal value 512 and stores the result in the first global register.

Memory operations consist of load and store. Memory instructions take two arguments, a source and a destination.

For example:


st %r5, [%r4+8]
ld [%r4 + 8], %r6

In the above examples, the first calculates a memory address from adding 8 to the value in register 4. The value found in register 5 is then stored in the memory at the computed memory address.
The second exmaples performs the reverse operation. The data located at the computed address is loaded into register 6.

In addition to integer arithmetic and memory instructions, there are control flow instructions and floating point (real numbers such as pi) instructions.

Function Calls & the Stack:
With over 500 registers, who needs a stack? It turns out that, like the x86 discussed previously, the SPARC utilizes a stack for keeping track of function calls. However there are some important differences in the way the stack and function calls work between x86 and SPARC, some of which has been alluded to already in our discussion of registers.

The stack on SPARC, implemented in main memory, starts at high memory addresses and grows toward lower addresses, making the "bottom" of the stack at the top and the "top" of the stack at the bottom. When we need to allocate space on the stack we will decrement the stack pointer, so that it is pointing at a lower address in memory. Though unintuitive, this should be familiar from our discussion of x86.

Similar to x86, the stack is a place to record tops and bottoms of stack frames as the frames are created when functions are called. Where things begin to differ is the handling of parameters that are passed to called functions. Due to the relatively large number registers and the windowing technique inherited from Berkeley RISC, up to six parameters can simply be placed into registers rather than placed on and retrieved from the stack. This is particularly beneficial given the speed advantages of register access vs memory access. Given the lack of a direct address mode and the resulting requirement of multiple instructions to accomplish a single memory access, the ability to pass parameters via registers becomes doubly important.

When a function is called, generally control is transferred via either the jumpl or call instructions. call writes the value stored in PC (its own address) into %o7, to be used by the called function as the return address. jumpl writes the return address into a register specified in the instruction. However, the primary difference between jumpl and call is their respective addressing modes, with call being PC-relative, and jumpl being register-indirect. A call instruction will contain an offset from the current PC value. jumpl will specify a register and possibly an offset.

Once control is transferred, the first instruction in the called function is a save instruction like the following:


save    %sp, -120, %sp

One thing that the save instruction does is manipulate a special register that keeps track windows. When executed, save decrements (yes, decreases) the Current Window Pointer, or CWP.

Remember from our discussion of registers that %sp and %o6 are synonyms for the same register. Also recall that the caller's %o6 becomes the callee's %i6. This means that the caller's stack pointer becomes the callee's frame pointer. In other words, the top of the caller's frame becomes the bottom of the callee's frame. The callee's first instruction, save, creates the new frame by decrementing its own stack pointer. How can the save instruction decrement the newly acquired %sp (%o6) if it doesn't contain a known value? What isn't immediately obvious, but is absolutely crucial, is that in the above save instruction, the two references to %sp, are actually two *different* registers. What is happening is that between the first %sp reference, and the second, the CWP is getting incremented, so the second %sp refers to a register in the new window. The save instruction subtracts a value from the outgoing %sp and stores it in the incoming %sp, thus creating a new stack frame with space allocated.

When we're ready to return to a calling function there are a number of cleanup steps that must be accomplished. First, remember that the steps required to call and execute a function are played back in reverse. This means, among other things, that the input registers, %i0-%i7, will become the output registers of the caller. If there is return value currently stored in the local registers or our stack frame, it should be placed in an appropriate *input* register. In order return control to the caller, we must jump back using the address the caller placed in our %i7. This is accomplished with the synthetic ret instruction. The ret actually accomplishes a jumpl instruction similar to


jumpl %i7+8, %g0

Which causes a jump 8 bytes from the address stored in %i7, storing the return address in %g0. Since we're returning from a called function there's no need to store the return address. This is why %g0, which is hardwired to zero, is specified as an argument. The return address is discarded. After ret, increments the CWP, shifting us back to the previous window. Since the previous %sp/%o6 are now visible, the previous stack frame is effectively destroyed.

There are two more issues regarding function calls and the stack, which should be mentioned briefly for the sake of completeness. First, what if there are more than six parameters to pass to a function? In this case, the additional parameters must be stored on the stack. Also, since there are a finite number of registers, there are a finite number of register windows. This is the number of general registers, less the globals, divided by 16, to be exact. What happens if we are so many function calls deep that we run out of windows? In this case, the CWP reaches zero, causing a trap, which causes the next register window to be saved to the stack before being used.

Okay, that was a long set up, but it was necessary establish a context for the program we're going to look at. Again, this is the same simple program we looked at in the x86 demo.

Here is the program:


int a=4;
int b=5;

int adder(int d, int e)
{
    int c;
    c = d + e;
    return c;
}

int main()
{
    int c;
    c = adder(a, b);
}

This program has a main() and a function called adder(). main() calls adder(), passing the values of two global variables as parameters. adder() then adds the values passed into it and returns the result. We will look at how the parameters to be passed are stored in registers and how the adder() function is called, creating the new stack frame. Then we'll see how the result is returned, the stack frame is destroyed, and how control is transferred back to main().

Again, gcc can produce the compiled, but not yet assembled program like so:


gcc -S adder.c

The assembly-language output is saved in adder.s.

Let's take a look:


    .file "adder.c"
    .global a                #our global variable a
    .section ".data"
    .align 4
    .type a, #object
    .size a, 4
a:
    .long 4
    .global b                #our global variable b
    .align 4
    .type b, #object
    .size b, 4
b:
    .long 5                  #the values for a and b are written directly in the assembled program
                         #rather than being initialized at runtime.
    .section ".text"
    .align 4
    .global adder
    .type adder,             #function
    .proc 04
adder:
    !#PROLOGUE# 0
    save %sp, -120, %sp      #create a stack frame for adder()
                         #and decrement window pointer
    !#PROLOGUE# 1
    st %i0, [%fp+68]
    st %i1, [%fp+72]         #store our incoming parameters to the stack
    ld [%fp+68], %i5         #before loading them into working registers
    ld [%fp+72], %g1         #where we will compute the answer
    add %i5, %g1, %g1        #add the thing in %i5 to the thing in %g1, storing
                         #the answer in %g1
    st %g1, [%fp-20]         #store our answer
    ld [%fp-20], %g1         #getting ready to return, load return value into %g1
    mov %g1, %i0             #put return value into %i0, caller's %o0
    ret                      #jump to previously saved return address
    restore                  #Shift the register window back to main()'s registers
    .size adder, .-adder
    .align 4
    .global main
    .type main, #function
    .proc 04
main:                    #***************Execution starts here.*********************
    !#PROLOGUE# 0
    save %sp, -120, %sp      #create a stack frame for main()
    !#PROLOGUE# 1
    sethi %hi(a), %g1        #a time saving shortcut for loading a 32 bit constant
    or %g1, %lo(a), %o5      #straight to a register without memory access
    sethi %hi(b), %g1        #more on these later
    or %g1, %lo(b), %g1      #
    ld [%o5], %o0            #load our 'a' into out register 0
    ld [%g1], %o1            #load our 'b' into out register 1
    call adder, 0            #call adder with 0 offset
    nop                      #delay slot related to pipelining--more on this later
    mov %o0, %g1             #put adder()'s result into %g1
    st %g1, [%fp-20]         #save %g1 to the stack
    mov %g1, %i0             #end of function so save return value to %i0
    ret                      #return control to caller
    restore

Here is a synopsis of how the adder program is executed on the SPARC cpu.
--The program starts by executing main.
--main()'s stack frame is created by subtracting some value from the caller's %sp, shifting the register window, and storing new stack pointer in main()'s %sp
--A series of shortcuts is used to load a and b as 32-bit constants into registers using immediate mode, requiring only two instructions each.
--Transfer to control to the address adder() with 0 offset.
--A delay is required after the call instruction since other instructions are in the pipeline. If the compiler can't find a useful instruction not related to adder(), it put's in a nop.
--Once in adder(), we set up a new stack frame
--Move our incoming parameters around using the stack as an intermediate storage area
--Perform the addition, storing the result in %g1
--Store the result from %g1 onto the stack space allocated for variable c.
--Before returning from adder(), the return value should be loaded from c into a register
--The return value should be moved into one of the callee's input registers which will become the caller's output register after the return.
--Ret jumps back to previously saved return address
--Restore increments the CWP, shifting the register window and popping the old stack frame
--Back in main(), the return value is stored in the space on the stack allocated for c.
--Prior to exit from main(), result from adder is relocated to a register that will be accessible by the caller.
--Control is transferred to the caller, and the register window is shifted back once again.

Some additional notes:
Global variables a and b already have values at compile time, which allows them to be treated as constants by the assembler. This allows the compiler to optimize the program by loading their values straight into registers without the overhead required for memory access. However, SPARC instructions are all 32-bits wide. By the time you subtract bits for the opcode, register reference, and other things, only 22 bits remain for an immediate value (depending on instruction format). There is no way to place a 32-bit integer in 32-bit instruction as an immediate value. This is where the sethi and or instructions come in. sethi loads a 22-bit immediate value into the upper 22 bits of the lower word of the specified (64-bit) register, and clears the lower 10 bits to zero. Then, the or instruction ORs a 10-bit immediate value with low 10 bits of the specified register. This saves time by eliminating the multiple instructions required for a register-indirect memory access, as well as the inherent delay in accessing slower system memory vs faster cpu registers.

Due to the SPARC's pipelining--the technique of having multiple instructions "in flight" at the same time, a delay slot must be incorporated between a call instruction and actual transfer of control to the called function. This creates a bubble in the pipeline, allowing us to transfer without affecting the instructions that will come after. The compiler will generally attempt to place an instruction in this slot which does not depend on the outcome of the called function. If no such instruction can be found, then a nop is used, which has no effect.

For more information:
The SPARC v9 Architecture Manual(sun.com)

PANIC! UNIX System Crash Dump Analysis(amazon.com)
This book, though out of print, is an excellent reference on SPARC architecture. Even if you're not interested in crash dump analysis, just skip to part 2, "Advanced Studies." In addition to coverage of program execution, it goes on to cover topics related to any modern operating system, including the kernel, virtual memory, file systems, and others.

A paper(ZIP file!) by David Litchfield covering overflows on SPARC. A brief introduction to execution on SPARC, before going on to exploiting buffer overflows to gain control of execution.

Execution on Intel x86 CPUs

2006-08-22T13:45:00.003-04:00

I know I said back in May that I was going to post a couple of articles about CPU execution. Well, I got busy and stuff, then three months happened, and now it's August. Anyway, here it is.

I decided to throw together a brief demo of how x86 cpus are organized, and how they execute instructions.

In terms of register organization, x86 CPUs essentially fall under the 2nd category described in the Mano[1] text--that of general register organization. They have six general purpose integer registers, 4 of which can be accessed 8, 16, or 32 bits at a time, and a number of special purpose registers such as floating point registers, and others that are beyond the scope of this discussion. In addition there are two registers that are essential to stack operations: the base pointer and the stack pointer. Although x86s don't utilize a stack in the same way as the "stack organized" CPUs Mano describes, the concept of a stack is still central to x86 execution. We can see how instructions are executed by looking at a simple program:


int a=4;
int b=5;

int adder(int d, int e){
  int c;
  c = d + e;
  return c;
}
int main(){
  int c;
  c = adder(a, b);
}

This program has a main() and a function called adder(). main() calls adder(), passing the values of two global variables as parameters. adder() then adds the values passed into it and returns the result. It is worth looking at how a function is called and a value is returned, because the stack makes it possible.

One very important thing to understand and keep straight regarding the x86 stack implementation is that the stack is upside down. What I mean by this, is that as the stack grows as things are pushed on to it, it grows toward lower (smaller) addresses. This is not intuitive. If the base pointer and stack pointer both contain address 10, and an item is pushed onto the stack, the stack pointer will then contain address 9. So, a reference to the "bottom" of the stack or a location "lower" on the stack refers to something at a higher address. Something "higher" or at the "top" of the stack is at a lower address. I just picture the stack as a stalactite growing from the ceiling of a cave.

In order to see how this program is executed we can look at the assembly language produced by the compiler. You can tell gcc to compile the program but not assemble it with the -S option:

gcc -S adder.c

The assembly-language output is saved in adder.s.

For reasons I'll explain below, we're going to add one more option to gcc. This will result in output that is easier to read.

A simple incantation:


gcc -mpreferred-stack-boundary=2 -S adder.c

Yields our compiled, but not assembled, program, adder.s, so let's take a look.


  .file    "adder.c"
.globl a                        #our global variable, a
  .data
  .align 4
  .type    a, @object
  .size    a, 4                 #It's 4 in size because an int on a 32-bit x86 is 32 bits.
a:
  .long    4
.globl b                        #our global variable, b
  .align 4
  .type    b, @object
  .size    b, 4
b:
  .long    5                    #the values for a and b are written directly in the assembled program
                                #rather than being initialized at runtime.
  .text
.globl adder
  .type    adder, @function
adder:                          #the adder() function
  pushl    %ebp                 #Save the caller's base pointer on the stack
  movl    %esp, %ebp            #make the old "top" the new "bottom"
  subl    $4, %esp              #allocate 4 bytes on the stack for local variable c
  movl    12(%ebp), %eax        #move the thing stored 12 bytes "below" the base pointer into register A
  addl    8(%ebp), %eax         #add the thing stored 8 bytes "below" the base pointer to the thing in reg A
  movl    %eax, -4(%ebp)        #store the answer in the space we allocated on the stack earlier

  movl    -4(%ebp), %eax        #we're about to destroy the stack current stack frame, so better save our
                                #return value in a register.

  leave                         #this destroys the current stack frame by reversing the first two steps:
                                #move %ebp to %esp, making the current "bottom" the new "top",
                                #then then pop the saved base pointer back into %ebp

  ret                           #the previously saved instruction pointer is popped from the stack and
                                #execution returns to where we left off in main()
  .size    adder, .-adder
.globl main
  .type    main, @function
main:                           #***************Execution starts here.*********************
                                #The next two steps are always used to step up a new stack frame for
                                #The current function:
  pushl    %ebp  #<-------------%ebp is a register containing the address of the "bottom" of the stack
                                #we're going to save it by pushing it onto the stack   
  movl    %esp, %ebp  #<---------make the current stack pointer the new "bottom"    
  subl    $4, %esp              #Allocate some space on the stack, specifically 4 bytes, for local variable c
                                #by subtracting 4 from the stack pointer.  On x86 the stack grows upside down,
                                #so it starts at higher addresses and grows toward lower ones.    
  movl    b, %eax               #Move our global variable, a, into the A register   
  movl    a, %edx               #Move b into the D register    
  pushl    %eax                 #Preparing for the function call to adder() we need to push   
  pushl    %edx                 #the parameters we'll be passing onto the stack

  call    adder()               #the instruction pointer is pushed onto the stack and execution jumps to adder   
  addl    $8, %esp              #clean up the the stack   
  movl    %eax, -4(%ebp)        #save the return value, previously saved in the A register, on the stack for later use   
  leave                         #we're done with main, so destroy the stack frame   
  ret

So here is a synopsis of how this simple program gets executed on an x86 CPU:
--The program starts by executing main.
--A stack frame for main() is set up by saving the contents of the base pointer register to the stack, then moving the contents of the stack pointer to the base pointer.
--Four bytes are allocated on the stack, for the local variable, c.
--The values that will be passed to adder() are located in global variables, which are located in memory somewhere other than the stack. In order for the values to be used by adder(), they must located on the stack. They must first be moved to registers before they can be pushed onto the stack.
--From there they are pushed onto the stack for use by the soon to be called adder() function.
--When the adder() function is called, the current instruction pointer is saved to the stack (implicitly by the call instruction) and execution jumps to adder.
--Once in adder, the steps to set up a new stack frame are repeated.
--Space is allocated on the stack for adder()'s local variable, also called c.
--Adder then grabs the numbers to be added from "below" its stack frame and performs the addition, saving the result to the the space allocated on the stack for local variable c.
--Before returning to main, which will destroy the current stack frame, the contents of the local variable, c, are moved from the stack into the A register for use by main.
--Adder's stack frame is then destroyed and execution returns to the point where it left off in main.
--Adder's return value, previously stored in the A register is saved to the stack for further use.
--With nothing left to do, the leave instruction destroys main()'s stack frame and the ret instruction returns control of execution to the calling program.

Some additional notes:
This assembly code uses AT&T syntax. Among other things, it is characterized by specifying a source register or address, followed by destination. Alternatively, there also is Intel syntax, which does a number of things differently. However, the finer points of the syntax are really dependent on the assembler being used and aren't really tied to the hardware.

It also is worth noting that there are a number of operations that may seem unnecessary or less than optimal. This is attributable to several things. First, this is an artificially simple program. It adds two numbers together and then quits. The adder() function moves the result from the A register to the location on the stack allocated for c. We told it to do this when we said 'c='. It then moves the contents of c into the A register because it has to. adder()'s stack frame is about to be destroyed along with any local variables it may have. Also, there are a number of different ways that a high level program can be implemented in low level instructions and still accomplish the same effect. The assembly code generated is very compiler-dependent.

Lastly, recent versions of gcc, when compiling for x86, like to align the stack on a 16 byte boundary. This results in lots of extraneous stack allocations and housekeeping. Though this is almost certainly some sort of optimization for modern CPUs and is done for a very good reason, it makes for very difficult-to-read assembly code. For this reason, we used the -mpreferred-stack-boundary=2 option to align the stack to a 2^2 byte boundary. This made the stack 4 bytes wide rather than 16.

[1] This demo was originally written in the context of an undergraduate Computer Architecture class and is intended to be tied to relevant discussions from the computer architecture text by Morris Mano.

Computer Processor Architecture

2006-05-12T22:36:00.001-04:00

I'd like to take the opportunity to indulge myself. I have a couple of articles that I have written for another purpose that I'd like to post as a two-part series. They're fairly technical, and, as such, may not be of general interest to most people. However, if you've ever been curious about how a computer's processor, or CPU, works, then they may be worth taking a look at.

In the articles, we'll take a look at a simple C program and analyze how that program gets executed on two different types of CPUs. There are a few things that these articles leave to the reader to understand. Among them are a general familiarity with C-style programming syntax. Also assumed is a basic understanding of CPU organization, especially the notion of registers. An essential concept to both articles is that of a stack, as it relates to data being organized in a computer's memory. Lets take a moment to discuss these ideas.

A CPU must have registers, which are very small but fast memory locations built right into the cpu. They are designed to hold operands which are used in the CPU's operations. For example, if a processor with two regisers were instructed to add 1+2, it might store 1 in one register, and 2 in the other. Then having quick access to the operands, it can add them and store the result in one of the two registers. Registers can be used in other ways, too, such as storing the memory address where a paricular operand is to be found.

A stack, as it relates to computer memory, simply is an organizational structure for data. It is exactly like the spring-loaded plate warmer at a buffet. After taking a plate of the stack, the stack rises to present the next plate. A plate can be added to the stack, causing the stack to be pushed down, accomodating the new plate. A stack data structure works the same way. A stack can be accessed with a "push," "pop," or "peek" operation. A data item can be pushed onto the stack, or popped off of it. Some stacks may also allow a peek at the next item without removing it. Less strict stack implementations allow accesses at arbitrary locations within the stack. For obvious reasons, a stack is sometimes characterized as being a last in first out, or LIFO, data structure.

Briefly, I'll introduce the program we'll look at in the CPU articles. This is a simple program that adds two numbers. The program follows:


int a=4;
int b=5;

int adder(int d, int e){
  int c;
  c = d + e;
  return c;
}
int main(){
  int c;
  c = adder(a, b);
}

This program has two integer variables, a and b, which are set equal to 4 and 5 respectively. The program starts running at the main() function. Main has an integer variable, c, which will be used to hold the result from the addition. From within main(), another function, adder(), is called to perform the addition. The a and b variables are sent to adder() for addition. adder() receives the contents of a and b storing them in its own variables, d and e, respectively. It also has its own integer variable, c. d and e are added and stored in c. Then the value stored in c, is returned to main(), from whence adder() was called.

There, in about 3000 characters, is a very brief introduction to a few concepts that will be central to the next couple of posts. Like I mentioned, these are somewhat of an indulgence for me. Though they may not be of general interest, this is a topic that I find very interesting, so hopefully somebody else will, too.

Keyboard Tray Lighting

2006-05-09T19:04:00.000-04:00

The desk that I use at home has a lap drawer that pulls out to be a keyboard tray. It works pretty well and is a nice desk, but one thing that has always bugged me is that when I pull out the keyboard the desktop casts a shadow over the top few rows of keys. I'm always having to bend my neck down to get a good look at the keys to make sure I've got my hands lined up in the right place. I always thought it would be great if I could mount some sort of small light under the desktop that would illuminate the keyboard. I know there are really bright LEDs available in various colors. including white, but I didn't want to buy loose LEDs and try to inegrate them into a lighting system. I didn't want to have to deal with attaching them to something, converting voltage, etc. Then I remembered cold cathode tubes, which are readily available for people who trick out their computer cases (I've used them before). I checked on directron.com and found a kit that included two 12" white tubes which would be perfect. Since they are for use inside a PC they use 12 volts DC which I could easily supply using a 12V AC/DC wall adapter.

After ordering and receiving the lights, I headed to Radio Shack to pick up a few parts (it wasn't quite that easy--there was quite a bit of thinking involved in the mean time). I would need a power supply, some sort of spring loaded switch (like a button or lever), and a way to connect the power supply to the lights.

Radio Shack carries a variety DC power adapters in varying voltages, that all have a modular connector on the end to accomodate a variety of available power tips, available separately, one of which hopefully will match your electronic device. You get to pick one power tip for free with the purchase of an adapter. In the electronic compenents section of the store, among other things, there are DC power jacks ready for soldering that are available in surface mount (for mounting in a box or enclosure) or inline (for hanging off the end of a cable) form factors. They are coded with different letters so you can get matched plug and jack. It turns out that the same letter codes are also labeled on the power tips that go with the DC adapters. This makes finding and choosing an appropriate DC adapter and power tip to match your project very easy. I went with the "M" size. No particular reason--it just looked like a good size. The smallest power adapter they had in 12V DC was 500ma, which is unfortunate because my lighting kit only draws a meager 5ma.

For a switch, I chose a lever switch. My thinking was that as I slid the keyboard drawer in, the lever would get pushed downward by some object which I would attach to the underside of the desktop, like a wedge of wood or some such. I considered using a push button type switch, which I could attach to the backside of the drawer and would be pushed in when the drawer was fully closed. However I was worried that the spring in the switch would be stiff enough that it would cause the drawer to spring back out, leaving the switch in its "on" position.

Here are the parts and part numbers that I used:

Mini SPDT Lever switch: 275-0016

Inline power jack: 274-1577

AC/DC Adapter: 273-1773

Power tip for AC/DC adapter: 273-1716

Cold cathode lighting kit from directron.com

Here are some pictures showing what I did, along with some descriptions.

In this picture, you see the mechanism responsible for tripping the lever switch when the keyboard drawer would be closed. I spent the most time figuring this part out. I was thinking of using a wood block or wedge or something. The problem is the amount that the object would have to hang down must be very precise. It couldn't hang down low enough to obstruct the free movement of the keyboard drawer. On the other hand it must hang down enough to cause the lever switch to be pressed down completely, opening the circuit. Based on my measurements, that only gave me a margin of about 1/8" to play with. Then I realized that by using a strip of metal I could flex and bend it until I had the clearance just right. I decided to use one of those metal blanks that covers the unused expansion slots on the back of a PC. Using fairly small screws, I attached it to the underside of the desk. I had measured and made marks earlier, so I knew right where it needed to go. A pair of channel-locks where ideal for bending the metal into an appropriate shape.

In this picture you see the wiring harness with my inline power jack soldered onto it, as well as the lever switch. The lighting kit came all wired together with a switch and power connector ideal for use inside a PC. I chopped those off and soldered on my own.

Here, you see the miniature lever switch tacked onto the rear end of the side of the keyboard drawer. The switch is oriented so that the lever will be pushed down as the drawer is slid in. You may also notice my extra crispy heat-shrink tubing. I was lazy and didn't feel like getting out the heat gun, so I used a lighter instead.

In this picture, you see the actual lights attached to the underside of the desktop along with the power inverter. The lights operate on 300V DC, and the power inverter converts the 12V DC to to voltage appropriate for the lights. The power wire is easily attached and dettached from the power inverter. This is so that the keybard tray can be completely removed if necessary. The power inverter is attached with velcro and can be repositioned if necessary. I'm not really satisfied with how I attached the lights. I used a staple gun to attach zip ties in the spots where I wanted the lights to be attached. Then I cinched the zip ties down on the lights. Zip ties are kind of ghetto, but I didn't want an extremely permanent solution like epoxy or anything like that.

Here is the power strip tucked away inside the cabinet of my desk that controls all of my lights. All the lights on and around my desk are plugged into this power strip, such that the switch controls them all.

In this picture the keyboard drawer is pushed all the way in and the light is off.

In this picture the keyboard drawer is pulled out, and you can see the glow of the white lights on the keyboard and trackball.

Not counting preliminary head scratching and trips to Radio Shack, the project took about two hours. I'm really pleased with how it turned out--it solves the problem perfectly.

Middle Finger

2006-04-29T17:35:00.000-04:00

I seldom write new posts, because I seldom have things I want to write about. My wife says I should write about things like this, so here you go.

Friday, when I was driving home from work, I was merging onto a very busy Highway 32 from I-95. Highway 32 is a six lane highway--three each way. Just as I was getting into the merge window, a guy in the center lane changed into the right lane right where I needed to get. I was already going faster than him, so there was just enough room to slice in right before my merge lane ran out. It was close, but since I was already going faster that he was, he didn't have to slow down. It definitely was not a cutoff. Evidently he though it was because right after I sliced in, he proceeded to flash his (I assume) bright lights at me.

The distance between us continued to grow, and within a few seconds I was able to make my way over to my rightful place--the far left lane. The other guy must have realized that bright lights are ineffective as an offensive tool at 3:00 pm on a sunny afternoon. I noticed in my peripheral vision that he had switched back to the center lane, caught up with me, and matched my speed. We were now neck-and-neck. I was certain he had arrived to deliver yet further vengeance upon me.

Not wanting to give him the satisfaction, I continued to look ahead rather than at him. He could not communicate his hateful message if I refused to receive it. Eventually, however, I surrendered to my curiosity. Looking to the right, sure enough, I was greeted by a middle finger. Attached to the middle finger was a very angry man. The man was mouthing something at me, but with two panes of automotive glass, four feet, and 74 miles per hour worth of rushing wind between us, I was unable to make out his message. I am about 90% certain it involved the word "fuck."

The thing is, I wasn't angered by this childish display of insolence. It made me envious. What simple problems he must have. If the worst thing that I had to worry about all day long was some guy cutting in front of me in traffic, life would truly be good.

The Google Calculator

2006-02-26T19:23:00.000-05:00

If you have never used google's built in calculator, now's a good time to take a look at it. It's quite handy.

Here's a quick introduction.

You can simply type your mathematical expression into Google's search bar to get the answer.

For example, you can obvious things like

(3*768)/(456.7+28)

to obtain the following result

(3 * 768) / (456.7 + 28) = 4.75345575

with 8 decimal places of precision.

You can also do less obvious things like perform unit conversion:
For example,

7 miles to kilometers

gives:

7 miles = 11.265408 kilometers

and

3 gallons to cups

yields:

3 US gallons = 48 US cups

(I think google returns localized results based on your ip address, so the result to conversion queries may vary)

But the fun doesn't stop there.
Entering the phrase "8670 to binary" yields:

8 670 = 0b10000111011110

and,

0b10000111011110 to octal:

gives

0b10000111011110 = 0o20736


0o20736 to hex:

gives

0o20736 = 0x21DE


0x21DE to decimal:

gives

0x21DE = 8 670

Base conversion appears to work only for integers, sadly. How cool would floating point decimal to binary conversion be? Very.

Anyway, give it a try--it's really handy. And if you use a web browser that has a google search box built in, like firefox, then you don't even have to go to Google's site, saving a few keystrokes.

For lots more info, see the Google Calculator help page.

Anonymous Comments

2005-05-06T10:58:00.000-04:00

So I recently discovered a setting in my blogger.com profile that determines who can post comments. Previously, it had been set to members only, which I think is dumb. I have now changed it to "anyone." Anonymous comments should now be possible.

Portable hard drive

2005-05-06T08:23:00.000-04:00

I had an old laptop hard drive laying around that I bought off of someone for cheap. It really wasn't serving any purpose and was just going to waste. Back when I had my Dell Inspiron 8200 I got an extra hard drive tray on Ebay and it was handy to swap the spare drive in and out to run other OSes and such. However since I got my current notebook, the 10gb hard drive has just been sitting on the shelf. Enter the USB 2.0 enclosure. I got this thing form Newegg.com for about $20 shipped. It came in yesterday. This thing is amazing. For one thing it took all of five minutes to slide the hard drive in and screw in two eye-glasses sized screws. It was so easy your mom could do it. Second, it's really small. The picture doesn't do it justice. It fits in my back pocket. If you've never seen a 2.5" notebook hard drive, then let me tell you they're amazingly small. This external enclosure adds very little to the hard drive's dimensions. Third, I just plugged it in and it worked automatically. My laptop is dual boot. Both in my preferred distrubution of Linux as well as Windows XP, the drive was automagically recognized. As a matter of fact, I had forgotten that I had Debian Linux installed on that hard drive previously, so when I plugged the drive in and the partitions were recognized, there was all the data and OS files that had been there previously. For the record, if you want to be able to read (and write) the maximum number of different file systems, whether it's Windows, Mac, or Linux, or others, Linux is the way to go. The Linux kernel has support for more than about a dozen or so different filesystems.

The other cool thing is how the drive is powered. Since it's USB, and notebook harddrives are so low-powered, it's designed to take power off the USB port it's pluged into. This worked great for me, and my understanding is that most USB ports on notebooks and computers are cable of providing adequate power. However, in the case that your notebook's USB ports are under powered, there is a second, sort of "vampire" USB plug on the cable that you can plug to into a second port to supplement the power requirements. I didn't have to use the second plug.

Anyway, this thing is really cool, and for $20 I can now put this hard drive to good use backing up my personal files or shuttling large files from one computer to another. Did I mention it's really small?

Why Digital Rights are an Illusion of Freedom

2005-02-09T19:25:00.000-05:00

Recently, in a class I am taking (of the web-based distance learning variety) there was a sidebar discussion regarding the Apple iPod. The people who have the iPod were going on and on about how great it is, how it's easy to use and how you can buy songs at the iTunes Music Store.

I don't have an iPod, and although I think they're a brilliant example of industrial design, with their clean lines and intuitive interfaces, there are a number of issues I would need to understand better before deciding to purchase one for myself. Most of these issues revolve around a technology called Digital Rights Management, or DRM.

Brief Description of DRM
DRM is technology that enables copyright owners, namely the big four record companies, to ensure that songs you purchase online at stores like the Itunes Music Store don't end up online being swapped in file trading networks. This sounds resonable, right? After all artists (i.e., record labels) should be compensated for their work.

Before we decide, however, let's take a quick look at how DRM works. There are a variety of DRM schemes, each working differently, and each have more or less restriction. However, here's a generic description of how it works. A song that you buy from an online music store is downloaded to your computer in an encrypted format. This means that you can't unlock it unless you have the right key. Of course the key has to be provided by the store where you bought the song. Naturally the key is tied specifically to that particular song, so it won't work with any other song. It doesn't stop there, though. You could simply upload a copy of the key along with a copy of the song into the ether, then everybody would be able download it, unlock it and play it, right? Right. And that just wouldn't do. So the key also is tied to your computer or music player as well, so it won't work on another computer or music player.

Historic Problems with DRM
Historically, there have been a few issues with DRM that have prevented its acceptance among consumers. A major problem has been with compatibility. Music purchased from an online music store would require a special program to play it back. Music players from different online stores wouldn't be compatible. Music purchased online would almost certainly not play on a portable digital music player. Lately, Microsoft is hard at work to solve the compatibility issue. They've started the "Plays for Sure" logoing program where music players and online vendors that display the "Plays for Sure" logo are supposed to be compatible with one another. At the heart of this program is Microsoft's own Windows Media Player 10. You wouldn't accuse Microsoft of committing an act of altruism, would you? Me either. It doesn't even bare mentioning that music purchased from the "Plays for Sure" logo partners won't play back on competing operating systems like Mac OS or on Linux.

Another issue was pricing. It took some time to arrive at a pricing model that was attactive to consumers. One model is a subscription that allows a certain number of downloads per month. Another is a subscription that allows all-you-can eat listening online, but no downloading. Yet another, as is the case with the iTunes Music Store, is a per song and per album pricing structure. In this scenario the customer pays a certain amount for each song downloaded, or, alternatively, a certain amount for an entire album. If the entire album is desired, the price is usually more economical than for each individual song.

The big problem, however, that has prevented wider acceptance of DRM restricted online music, is with the restrictions themselves. Nobody wants to pay good money for music that can only be listened to online and not downloaded, or for music that must be kept in a virtual lockbox and can't be copied to other computers or devices. It is here, that Apple has been able to make serious inroads with their iTunes Music Store. Of all the competitors in the fledgling pay-for-download music industry, the restrictions placed on Apple's iTunes music are perhaps the least restrictive of all. Recently they've been relaxed even further to allow playback on up to five separate computers. This is in addition to the ability to burn CDs and copy to an Apple iPod (but no other type of music player of course).

Digitally Restricted Freedoms
Herein lies the rub, as well as the purpose of this article: When you buy DRM-restricted music (or anything else for that matter) you are surrendering certain freedoms that we historically have taken for granted. I think folks are being duped. As DRM schemes become more broadly compatible, easy to use, more flexible in terms of restrictions, the easier it becomes to give up those freedoms.

To put this into perspective, lets consider the purchase of a physical audio CD as the basis of comparison. Under the fair-use provisions of our (U.S.) copyright laws, it is acceptable to do a number of things with a music CD you have purchased. You can make copies of it. You can even make multiple copies of it, for example, one for upstairs, one for downstairs, one for the car, and even a copy to listen to at work. Maybe you even want to put the original away for safe keeping. You can also extract the songs from the CD into mp3 format for playback on your mp3 player and any and all of your computers. That, too, is legal and perfectly acceptable (note that at this point legal purists will point out that can be no case presidence for fair-use--each instance must be tested by the court). With DRM-restricted music, you usually are afforded only some subset of this flexibility. As I mentioned before, Apple has afforded a great deal of flexibility with their for-purchase music, giving the illusion of freedom.

I want to make sure that last point isn't lost. Illusion. Regardless how unrestrictive a vendor's DRM is, any "freedom" it affords is only an illusion. It is for that reason that customers of iTunes are being duped. In general, people tend to think that as long as the DRM doesn't "get in my way," then it's as good as being unrestricted. That still doesn't change the fact that their perception of freedom isn't real.

Put it this way--If I were to put you in prison, you most certainly would be upset with restrictions on your freedom. Even if I said you were free to move about as you wish within the confines of the prison, the fact would remain that you are in prison and cannot leave. Now, what if I expanded the walls of the prison so that you would have greater freedom of movement? I could even expand the walls so far that you could not see them. Would you not still be in prison, unable to leave? You could move about in any direction as far as you could see. For all intents and purposes you would be free, save for the fact that you couldn't leave. Regardless how unrestrictive your confines may be, your perception of freedom would be an illusion.

Some may argue that the difference between my analogy and reality is that with the music, participation is optional, and in the analogy participation is compelled. Certainly the prison metaphor isn't intended to be extended to the smallest detail. However the comparison is still valid. It boils down to this. With DRM-restricted music, whenever you play back the songs you have purchased, you must ask permission, either directly or indirectly. Remember the files are encrypted not by you but by the vendor. You cannot decrypt the files for playback without a key issued by the vendor. Each time you play the file, the playback software must verify that the key is valid, has not been tampered with, and has not expired. And of course the key can be revoked at any time. So even if you can play the files when or where you want, in principle you still must ask permission each time you do. That is not true freedom--it is only the illusion of freedom at best.

As a final thought, the open source sofware community likes to highlight the differences in the way we use the word free. They say there is "free as in beer" and "free as in speech." I'm not saying music should be free as in beer--artists (i.e., record companies) should be compensated. But they should be free as in liberty. Once I purchase a song or CD or book or pair of jeans, I think I should be able to use it how & where I want without asking anyone's permission, so long as those uses are legal. Don't you?

The Shadow File

2004-12-15T22:20:00.000-05:00

This is the first post to The Shadow File. I suppose a brief explanation is in order. What is The Shadow File? In Unix and Linux, an algorithm is used to create a one-way hash of a user's password. This hash, along with the username and other information used to be stored in /etc/passwd. In many modern *nix systems, the password hash has been removed from the passwd file and is now stored in a separate file, because the passwd file is world-readable. This way, an attacker would have to obtain both files in order to attempt to recreate the password. The file that contains the password hashes is /etc/shadow. Now you know.

The things that will fill this page include my humble observations and musings on whatever projects I may be working on or anything at all that may stimulate my thoughts. So prepare to have your attention captured. I'm sure you won't be able to pry your eyes away.