Two Steps Forward

My blog on software development, the universe, and everything.

An Interview with Olaf, Part II

Note: Read part one for more context first!

Ah yes, the old punch a hole in the floppy to double the storage trick. Unfortunately, my drive was one sided or something, so it didn’t work for me, but I had friends who did that. If only it were so easy to increase storage capacity in these days!

How do you store your work environment customizations? Do you have a dotfiles repo or something similar?

I keep my dotfiles repo at I use mostly the same setup for both home and work. For my personal use I have an Ansible setup that will install my dotfiles on other machines that I need to work on. So, it’s quite portable and I can develop in a familiar environment in most situations.

Any tips or tricks?

I have my dotfiles running Continuous Integration via GitHub Workflows. So, I can tell immediately if I’ve broken something. I find it helpful to take care of little issues as they come up rather then when I’m setting up a new account. That’s not the time I want to be fixing things.

And why did you hate the Fortran 77 class?

The class consisted of sitting in a dark amphitheatre early in the morning and watching a guy in mock turtleneck scribble on transparencies that we viewed via his overhead projector. From where I was sitting it looked like he was just phoning it in. It was dull, there was no effort to make things interesting and it wasn’t clear to me why I’d want to be learning this language at all. I don’t remember anything that I learned in this class other than the fact that I probably didn’t want to write software. So, after I took a few years off from University, I switched over to the Humanities. I learned Greek and Latin. Those languages seemed to have way more day to day applications for me than Fortan 77. I also began enjoying my education.

And are there any interesting intersections between your experiences with the humanities and programming?

There’s some overlap between learning natural languages and machine languages. If you can sit down and learn Ancient Greek, you can probably sit down and learn to do some programming. In both situations you’re learning to parse constructs which, at the outset, probably seem foreign, but begin to make sense once you learn the rules. Natural languages are much more expressive and the rules in natural languages have more exceptions, so if you can master reading a new natural language there’s a lot you can apply to breaking down a machine language.

I also find with both of these that there’s a difference between a passive and active understanding of the language. Just because you can read Latin doesn’t mean you’re just as proficient at writing it. The same can be said for machine languages. Reading and writing are two different skills. Clear and succint writing is a valuable skill in both natural and machine languages.

I have to confess I’m really curious why you went the HTML route to digitize the book instead of using something like a word processor. Was there a particular reason behind the decision, or perhaps was it something like “HTML and web browsers look cool, so….”?

At the time, you could still use Gopher on the University network, but web sites were starting to become popular. I was in the Classics department and had, with some others, also been looking at digitizing the University’s ancient coin collection. There was some friction with getting getting access to the coins. If we had pushed harder we may have been able to make that happen, but it quickly became less fun as we tried to get the right permissions. It reminds me of when a Professor of mine encouraged me to ask a local manuscript collector if it would be possible to see a particular parchment that was in his collection. The collector looked at me and said, “possible, but not probable.”

And why on earth was this guy building a server in a fridge? For cooling purposes? If I learn nothing else from this interview I’d love to hear about that. [If you happen to have any pictures, I’d love to share them as well.]

As much as I’d like to have some real insight here, I think the form factor of the fridge was close enough to that of a server rack that it was just a quirky place to build his project. I’m pretty sure it had been gutted and no longer contained a compressor, so I don’t think he would have been able to use the fridge to actively manage CPU temperature. I remember two things that he told me. One was “you should really buy a book on Unix”. (This was told to me when I dropped by his office to ask for help.) The other thing he said was “Windows is for jerks”.

Do you remember any of your early experiences with CPAN, reinventing or discovering existing wheels?

A number of years ago I was spending my free time playing in a band and recording songs. Initially we shared our music on, but the real action ended up being on MySpace. (I remember how Tom was my first friend – he was everybody’s first friend). In the early days of MySpace things were not all that locked down and you could automate stuff fairly easily. I found WWW::Myspace on CPAN, which was a module which used WWW::Mechanize to automate using MySpace. Since there was no actual API, this was as close as you could get to having an API. I wrote a script which used WWW::Myspace to get, for example, a list of people who were friends with the Dixie Chicks (now The Chicks) so that I could add them as friends of my band. Some people would inevitably add you back as a friend and your network would grow.

As I was doing this I also wrote a kind of data model for MySpace. It allowed you to log in to MySpace with an arbitrary number of accounts and keep information about your friends in a database. This was necessary so that you wouldn’t try to add the same friends twice.

I was getting really into this friend adding. I bought an old 386 from a used computer shop and I set it up in my office so that I could have a machine dedicated to this hobby. It would run for hours at a time. I have even had it sleeping at random intervals to make it look more human. Eventually MySpace added CAPTCHAs to the process at random(ish) intervals. So I had to tweak my script to stop if it detected a CAPTCHA and launch a FireFox window so that I could manually enter the CAPTCHA and then enter the correct keystroke so that the script could continue. Using this method I eventually got over 20,000 friends for my band.

You may be asking yourself why am I spending so much time on this story. As it happens, as I was working on the MySpace code, I contacted Grant Grueninger, the author of WWW::Myspace and told him what I was working on. He said I should publish it as a CPAN module. He was really encouraging. So I got a PAUSE account and started uploading. He taught me about SVN and got me started using version control for the first time. I collaborated with him on the project and also uploaded my own modules. I learned a lot from him. He got me started with Open Source Software and made me realize that it wasn’t a scary thing. It could be a very fun, collaborative and creative way to spend your time.

I remember when I submitted my first bug report. I got a message back from the author thanking me for the report and taking a few minutes to explain some things. I hadn’t been expecting to hear back at all and to get a really kind response back was very encouraging for me. I don’t think it took a lot of time out of his day, but it made me feel like this was something that I could do and feel good about.

How did you get the motivation to write MetaCPAN?

I had just gotten the first iPhone. They weren’t being sold in Canada or officially supported on any mobile networks here, so I had to drive to Buffalo to pick one up there. I got it back home to Toronto and eventually decided to jailbreak it and install the Cydia app store. I still have that phone and, to this day, when you boot it up, you see a pineapple with a bite out of it rather than an apple. I loved that phone. It was such a pleasure just to hold it in your hand.

Eventually when Apple opened things up I wanted to write an iPhone app. I decided that I wanted all of CPAN’s documentation on my phone and in one app, so that I could browse docs on the subway, where there was no cell signal. I was commuting to work on the subway a lot at the time and I figured browsing random CPAN modules was a good way for me to learn.

I eventually wrote iCPAN along with Mark Jubenville. The app got a lot of the documentation out of CPAN and stored it in an SQLite database, which my Objective-C app could then display on your phone. I had to upload a new release anytime I wanted to update the docs, but you had to start somewhere.

As I was writing this I came to realize that what I really wanted was an API which already had the data I was looking for. I was at a meeting of the Toronto Perl Mongers and somebody mentioned this new thing called ElasticSearch (now Elasticsearch), which was basically an API in a box. I went home and started playing with it. After maybe 8 weeks of evenings and weekends I had the first version of the MetaCPAN API. I remember asking for help on the ElasticSearch IRC channel in the early days. Two people in particular were very helpful to me. One of them (Clinton Gormley) went on to become head of Elastic Stack development at Elastic and the other (Shay Banon) became Elastic’s founder and CEO.

The first version of MetaCPAN was basically just ElasticSearch exposed to the Internet, running in a tiny cloud instance. The funding was paid for by folks from who handed me some cash one evening at a pub. A friend from, Mark Jubenville, built a JavaScript front end to demonstrate that the API actually worked. It had customizable author profiles. Once we published the URL, people almost immediately got in touch to find out how they could customize their author profiles. From there the interest continued to pick up.

I had to rush to build MetaCPAN since our second child was on the way and I wanted to get something together before my free time disappeared completely. By the time our little girl was born, the project was already up and running and kind of took on a life of its own. I was able to step back a bit and it became more of a collaboration. It also became pretty clear that more people were interested in MetaCPAN than iCPAN. So, my attention shifted away from iOS apps and over to OSSaaS (Open Source Software as a Service).