My First Ruby Project

The Project

Typically, even my pet projects have a specific goal in mind. I don't write a spec for my personal projects, there's just a, maybe slightly fuzzy, goal. Generally I want something that will be useful. I'll work on a project until:

• The project does everything I hoped for. Maybe even more!
  
• I run out of spare time to work on it. This doesn't happen as often as you may imagine. I really enjoy coding. So, after a tiring day of coding at the office, I come home to relax and work on one of my pet projects. If it was any different, then I have no right to call my blog My Geekdom.
  
• I decide that I did a poor job picking a pet project as they're supposed to be fun and this projects isn't. So long!
  
• I get stuck such that proceeding will require more effort than I feel it's worth. That said, hopefully I learned something working on the project.
  

But, unlike my typical pet projects, my goal with my new project is rather amorphous. I want to analyze some data. At time of writing this, I have just shy of 5K records totally 181MB of data.

Leaving the details for a later posting, each record is a snapshot of the state of my Mac OS X computer. I've been collecting, and am continuing to collect, samples every five minutes since the beginning of March. At least every five minutes that my computer isn't sleeping or hanging.

I decided to collect the samples because I was getting downright frustrated with the performance of Mozilla's Firefox browser. For reasons that are off-topic for this posting, I feel a strong interest in making sure that Firefox continues to be successful.

For years I thought Firefox was the bomb. Starting about 9-12 months ago I started to notice that Firefox was routinely using all the resources on my computer. This concerned me because Google's Chrome browser was starting to have enough functionality to be a primary browser.

Being that I wanted Firefox to stay competitive, I decided to try to stay with it. As Mozilla changed their release model, I started to use Aurora and Firefox Beta and provide lots of feedback. The newer versions also seemed to consume a more reasonable amount of my computer's resources. Being that the Firefox team was working hard at fixing performance problems, especially memory leaks, I stayed with Firefox. (See Reference: Articles About Firefox's Memory Leaks And Repair Efforts for information about FF's memory leaks.)

Org-Mode + t Makes Tweeting Easy

Update 2012-06-04 Mon 00:57

You can download the code from its Github repository.

Motivation

After yesterday's ranting about Octopress, it occurred to me that while I now own the content of my blog postings, I don't own the content of my tweets. I'm sure that there's some way to convince Twitter to give me my tweets, but why should I rely on them to store my tweets.

The obvious answer is that my tweets should be created on my computer and then sent to Twitter.

It should be noted that I'm not interested in implementing a full Twitter client. There are more than enough of those. I'm interested in having my tweets, or at least the bulk of my tweets, in a simple and easily accessible format.

The Answer Must Be Cheap

While I want to author and store my tweets locally, it is not a high priority task for me. Whatever solution I come up with must add little or no overhead to my tweeting workflow, must be simple to implement, and cannot require ongoing tinkering to keep working.

Being that my relationship with Org Mode is rather simple: The more I learn about it the more of my life I want to be based on Org Mode. Org Mode just seems to make everything easier. So I decided to use Org Mode for the author, publication, and storage of my tweets. The system I'm describing has these properties:

• It took me just a few hours to design, implement, test, and begin using it for tweeting.
  
• Besides not adding overhead to my tweeting, it has made tweeting dramatically simpler.
  
• The entire implementation is a single file of elisp that is 120 lines long. 50 of those lines are standard Emacs header comments that are automatically created. The implementation, along with its documentation, is only 70 lines long. While a small code base isn't one of my goals, not needing ongoing tinkering is. It is hard to imagine that 70 lines of code will require too much tinkering.
  

Automatefd Gvoice And Gmail To Gcal

I seem able to hold incredibly large amounts of technical details in my head. In contrast to this, I'm excited when I can remember what year my kids were born in. From my viewpoint, this is just the way I have always been. From the viewpoint of people in my personal life, it can be frustrating.

Being the technophile that I am, I've incorporated many electronic assistive technologies into my life. In 1989 I got my first PDA. It was about the size and weight of a half-brick. But I could carry it wherever I went and it would ring to remind me to do things. What a deal!

My newest electronic assistive technology is a rather complete phone and SMS management system. Its features are:

• I have a permanent record of every incoming or outgoing call or SMS.
  
• My mobile phone rings whenever I get an incoming call or SMS.
  
• My home phone rings whenever I get a call.
  
• My mobile rings and gives me a notification whenever I get a call or SMS.
  
• My computer rings and gives me a pop-up whenever I get a call or SMS.
  

This blog entry is unfinished. It will be updated or followed up with a second blog entry shortly.

Blog In A Box (actually, a repo)

As I have been developing my blog's technology, I have been surprised at the lack of a pre-packaged blogging solution for Emacs' Org Mode. Org Mode is simply a wonderful tool for organizing thoughts, ideas, text, links, etc… While Org Mode has a rather difficult learning curve, its motto, Your life in plain text., seems to be accurate.

Spurred by ideas from 0xDECAFBAD.com and code from NanoBlogger (NB), I've gone about creating an Org Mode-based blogging system. And, if you are reading this, it is actually working. While NB and Org Mode do all the heavy lifting, I've simply added some glue to hook them together. What I have is far from a perfect solution, but it seems to work pretty well.

There is a bit of a UI problem in that the Org Mode editing has a general look-and-feel like that seems like a typical Org Mode editor with a NanoBlogger posting jammed in. That's not surprising as, to save development time, I've basically just jammed a NanoBlogger posting into an Org Mode buffer.

The other problem that I have with the system is that there is no way to automatically detect the difference between postings that have been published and those that have not published (ie: drafts). Unless you take extra care when publishing your blog, all postings, whether or not they are drafts, will be posted.

Once I've fixed these two problems I will have a simple but reasonably functional blog system. By "simple" I mean that it is based off of NanoBlogger, a minimal blogging platform. That being said, the goals of NB are simple goals compared to most blogging systems. If you want a richer blogging system then NB isn't for you.

The glue that I've written to connect NB with Org Mode is a shell script and some elisp code. Combined they are only a few hundred lines of extensively-commented code. The entire NanoBlogger installation, living up to its name, is only 844K.

Being that NB is so very compact, my shell scripts and elisp code are small, everything is in plain text files, and that Github provides simple and free hosting of both the blogging system and the blog, I'm going to merge my entire blogging system, minus Emacs, with my blog and put it into a single Github repository.

Once that is done, anyone who wants to try my blogging technology should only have to do a few simple steps:
# Fork my blog to their own Github repository.
# Clone their copy of my blog to their computer.
# Delete my blog postings (I'm assuming people will want to create their own postings and not republish mine).
# Run emacs and load the single elisp glue file I've created.
# Start blogging.

I believe that it will take less than 10 minutes to get a blog up-and-running. That is longer than it takes to get a hosted blog on Blogger but much, much faster than most self-hosted mechanisms such as MovableType. The only prerequisites for this blogging system are

• Bash and the other common shell script utilities that are required by NanoBlogger such as sed and awk.
  
• A reasonably modern GNU Emacs. It comes with Org Mode pre-installed.
  
• The git command-line program.
  

Being that this blogging system is only aimed at Emacs users, it is certain that they'll have Emacs already installed on their machine. It is also very likely that they'll have bash and the shell script utilities. Maybe they'll have git or maybe they'll need to install it. It is a simple installation.

There is no need to install a local web server for staging. Provided you have versions from this decade, there is no need to ensure you have the right version of any of this software. Managing versions of installed software can be a major pain for many systems (eg: Python based-apps). You also have full access to your blog's data without having to export anything (everything is already in plain text files) or provide a credit card (Github is free).

My New Blogging Technology

As I've discussed in previous postings about this blog's technology, this is a static blog written with NanoBlogger. NanoBlogger is a CLI driven blogging system. Alas, I have found the NanoBlogger manual to be difficult to follow and the syntax of the CLI to be non-intuitive. (See the footnote here regarding the existing CLI.) On the upside, NanoBlogger is a CLI system which makes it easy to modify.

I started out with a few small helper scripts, eventually merging them into a single helper script. As I was using that script and learning more about NanoBlogger, I found myself writing smaller helper scripts for my big helper script. Eventually, I was never calling my original, large helper script directly.

At this point it became clear that some refactoring was needed. So I refactored. I came up with a single shell script call nbc.sh that does all the work and a series of shell scripts that simply call nbc.sh.

The key idea behind nbc.sh, or simply nbc, is that the scripts that call it are trivial. By "trivial" I mean that each script is only a few lines long, all the side scripts have identical content, and, if not for the semantics of Git, would simply be multiple symbolic links to the same program.

As you can see below, the code for the side scripts simply stores the command name, $0, of the side script that was called and then calls the nbc.sh script that is in the same directory as the side script.

#!/bin/sh

# Code to load include file.
ORIG_ARG_0="$0"
NBDIR=${0%/*}
. ${NBDIR}/nbc.sh

This means that as I continue to add functionality to nbc, all I need to do is copy one of the side scripts to the filename of the new command I added.

While I won't discuss the exact naming scheme used for these side scripts in detail in this posting, I will say that I have tried to systematic in how the scripts are named. Once you learn a few abbreviations (eg: "e" for "Entry" or "Entries", "u" for "Update") and the rules for combining those abbreviations (standard RPN), you will know the names of all the commands, what each command does, and the type and order of arguments each command takes.

I still need to write richer documentation for nbc and its side scripts, but I can finish this posting off with some concrete examples.

NB supports an "update" command that essentially tells NB to rebuild the blog from scratch. The syntax of this command is:

nb update

My nbc version of this command is:

nbu

All of the commands for nbc start with the letters "nb". The letter "U" is the abbreviation for "Update". So "nbu" tells nb to run the update command. As I said, the code for nbu and the other side scripts are trivial and given above. As far as examples go, nbu is not very exciting. But the nbc's CLI for creating tags is a bit more exciting.

NB's syntax for creating tags, as documented in its manual, is:

nb --title <tag-name> add tag

Gosh! That's both tough to remember and tough to type. In nbc, the command for adding a tag is nbga. All commands start with "nb". After that, the "g" means that we are talking about "taGs". The "a" stands for "Add". The syntax for the command is

nbga <tag list>

Besides being much easier to type and remember, it allows a tag list as compared to a single tag name. That is, you can create multiple tags in a single command rather than needing to call nb once for each new tag.

In NBC, a tag list is a comma-separated list of one or more tag names. So "tag1" is a tag list and "tag1,tag2,tag3" is also a tag list. If you prefer you can space-separate the tag list but you need to ensure that you pass the entire tag list as the first argument to nbga.

As the operations become more complex, NBC's simplifications become greater and more obvious. My next posting about NBC will discuss some of these simplifications.

If you wish to use NBC, look at the source code, or tell me why NBC sucks, you can get the scripts at https://github.com/Neil-Smithline/nbc.

This Blog Is Now Hosted For Free (and it's slashdot proof too)

While there are many free or nearly free blog hosting sites available, most of them come with a long list of restrictions. Either they restrict you to using their blogging software, throw ads on your pages, give you the "unlimited bandwidth" plan that they'll cut off if your site gets too much traffic, limit you to only the ugly themes, etc.

But www.neilsmithline.com is now hosted on a site that charges me nothing, doesn't put ads on my pages, couldn't care less about the theme I use, and, should I ever be lucky enough, survive slashdotting. The only restriction is that my blog has to be static. Being that I already had a static blog (see previous posting), this was no problem for me at all. I've used lots of dynamic blogs (Drupal, WordPress, Joomla, Blogger, Movable Type, etc.) and just found that "dynamic" is a short way of saying that the blogging system is going to do what it wants and not what I want.

Along with the free hosting I get automatic version control, well-managed collaboration (not that I have any collaborators), data redundancy, eternal archiving (I probably won't need it that long but it's nice to know that nothing will get lost), and the ability to get a copy of my data onto my local machine whenever I want it.

My choice of blogging technology was largely influenced by these two blog postings, 1 and 2, and email conversations I had with their author, Les Orchard. While Les went with Jekyll, I preferred the simpler NanoBlogger. Les uses Github.com for versioning his blog but hosts his blog on his own site as he wants full control of what is published. I simply couldn't agree with him more.

When I moved to my new blogging framework, I quickly got my blog up and running using NanoBlogger for site generation and sftp to publish it to my VPS. Today I investigated replacing sftp with Github and its notification framework. This would essentially mirror Les's blog system except that I am running NanoBlogger locally while he runs Jekyll.

As I was investigating using Github for transportation and archiving of my blog, I decided to take another look at Github Pages. And then, like a lightning bolt, it occurred to me. When I publish my blog by pushing it to Github, if the repository has the magic name of neil-smithline.github.com, then Github would publish my blog at http://neil-smithline.github.com/ automatically. There simply was no benefit to serve my blog from my VPS as Github would do it far better than I could on my VPS.

The only problem I foresee is that I run more than just my blog on www.neilsmithline.com. But the other applications I run are only for my private use so I can just map them to another sub-domain such as whatever.neilsmithline.com. For example, I've been playing with a cloud-based bookmarking system, Asaph, that allows you to run your own cloud bookmark server. But I can access it at and, if you have an interest, can view it at: http://www2.neilsmithline.com/asaph/.

All-in-all, it's like a nerd's dream come true :-D

UPDATED: Sat Feb 11 13:12:02 EST 2012

There seems to be another problem with hosting my blog on Github. While practically all of Github requires SSL, there appears to be no way to get a Github hosted website to use SSL. Thinking about it, the task seems inherently difficult. Github makes all of its sources file public but you must protect your site's SSL private key. Currently, all site information is stored in the Github repository. A private SSL key could only be added if Github updated their configuration to allow the key to be stored out-of-band (ie: not in the repository).

Nastiest Hack I've Seen In Awhile

I was reading about SL4A, also known as Android Scripting Languages. They are a set of quick-and-dirty ports of programming languages to the Android that interact with the Android OS. I just stumbled upon this "issue page" (it's more like a forum than a bug report).

The developer is trying to run something on his Android, which is basically Linux, that requires superuser permissions. As the responses started to build upon each other, one user suggested starting his shell script with the following lines (the actual posting was in Android's flavor of Python but I've converted it to bash(1) to make it more accessible):

#!/bin/sh
if [ "$USER" != "root" ]; then
    exec sudo $SHELL -c "`cat /proc/$$/cmdline | tr '\000' ' '`" root
fi

I simply couldn't help being entertained at how revolting but equally brilliant that code is. But, I started thinking (always something dangerous) about how much I hate having to deal with stupid commands. For example, this is the output of running apt-get(8) without proper permission:

$ apt-get install foobar
E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)
E: Unable to lock the administration directory (/var/lib/dpkg/), are you root?

Being as I often write shell scripts with calls to sudo(8) in the script, I decided that imitation is the best response to being revolted (I know my mom said something like that :-).

So I expanded upon the existing code, making it a full shell script utility. You can look the code or grab a copy at my GitHub page in the rerunasroot repository. I'll be using it in the future.