mattberther.com

Removing duplicate messages from Outlook

2012-09-20T00:00:00Z

I recently learned that Outlook for Mac had been uploading multiple copies of the same message to the Exchange server. At final count, I had approximately 280,000 email messages sitting in my “Archive” folder on the server. As you can imagine, this caused tremendous download times for resynchronizing my folders.

I looked for tools that could purge the duplicates for me, but had a tough time getting most of them to work on Microsoft Outlook 2010. I set out to try and solve this problem by creating a simple C# app that would iterate through my archive folder and identify and remove duplicate items.

I chose to parse the messages in two different ways to make sure that I was able to remove as many duplicates as possible. The first scan removed every message that had a duplicate message id. The second scan removed every message that had the same sender email, subject, and sent time.

This technique worked remarkably well. My archive folder now has less than 70,000 messages in it, which means that approximately 75% of the messages in that folder were deleted as duplicates.

I’ve made my source code available at github for anyone that is interested in using and/or forking the project.

Please keep in mind that there are no warranties with the code. It worked well for me; your mileage may vary.

Chrome extension for instapaper

2012-09-14T00:00:00Z

I use instapaper.com as my read later service. I have installed the Chrome add-on to allow me to quickly tag an article to read later. Also, I have configured it as my read later service in Tweetbot, which allows me to quickly send articles to it for later reading.

The one thing that has always bugged me about the instapaper website is that it does not open links in a new tab/window. To get around this, I set out to create a Chrome extension. This is what I came up with.

// ==UserScript==
// @name        Instapaper New Windows
// @namespace   http://mattberther.com
// @description Open Instapaper links in a new window
// @include     http://www.instapaper.com/*
// @require     http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js
// ==/UserScript==

(function() {
    function loadJQuery(callback) {
        var script = document.createElement("script");
        script.setAttribute("src", "http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js");
        script.addEventListener('load', function() {
                var script = document.createElement("script");
                script.textContent = "(" + callback.toString() + ")();";
                document.body.appendChild(script);
            }, false);
        document.body.appendChild(script);
    }

    function main() {
        $("a.tableViewCellTitleLink").attr('target', '_blank');
    }

    loadJQuery(main);
})();

Copy the code above and save it to a location on your computer; I called mine instapaper.js. The latest versions of Google Chrome no longer allow you to add extensions from a third party source (like your own computer) by simply clicking on the javascript file. To install the extension, open the extensions window in Chrome and then dragging the file you created onto the window.

Once the extension is activated, any links from your unread list in instapaper.com will open in a new window.

Validating HABTM relationships with Rails 3.x

2012-09-09T00:00:00Z

There comes a time as you build up a rails application that you end up using the has_and_belongs_to_many (HABTM) macro. This macro is an easy way to create a many-to-many relationship between two of your ActiveRecord models.

In some cases you may want to validate that association. However, the traditional methods for validating rails models do not work.

The unit tests below described how I wanted the relationship to function.

class ProjectTest < ActiveSupport::TestCase
  setup do
    @project = Project.new()
  end

  test "may have many developers" do
    4.times { @project.developers << FactoryGirl.create(:developer) }
    assert @project.save
  end

  test "must have at least one developer" do
    @project.save

    assert_equal 1, @project.errors.count
    assert_not_nil @project.errors[:developers]
  end
end

In my case, I was hoping to validate that each project had at least one developer associated to it. Initially, I coded my models to make the first test pass.

class Developer < ActiveRecord::Base
end

class Project < ActiveRecord::Base
  has_and_belongs_to_many :developers
end

To make the second test pass, I tried to implement a custom active record validator.

class Project < ActiveRecord::Base
  has_and_belongs_to_many :developers

  validate :minimum_number_of_developers

private
  def minimum_number_of_developers
    errors.add(:developers, "must have at least on developer") if developers.count < 1
  end
end

This, however, does NOT work with HABTM relationships. The way that these relationships work is that the associated property is not available until after the record is saved.

To get around this, we can validate as part of the after_save callback. Validating here and returning false from the callback will rollback the entire transaction.

class Project < ActiveRecord::Base
  has_and_belongs_to_many :developers

  after_save :validate_minimum_number_of_developers

private
  def validate_minimum_number_of_developers
    if developers.count < 1
      errors.add(:developers, "must have at least on developer")
      return false
    end
  end
end

The test passes with the code above.

Fixed Position Footers

2011-12-08T00:00:00Z

Posting mostly for my own reference…

One thing I find that I need to do a lot is position a footer bar across the bottom of the page. The most common way to do this is to set a fixed position on the element and anchor it to the bottom using this css:

 #footer {
      width: 100%;
      position: fixed;
      bottom: 0;
      height: 75px;
 }

Unfortunately, this doesnt work quite right in IE. When using this style definition in IE, the footer gets locked into a specific position in the viewport and when you resize from the corner anchor the footer does not move with the window.

The ~~proper~~ cross-browser way to declare a fixed position footer is to use negative margins, like this:

 #footer {
      width: 100%;
      position: fixed;
      top: 100%;
      margin-top: -75px;
      height: 75px;
 }

This appears to function properly in every browser I’ve looked at so far.

The Software Behind the Site

2011-12-06T00:00:00Z

Several times over the past months, I’ve received questions about the software and setup that I use to run the mattberther.com blog and related pages. Since the site recently underwent a dramatic change in tooling, I want to detail what I chose and why.

Before the change, the site was powered by wordpress. I had my own rackspace virtual server that I was using to host the apache and mysql server servers required by wordpress. Wordpress is not a bad piece of software, but what I realized is that it tries very hard to be all things to all people.

Enter the blog engine Im using now: toto

Toto’s philosophy is akin to mine: use the best tool for the job. Toto’s website states that “everything that can be done better with another tool should be”. To that end, toto doesn’t use complicated web frameworks. It doesn’t use a database. There’s no built-in commenting support. If you want comments, you’ll use disqus. It also relies on git for version control. When you combine toto with a Heroku account, you also use git to deploy the site. Its designed to be used with a proxy cache for high availability and fast response times.

Migrating the posts from the mysql database to the text files proved to be relatively straightforward, using a simple ruby script to iterate over the rows and format a text file that matched toto’s expectations. Importing the comments into the disqus platform was equally straightforward using their JSON API.

I much prefer the simplicity of this new setup. The entire blog engine weighs in at about 300 lines of code. If running a blog without putting your hands on the metal and being able to control every nuance of your blog platform appeals to you, then certainly take a look at the toto/heroku combination. If not, then I believe that wordpress is a fine solution.

"Gitting" TFS out of your way

2011-11-29T00:00:00Z

More and more, I have developed a passion for the Git source control system. I love how Git stays out of my way, until I need to use it. Git offers a very easy way to test things out, whilst utilizing the benefits of source control. In the traditional, connected model of source control, experimentation proves to be somewhat difficult because you don’t want to corrupt your main development line. Branching and merging with most other systems is a nightmare at best. With Git, branches are very cheap and merging is virtually painless.

I use Git for side projects as well as a few of the open source projects I am a part of. However, during the day, my organization uses Microsoft’s Team Foundation Server for source control. I find that in a connected source control model, I am constantly waiting on TFS to catch up. Either it’s out too lunch, or it needs to download and checkout a file before I can edit it. Experimentation is tough, for the reasons I mentioned earlier. Surely, there has to be something better, while still keeping TFS in the organization.

Enter Git-TFS… Git-tfs is a two-way bridge between Git and TFS created by Matt Burke. It allows me to clone a TFS repository and use Git for source control (without the hassles of being tied to a TFS server). This means: 1) no more waiting on TFS, 2) no more locked files, 3) no more requiring network connectivity to work on a project. At a time that I believe a development effort is ready to be committed to the main line, I use git-tfs to push my changes to the TFS server so that the rest of the team can get them.

Sound interesting? Here’s how to get started:

First step’s first. You need to have a Git installation on your machine. I use msysGit 1.7.6 preview 20110708, but this should work with newer versions as well. I just used the default installation options, specifically, I chose to use the Git Bash only when prompted about how to use Git from the command line. I chose this option because it did not otherwise modify my system.

Once you have a Git installation, you need to install the git-tfs plugin, which is available on github at https://github.com/spraints/git-tfs. Installing the plugin is pretty straightforward. You can choose to either download or build it yourself. I went with the download option and extracted the zip file to c:\git-tfs. Once the files are in place, git has to know how to find them. The easiest way to do this is to add c:\git-tfs to your path (Advanced System Settings | Environment Variables).

At this point, you are ready to clone a TFS repository using git-tfs. You can do this one of two ways:

git tfs clone http://tfs:8080/ $/TeamProject/folder

git tfs quick-clone http://tfs:8080/ $/TeamProject/folder

At this point, you are ready to open your project. If you’re using Visual Studio (which you likely are), you may still have the TFS source control bindings in place. There’s several ways to handle this. You could 1) disconnect your network cable while starting the project, 2) update all of the projects to remove the source control bindings, or 3) install GoOffline.

Option one is obviously less than desirable. Option two is reasonable, however, we are assuming that others on the team continue to use TFS and need to have the bindings in place. That leaves the GoOffline solution. GoOffline is a free Visual Studio add-in that adds a Go Offline button to the Source Control menu. When the solution is in offline mode, any file renames or moves happen without communicating with the TFS server. Perfect!

I prefer to work in feature branches (sometimes also referred to as topic branches). Again, branches are cheap in Git and this allows me the ability to experiment while enjoying the comforts of source control and not forcing experimental changes on my team. My typical workflow with git-tfs looks something like this:

# create and checkout a new branch for feature
git checkout -b feature_name

# write tests and code

# commit my changes with a meaningful commit message
git commit -am "meaningful commit message"

# repeat the code/commit process until the feature is complete

# when ready to commit the changes to the TFS repository, first sync any newer changes from the master

# switch to the master branch
git checkout master

# pull new changes from the TFS server
git tfs pull

# switch back to the feature branch
git checkout feature_name

# import the changes from the master branch onto our feature branch
git rebase master 

# last but not least, push to TFS
# optionally, add --build-default-comment which will create a default commit message
git tfs checkintool

The checkintool command brings up the TFS commit dialog which allows you to associate checkins with work items and modify the commit message. Importantly, the push to TFS is done as a single commit, so regardless of how many times you commit to your git repository, the change pushed to TFS reflects only the final result and not the journey it took to get there.

I have only been using this workflow for a little while now, but for now, I am very happy with it. Your mileage may vary. Please let me know in the comments if you’re using this and how it’s working for you.

Goals and Goal Setting

2011-11-27T00:00:00Z

Goals and Goal Setting:

In the book What They Don’t Teach you at Harvard Business School, Mark McCormak discusses the importance of setting goals. The students in the 1979 Harvard MBA program were surveyed and asked “Have you set clear, written goals for your future and made plans to accomplish them?” Only three percent of the graduates had written goals and plans; 13 percent had goals, but they were not in writing; and 84 percent had no specific goals at all.

Ten years later, the members of the class were surveyed again. The 13 percent of the class that had goals were earning an average of twice as much as the 84 percent who had no goals at all. However, the three percent that had clear, written goals were earning on average, ten times the other 97 percent combined.

For many of us, during this time of year, we begin to look at performance reviews for our team members. An important part of a good performance review are effective goals for the coming year. I am certainly not a management expert and most of the ideas that I will share with you today are not my own. However, I do see value in each of the ideas. For me, these ideas make the goal setting process easier. I certainly hope that they can do the same for you.

Top Tips:

Strong relationships with directs is paramount
- Many management leaders propose that having weekly one-on-ones is the single most important way to cultivate a positive relationship with your direct reports.
- 10/10/10 agenda
  - 10 minutes for the direct to discuss whatever they want
  - 10 minutes for you to discuss whatever you would like with the direct
  - 10 minutes to discuss business and progress toward goals
Create MT goals (measurable and timely) and use SMART to validate them
- Examples:
  - By April 30, 2011, consolidate the presentation models used by the company’s applications.
  - Integrate the new ERP system with the CRM system by October 31, 2010.
Create goals that intersect business and individual objectives
- Examples:
  - By April 30, 2011, combine multiple automated test frameworks into a single framework and increase coverage of automated tests by 20% using the consolidated framework.
  - Complete Microsoft developer certification 70-515 by April 30, 2011.
Emphasize cross-team collaboration
- Examples:
  - Participate in a mentorship program to grow understanding of our organization’s business by April 30, 2011.
  - Participate in a mentoring program to enhance leadership and communication skills by December 31, 2009.
Create personal goals
- Examples:
  - Run a 5K race in less than 32 minutes by April 30, 2010
  - Receive a belt promotion in my chosen martial art by December 31, 2011
Be flexible

“I’m much more likely to accomplish something if I write my goals down and share my goals with others. Recruiting the support of other people also helps keep me on track.”

“What was/is valuable to me is taking into account what I want to do when setting goals. That results in my engagement right from the start.”

“I think the addition of a personal goal is a great idea. It’s good for promoting personal improvement and makes the goal setting process less painful.”

I have used this model for a number of years. The quotes above are a few pieces of feedback that I received from my team regarding these ideas.

Cygwin - unable to remap to same address as parent

2011-11-22T00:00:00Z

I find the windows command prompt somewhat limiting and have never really been able to make the leap to Powershell. Personally, I like to use a Cygwin shell for command line work. I am comfortable in a unix shell with previous Linux and Mac experience. I’m not an expert by any means, but I can get by.

Recently, I installed the Cygwin shell on my 64bit Windows 7 system. After installing the developer tools and trying to compile ruby 1.9.2, I found that I had a tremendous amount of problems building. Id see numerous errors that stated:

unable to remap file to same address as parent
ruby ### fork: child XXX - died waiting for dll

The most commonly referenced solution was to rebase by doing the following:

Start –> Run
cmd.exe
cd c:\path\to\cygwin\bin\
ash.exe
./bin/rebaseall
Reboot

However, this did not work for me. I found a message on the cygwin mailing list stating that the gems may not add information to the proper paths and that we must create the list manually.

To do this, first from your cygwin shell do:

find /lib/ruby/gems -name '*.so' > /tmp/ruby.gems.local.so.lst

Then, follow steps the steps above, replacing step five with:

./bin/rebaseall -T /tmp/ruby.gems.local.so.lst

Since performing this, I have not seen any more of the remap file errors. Your mileage may vary though.

Hidden Gems in Code

2011-01-26T00:00:00Z

This gem was spotted in some code that my team was working on today. It made us chuckle anyway. :)

function GetAgeOptions(localization) {
  var ageArray = new Array();
  ageArray[0] = "Nick Rocks";
  ageArray[1] = "Ben Rocks too";

  return ageArray;
}

ListExtensions

2010-01-20T00:00:00Z

I prefer to use the abstract IList<T> interface in my public APIs, rather than passing around concrete List<T> implementations. However, the one thing I’ve always missed was the awesome .ForEach syntax which was available on the List<T> implementation.

Today, I realized that I can use extension methods to add methods to interfaces.

public static class ListExtensions
{
    public static void ForEach<T>(this IList<T> source, Action<T> action)
    {
        foreach (T item in source)
            action(item);
    }
}

This now allows me to do things like:

public static void Main(string[] args)
{
    IList<string> items = GetItems();
    items.ForEach(Console.WriteLine);
}

private static IList<string> GetItems()
{
    return new List<string> { "abc", "123" };
}

I much prefer this syntax to the foreach construct.

The SwappableFile

2010-01-04T00:00:00Z

I was doing some testing on a javascript abstraction that we created earlier. In order to test that the validation was being properly called, I wanted to drop in a new target javascript and then use WatiN to execute and validate the various calls to that javascript.

I came up with (what I think is) a neat semantic for doing this. The SwappedFile implements the IDisposable interface, so you can wrap it up nicely in a using statement and make sure that the original file is reverted when completed.

class SwappedFile : IDisposable
{
    private readonly string originalFile;

    public SwappedFile(string originalFile, string newFile)
    {
        if (String.IsNullOrEmpty(originalFile)) throw new ArgumentNullException("originalFile");
        if (String.IsNullOrEmpty(newFile)) throw new ArgumentNullException("newFile");

        this.originalFile = originalFile;

        if (!File.Exists(String.Format("{0}-original", originalFile)))
        {
            File.Move(originalFile, String.Format("{0}-original", originalFile));
        }

        File.Copy(newFile, originalFile, true);
    }

    public void Dispose()
    {
        File.Delete(originalFile);
        File.Move(String.Format("{0}-original", originalFile), originalFile);
    }
}

Usage is simple:

using (new SwappedFile("original-file.txt", "new-file.txt"))
{
    // some operations using the new file
}

I kinda like this, plus it beats having all of my code cluttered up with file manipulation statements.