Communicate with certainty
and your Voice will be heard.

Category Archives: web development


Search is the compass of the internet. It guides us to the content that we are really looking for and helps avoid the stuff we don’t really care about. Or at least that’s how it is supposed to work. It turns out that beyond just the complexity of installing and configuring a search server, it can also be difficult to account for the various use cases of your search tool. Lets take a quick look at how The Mechanism engineers were able to tackle this challenge when building a restaurant search application for SafeFARE.

The good folks at foodallergy.org enlisted our services to build a restaurant search application that will allow users to find allergy-aware restaurants based on any combination of 9 criteria. Using the Ruby on Rails framework and Sunspot Solr (a Ruby DSL for the Lucene Apache Solr search server) we built this search app, and learned a few things on the way.

If a user searches for restaurants in a ZIP code should we only return restaurants within that ZIP code, or should we include restaurants from other nearby ZIP codes in our search results? And if we include other ZIP codes, how many other ZIP codes? How should we order the results? These and other similar questions helped up to come up with the structure of our search controller.
Figure 1.1

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
if params[:search].present?
@search = Restaurant.solr_search do
fulltext params[:restaurant_name] # runs a full text search of
with(:approved, :true) #facets approved restaurants
if params[:cuisine_search].present? #user also entered cuisine preference
any_of do
params[:cuisine_search].each do |tag|
with(:cuisines_name, tag) # facet by matching cuisines
end
end
end
if params[:address].present? || params[:city_search].present? || params[:state_search].present? || params[:zip_search].present?
#if any location fields are present, geocode that location
with(:location).in_radius(*Geocoder.coordinates(whereat), howfar)
#facet based on user given location,
end
order_by_geodist(:location,request.location.latitude,request.location.longitude)
@restaurants = @search.results
end

It took us about a week but we were finally able to come up with enough if statements to cover every one of the 362,880 possible combinations of search queries. Figure 1.1 is a small sampling of how we implement search when a user types in a restaurant name, cuisine preference, and restaurant location. First we search the solr index for whatever the user enters in the restaurant_name field, then cut that list down to only the approved restaurants, then we check to see if the user also entered a cuisine preference, if so we facet our list down to restaurants that match that cuisine, if the user did not enter a cuisine, we skip that step, then we check if the user entered a location that they would like to search like a city, or state, and we facet our list down to only restaurant’s in that area. Using this strategy we can create sort of a Venn diagram that allows us to drill down only to the information that we want, and point that result to the restaurant variable. To increase the functionality of the site, The Mechanism engineers implemented an IP lookup to automatically detect the IP address and location of the user, and order search results by how close the restaurant is to the user.

A second major challenge that many developers face when using a search server is deployment. In order to use solr in a production environment, you will need a Java app servlet like Tomcat or Jetty, and you will need an instance of Apache Solr. Developers may consider installing standalone versions of Tomcat and Solr Sunspot depending on their hardware capabilities, but sunspot comes bundled with a Jetty server which can be used in production by running the command

1
RAILS_ENV=production rake sunspot:solr:start

And voila! we have implemented an advanced search tool that will help users find allergy-aware restaurants all across the nation and may even save somebody’s life one day.

Until recently I considered writing tests for my applications much like reading Dickens in high school: boring, repetitive, hard to understand, and yet for some reason a total necessity.  What’s more, I wrote tests about as frequently as I read Dickens – and I’ve never read any Dickens.  My meandering point here is that Test Driven Development (TDD) seems to be the standard of the rails community, yet I don’t know a single person who actually does it.  With that it mind, I decided to develop our last client’s application with tests written to the best of my ability.  At first I thought I was burying my productivity in the minutia of each test, but I ended up learning quite a few things, including the value of testing.

Tools

I guess the first thing to do here is to describe my tools.  To test with Rails I used RSPEC, Factory Girl, and Capybara.

RSpec is a testing tool for Ruby.  Baked into the gem is a rich command line program with detailed error reporting. The beauty of Ruby, and RSpec, is that it enables you to write human readable tests that tell a story. For instance:

1
2
3
4
5
it 'has a list of employees' do
employee = Employee.new('John', 'Smith')
company = Company.new([student])
expect(company.employees).to include(student)
end

Though testing with RSPEC ensures that our models behave the way we expect, there is a serious problem with this approach: It takes forever. Testing any sort of interesting behavior not only involves extensive amounts of setup to your environment, but a tremendous amount of code to create various instances of your model. This is where Factory Girl comes in.  Factory Girl allows a tester to create factories that create multiple records for a model with some generic attributes that are able to be overridden as needed.  This means that creating a unique record is as simple as:

1
FactoryGirl.create(:employee, name: "Jon Snow").

My last tool in my little testing toolkit is Capybara. Where RSpec is a way to test your models, Capybara allows your application to test external behavior. In other words Capybara provides a simple way to test user stories and general behavior. Heres an example:

1
2
3
4
5
6
'When I sign in' do
visit user_sign_up_path
fill_in 'Login', :with => 'DavidBowie@spidersfrommars.net'
fill_in 'Password', :with => 'whereWereTheSpiders?'
click_link 'Sign in'
end

 

So What’s The Point

When I started, I assumed that testing would help to prevent bugs. While this is likely the case, I found that using TDD did something far more valuable. By letting tests to drive the code, developers are forced to conceptualize their application before a single semi-colon is ever written. This forces developers to step through their own logic, which in my case, is full of inconsistencies and little things that I had not thought through fully…likely because I have never read Dickens.

The Thinking Mechanism is a series of weekly posts, published on Fridays, covering the ideas The Mechanism is thinking and talking about with our peers and clients.

This week members of our London team have been working out of the New York office leading to many interesting conversations about technology, development and the future of creative digital work. Mostly because during this week, and next, a lot of industry changing announcements are taking place. This week the D9 Conference is happening and next week is Apple’s World Wide Developers Conference.

Here are some of the things we’ve learned so far:

• In a rare and unexpected break from their absolute secrecy Apple pre-announced the content of their WWDC keynote. We now know Steve Jobs will be back for the presentation. There will be no new hardware, certainly no new iPhone announcement. The keynote will focus on software, specifically Mac OS X LioniOS 5, and will introduce iCloud, a potential replacement for MobileMe and music locker. They also released iPhone versions of the iWork suite of apps ahead of the conference. They have certainly gone out of their way to manage expectations. If all the rumors floating about are to be believed Apple is up to something big.

• Microsoft demoed Windows 8 (video) which introduces a new tile-based interface based on the Windows Phone.

Twitter introduced the Follow button, which allows one click follows without having to leave the page you are in. It also introduced photo and video sharing within Twitter (video,) a full of potential extension to the service that transforms hashtags beyond keywords and trends into galleries. Twitter may even be baked into iOS 5.

• Google releases the +1 button to the web. One more button to add to pages, to blogs, to the online ecosystem. Although it comes late to the social media party the +1 button has the advantage of actually influencing the Google search algorithm, possibly leading to improved SEO. For that reason alone it may be embraced.

• Former Google CEO and current Board member Eric Schmidt introduced the phrase “The Gang of Four,” or how he refers to Google, Apple, Amazon, and Facebook. This cabal of frenemies has highly successful partnerships in some areas and aggressively competitive pursuits in others. And together they are inventing the future. Notably absent from Schmidt’s “Gang” – is Microsoft…

• If you were in Jonathan Kaplan’s shoes and experienced Cisco buying your Flip line of consumer cameras only to abruptly discontinue them, what would you do? He is going into high-tech grilled cheeses.

• And speaking of dairy products, is White Power Milk a joke, performance art, political satire, a student project, a viral for Yakult? Only Nate Hill knows for sure.

The Thinking Mechanism is a series of weekly posts, published on Fridays, covering the ideas The Mechanism is thinking and talking about with our peers and clients.

In the U.S. this weekend is Memorial Day weekend, the official start of summer. Many people travel to visit family and friends or just to take their first trip to the beach. In the spirit of the holiday weekend we are going to do something different. Instead of sharing the items we’ve been talking about we are going to introduce you to two services we love and share items that you could enjoy while commuting to your destination or while taking some deserved time off.

Once you discover Instapaper you wonder how did you manage without it. Created by Marco Arment, co-founder of Tumblr and coffee aficionado, Instapaper is a simple tool to save web pages to read later. The text is stripped out of any web page and becomes available, via apps on most mobile devices, for you to read when you have the time. It is also available online through the Instapaper website. Instapaper is time-shifting for text, TiVo for words.

Longreads is the perfect compliment to Instapaper. Founded by Mark Armstrong, Longreads posts links to new stories every day — they include long-form journalism, magazine stories from your favorite publications (The New Yorker, Esquire, The Atlantic), short stories, interview transcripts, and even historical documents. The site has a brilliant search feature that allows you to filter articles based on length, so you can find the perfect article to read in the amount of time you have available.

In the age of Twitter and Facebook status updates, these two services encourage long-form reading. Here are some of the articles we’ll be reading this weekend, as discovered through Longreads:

Error Message: Google Research Director Peter Norvig on Being Wrong
(Kathryn Schulz, Slate, Aug. 3, 2010)
Time to read: 16 minutes (4,050 words)
Norvig explains what happens when a company (in this case Google) takes an engineering-centric approach to its products and business. First, it means that errors are actually a good thing.

Apple & Design: The Man Who Makes Your iPhone
(Frederik Balfour and Tim Culpan, Businessweek, Sept. 9, 2010)
Time to read: 21 minutes (5,204 words)
Foxconn founder Terry Gou might be regarded as Henry Ford reincarnated if only a dozen of his workers hadn’t killed themselves. An exclusive look inside a postmodern industrial empire.

Later: What Does Procrastination Tell Us About Ourselves?
(James Surowiecki, The New Yorker, Oct. 11, 2010)
Time to read: 14 minutes (3,574 words)
Take comfort in this exploration of the “basic human impulse” of putting work off.

Master of Play: Shigeru Miyamoto, Nintendo’s Man Behind Mario
(Nick Paumgarten, The New Yorker, Dec. 13, 2010)
Time to read: 37 minutes (9366 words)
Jamin Brophy-Warren, who publishes a video-game arts and culture magazine called Kill Screen, told me that there is something in the amplitude and dynamic of Mario’s jumps—just enough supernatural lift yet also just enough gravitational resistance—that makes the act of performing that jump, over and over, deeply satisfying. He also cited the archetypal quality of Mario’s task, that vague feeling of longing and disappointment which undergirds his desperate and recurring quest for the girl. “It’s a story of desire,” Brophy-Warren said.

Twitter Was Act One
(David Kirkpatrick, Vanity Fair, March 3, 2011)
Time to read: 18 minutes (4,543 words)
“The Facebook Effect” author David Kirkpatrick on another Silicon Valley superstar—Twitter and Square founder Jack Dorsey. In submitting to his first in-depth profile, we learn about the events the led to him stepping down as CEO [since then he has returned to Twitter as CEO], his long-term goal (to become mayor of New York City), and his earliest career experiences.

Cranking
(Merlin Mann, 43 Folders, April 22, 2011)
Time to read: 12 minutes (3,068 words)
A disappearing dad with a looming book deadline examines his priorities, and promises changes.

Håkon Wium Lie

Photos and discussion of CERN and Tim Berners-Lee. How the web was actually developed by Tim at CERN. Pretty hilarious.

CSS a response to potential destruction of HTML's markup language with presentation elements. Currently CSS still relies too heavily on images and the occasional table hack.

HTML5 + CSS3 = WEB8. CSS3 is not a single specification, a series of specifications.

CSS1 could style text in an element. Some borders. A revolution when it came out.

For this presentation, Håkon is using a special build of Opera that isn't publicly available.

1
text-shadow

. Web font: Safari, Firefox and Opera.

Håkon believe web fonts is going to be the big new thing. Will create a pre- and post-font Web.

1
@import url(path/to/font);

1
font-family:Cardiff;

IE is the only one that does not not support this. Argument now breaks out between Håkon and Pete LePage

IE has supported a broad range of fonts since IE 4, but .EOT. However, the above @import is all TTF. IE uses .EOT because font foundries want support for DRM. Check out Fontdeck and Typekit.

At Standards.Next, IE pledged support to add support to TTF along with EOT.

rgba()

1
background: rgba(0, 0, 255, 0.4);

See Molly's presentation.

A review of Opera's support for the CSS 3 features discussed by Andy Budd.

Opera Unite

Turns browser into a server. Allows people to access your machine via

1
http://

. Can be quite useful.

It's not the cloud. This service is close to machine and useful for local collaboration. Millions of phones use Opera. Peer-to-peer model. Can take picture and easily

Question/Answer

Q: When will the geolocation build be integrated with the shipping version?

A: Not sure; no date yet. Will probably not occur until the standard is more fixed.

Pete LePage
Senior Product Manager, Developers & Security for IE

IE 8

IE 8 is a hybrid, has two engines; IE 9 will have three engines: IE 7, IE 8 and IE 9. This way can ensure site works in future versions of the browser.

IE 8 supports session and local storage (see presentation by Marcus Lofthouse to the New York Web Standards Meetup). Added addition to spec: a way to remove all.

Mutable DOM prototypes: take a DOM element and add properties/methods to it. Add method to

1
img

. There are a number of things not implemented in IE 8. Can write a chunk of JavaScript that adds functionality to browser. This will be used by people writing the JavaScript frameworks (jQuery, Dojo).

Native JSON support.

Network events

CrossDocumentMessaging and CrossDomainRequest

Selectors API

Fixed getElementById.

Demos at ie8demos.com (browser-sniffs and only wants to work in IE 8).

IE 9

  • Faster; speed is important. Finally comparable to FF.
  • A lot of work on standards-support. Will implement border-radius. Acid3 score went from 20 to 32. IE 8 Current CSS3 support is vertical-text. Hope to more than double it in IE 9. ;)

Questions/Answers

Q: Why always trying to play catch up? Why not just adopt Webkit?

A: A lot of things that make adopting Webkit or Gecko more difficult than it seems.

Q: Any release date for IE 9?

A: No dates as of now

Q: Still trying to get Canvas out of the spec for HTML5?

A: Not sure; might be a miscommunication

Q: Any major updates planned for IE 8?

A: Probably not; next major version will be 9. Security updates will be released for 8, but the rendering engine will not change.

Molly Holzschlag

RGB: Additive synthesis

Red, Green, Blue: Add colors to get result color. In print world, subtractive. Remove color to get result. R + G + B = white

RGB color notation

RGB in percentage values or RGB in integer values. Not equivalent; so don't use both (mix in a single rule)!

NOT

1
rgb (60, 100, 50%);

Firefox and Opera will ignore the incorrect rule. Safari and IE (5–now) will attempt to correct and end up with a severely washed-out result.

HSL color

Hue, saturation and lightness. Easier and more intuitive way to get to colors we want as designers. Value from 1–360, then adjust saturation and lightness.

  • Hue value comes first, using the hue's angle on the color wheel as represented by an integer.
  • Saturation comes next, using the percentage of saturation you'd like
  • Lightness is next, using the percentage of lightness you' like

HSL advantages

  • Not CRT specific
  • Easier, more intuitive to work with
  • Working with a set of colors allows for easy palette creation
  • Reduces errors
  • Reduces dependency on charts and other value lookup tools

As adjust saturation, lightness, base-hue remains the same.

RGBA and HSLA

Alpha transparency for both RGB and HSLA.

Notation is the same in both. a value is alpha transparency from 0–1.

Allows for fully opaque to fully transparent.

1
hsla(0, 100%, 50%, .25)

Supported by Firefox, Webkit and Opera.

Questions/Answers

A: Best article out there on HSLA is Wikipedia. Useful to have the color wheel.

Q: Why is a value decimal instead of percentage?

A: Not sure. No one remembers a discussion about it.

Q: Is this part of the spec?

A: Yes. It is all part of the CSS3 color module.