gabriel mcgovern


Archive for the ‘work’ Category

The joy of compression

Tuesday, August 5th, 2008

I gave a little talk at our monthly “All Developers Meeting” today. The topic was web optimization. Recently we setup http compression on our servers and I decided to take a look at how much bandwidth we save via all of our optimization/compression techniques. Using our academic programs page as a sample, I ran the numbers and found that we are saving from 1-6.5 terabytes annually. Here are my notes for the presentation:

No Optimization - Huge/bad/gross

No css or styles inline. Tables for layout. Scripts directly on page. Won’t even  bother showing an example.

Step 1. Separate code - 283 KB

Style is separated from content (separate .css). Behavior is separated from content (separate .js). Images are used wisely ( for instance 1px background image that repeats horizontally)

Step 2. Compress images, then compress again - 239 KB [44K less]

Choose the correct file type for each image (JPG for photos, PNG > GIF). Compress each image (using photoshop). PNG images can be compressed again (pngslim removes meta and shuffles bits - like defrag)

Step 3. Minify styles and scripts - 158 KB [81K less]

View compressed script library, or layout css. For Javascript we use JSMin (removes comments white space. Need well-formed code.) There are other algorithms like packer, but they requires non-trivial client-side processing time to uncompress the code. For CSS we use CSStidy ( removes white space., compresses colors, combines styles)

Step 4. Enable HTTP compression - 70 KB [88K less]

Note that this is our live version! Text is compressed by server and uncompressed by browser. Unsupported browsers will receive uncompressed version (wiki). Browser support: Internet Explorer (>= 4.0) FireFox (all) Netscape (>= 4.06) Opera (>=5.12) Images are not compressed further (doesn’t result in smaller size).
Minified files compress better then original.

Total Optimization

283 KB original
70 KB compressed
—————————
213 KB saved [That over a 2/3 savings]

And it adds up quickly!

Over the past year (Aug 4, 2007 - Aug 4. 2008) the site received:

  • 32,550,230 Pageviews from
  • 2,213,198 Absolute Unique Visitors

If 13KB was saved per pageview, then:

213 kilobytes * 32,550,230 pageviews = 6.5 terabytes of bandwidth saved

Or, with best-possible caching:

First page: 213 kilobytes * 2,213,198pages = 449.6 gigabytes
Subsequent pages: (32,550,230-2,213,198) pages * 20KB (html+images) = 578.6 gigabytes

So…

Between 1 and 6.5TB saved annually (not to mention the amount of time saved for our end-users)

A List Apart - the survey

Tuesday, July 29th, 2008

A List Apart is currently conducting their yearly survey:

The Survey, 2008

Calling all designers, developers, information architects, project managers, writers, editors, marketers, and everyone else who makes websites. It is time once again to pool our information so as to begin sketching a true picture of the way our profession is practiced worldwide.

All web professionals should take a few minutes to help them get a bigger dataset.

RSS autodiscovery on every page (of a static site)

Thursday, July 10th, 2008

So you have some feeds and want browsers to be able to find them using autodiscovery.

The problem

Unfortunatly, you have static pages and would have to update each one to add the <link> tag. At PCC, we have a site composed of tens-of-thousands of pages. Some are static, some are dynamic applications we can alter, some are closed vendor products.

The solution

Like I said - we have many different types of pages. However, they all use the same JavaScript library!

JS is really good at inserting content into the page. We use the jQuery library, so adding a feed is as easy as:


/* add feed autofind! */
$("head").append('<link rel="alternate" type="application/rss+xml" title="News Releases | Portland Community College" href="http://www.pcc.edu/about/feeds/news/" />');

We can now add/remove feeds at a later date without opening every page on the site. If desired, the script could check the URL and only display the feed on appropriate pages, but for now lets put them everywhere.

Final Thoughts

I looked around the web, but didn’t see much discussion on this meathod. It seems to work in all modern browsers, but let me know if you find any problems.

Safely using FeedBurner

Wednesday, July 9th, 2008

So you have some feeds and you want to track subscriptions using FeedBurner.

The problem?

FeedBurner is a great service and probably will stay that way since it was recently acuired by google. However, it is still a 3rd-party service and that’s somthing you should always be worried about.

You see, when someone subscribes to a burned feed, they will be pointing the agregator to: <feeds.feedburner.com/your-feed>. This means that you will loose all of your subscribers if any of the following happens:

  • FeedBurner goes down, closes shop, or starts charging an outrageous fee. (unlikely)
  • The person who set up the account forgets the password, or becomes a disgruntled employee. (possible)
  • A better service comes along and you decide to switch. (very likley)

The solution

The simple solution is to have users subscribe to a feed on your domain.

  1. Create a folder structure for the feeds - like: /feeds/news/ and /feeds/videos.
  2. In each folder create an index page using you favorite server-side language - index.php, index.cfm, etc. This gives us added flexibility. We can even change the language at a later date!
  3. Use this page to mirror the FeedBurner feed. below is an example in ColdFusion. In PHP you will want to look into cURL:

<!----------------------------------------------------------------------------------------------------
-  This is a placeholder for a feed.
-  Make sure to link to the folder and not this .cfm page for even more flexibility.
-  Created:   2008-04-09 Gabriel McGovern
-  Modified:
----------------------------------------------------------------------------------------------------->

<cfhttp url="http://feeds.feedburner.com/pcc-videos/">
<cfoutput>#cfhttp.filecontent#</cfoutput>

Examples

Check out the feeds we offer for Portland Community College. Each is being run through FeedBurner, but you subscribe to a URL in our domain. If we decide to change services - all we have to do is update the index page. You as a subscriber would never even have to know.

Extracting DVD Subtitles

Wednesday, July 2nd, 2008

Problem

  • Our foundation has a great little video produced by an outside complany and they want us to put it online, but…
  • We require that all online video includes captioning for accessibility (dfxp format) and…
  • They only have a DVD of the video with captions (not original text).

Solution

Well we could re-transcribe the video, but that seems like a lot of redundant work since someone already took the time to make the captions for the DVD. The only problem is that DVD captions are stored as bitmap images rather then acctual timed- text.

Step 1: OCR the DVD subtitles

OCR stands for “Optical Character Recognition”. Your scanner may have this for decoding paper documents. Because the subs are actually images - OCR is needed here too. There is a cool little application called SubRip that does exactly what we need.

You point to the correct DVD file and it will attempt to transform the subtitle images into text. Since subtitles can come in all different fonts, languages and colors - the process is not completly automatic. From time to time it will ask you to identify certian charater(s). Think L vs. I and m vs. rn.

Step 2: Convert the subs to dfxp

SubRip does a pretty good job, but you will still need to clean up the output and transform it into the correct format.

SubRip outputs an .srt file where the captions look like:
1
00:00:06,373 --> 00:00:08,933
Whether we realize it or not, we need
each other.

We need them to be in dfxp format like:
<p begin="00:00:06.37" end="00:00:08.93">Whether we realize it or not, we need each other.</p>

This is a job for regular expressions!

  • Replace the entry number (1,2,3) with a marker: /n/d+/n becomes ~
  • Then grab the srt timecode and text info:
    ~(\d\d:\d\d:\d\d),(\d\d)\d --> (\d\d:\d\d:\d\d),(\d\d)\d
    ([^~]+)
  • This becomes:
    <p begin="$1.$2" end="$3.$4"><span tts:fontStyle="italic" >Speaker</span><br/>$5</p>

Step 3: Check your work

Now, you just need to make sure that it all worked as expected.

For this example, I found that the captions were actually off about 4 seconds. This could have been due to editing differences between this DVD and my final FLV.

I also found some minor mistakes that the OCR had created.

Luckily, both of these wer easy to fix manually.

Firefox 3 is available…

Tuesday, June 17th, 2008

Or, it would be - if 2 million people weren’t trying to download it all at once.

For weeks the good people at mozilla have been pushing today as “World Download Day”. They even had a site where they asked people to pledge to download, all in the hopes of setting a world record. Early today, they announced that the official starting time would be 10am (PST).

Unfortunatly

It looks like everyone wants to be first. This has resulted in a self inflicted DOS attack. All their relivant sites ark borked including: www.firefox.com, www.spreadfirefox.com and www.mozilla.com/firefox.

You think with all the planning, that they might have expected this.

WebVisions: Overcoming Chaos - Designing the Future Web

Friday, May 23rd, 2008

Jeffery Veen

Second (and final keynote)

…but first some props to the board

The Past

First spoke at WebVisions 5 years ago in 2003.

Worked at HotWIre 16 years ago. Then onto adaptive path for measure map. Bought by google where he worked on the great analytics.

3 weeks ago he split from Google….

Now back to 1974

(before I was born)

  • OPAC crisis where gas spiked to 81cents
  • Nixon left office
  • You could see Elvis OR kiss
  • Ma bell split (almost all back together)
  • First published reference to Internet (packet switch)

He was first introduced to Pong. Had the revelation that you could interact with the TV instead of just consume. Never looked back.

Historical background of data visualization

From 1851 cholera outbreak, to Neapolitans march, to challenger shuttle data.

Need to make stories as clear as possible.

Data

  • Simplicity of data on google analytics - from that scene in Indiana Jones where the plane flies over the ocean, leaving a single red line.
  • Statistics can feel abstract - chrisjordan.com

Interaction

  • Apply tools so that users can draw their own conclusions.
  • Stamen Design is a good example of interactive data visualization.
  • Allow people to follow their own trails.

Participants

  • Need to know your audience. Can be scary.
  • Variety of needs,
  • Of scale,
  • Of locations.
  • Identify needs - then design a product to solve problem.
  • A lot of time talking to a lot of people. (then pull out themes)
  • Stack themes on features. Are the needs met? Gaps? Too much?

Nice Typography:

Gotham

WebVisions: Convergence 2.0

Friday, May 23rd, 2008

Tjeed Hoek

I actually ended up in this session by chance. My original choose “Designing Social Media Interfaces” was scheduled for this room, but was canceled.

But, no regrets. Tjeed seems like an interesting speaker (just look at that dreamy photo :)

FrogDesign

Tjeed works at frog design. Started as a typical physical design company - now doing more and more digital interfaces.

Finds design can make products simpiler AND more complicated.

Convergence

Uses iPhone as a prime example. Not only is the phone more then just a phone - the coolness relies on an entire ecosystem of other things (iTunes, network, ipod, google map). And the design of each of the other items in this ecosystem is essential to the success of the core product.

Also notes the convergence of real-world and digital products.

Research-driven design process

Who are we designing for? What are trends in the world? What do users do before and after visiting your site?

  • Foundational - Ethnographic.
  • Generative - Card sort and concept structure.
  • Evaluative - Concept testing and user testing.

Still need to try your designs out with real people. Often overlooked step. And this does not just mean testing with your coworkers and mom.

  • Do lots and lots of iterations
  • …but dont loose site of main features and values.
  • Feature creep is easy with these converging products.
  • Just adding features is not enough - each feature need to to work well.
  • Being consistency is not enough -need unique, right experience.
  • Attention to detail on usage flow is critical. look at all steps, nit just beginning and end.
  • Beauty in initial appearance remains important.
  • But, beauty and elegance in motion, in the dynamics of using, is becoming more important. (don’t go overboard)
  • Need both nerds and rock stars. Need to bridge the gaps between (shows cool chart)
  • Concurrent process with synchronized workflow is optimal (another cool chart)

WebVisions: Starwars kid is your new bicycle

Friday, May 23rd, 2008

Andy Baio

Trip back to 1996 (Mr T ate my balls website). Creator just thought it was funny at the time. Linked to yahoo, that at the time had a list of 300 “ate my balls” pages”

New kind of ingenious stupidity – welcome to the meme.

Not a huge leap to this!

18M Americans have now been RickRolled.

Other Mentionables

  • Tourist guy
  • Hamster dance
  • Badger badger
  • Lolcats
  • Peanut butter jelly time
  • All your base

Memes

Out of contect and by themselfs they make no sence.

Together they tell alot about community.

Meme - Dawkins (the selfish gene)

Predate the intenet (andre has a posse)

3 characteristics

  • fadelity (must be reporducable)
  • Fecundity (esialy creatible)
  • Longevity ( able to stay around)

Folklore

Remixable - meme is the new folklore.

Reasons

Why to people creat, propigate and remix?

We may have no choice - very powerful or gets stuck in our head like an earwig.

or…

  • Commercial (viral ads)
  • Activism (obama girl, peta, jibjab)
  • Self expression! (respect, freshness fetish)

StarWars Kid

led him to post star wars kid on his site (waxy.org) in 2003. Coined term “starwarskid” (file name). At that time it had only been posted twice before.

Has now been viewed 900m - not all on his site of course. But his hits spiked. Over a million hits in one day. 500,000 downloads in a day. Interesting to see that most traffic did not have a referrer (people passing link from email). Took weeks for the media to pick it up.

Identified kid and - did interview. He didn’t seem to mind much.

Then people started getting vindictive…

they decided to have a fund raiser and raised over $40,000 $4,300 for him to buy an iPod.

Barack Obama is your new bicycle

Simple but interesting site. Posted it on twitter. Jumped to 500,000 page views the next day. Spread instantly.

Most traffic from social networks.

Remixes began immediately.

Media picked it up 2 days latter.

What Changed?

  • New tech (no need to post and host yourself).
  • Media is now watching.
  • Lots of memes. Twice the people on the internet.
  • Lots of mixing. With new tech remixing is easy. (makes contribution easy)
  • Even the silliest meme become part of the shared experience.

Just this morning:

Wezer’s new single Pork and beans.

.

.

.

a great ending for a great presentation!

WebVisions: CSS Trasformations

Friday, May 23rd, 2008

SchmittChristopher Schmitt

Wait I have his book! CSS cookbook

  1. Expectations limit design
  2. Browsers limit design (each has its own internal stylesheet)

Then a great story about an interview with Apple and a “hard” question. It all boils down to designing the American flag with CSS (which was something he had, by chance, done previously).

Semantic markup

Using appropriate html for appropriate element!

Only 91 elements! (but most people can only remember 40-50 of hand). Some people in the room didn’t even do that well.

Then goes on the show code he used to create the flag - lists all 50 states in ordered list.

Bad (not to do

  • <p>&nsbp;</p>
  • <p> … <br><br> … <br><br> … <br><br> </p>
  • and so on…

Back to flag

Setting up easel

Recommends using css reset.

For Flag he only resents elements that he needs to style. Uses minimal divs.

Clearing out the Content

display:none;

Stripping

  • Absolute positioning - allows for position that is ignored by other elements. However, screws up when fonts or other elements are resized.
  • Relative positioning - starts at normal position then moved.

He positions 13 colonies (stripes) absolutely inside realativly positioned easel.

Uses attribute selectors to pinpoint each colony, color and position. IE 5and 6 don’t support - will need javascript to mimic (Jquery is recommended).

Pooling the Stars

z-index is like layersin photoshop. Uses z-index to position blue union area above stripes and -9999 hack to move text off screen.

Stars

Block vs inline.

Uses extra <i></i> (bad form, he admits) to work as element to position star image. Should have added extra info such as year <i>1800</i>.

Texture

Alpha transparency PNGs. Hack for older versions of IE.

Overlays each area with fixed alpha overlay. This is rather then overlay entire flag with z-index image. That would block all links. As it stands all stripes and stars are clickable!

Streached

Reason for pixels vs em? easier… but em based layout (divs not just text) would allow it to be stretched. Talks about setting text to 10px for easy em use.

EM is based in typography. Hard to match up for divs. And what about the images used for stars.

Switches stars to text. Finds star in ZapDingbats. Also, uses CSS Generated content. So this is limited use - wont currently work in IE or without font.

PS

He didn’t Take the apple job - moved to be with his (now ex) girlfriend.