This presentation was part of John Caserta's Fusion Arts Workshop at the Rhode Island School of Design on 7/31/2008.

The Web

The web is like a giant collection of unorganized books that just appeared and continued to grow exponentially. The books (read web pages) in this collection make reference to other books in the collection, but that's about it. In a library, some degree of spatial relevance is maintained across the collection. On the web, however, no spatial relevance is maintained between pages.

In the 1990's, existing software companies and startups saw the need and opportunity to index this growing collection. Companies like Yahoo and Google, for example, wrote software to index them, and made their results available to the public through a web interface. These services direct users to the actual documents thought to be of interest or relevance to the user.

The pages that web search engines direct users to are again interfaces to large, often rapidly growing repositories of text and media. Sites like CNN and the New York Times site present the end user with a single 2D web page that they must use to access the content they contain. The amount of information represented on these index pages is limited by both the network bandwidth and the capabilities of the web browser.

Interacting With Digital Archives

Experiment: Crawling CNN

In 2007, curious about the relationship between the content shown on sites like CNN and the actual content they contain, I wrote a program to crawl a few web sites, including Starting with the main CNN web site, my program saved an image of the page, and followed each link. Images of these pages were again saved, and the process continued until every accessible link on was visited and saved as an image.

After 36 hours and tens of thousands of images later, I had the entirety of on my hard disk, in image form. Visually scanning through this collection (now no longer limited by bandwidth or rendering time required by the web browser), I stumbled across a number of pages that interested me, pages that I wouldn't have otherwise found when visiting the site directly using the web browser.

Diversion: The Web Browser

What if you wanted a different way to view, search, and ultimately experience web content? From an engineering perspective, we need to look to the browser itself. What is the browser? What exactly is it doing for us?

When we visit a web page, our computer is requesting data from another computer that can be anywhere in the world. That computer gives (or tells us where to find) the following types of data that are used by the web browser to compose the final web site:

  • html: Represents the basic structure of the data we will see. Contains links to media (images, videos, sound). Sometimes contains formatting.
  • css: Optional formatting information for the page.
  • javascript: Enables application-like interactivity on pages after they have been downloaded.
  • multimedia: (images, video, sound, etc.)

The role of the web browser is to collect all of these pieces (including downloading images, video, and other media file types such as Flash), and present us with the final result, properly formatted and structured. This final representation is extremely difficult to obtain without a web browser.

As a result, the browser knows important things about the relationship between the downloaded data (html, css, etc.) and the final visual product. E15 uses an embedded web browser in order to obtain access to this structured data.

The Problem with 2D

One problem with the way data is presented to us when we use the web browser is that it doesn't really allow users to see anything more than the current page. Even though there really isn't an inherent spatial relationships between things on the web, each individual user has their own mental associations about the pages they visit. Web history is an example.

3D browsers have been considered for quite some time, although no compelling examples currently exist. The most recent attempt is a browser called Pogo from AT&T (of all places). Pogo allows users to see collections of pages and history leveraging the third dimension.

While Pogo provides a pseudo-3D experience on a page basis, Firefox browser plugins like Piclens allow users to rapidly scan through image collections contained within a specific domain. Plugins such as these are often particularly inflexible, allowing users to customize none or very few parameters relevant to the content presentation.

E15 was designed as an application that allows users to easily grab all types of data and media from the web and throw it into a flexible 3D context where it can be thrown around and manipulated, providing a more sloppy, playful, experimental interface to traditionally inflexible content.

E15 Demos

  • Simple page loading. Demonstrate ease of extracting page content (images, for example), and throwing it into the 3D E15 context:

  • Loading/Playing videos from YouTube:

  • Facebook: Loading profile information:

  • Facebook: Augmenting the visualization with scraped data (message headers from users inbox, data that cannot be obtained by leveraging the Facebook APIs):

  • Searching/browsing a large collection of pages from collected every two hour period over the course of the past 3 months:

Input Devices

The mouse was designed in the mid 1960's for two dimensional interfaces. In an attempt to move away from devices like the mouse to those with greater degrees of freedom, E15 was built with support for a 6 degree of freedom CAD mouse.

Further experimentation became possible through the use of the iPhone SDK and its easily accessible multi-touch interface. I sketched up a collection of multi-finger mappings between the iPhone and E15, allowing me to navigate and interact with E15 content in a more playful manner.

Miscellaneous Interface Examples

Presentation Images

e15_fusionarts.txt · Last modified: 2011/09/13 15:13 (external edit)
©2008 BuzaMoto industries.