Introduction

This document is the result of discussions and personal explorations of the projects listed below, and may serve as a resource for curious people about how various (open source) creative multimedia infrastructures are built.

Ogre 3D

Description

Pure Data

Description

From the Pure Data Wikipedia entry:

“Pure Data (or Pd) is a graphical programming language developed by Miller Puckette in the 1990s for the creation of interactive computer music and multimedia works.”

Pure Data is written in C, and is cross-platform.

Overview

As described by Miller Puckette, the author of PD, PD is essentially an extension of the “Max” paradigm. So, in the mid 1980s, there was Max, which was used to control audio hardware, as opposed to generating audio signals itself. PD came in 1996, extending the Max idea to include the manipulation and generation of audio signals. This audio functionality was ported into the commercial version of Max in 1997, creating the “MSP” audio component of “Max/MSP”, sold by software company Cycling '74.

After diving into the PD source code, I quickly realized how tightly integrated the system itself is with the available methods of allowing users to write their own custom Max objects (referred to as externals).

General Process

Many of the objects that come as part of PD (and Max/MSP, for that matter), are structured pretty much the same way that externals are developed. When developing an external, for example, users are expected to implement a collection of methods corresponding to various behaviors: when the object receives an input, gets created, deallocated, etc. Because most of the PD internals are structured this way, all that's left for the PD runtime to do is to manage the collection of those objects and make sure the objects get their messages sent to them. The PD source feels like a museum, where everything is positioned with clear intent and without ostentation.

Performance Considerations

I can't really comment about the performance characteristics of PD. As it is written in pure C, and allows users to write their own objects in C as well, I'd say it probably performs just about as well as it should.

References

Max/MSP on Wikipedia.

Max/MSP external programming guide.

GEM

From the GEM Wikipedia entry:

“GEM (Graphics Environment for Multimedia) is a collection of externals (libraries) that provide OpenGL graphics functionality to pure data, a graphical programming language for real-time audio processing.”

Description

Overview

General Process

Performance Considerations

References

The GEM page.

Context Free

Description

From the Context Free site:

“Context Free is a program that generates images from written instructions called a grammar. The program follows the instructions in a few seconds to create images that can contain millions of shapes.”

Overview

Processing

Description

From the Processing.org site:

“Processing is an open source programming language and environment for people who want to program images, animation, and interactions. It is used by students, artists, designers, researchers, and hobbyists for learning, prototyping, and production. It is created to teach fundamentals of computer programming within a visual context and to serve as a software sketchbook and professional production tool. Processing is an alternative to proprietary software tools in the same domain.”

Written in Java, Processing is cross-platform.

Overview

At the heart of Processing is the ”.pde” file, which is the file format in which Processing “sketches” are written. The methods and variable declarations contained within these types of files maintain a very intimate relationship with the Java programming language, the language in which Processing itself is mainly written. A brief discussion about the nature of the “language” contained within the pde files can be found on Ben Fry's blog. I've always described Processing as syntactic sugar on top of Java, and will continue to do so here.

One characteristic of the Processing infrastructure that can be seen in other systems is the way it provides users with seemingly more direct access to drawing pixels on the screen. Instead of having to write a visual application the “Java way”, that is, by creating the windows and graphics contexts necessary to draw lines and circles, the application abstracts away those hassles and provides simplified methods that Java provides (in one form or another). I've always thought about this architecture in terms of a pyramid, with a simple Processing function like rect() at the top, and increasing amounts of complexity hidden below as one gets closer to the operating system.

General Process

When using the Processing editor to create pde files, how does the application produce a running Java application from that code?

The first thing to note is that pde files are first translated into actual ”.java” files, then compiled with a traditional Java compiler. So, how does this translation happen?

First, Processing first takes the code written by the user (the pde), and transforms it into an abstract syntax tree (AST), using a grammar that is a subset of the Java grammar. This resulting data structure is used to generate a valid Java source file (.java). This file is then complied with a traditional Java compiler to produce the application.

Those familiar with the Processing IDE can look at both of these files following the process of “Exporting” the application using the editor. The .pde file and generated .java file are contained within the generated folders.

Application Structure

This Java transformation process is only part of the Processing story. As mentioned earlier, the system involves a lot of infrastructure to abstract away the setup required to start drawing pixels to the screen. One question now is, how does the .java file generated from the pde file insert itself into this collection of classes that are doing the heavy loading?

As Processing applications are most often written in order to generate animations, they need to have some of the same structure that most graphics applications have – one main loop that drives the animation, often running as fast as the operating system will allow. Processing applications are indeed structured in the same way – there's a main loop in each one that gets called as often as is necessary (although Processing allows users to configure this rate).

Each Processing application is a subclass of “PApplet”, which is again a subclass of Applet. Applet contains general mechanisms for controlling the execution of the application (it contains start() and stop() methods, for example). The generated Applet leverages the Java inheritance mechanism to allow it to provide the main Applet render loop. In other words, the PApplet class contains a Java method – public void draw(), that is overridden in the generated subclass of PApplet that the pde transformation process emits. So, when running, the PApplet class tries to call its “draw” method as often as is necessary, which just happens to reside in the generated PApplet subclass. The methods in this subclass (rect(), and ellipse(), for example) are actual java methods that reside in the PApplet class itself (e.g. public void rect(…). These methods draw into the PApplet graphics context, and, at the end of each iteration of the draw() loop, the contents of that graphics context are drawn to the screen.

Performance Considerations

Built with Java, Processing gets a lot of performance optimizations for free (in a sense), as the engineers at Sun are (presumably) continuing to improve the flexibility and performance of both the Java Virtual Machine and the class libraries that are used to draw pixels to the screen.

There are drawbacks to using Java, however. At the end of the day, Java is an interpreted language, which means that the operating system cannot “just execute” the class files generated by a Java compiler like it can a binary executable written in assembler. Executing the code contained within Java class files (in the form of bytecode) requires additional overhead, plain and simple. Indeed, there are ways to compile Java bytecode directly to machine code to avoid some interpretation overhead, but even using a natively compiled set of Java (application and system) classfiles would require a graphical (window-based) application to (perhaps unnecessarily) execute code that would be unnecessary when simply writing a native application and directly using native OS functionality.

SuperCollider

Description

From the SuperCollider Wikipedia entry:

“SuperCollider is an environment and programming language originally released in 1996 by James McCartney for real time audio synthesis and algorithmic composition.”

Overview

ChucK

Description

From the ChucK Wikipedia entry:

“ChucK is a concurrent, strongly-timed audio programming language for real-time synthesis, composition, and performance.”

Overview

References

NodeBox

Description

From the NodeBox site:

“NodeBox is a Mac OS X application that lets you create 2D visuals (static, animated or interactive) using Python programming code and export them as a PDF or a QuickTime movie.”

Even though NodeBox is written in Python, it maintains an intimate connection with OS X functionality, and is only available on OS X.

Historical Notes

NodeBox is based on Just van Rossum's DrawBot.

Overview

OpenFrameworks

Description

Overview

app_infrastructures.txt · Last modified: 2011/09/13 15:14 (external edit)
 
 
©2008 BuzaMoto industries.