transmote speaks…

<– 2D Marker Tracking | Loading Collada Models –>
 
So, you want to create augmented reality in a browser. You saw the GE ad, you tried stuffing your cat into a USPS box, and now you want a piece.

This tutorial will run you through the basics of getting an augmented reality application up and running. You might want to try the 2D tutorial first, to get a handle on a Papervision-less FLARManager application. Or, you might just want to dive right in.

FLARManager Tutorial: 3D and Augmented Reality

Download source for this tutorial here:
FLARManagerTutorial_3D
and place it in the root of the /src/ FLARManager folder.
Print out any of the ‘pattNNN.png’ markers in this folder to use with the tutorial.
 
 
This tutorial will demonstrate how to draw a cube wherever FLARToolkit sees a marker. We’ll start with just one cube for now; things get more complex when you want to support multiple cubes tied to multiple markers.

First, we create a FLARManager instance, passing in the path to an external xml configuration file. For now, we’ll use flarConfig.xml (located in the /resources/ folder in the FLARManager download).
this.flarManager = new FLARManager("../resources/flar/flarConfig.xml");

Once you’re comfortable with the basics of FLARManager, you can edit this config file, or create your own, as you see fit. More information on configuration options lives here.

We want to see the video capture, so let’s add it to the stage. FLARManager creates a default video source all ready to go, though the source can also be modified via the configuration file.
this.addChild(Sprite(this.flarManager.flarSource));

FLARManager uses an event model to notify any interested parties of newly-detected markers, changes in already-detected markers, and marker removal. We add FLARMarkerEvent handlers to respond to these changes.

this.flarManager.addEventListener(FLARMarkerEvent.MARKER_ADDED, this.onMarkerAdded);
this.flarManager.addEventListener(FLARMarkerEvent.MARKER_UPDATED, this.onMarkerUpdated);
this.flarManager.addEventListener(FLARMarkerEvent.MARKER_REMOVED, this.onMarkerRemoved);

We’ll write our event handlers in just a a bit. First, we wait for FLARManager to initialize before setting up the Papervision3D environment.

this.flarManager.addEventListener(Event.INIT, this.onFlarManagerInited);

Setting up Papervision3D

once FLARManager has finished initializing, we can set up the Papervision3D environment:

private function onFlarManagerInited (evt:Event) :void {
    this.flarManager.removeEventListener(Event.INIT, this.onFlarManagerInited);
 
    this.scene3D = new Scene3D();
 
    // initialize FLARCamera3D with parsed camera parameters.
    this.camera3D = new FLARCamera3D(this.flarManager.cameraParams);
 
    this.viewport3D = new Viewport3D(this.stage.stageWidth, this.stage.stageHeight);
    this.addChild(this.viewport3D);
 
    // flip display to match mirrored source
    this.viewport3D.x = this.stage.stageWidth;
    this.viewport3D.scaleX = -1;
 
    this.renderEngine = new LazyRenderEngine(this.scene3D, this.camera3D, this.viewport3D);
 
    this.pointLight3D = new PointLight3D();
    this.pointLight3D.x = 1000;
    this.pointLight3D.y = 1000;
    this.pointLight3D.z = -1000;
 
    this.addEventListener(Event.ENTER_FRAME, this.onEnterFrame);

Papervision has to re-render the scene every frame, so we add an ENTER_FRAME event handler in which we’ll do that. (We’ll get to that part in a sec.)

FLARToolkit camera parameters

This section is extra credit, but thought you might want to know about that FLARCamera3D line…

FLARToolkit compensates for distortion caused by the camera lens by referring to an external camera parameters file. In the FLARToolkit download, this is named ‘camera_para.dat’; in FLARManager, the file is named ‘FLARCameraParams.dat’. FLARToolkit creates a special FLARCamera3D object (that subclasses Papervision3D’s Camera3D) that incorporates this correction into Papervision3D’s display.

FLARCamera3D requires the information from the camera parameters file, but FLARManager cannot provide this until it has loaded and parsed this file. Therefore, we wait to initialize the Papervision3D environment until after FLARManager has initialized. Once there, we pass the parsed camera parameters data (stored in a FLARParam instance) to the new FLARCamera3D.

Create the Cube

Let’s set up a single Cube instance that we’ll map to the detected marker:

    var cubeMaterial:FlatShadeMaterial = new FlatShadeMaterial(this.pointLight3D, 0xFF1919, 0x730000);
    var materialsList:MaterialsList = new MaterialsList({all: cubeMaterial});
    var cube:Cube = new Cube(materialsList, 40, 40, 40);
    cube.z += 20;
 
    // create a container for the cube, that will accept matrix transformations.
    this.cubeContainer = new DisplayObject3D();
    this.cubeContainer.addChild(cube);
    this.scene3D.addChild(this.cubeContainer);

We place the Cube inside of a DisplayObject3D so that we can lift the Cube 20 pixels (in the z-axis) above the center of the marker, so it appears to be sitting on the marker instead of embedded in it. If we moved the Cube up 20 pixels and then applied the transformation matrix directly to it, the 20px z-coordinate would get overwritten, and the Cube would appear stuck in the middle of the marker.

Responding to FLARMarkerEvents

Now that FLARManager and Papervision3D are both set up, we can make the two work together by drawing Papervision3D objects when handling FLARMarkerEvents coming from FLARManager.

private function onMarkerAdded (evt:FLARMarkerEvent) :void {
    this.cubeContainer.visible = true;
    this.activeMarker = evt.marker;
}
 
private function onMarkerRemoved (evt:FLARMarkerEvent) :void {
    this.cubeContainer.visible = false;
    this.activeMarker = null;
}

For the purposes of this tutorial, we’re simply toggling the visibility of the Cube as a marker is added and removed from the camera’s view. We’re also keeping track of the active FLARMarker, from which we’ll extract and apply the transformation matrix that makes the Cube appear to be tethered to the marker.

For more on FLARMarkerEvents, see the 2D tutorial.

Updating the Cube

As mentioned earlier, Papervision has to re-render the scene every frame. We’ll take advantage of this opportunity to apply the latest transformation matrix to the Cube, to make it look like it’s Augmenting our Reality.

private function onEnterFrame (evt:Event) :void {
    this.cube.transform = FLARPVGeomUtils.convertFLARMatrixToPVMatrix(this.activeMarker.transformMatrix);
    this.renderEngine.render();
}

Note that the FLARMarker instance we’re tracking as this.activeMarker will be updating itself continuously; FLARManager handles this for us. We could listen for changes as MARKER_UPDATED events, but since we have to render the Papervision scene every frame, we can just grab the updated information from this.activeMarker while we’re at it.

You might also notice that long, ugly method call, FLARPVGeomUtils.convertFLARMatrixToPVMatrix. (Unfortunately at times like this, I’m a firm believer in self-commenting code.) FLARToolkit uses a different coordinate system than Papervision (and Flash 3D, and Away3D, and Sandy, and Alternativa), so we have to convert FLARToolkit’s transformation matrix into numbers that make sense to Papervision before applying the matrix to the Cube. Coincidentally, FLARPVGeomUtils has just the method for us!

That should take care of it — you should now see a colored cube perched gently on the marker in your hand!

 
To handle multiple cubes and multiple markers, we have to be smarter about managing all the DisplayObject3D instances and active FLARMarker instances. One way to do this is laid out in this example:
FLARManagerExample_PV3D

Also, FLARManager supports arbitrary aspect ratios. No need to stick with 4:3! Here’s a 16:9 example, if you want to get all cinematic:
FLARManagerExample_PV3D_Wide
 
 
<– 2D Marker Tracking | Loading Collada Models –>