FLARManager: AR and Alternative Controllers

i’m also participating in a panel on “inspiration”, along with Brendan Dawes, Tali Kraokowski, and Craig Swann. pretty noble topic…!

Source + Imagination

Super Early Bird pricing is now available — get em while they’re hot!

see you in toronto…

those of you who’ve tried to fullscreen an AIR app in OSX know about the annoying grey bar across the bottom of the screen. apparently, that’s a status bar. i guess you can even set status messages into it. handy, perhaps, if you’re making a browser entirely in AIR. but i think most of us just want it to go away.

you can do this by adding showStatusBar="false" to your mx:WindowedApplication tag.
(you can also add status with status="who needs this stupid grey bar?".)

i found this out while playing with mx:Windows. all NativeWindows have a statusBar by default. how bout that.

-eric

http://www.ardevcamp.org/

my session will address the web component of AR, and will tie in with discussions on mobile AR, standards, experience design, and more. i believe (free!) registration is still open, on the site above. stop by!

v0.61 corrects those issues, thanks to some help from jim alliban. download it here:
http://words.transmote.com/wp/flarmanager/

what is this?

FLARManager is a lightweight framework that makes it easier to build FLARToolkit augmented reality applications for Flash. it is compatible with a variety of 3D engines, and provides a more robust event-based system for managing marker addition, update, and removal. it supports detection and management of multiple patterns, and multiple markers of a given pattern.

On to version 0.6. 1.0 is around the corner…
download here:
http://words.transmote.com/wp/flarmanager/

what is this?

FLARManager is a lightweight framework that makes it easier to build FLARToolkit augmented reality applications for Flash. it is compatible with a variety of 3D engines, and provides a more robust event-based system for managing marker addition, update, and removal. it supports detection and management of multiple patterns, and multiple markers of a given pattern.

what’s new this time around?

[customization]
FLARManager’s smoothing and adaptive thresholding display algorithms are now broken out into separate classes, allowing developers to implement their own algorithms. see Inside FLARManager: Customization for more info. additionally, custom smoother and threshold display algorithms can be implemented via flarConfig.xml, to allow fine-tuning without recompiling.

[better 2D support]
added .x, .y, .z, .rotation2D, .scale2D, and .matrix2D properties to FLARMarker, for easier access to 2D transformation properties that used to be hidden inside FLARToolkit’s 3D transformation matrices.

[better camera management]
changes to FLARCameraSource provide better camera management, with less of a chance that the wrong camera will be auto-selected.

[FLARManager deactivation and disposal]
improved support for de- and reactivating, and for completely removing FLARManager from memory.

[native mirroring]
developers no longer need to flip the viewport to achieve mirroring. when FLARManager.mirrorDisplay is set to true (as it is by default), FLARManager internally mirrors the matrices generated by FLARToolkit.

 

custom adaptive thresholding and smoothing

FLARManager’s smoothing and adaptive thresholding display algorithms are now broken out into separate classes, allowing developers to implement their own algorithms. see Inside FLARManager: Customization for more info. additionally, custom smoother and threshold display algorithms can be implemented via flarConfig.xml, to allow fine-tuning without recompiling.

as a result of these changes, FLARManager.adaptiveThresholding, .adaptiveThresholdSpeed, and adaptiveThresholdBias are deprecated and removed. developers should instead specify adaptive threshold speed and bias in a <thresholdAdapter> tag, as described here.

thanks to eugene zatepyakin and matt reynolds for their suggestions and contributions to the still-in-process IntegralImageThresholdAdapter and HistogramThresholdAdapter algorithms.

better camera management

FLARCameraSource now activates the first camera found, instead of defaulting to ‘USB Video Class Video’. if a camera reports no activity after a few seconds, the next camera in line is automatically selected. this continues until an active camera is found, or all camera options are exhausted. also, developers can manually specify a camera index (from Camera.names) via FLARCameraSource.cameraIndex.

better 2D support

added .x, .y, .z, .rotation2D, .scale2D, and .matrix2D properties to FLARMarker, for easier access to 2D transformation properties that used to be hidden inside FLARToolkit’s 3D transformation matrices. updated FLARManagerTutorial_2D and FLARManagerExample_2D to demonstrate these new properties.

updated 3D engines

the FLARManager distro now includes Away3D 3.3.4 (featuring vector shape primitives, like text) and Papervision3D 2.1.920 (featuring better COLLADA support).

more configuration options

in addition to exposing custom smoothing and adaptive thresholding in flarConfig.xml, developers can also specify the pixel size of a pattern at which it generates a transformation matrix of a scale of 1.0, via <pattern size='N'>; and can tweak FLARManager performance using <flarManagerSettings minimumLabelSize=’N'>. see the documentation page for more details.

native mirroring

developers no longer need to flip the viewport to achieve mirroring. when FLARManager.mirrorDisplay is set to true (as it is by default), FLARManager internally mirrors the matrices generated by FLARToolkit.

improved stability in tutorials/examples

a condition in a number of the tutorials and examples caused content to be absent from the screen, even when the marker was clearly present. this is fixed, by adding a handler for FLARMarkerEvent.MARKER_UPDATED events in those tutorials/examples.

FLARManager deactivation and disposal

FLARManager.activate(), FLARManager.deactivate(), and FLARManager.dispose() give developers more control over when FLARManager is consuming resources.

FLARManager.verbose

setting FLARManager.verbose to true allows developers to keep a closer eye on changes to FLARManager's configuration via trace() output.

just realized i never posted this on my own site.

i was recently contacted by InsideRIA.com to write an in-depth article about FLARToolkit and FLARManager. you can find it here.

another upgrade, hot’n'fresh for you.
download here:
http://words.transmote.com/wp/flarmanager/

what is this?

FLARManager is a lightweight framework that makes building FLARToolkit augmented reality applications easier. it is compatible with a variety of 3D engines, and provides a more robust event-based system for managing marker addition, update, and removal. it supports detection and management of multiple patterns, and multiple markers of a given pattern.

what’s new this time around?

[adaptive thresholding]
makc and saqoosha have been experimenting a bit with adaptive thresholding, so i decided to give it a go as well. markers are now detectable in varying and low illumination — great for web apps, where developers have no control over the end user’s lighting. (the image above demonstrates this feature in action.)

[support for Alternativa3D, Away3D, and Sandy3D]
FLARManager now supports the above three frameworks, along with the existing support for Papervision3D and Flash Player 10 3D. Thanks to makc and rob bateman for their assistance with matrix and camera conversions, and to holde for help with Away3D lighting.

[tutorials and examples]
the Inside FLARManager section in the nav bar above provides access to documentation and tutorials about FLARManager, as well as links to FLARToolkit-related content. simple tutorials are now available for basic 2D, 3D, and collada applications.

[optimization]
some changes under the hood, accompanied by extensive testing and profiling, allow FLARManager to run faster and more efficiently than before. additonally, a number of features are now exposed to the developer to tweak performance on an application-by-application basis. stay tuned for a detailed writeup on this.

[FLARProxy update]
FLARProxy allows developers to simulate marker detection with a mouse and keyboard. The functionality has not changed in this version, but it can now be activated via the external config file.

 

FLARManager constructor change

completing the move away from specifying application settings in source code and toward using an external configuration file, the path to the config file should now be passed directly into FLARManager’s constructor:
this.flarManager = new FLARManager("../resources/flar/flarConfig.xml");
if you still wish to specify application settings in source code, use FLARManager.initManual.
also, FLARManager.initFromFile is still supported, for legacy applications.

support for Alternativa3D, Away3D, and Sandy3D

FLARManager now supports the above three frameworks, along with the existing support for Papervision3D and Flash Player 10 3D. Thanks to makc and rob bateman for their assistance with matrix and camera conversions. use of these frameworks is subject to each of their licenses.

FLARManager comes with examples for all of the supported 3D frameworks:

• FLARManagerExample_Alternativa3D.as
• FLARManagerExample_Away3D.as
• FLARManagerExample_Flash3D.as
• FLARManagerExample_PV3D.as
• FLARManagerExample_Sandy3D.as

FLARManagerExampleLauncher.as

adding more examples and tutorials created just too many runnable Application classes, so i consolidated them into one class. FLARManagerExampleLauncher.as is now the only Application class; uncomment the example or tutorial you want to run.

FLARMarkerOutline removed

FLARMarkerOutline was deprecated in v0.4; it is now removed completely. use FLARMarker instead.

FLARProxy update

FLARProxy allows developers to simulate marker detection with a mouse and keyboard. The functionality has not changed in this version, but it can now be activated via the external config file.

adaptive thresholding

makc and saqoosha have been experimenting a bit with adaptive thresholding, so i decided to give it a go as well. markers are now detectable in varying and low illumination — great for web apps where developers have no control over the end user’s lighting.

use FLARManager.adaptiveThresholding to toggle this functionality; it is set to true by default. adjust FLARManager.adaptiveThresholdingSpeed and FLARManager.adaptiveThresholdingBias for finer control over this functionality.

preliminary sample blurring

per a suggestion by jeffery bennett, frames sent to FLARToolkit for marker detection are blurred slightly before processing. this feature can be adjusted and disabled via FLARManager.sampleBlurring. thanks jeffery!

marker removal delay

while smoothing can help reduce jitter, slightly delaying removal of the marker can help reduce flickering. use FLARManager.markerRemovalDelay to adjust the length of time between marker removal and the MARKER_REMOVED event dispatch.

error reporting

by applying an ErrorEvent listener to your FLARManager instance, you can catch any errors that might arise in FLARManager’s initialization. currently, the only such error is a failure to acquire a camera.

this.flarManager.addEventListener(ErrorEvent.ERROR, this.onFlarManagerError);

all of the examples (not tutorials) demonstrate how to handle ErrorEvents dispatched by FLARManager.

collada tutorial

the people have asked for a simple tutorial demonstrating how to load a collada model for use in an AR app. here you go.

The Sputnik Observatory collects video interviews of many leading thinkers on a broad range of topics, and enables viewers to follow a stream-of-consciouness path through the themes that link the conversations together.

 
For example, cartoonist Scott McCloud talks about facial recognition:

 
which could lead to physicist Janna Levin discussing chaos and disorder:

 
and then to neuroscientist Edmund T. Rolls explaining how bacteria can decide who we get it on with:

Jonathan says:

The central premise of the Sputnik project is that everything is connected to everything else, and that topics and ideas that may seem fringe and even heretical to the mainstream world are in fact being investigated by leading thinkers working in fields as diverse as quantum physics, mathematics, neuroscience, biology, economics, architecture, digital art, video games, computer science and music. Sputnik is dedicated to bringing these crucial ideas from the fringes of thought out into the limelight, so that the world can begin to understand them….There are about 200 videos on the site today, and there will be thousands more added over the coming weeks, months, and years.

I could waste hours learning here…if i didn’t have freelance work to attend to. Back to it…

i’m pleased to announce the release of a new-and-improved version of FLARManager! this one has been a while in the works, due in part to general busy-ness, in part to my laptop threatening death, and in part to the fact that there are SO MANY NEW FEATURES.

download here:
http://words.transmote.com/wp/flarmanager/
(NOTE, this is now pointing to version 0.41, which is required for the new tutorials.)

quick description of FLARManager:
FLARManager is a small framework that makes building FLARToolkit applications easier. it also decouples the marker-tracking functionality from Papervision3D, and provides a more robust event-based system for managing marker addition, update, and removal. it supports detection and management of multiple patterns, and multiple markers of a given pattern.

 
here’s a rundown on the most interesting new stuff:
 
[matrix-level smoothing]
FLARManager now applies smoothing to the matrix values reported by FLARToolkit for detected markers; this helps to clear up jitter in rotation, scaling, and position. developers can set the amount of smoothing, and easily implement their own smoothing algorithms.

[external configuration]
devs now have the option to specify many configuration settings via an xml file that does not require app recompilation. among the settable features are capture and display width/height, display mirroring, matrix smoothing, and pattern and camera param file paths.

[easier access to detected marker data]
FLARMarker instances, accessible via FLARMarkerEvents, now expose (x,y,z) location (via a Flash Player 10 Vector3D object), rotation on the x, y, and z axes, and the (x,y) coordinate of the marker’s centerpoint. this opens the door to applications that use FLARToolkit to create new user interfaces and controllers, in addition to its ability to produce traditional augmented reality applications.

[documentation! (just a start...)]
i finally got around to generating ASDocs for FLARManager; this Javadoc-style documentation is similar in format to Adobe’s Flex Language Reference. i still have a ways to go to make these docs more useful, but…it’s a start. you can find the docs in the html_docs folder; just open index.html in a browser. i’ll be putting them online in the near future.

[commercial license]
ARToolworks, the group managing the commercial license for FLARToolkit, will be offering a commercial license for FLARManager very shortly. we are just wrapping up logistics now, and i’ll post more about this shortly; in the meantime, please contact me via the link at the right of the titlebar above for more information.

 

FLARCameraSource.as

• mirroring is now implemented; see FLARManager.as below for more info.
• full-resolution video: video can be displayed at full capture resolution. however, when the source image is somewhat dark, FLARToolkit slows down substantially. i’m still working on this one; in the meantime, FLARManager still defaults to capture 320×240 video.
• developers can now explicitly specify capture width/height and display width/height. please note that if you change display width and height and you’re mirroring your display, you should ensure that your application is mirroring along the center of the display width, not just the center of your stage. otherwise, your marker results will likely be offset in the x-axis.
• downsampleRatio now does just that — specifies how much to downsample the captured video before sending to FLARToolkit. it no longer has any effect on the displayed video.
• resultsToDisplayRatio provides a value for scaling FLARToolkit results to fit the specified display size. this is mainly used internally within FLARManager; developers should not need to access this (but it is available via a public accessor).

FLARLoaderSource.as

the changes here follow the changes in FLARCameraSource.

• full-resolution display: the loaded source can be displayed at full resolution. this works well, and unlike FLARCameraSource, does not need further iteration.
• developers can now explicitly specify capture width/height and display width/height.
• downsampleRatio and resultsToDisplayRatio changes; see FLARCameraSource.as above for more information.

IFLARMatrixSmoother.as

matrix smoothing can be set via an external xml config file (see External configuration file below), or via FLARManager.smoothing and FLARManager.smoother. developers can use the default smoother, FLARMatrixSmoother_Average.as, which simply averages matrices over the number of frames specified by FLARManager.smoothing, or write their own implementations of IFLARMatrixSmoother.as.

External configuration file

developers can now pass a number of configuration settings to FLARManager via FLARManager.initFromFile(). these settings include:

• source width/height
• display width/height
• capture framerate
• capture downsampleRatio
• loader file path, for using FLARLoaderSource.as
• display mirroring
• FLARManager.updateMarkerThreshold
• smoothing value
• camera parameters file path
• list of pattern files
• pattern resolution
• patternToBorderRatio
• pattern minimum required confidence

initializing FLARManager via an external xml config file is now the preferred method. initialization via an IFLARSource, path to camera params file, and Vector of FLARPattern instances is now deprecated, and will be removed from the FLARManager.as constructor in the next release.

FLARMarker.as

• FLARMarkerOutline functionality has been rolled into FLARMarker, and FLARMarkerOutline.as is now deprecated. there is no longer any need to access FLARMarkerEvent.marker.outline.someValue; just access FLARMarkerEvent.marker.someValue.
• vector3D offers (x, y, z) coordinates of a detected marker.
• rotationX, rotationY, and rotationZ provide rotation of a detected marker along each of the three axes.
• sessionId provides a unique id for each detected marker in a session.
• FLARMarker instances were not being properly garbage collected in the previous version of FLARManager; they are now.

FLARManager.as

• initFromFile() allows developers to initialize FLARManager via an external xml config file. see External configuration file above.
• mirrorDisplay allows developers to turn horizontal display mirroring on and off easily. this can also be achieved via an external config file.
• markerUpdateThreshold is now exposed; this value specifies the maximum amount of motion (in the xy plane) a marker can move before it is read as a new marker. it defaults to 20px; increasing this value can help to keep fast-moving markers from flickering.
• smoothing sets the number of frames over which FLARMatrixSmoother_Average averages marker matrix results. lower values result in crisper motion with more jitter; higher values result in slower, smoother updates. set to 0 to turn off smoothing completely.
• smoother can accept custom smoothing algorithms, as implementations of IFLARMatrixSmoother.

miscellaneous changes

• Flash 3D marker tracking is greatly improved. turns out that FLARToolkit and Flash 3D have a different z origin; FLARGeomUtils.convertFLARMatrixToFlashMatrix now adds an offset when translating matrices from FLARToolkit to Flash 3D. non-4:3 aspect ratios in Flash 3D are currently broken, however
• 12 new sample patterns are included in the resources/flar/patterns folder. these are 4x4 patterns, and FLARToolkit is able to distinguish them as separate patterns quite successfully.
• ASDocs can be found in the html_docs folder; just open index.html in a browser. i'll be putting them online in the near future.

bugs

there are a LOT of new features here, so there are likely bugs! please help me squash them. there are a few minor bugs lurking here and there, but nothing major that i've seen (with the exception of non-4:3 aspect ratios being broken in Flash 3D). please let me know if you find anything!