This blog entry was posted by peter on October 27, 2007.
It is tagged with InstantPlayer, instantreality, JavaScript, MIDI, reacTIVision, tracking, and X3D.
So far there is 1 comment. Feel free to add one.
The previous entry chronologically is A V4L2 camera engine for reacTIVision.

This article describes how to use the reacTIVision tracking framework in combination with the InstantPlayer X3D browser. It provides a solution based on reacTIVisions MIDI output, representing each fiducial by a node in the X3D scenegraph. The Fiducial X3D prototype node offers fields for position, angle and state of a fiducial, which can be routed like any other X3D field.
Everything described in this article was developed and tested under Linux. Theoretically this should run under Win and Mac but I simply did not try.
You need a working reacTIVison setup. Basically you need a camera working in your environment. If you are a Linux user and own a V4L2 compatible camera you might be interested in the article A V4L2 camera engine for reacTIVision. For further information on reacTIVision installation and setup please see the reacTIVision homepage.
You also need a working InstantPlayer setup. See the InstantReality homepage for more information and supported platforms. I am using the SuSE rpm on a Gentoo box (you can read about how to do this here).
This solution only works in the InstantPlayer X3D browser. The reacTIVision integration is based on MIDI and uses the non-X3D-conform MIDI input feature of the InstantPlayer, so other X3D browsers will not work out.
The necessary files for integrating reacTIVision into the InstantPlayer X3D browser can be downloaded here:
The package contains the following files:
The reacTIVision framework is capable of tracking the position and rotation of so called fiducials. In order to use these fiducials within an X3D world I have implemented two X3D prototypes called Fiducial and FiducialManager. Fiducial nodes are supposed to be used within a FiducialManager and will not work when used elsewhere in the scenegraph. The prototypes are contained in the FiducialManager_PROTO.x3d file.
The Fiducial prototype represents a fiducial as defined by the reacTIVision framework. The output fields of the Fiducial node represent the fiducial attributes and are updated by the FiducialManager. The input fields of the Fiducial node are meant to be used by the FiducialManager and should not be set manually. See the documentation of the Fiducial prototype for more information.
The FiducialManager is responsible for receiving and dispatching MIDI input events send by the reacTIVision application. The FiducialManager dispatches the incoming messages and forwards them to the corresponding Fiducial node. See the documentation of the FiducialManager prototype for more information.
To integrate fiducial tracking into the X3D scene simply load the Fiducial and FiducialManager prototypes and instantiate a manager, containing as many Fiducial nodes as you want to track (please note that the current implementation is limited to 17 fiducials). The following example contains a manager with four fiducials and Logger node for debugging output:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | <?xml version=“1.0” encoding=“UTF-8”?> <!DOCTYPE X3D PUBLIC “ISO//Web3D//DTD X3D 3.0//EN” “http://www.web3d.org/specifications/x3d-3.0.dtd”> <X3D xmlns:xsd='http://www.w3.org/2001/XMLSchema-instance' profile='Full' version='3.0' xsd:noNamespaceSchemaLocation='http://www.web3d.org/specifications/x3d-3.0.xsd'> <Scene DEF='scene'> <ExternProtoDeclare name='Fiducial' url='FiducialManager_PROTO.x3d#Fiducial' /> <ExternProtoDeclare name='FiducialManager' url='FiducialManager_PROTO.x3d#FiducialManager' /> <FiducialManager DEF='fiducialManager' > <Group containerField='fiducialsGroup'> <Fiducial DEF='fiducial0' index='0' /> <Fiducial DEF='fiducial1' index='1' /> <Fiducial DEF='fiducial2' index='2' /> <Fiducial DEF='fiducial3' index='3' /> </Group> </FiducialManager> <Logger DEF='logger' level='3' /> <ROUTE fromNode='fiducial0' fromField='visible' toNode='transObj0' toField='render'/> <ROUTE fromNode='fiducial0' fromField='angle' toNode='oi2' toField='set_fraction'/> <ROUTE fromNode='fiducial1' fromField='visible' toNode='transObj1' toField='render'/> </Scene> </X3D> |
The reacTIVision integration is based on the MIDI output functionality of the reacTIVision server. The output can be configured via an XML file which is passed to the server when started on the command line. The XML file for integrating reacTIVision into InstantPlayer is named instanMidi.xml and it looks like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | <?xml version='1.0' encoding='ISO-8859-1'?> <midi> <map fiducial=“0” type=“hfader” control=“0” min=“0” max=“1”/> <map fiducial=“0” type=“vfader” control=“1” min=“0” max=“1”/> <map fiducial=“0” type=“knob” control=“2” min=“0” max=“1”/> <map fiducial=“0” type=“note” note=“0” /> <map fiducial=“1” type=“hfader” control=“3” min=“0” max=“1”/> <map fiducial=“1” type=“vfader” control=“4” min=“0” max=“1”/> <map fiducial=“1” type=“knob” control=“5” min=“0” max=“1”/> <map fiducial=“1” type=“note” note=“1” /> <!— more fiducials … —> <map fiducial=“17” type=“hfader” control=“51” min=“0” max=“1”/> <map fiducial=“17” type=“vfader” control=“52” min=“0” max=“1”/> <map fiducial=“17” type=“knob” control=“53” min=“0” max=“1”/> <map fiducial=“17” type=“note” note=“19” /> </midi> |
The position and angle of a fiducial is mapped onto three succesive MIDI controller numbers. The fiducial presence is mapped to a note number corresponding to the index of the fiducial. This makes four <map> entries per fiducial.
Normally you should not need to edit this file.
You can use the provided test_reactivisionMidi.x3d file to test your reacTIVision - InstantPlayer setup.
First start InstantPlayer with the X3D test scene containing the FiducialManager. The FiducialManager initializes the MIDI input of the InstantPlayer called HID MidiIn:
1 | $ InstantPlayer test_reactivisionMidi.x3d |
You should see a windows that looks like the following, with two items a the bottom left labeled “F1” and “F2” representing two fiducials. Since we did not start the reacTIVision application yet, these items currently do not move.

After starting InstantPlayer the output of the following command should contain a HID MidiIn entry:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | $ cat /proc/asound/seq/clients
Client info
cur clients : 2
peak clients : 3
max clients : 192
Client 0 : “System” [Kernel]
Port 0 : “Timer” (Rwe-)
Port 1 : “Announce” (R-e-)
Client 128 : “MidiIn Backend” [User]
Port 0 : “HID MidiIn” (-We-)
Input pool :
Pool size : 200
Cells in use : 0
Peak cells in use : 0
Alloc success : 0
Alloc failures : 0
|
To make the items move, start the reacTIVision server in MIDI mode like this:
1 2 | $ ./reacTIVision -m /path/to/instantMidi.xml opening midi device: HID MidiIn |
Please make sure that reacTIVision is using InstantPlayer’s HID MidiIn as it’s MIDI input device, otherwise the tracking data will be send to the wrong application.
Using the fiducial number 0 and 1 from the reacTIVision website, you can now move and rotate the items by holding the fiducial markers in front of the camera.

Max said on September 04, 2008
I tried your example on Windows XP, but the HID MidiIn is not found after starting InstantPlayer. I am using Midi-yolk. Any thoughts? Can you change the MIDI source?