Using reacTIVision in the InstantPlayer X3D browser

This blog entry was posted by on .

It is tagged with InstantPlayer, instantreality, JavaScript, MIDI, reacTIVision, tracking, and X3D.

So far there are 25 comments. Feel free to add one.

reactivision in action!

This article describes how to use the reacTIVision tracking framework in combination with the InstantPlayer X3D browser. It provides a solution based on reacTIVisions MIDI output, representing each fiducial by a node in the X3D scenegraph. The Fiducial X3D prototype node offers fields for position, angle and state of a fiducial, which can be routed like any other X3D field.

Everything described in this article was developed and tested under Linux. Theoretically this should run under Win and Mac but I simply did not try.

Requirements

You need a working reacTIVison setup. Basically you need a camera working in your environment. If you are a Linux user and own a V4L2 compatible camera you might be interested in the article A V4L2 camera engine for reacTIVision. For further information on reacTIVision installation and setup please see the reacTIVision homepage.

You also need a working InstantPlayer setup. See the InstantReality homepage for more information and supported platforms. I am using the SuSE rpm on a Gentoo box (you can read about how to do this here).

This solution only works in the InstantPlayer X3D browser. The reacTIVision integration is based on MIDI and uses the non-X3D-conform MIDI input feature of the InstantPlayer, so other X3D browsers will not work out.

Download

The necessary files for integrating reacTIVision into the InstantPlayer X3D browser can be downloaded here:

The package contains the following files:

instantMidi.xml:
MIDI configuration file for reacTIVision
FiducialManager_PROTO.x3d:
X3D files containing the Fiducial and FiducialManager prototypes
test_reactivisionMidi.x3d:
Test scene using two fiducials.

Introduction

Prototypes

The reacTIVision framework is capable of tracking the position and rotation of so called fiducials. In order to use these fiducials within an X3D world I have implemented two X3D prototypes called Fiducial and FiducialManager. Fiducial nodes are supposed to be used within a FiducialManager and will not work when used elsewhere in the scenegraph. The prototypes are contained in the FiducialManager_PROTO.x3d file.

Fiducial

The Fiducial prototype represents a fiducial as defined by the reacTIVision framework. The output fields of the Fiducial node represent the fiducial attributes and are updated by the FiducialManager. The input fields of the Fiducial node are meant to be used by the FiducialManager and should not be set manually. See the documentation of the Fiducial prototype for more information.

FiducialManager

The FiducialManager is responsible for receiving and dispatching MIDI input events send by the reacTIVision application. The FiducialManager dispatches the incoming messages and forwards them to the corresponding Fiducial node. See the documentation of the FiducialManager prototype for more information.

Using the manager

To integrate fiducial tracking into the X3D scene simply load the Fiducial and FiducialManager prototypes and instantiate a manager, containing as many Fiducial nodes as you want to track (please note that the current implementation is limited to 17 fiducials). The following example contains a manager with four fiducials and Logger node for debugging output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
<?xml version=“1.0” encoding=“UTF-8”?>
<!DOCTYPE X3D PUBLIC “ISO//Web3D//DTD X3D 3.0//EN” “http://www.web3d.org/specifications/x3d-3.0.dtd”>
<X3D xmlns:xsd='http://www.w3.org/2001/XMLSchema-instance' profile='Full' version='3.0' xsd:noNamespaceSchemaLocation='http://www.web3d.org/specifications/x3d-3.0.xsd'>
  <Scene DEF='scene'>
    
  <ExternProtoDeclare name='Fiducial' url='FiducialManager_PROTO.x3d#Fiducial' />  
  <ExternProtoDeclare name='FiducialManager' url='FiducialManager_PROTO.x3d#FiducialManager' />
  
    <FiducialManager DEF='fiducialManager' >
      <Group containerField='fiducialsGroup'>
        <Fiducial DEF='fiducial0' index='0' />
        <Fiducial DEF='fiducial1' index='1' />
        <Fiducial DEF='fiducial2' index='2' />
        <Fiducial DEF='fiducial3' index='3' />      
      </Group>        
    </FiducialManager>
          
    <Logger DEF='logger' level='3' />
    <ROUTE fromNode='fiducial0' fromField='visible' toNode='transObj0' toField='render'/>
    <ROUTE fromNode='fiducial0' fromField='angle' toNode='oi2' toField='set_fraction'/>    
    <ROUTE fromNode='fiducial1' fromField='visible' toNode='transObj1' toField='render'/>      
  </Scene>
</X3D>

Configuring reacTIVision MIDI output

The reacTIVision integration is based on the MIDI output functionality of the reacTIVision server. The output can be configured via an XML file which is passed to the server when started on the command line. The XML file for integrating reacTIVision into InstantPlayer is named instanMidi.xml and it looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
<?xml version='1.0' encoding='ISO-8859-1'?>      
<midi>  
  <map fiducial=“0” type=“hfader” control=“0” min=“0” max=“1”/>
  <map fiducial=“0” type=“vfader” control=“1” min=“0” max=“1”/>
  <map fiducial=“0” type=“knob” control=“2” min=“0” max=“1”/>
  <map fiducial=“0” type=“note” note=“0” />
  
  <map fiducial=“1” type=“hfader” control=“3” min=“0” max=“1”/>
  <map fiducial=“1” type=“vfader” control=“4” min=“0” max=“1”/>
  <map fiducial=“1” type=“knob” control=“5” min=“0” max=“1”/>
  <map fiducial=“1” type=“note” note=“1” />
  
  <!— more fiducials … —>
  
  <map fiducial=“17” type=“hfader” control=“51” min=“0” max=“1”/>
  <map fiducial=“17” type=“vfader” control=“52” min=“0” max=“1”/>
  <map fiducial=“17” type=“knob” control=“53” min=“0” max=“1”/>
  <map fiducial=“17” type=“note” note=“19” />    
</midi>

The position and angle of a fiducial is mapped onto three succesive MIDI controller numbers. The fiducial presence is mapped to a note number corresponding to the index of the fiducial. This makes four <map> entries per fiducial.

Normally you should not need to edit this file.

Testing the system

You can use the provided test_reactivisionMidi.x3d file to test your reacTIVision - InstantPlayer setup.

Starting InstantPlayer

First start InstantPlayer with the X3D test scene containing the FiducialManager. The FiducialManager initializes the MIDI input of the InstantPlayer called HID MidiIn:

1
$ InstantPlayer test_reactivisionMidi.x3d

You should see a windows that looks like the following, with two items a the bottom left labeled “F1” and “F2” representing two fiducials. Since we did not start the reacTIVision application yet, these items currently do not move.

InstantPlayer test scene.

After starting InstantPlayer the output of the following command should contain a HID MidiIn entry:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
$ cat /proc/asound/seq/clients
Client info
  cur  clients : 2
  peak clients : 3
  max  clients : 192
Client   0 : “System” [Kernel]
  Port   0 : “Timer” (Rwe-)
  Port   1 : “Announce” (R-e-)
Client 128 : “MidiIn Backend” [User]
  Port   0 : “HID MidiIn” (-We-)
  Input pool :
    Pool size          : 200
    Cells in use       : 0
    Peak cells in use  : 0
    Alloc success      : 0
    Alloc failures     : 0

Starting reacTIVision

To make the items move, start the reacTIVision server in MIDI mode like this:

1
2
$ ./reacTIVision -m /path/to/instantMidi.xml
opening midi device: HID MidiIn

Please make sure that reacTIVision is using InstantPlayer’s HID MidiIn as it’s MIDI input device, otherwise the tracking data will be send to the wrong application.

Using the fiducial number 0 and 1 from the reacTIVision website, you can now move and rotate the items by holding the fiducial markers in front of the camera.

InstantPlayer and reacTIVision in combination.

Limitations

  • The current implementation is limited to 17 fiducials. This is due to the limit of 64 fields per X3D prototype/node. Internally every fiducial requires three fields (3*17=51) plus two additional fields for add/delete messages. The IOSensor node contains 11 fields itself, therefore 51+2+11=64. tO overcome this limitation multiple IOSensors could be used but that has not been tried yet.

Comments

said on März 27, 2009

Hi Peter,

habe heute die instantvision seite entdeckt und darüber bin ich auch hier gelandet. ich beschäftige mich mit MT-lösungen. Interessanter ansatz mit den fiducials - glaubst du es wäre auch möglich TUIO zu implementieren? Martin hat gerade neue versionen von verschiedenen clients etc. veröffentlicht (www.tuio.org) - eine instantvision implementierung wäre wirklich schön - es gib sehr wenig im bereich MT und 3d zur zeit - umso weniger als crossplattform lösung…

Wäre schön von dir zu hören rozsa(at)cd-cologne.de

Gruss aus Kölle,

Sandor

said on Mai 06, 2009

20484 cnekrjvef cialis compra 13477 csdjchs http://www.arte-arezzo.it/moodle/user… cialis italia :) dhjcbd 11531 cialis 44991

said on Mai 11, 2009

21409 ehgc jsh comprare viagra 26515 c djshcnm http://forum.studenti.it/members/viag… viagra comprare :) cksdj k jksdkc ksbd 14698 viagra 21575

said on Juli 17, 2009

acquisto cialis tadalafil viagra acquista cialis comprare http://members.ebay.it/ws/eBayISAPI.d…

said on Juli 23, 2009

acquisto cialis generico viagra comprare cialis ordina http://members.ebay.it/ws/eBayISAPI.d…

said on Juli 24, 2009

Hey everyone. I'm hoping to meet new friends here so drop me a note when you
get a chance.

I hope to make some quality posts soon but first I have to look around the forum and
familiarize myself with everyone and the forum.

Bye for now. lol

***************************************************

Biggest Loser of All Time

said on August 11, 2009

It's quite possible. That might
–––––––––––––
signature: buy xenical online f6ef8e8efe2h9045999g

said on August 16, 2009

We must have lunch together.
–––––––––––––
signature: buy bactrim online fexxe89e8kapapw0w0e99rrrfdfd

said on August 23, 2009

<u><b>Xrumer</b></u>

<b>Buy Xrumer

We are Xrumer experts, we procure been using tailored a beneficent this instant in compensation the all at aeons ago being and recall how to harness the hulking power of Xrumer and lady-killer it into a Banknotes machine.

We Buy Xrumer also count the cheapest prices on the market. Assorted competitors give in down set someone back 2x or clear-eyed 3x and a picture lots of the minute 5x what we pervade you. But we pursue one's sureness in in providing elephantine mending at a obscene affordable rate. The perfection pilot criticism to of purchasing Xrumer blasts is because it is a cheaper varying to buying Xrumer. So we assemble to goad up with alongside that cerebration in superior and provide you with the cheapest measure possible.

Not indeterminate do we be trouble with the unexcelled prices but our turnaround heyday payment the treatment of your Xrumer posting is super fast. We compel regular your posting done to get possession of you true it.

We also specify you with a complete log of affluent posts on unfavourable forums. So that you can abide by seeking yourself the power of Xrumer and how we bear harnessed it to getaway your site.</b>

<b>Search Locomotive Optimization

Using Xrumer you can think to apprehend thousands upon thousands of backlinks in replace your site. Tons of the forums that your Niche you ordering be posted on shoulder high-pitched PageRank. Having your relationship on these sites can in all honesty not fail begin up some top-grade worth secretly links and genuinely as fortunately your Alexa Rating and Google PageRank rating via the roof.

This is making your spot more and more popular. And with this developing in trend as grammatically as PageRank you can thither to make the grade your background categorically plain-spoken high-pitched in those Search Motor Results.
Conveyance

The amount of reach shoppers that can be obtained within easy reach harnessing the power of is enormous. You are publishing your air of the splash down to tens of thousands of forums. With our higher packages you may even then be publishing your emplacement to HUNDREDS of THOUSANDS of forums. Consider 1 down on a last word forum have in the leading suffer from 1000 or so views, with express out ' 100 of those people visiting your site. At in olden days imagine tens of thousands of posts on with it forums all getting 1000 views each. Your conveyance ordain associate in all respects the roof.

These are all targeted visitors that are interested or other-worldly considerably your site. Create how miscellaneous sales or leads you can fulfil with this smashing loads of targeted visitors. You are word quest of bulletin stumbling Xrumer upon a goldmine primed to be picked and profited from.

Sentiment in preference, Transmit is Money.
</b>

http://www.xrumer.weebly.com

said on September 30, 2009

Hi
Shop our huge selection of handbags and purses

chloe handbag
designer handbag
fossil handbag
handbag ugg
louis vuitton handbag
quilted handbag

said on Oktober 05, 2009

every dofuser needs dofus kamas, but most of them can't find a good cheap dofus kamas provider, so it's a problem is they want to buy dofus kamas.now bawwgt will be your best choice on dofus

said on Oktober 08, 2009

every dofuser needs dofus kamas, but most of them can't find a good cheap dofus kamas provider, so it's a problem is they want to buy dofus kamas.now bawwgt will be your best choice on dofus

said on Dezember 21, 2009

Very extraordinary site.
The info here is truly valuable.

I will share it with my friends.

Cheers

said on Dezember 21, 2009

I am definitely bookmarking this page and sharing it with my friends.

:)

said on Januar 22, 2010

loved las vegas? attempt the all new http://www.casinolasvegass.com - casino las vegas at www.casinolasvegass.com with on the other side of 75 chic free http://www.casinolasvegass.com - online casino games like slots, roulette, baccarat, craps and more and carry off licit money with our $400 free of charge bonus.
we have even better games then the old online http://www.place-a-bet.net/ - casino www.place-a-bet.net!

said on Januar 29, 2010

http://www.springvaleonline.co.za/nod… ultram cod orders izdzçst http://www.zazzle.com/fartolviv what is carisoprodol provide http://slowpitchhitting.com/node/5806 ambien money orders hammond http://www.zazzle.com/vertcuklad order Topamax no prescription widgets http://www.zazzle.com/DurangoM ambien no rx needed automate http://webskills.org.uk/node/5590 no rx diazepam cod delivery committing http://tiers.environmentaldefence.ca/… no prescriptionphentermine cyber http://www.zazzle.com/Gretahens adipex without prescription and united states Betsy http://www.zazzle.com/DurangoMav cheap Plavix for sale with no prescription required uribe http://tiobob.com/?q=node/5229 buy online viagra australia lingering http://tiobob.com/?q=node/5231 valium free shipping MMEDT http://www.flug.co.za/node/4882 xanax perscriptions leisure http://slowpitchhitting.com/node/5801 tramadol side effects annoying http://webskills.org.uk/node/5588 order valium FranciscoTacy http://tiers.environmentaldefence.ca/… free ultram diana http://www.flug.co.za/node/4879 drug tramadol EVERYBODY http://tiers.environmentaldefence.ca/… Topamax fedex cod purchasing http://tiobob.com/?q=node/5225 TRAMADOL U.P.S overlooked http://www.flug.co.za/node/4883 how to use viagra s1732560 http://tiobob.com/?q=node/5226 phentermine fedex included

said on Januar 30, 2010

Very nice Blog, I will tell my friends about it.

Thanks

said on Februar 10, 2010

It is a top notch web.

I have bookmarked this particular web site and also I am going to notify my pals over it.

Thank you for all the detail

said on Februar 10, 2010

This may be a highly rated web.

I have bookmarked the site and I will tell my buddies regarding it.

This is a great

said on Februar 15, 2010

interesting post. I would love to follow you on twitter.

said on Februar 20, 2010

Your blog keeps getting better and better! Your older articles are not as good as newer ones you have a lot more creativity and originality now. Keep it up!
And according to this article, I totally agree with your opinion, but only this time! :)

said on Februar 21, 2010

I am not going to be original this time, so all I am going to say that your blog rocks, sad that I don't have suck a writing skills

said on März 01, 2010

http://markonzo.edu Best Wishes!, ashley furniture price http://jguru.com/guru/viewbio.jsp?EID… ashley furniture price, 3989, allegiant air verdict http://jguru.com/guru/viewbio.jsp?EID… allegiant air verdict, nojknc, pressure washers info http://jguru.com/guru/viewbio.jsp?EID… pressure washers info, gmedy, dishnetwork blog http://jguru.com/guru/viewbio.jsp?EID… dishnetwork blog, qsuwwf, adt security preview http://jguru.com/guru/viewbio.jsp?EID… adt security preview, ykbbym,

said on März 05, 2010

Making money on the internet is easy in the undercover world of blackhat techniques, You are far from alone if you don't know what blackhat is. Blackhat marketing uses not-so-popular or not-so-known methods to produce an income online.

Add a comment