Rift Experiment #2: Binocular Vision

Once I had some basic AR working with the rift, I did a lot of research online and found that there were only a handful of teams out there that had published anything about their success or failure with merging the Rift and AR toolkits. A Korean team had it working in editor, and had the background convergence working, so I looked into their solution, but I couldn't get the same setup working, they weren't explicit in their documentation, and google translation of their websites wasn't perfect, so some language barrier too. A Japanese team had taken the free CV library and converted parts of it to accomodate the webcam behavior they needed, they didn't publish any videos of their work, so I'm guessing it worked in editor, but unsure if their solution would be ideal for me, nonetheless, porting over the CV code was very impressive. OVRVision, I think is a spinoff of this effort. It is a 2 camera rig, which you can buy for about $150 which includes Unity plugins and Dlls etc to get the binocular vision working in Unity with 2 webcams each capturing their own stream for use with the rift. Two other awesome sources I followe dup on were "Will Steptoe" who is a PhD student at University College London, and head of their computer vision lab. He managed to build a really cool 2 camera rig, he explained the cameras he used and why in great detail. What sets his work apart, is that he's using a mo-cap lab and suit to record his movement and coordinate this with what is going on with the rift and AR. He was able to use the suit to generate basic interfaces, as well as position spawned objects (like webpages) anywhere in his lab. Really slick stuff. Lastly I found a hack solution that worked with Vuforia webcam. If you search stereoscopic hack on the vuforia forums, you'll find his code with emulates a 2 camera rig using only one stream (which is all Vuforia natively supports). This seemed to work, but convergence was not working in the Rift with background live feed, so not very useful.

The image here is the tuscany demo with some of my AR user interface stuff ported over, notice the background stream is now binocular, convergence still not working though with the background feed, however 3D objects have awesome convergence.