First AR demo with 2 Camera Rig, Working!

Got a working AR build of a simple basketball game. In terms of Usability, this thing is miles from where it needs to be, but it works and it looks good!

I solved the projection matrix issue, it was a Metaio thing, I needed to fool it into adding its projection matrix to a second camera, and it worked great, now I have awesome convergence for both the background and 3D objects. The leap motion stuff now works pretty well too, can pick up and drop the ball no problem. I solved the issue of having 3D objects in both views properly, and of generating AR content in both views as well. The two background streams in this pic are from different sources (different cameras), but I also have different methods for displaying them. I needed to downsample the Metaio stream to 640 x 800 so it matched the other stream, but I think I can upres once I figure out how to set my own webcam resolution.

I built an exe version of this, but it's no small feat to use it: You need a LEAP Motion, Oculus Rift and 2 Webcams. I'll post this up on my website in a little bit. http://crevver6.wix.com/chriswrenar/

This version is using 3D instant tracking, so the court and hoop etc are anchored to that pile of stuff next to my computer. The second image (with all the white lines) is of me capturing the 3D tracking data.