Augmented Reality and Windows Phone 7

August 24, 2010 — 1 Comment

What is augmented reality?  Here is the augmented reality definition from Wikipedia:

Augmented reality (AR) is a term for a live direct or indirect view of a physical real-world environment whose elements are augmented by virtual computer-generated imagery. It is related to a more general concept called mediated reality in which a view of reality is modified (possibly even diminished rather than augmented) by a computer. As a result, the technology functions by enhancing one’s current perception of reality.

Here are some videos that I’m sure would make things a little more clearer than the definition above.

Augmented Reality Shooter Game

Rock ‘Em Sock ‘Em Robots (Courtesy Anthony Bartolo)

What About Windows Phone 7_

Implementing an augmented reality application on Windows Phone 7 should be possible because all the appropriate hardware will be included on all device such as camera, compass, accelerometer, powerful CPU, GPU & GPS.  The only thing it won’t have is a gyroscope but who knows that could change.

Only problem from a developer point of view is there are no APIs available to access most of this hardware.  You do have access to the camera via the Launchers & Choosers APIs available on Windows Phone 7.  Unfortunately this is very limited access via the CameraCaptureTask API.

You also have access to GPS via the System.Device.Location API and the current API implementation should suffice.  By default if you are writing an app you get access to the CPU & GPU (something needs to run your code 🙂

So what are we missing?  APIs to the compass and APIs to the accelerometer.  Thankfully all Windows Phone 7 hardware will support a compass and accelerometer.  You also need better API access to the camera like getting the raw frames from the camera directly into your application.

But Wait! There is hope!

I’m expecting all the above issues to be a ‘v1’ issue with Windows Phone 7.  Let’s face it, Microsoft needs to get this phone out the door fast as the competition is way ahead.

Where does the hope come from?  Well check out this picture from Justin Angel from his twitter account after he did a little hacking (I assume):


and his screen capture of the potential API


He posted this back in July 18 2010.  Then today (Aug 24 2010) I was taking a peak at the new XNA Creators Club and the new education roadmap for creating games on Windows Phone 7 using XNA.  Due out in October-November 2010 timeframe is Phase 3 of the educational series and at the bottom I noticed “Bonus: Augmented Reality Sample”


So in the end, we may just get access to the appropriate APIs and augmented reality on Windows Phone 7 may just be a reality for developers. So, since October – November 2010 is essentially ‘Holiday Season 2010’ (I’m already starting to see Christmas trees in stores) maybe the Windows Phone Developer team will give us a nice surprise at launch. Imagine apps like FourSquare or MLS or XNA Games on Windows Phone 7 using augmented reality! I for one can’t wait!

If it’s available, do you plan on using it?  Share your feedback here or let me know via my twitter account

One response to Augmented Reality and Windows Phone 7

  1. Pretty sweet, Mark. I had pretty much written this off for WP7 until I saw your post. Thanks!

Leave a Reply to James Ashley Cancel reply