Wednesday, November 18, 2015
A WORKING OUTLINE OF EXTENDED REALITY
A WORKING OUTLINE OF EXTENDED REALITY
This is an outline of the technology (and some of the content) of XR. (XR = eXtended Reality = VR + AR + MR + GR + …)
(Note: "GR" is a term I thought of a few seconds ago to designate "GIFTing Reality": possible holiday presents for anyone interested in VR and the other eXtended Realities. Please scroll down to the bottom of this outline for a special section on VIRTUAL REALITY HOLIDAY GIFT possibilities.)
We will be writing about these outline topics in QPORIT, with resource guides and reviews, for those new to XR as well as the experts. The articles are designed to provide an overview of the current status, and a way of looking into the future. We hope to make it easier to discover products and services, and find the resources needed to develop XR.
XR computing is the biggest advance in computer tech since the development of the personal computer.
The recent rapid development of head mounted audio and video displays, together with new sensors, including head mounted cameras, wearable and other IoT sensing devices, plus improvements in AI, faster wireless data transfers, and the continuing effect of Moore’s Law with processing units ever smaller and more powerful, make possible, now and through the next few years, dramatic breakthroughs in the applications of computing.
To understand the enormous scope of XR, here is a small table of some possible variations. It can be
Ø Isolated from the real world (VR) or integrated with it (AR, MR)
Ø Limited field of view (FOV) or full sphere panoramic 4π (360x180) FOV
Ø 2D or 3D
Ø Still image or video
Ø Linear or interactive
Ø Single user or collaborative
Ø Made to involve sight, sound, touch, balance, motion, position,,,
Ø Live, recorded (image or video), digitally generated, or a mixture
Ø Fictional, factional, or abstract
Ø About almost anything.
Below is a working outline of XR. We will be writing detailed stories about specific products, and surveying categories of products. Some products fall into more than one category. This outline is not complete! The outline will continue to change!
This is a working outline... not quite a first draft. There may be (aka are!) typos, spelling, formatting and other errors. Many things (categories, developers, products, services, companies, etc) are missing!
MOST IMPORTANT: If you have a product, production, or service which is covered here, please let us know, so we can include it with the related stories. Equally important, if your personal favorite categories are not represented, please let us know, so we can include them. We appreciate your input!
PLEASE SEND COMMENTS, CORRECTIONS, ADDITIONS, PRESS INFO, ETC TO
QPORIT@GMAIL.COM
SPECIAL TOPICS
FUTURE OF COMPUTING
SENSORY PERCEPTION
WHAT IS XR? VR? AR? MR? EYEWEARE?…
RECENT STORIES IN QPORIT
XR - EXTENDED REALITY (XR = VR + AR + MR + ...)
SOME VARIATIONS ON REALITY ...
LINKS TO ADDITIONAL RECENT STORIES ON XR
OUTLINE
Note: This story contains many links. In this era of rampant hacking, it is always possible for even a well established company to have compromised content. New sites may not be vetted. Beware and be vigilant! Use a powerful malware filter whenever scanning websites or downloading or accessing external content, especially code.
In this outline little, if any, distinction is made between products, services, etc, that actually exist (many do not), and products that are in beta, pre-beta, alpha, rumor, or fantasy stage.
The objective is to follow this outline (as it develops) with more detailed stories about particular products, projects, services, categories, and people – in reviews, previews, interviews, and my personal views … -- using words, photos, and 2D 3D and XR videos.
MAJOR COMPANIES WITH (COMPETING!) DEVICE / LANGUAGE SPECIFICATIONS
Facebook – Oculus
Google – Cardboard, Project Tango, Magic Leap, Android
Microsoft – HoloLens, Windows 10
SONY – PlayStation VR
OpenSource – OSVR
HTC – Vive
…
What's missing?
Apple has a few relevant patents, but no announced projects. IBM has some relevant research. Yahoo has not made significant noise, nor has Nintendo. Cisco is active in networking, of course, and IoT. Intel has 3D sensing devices as well as fundamental chips. Amazon has AWS cloud services and retail, but not currently a VR or AR device. Some companies are getting into XR by creating devices for the Cardboard spec.…
What's missing?
Apple has a few relevant patents, but no announced projects. IBM has some relevant research. Yahoo has not made significant noise, nor has Nintendo. Cisco is active in networking, of course, and IoT. Intel has 3D sensing devices as well as fundamental chips. Amazon has AWS cloud services and retail, but not currently a VR or AR device. Some companies are getting into XR by creating devices for the Cardboard spec.…
HARDWARE
AYE!WEARE
THE IDEAL! EYE!WEARE
INPUTS to the system
OUTPUT to the user, to remote devices, storage, other users….
FUTURE EYEGLASSES
…
VR HEAD MOUNTED DISPLAYS (HMD)
Oculus Rift
https://www.oculus.com/en-us/
Oculus/Samsung Gear VR
https://www.oculus.com/en-us/
OSVR
http://www.osvr.org/
Cardboard
Cardboard compatible hardware
…
Project Tango (mounted on another device)
https://www.google.com/atap/project-tango/
HTC Vive
http://www.htcvr.com/
PlayStation VR
https://www.playstation.com/en-gb/explore/ps4/features/playstation-vr/
AR
Google Glass (v2?)
MR
HoloLens
https://www.microsoft.com/microsoft-hololens/en-us
Magic Leap
http://www.magicleap.com/#/home
Oculus Rift with camera
Gear VR with pass-through camera
Cardboard with pass-through camera
BRAINWAVE
…
CONTROLLERS
HANDS & DISTANCE SENSORS
Oculus Touch
Leap Motion
https://www.leapmotion.com/
Xbox controller
…
BODY AND MOTION
VirZoom
VOICE
Cortana (MS)
http://windows.microsoft.com/en-us/windows-10/getstarted-what-is-cortana
Google Now
…
DISTANCE SENSORS
Intel
Project Tango
HTC Vive
Oculus
Kinect
Hololens
Plus stereo cameras, most hand controllers, etc
OTHER SENSORS
Visual
Audio
Touch
Body Motion
Mental
Biometric
Environmental
…other
CAMERAS
VIDEO
PANORAMIC 3D
Jaunt
http://www.jauntvr.com/#about
Samsung Beyond
Google GoPro Jump
https://www.google.com/get/cardboard/jump/
Nokia Ozo
https://ozo.nokia.com/
PANORAMIC 2D
PixPro 360
MOTION CAPTURE
Salto
http://salto3d.com/
STILL PANORAMIC 3D
Matterport
http://matterport.com/
EMBEDDED
Most HMD’s will (or at least can) have 3D video cameras and distance sensors embedded
SOFTWARE
MIDDLEWARE
Unity
http://unity3d.com/
Unreal
WebVR
Havok (for WebVR)
CryEngine (for WebVR)
http://www.crytek.com/cryengine
…
OS & LANGUAGES
Android
http://developer.android.com/index.html
Windows 10, C, C#, C++, DirectX 12
Java
https://www.java.com/en/
JavaScript
Three.js
OpenGL
WebGL
…
VIDEO EDITING &/|| PANORAMIC STITCHING
Premiere Pro CC
http://www.adobe.com/products/premiere.html
Magix
Kolor
KcPano
…
READING & WATCHING
BOOKS
FICTION
READY PLAYER ONE
NON-FICTION
HOW-TO
OCULUS RIFT IN ACTION
UNITY VIRTUAL REALITY PROJECTS
LEARNING VIRTUAL REALITY
STRATEGY
…
NEWS
WEBSITES
QPORIT
www.qporit.com
NEWSLETTERS
QPORIT’S Augmented & Virtual Reality Daily
http://paper.li/QPORIT/1433372865
TWEETS
https://twitter.com/search?q=virtual%20reality&src=typd
@qporit
@AltspaceVR
@ARVRNews
@comogard
@ConvrgeVR
@GearVR_us
@jema_vr
@KaleidoVR
@virtrealitytime
@oculus
@official_gdc
@otoy
@riftcentral
@Riftmax
@RtoVR
@stanfordvr
@unity3d
@UploadVR
@virtrealitytime
@VRChatNet
@vreducation
@Vrideo
@vrlosangeles
@vrreporter
@wearvr
CLASSIC MOVIES & VIDEO RELATED TO VR
“LAWNMOWER MAN”
“INCEPTION”
“CAPRICA”
“AVATAR”
…
XR CONTENT & CONTENT DEVELOPERS
TARGET HMD PLATFORMS
CURRENT VR
GEAR VR
CARDBOARD
TANGO
OSVR
IN DEVELOPMENT VR MR
OCULUS RIFT
HOLOLENS
PLAYSTATION VR
MAGIC LEAP
HTC VIVE
GENRES
TYPE
Generated by software
Photographic/Video
Mixed (Digital + Environment)
FORMAT
Full spherical (360x180 = 4 ) or partial (eg hemisphere, 360 band)
2D or 3D
Still image or moving video
CONTENT FORMS, TOPICS, APPLICATIONS
Linear (Drama, Comedy, Documentary …)
“Intimate Strangers”
Interactive (Games, Puzzles …)
Transmedia
Story + Data or Interactivity
Live + Digital
“Sleep No More” (testing VR)
Scientific visualization
Scientific research
Instructional
Remote communication
Teleconferencing
Collaboration
Social spaces
Art
There are applications in virtually every industry
…
HOSTING
YouTube
Panoramic videos
Netflix on Gear VR
Movies (2D, standard aspect ratio, only)
Play Store
Apps for Cardboard and Gear VR
Oculus Store
Apps
Panoramic 3D still images
Panoramic 3D videos
Samsung Theaters & Milk VR
User loaded 3D panoramic images and videos
Im360 Theater on GearVR
User loaded 3D panoramic images and videos
YouVisit
WearVR (supporting many VR formats)
http:/www.wearVR.com
EEVO
https://eevo.com/
…
STREAMING
NextVR
im360
im360.immersivemedia.com
…
SOCIAL SPACES
AltspaceVR
http://altvr.com/
JanusVR
Converge
http://www.convrge.co/
Riftmax
http://www.riftmax.com/
VR chat
http://www.vrchat.net/
…
DEVELOPERS
vrse
http://vrse.com/
Samsung
Oculus Story Studio
https://storystudio.oculus.com/en-us/
IM360
http://im360.immersivemedia.com/watch/im360/
NextVR
http://www.nextvr.com/
Google Spotlight Stories
YouVisit
http://www.youvisit.com/
Almost every Hollywood movie studio is testing and experimenting with XR
Almost every Hollywood and FX house is testing and experimenting with XR.
Almost every game development studio is testing and experimenting with XR
SERVICES
RENDERING
Otoy
https://home.otoy.com/
LEASING RIGS
Jaunt
http://www.jauntvr.com/#about
STREAMING
Im360
http://im360.immersivemedia.com/watch/im360/
NextVR
Stream VR
https://streamvr.com
https://streamvr.com
CLOUD STORAGE AND PROCESSING
MS Azure
Amazon AWS
Google
…
GENERAL RETAIL
Amazon
B & H
Best Buy
…
PROFESSIONAL
ACCOUNTING
STARTUP BUSINESS MANAGEMENT
PR AGENCIES
AD AGENCIES
LEGAL SERVICES
Entrepreneurial and business
IP (Patent, Trademark, Privacy …)
Liability
Entertainment Law
MEETUPS AND MEETINGS
CONFERENCES
CES
January 6-9, 2016
https://www.cesweb.org/
SUNDANCE FILM FESTIVAL (VR?)
SUNDANCE FILM FESTIVAL (VR?)
January 21-31, 2016
VISION SUMMIT 2016
February 10-11, 2016
http://visionsummit2016.com/
GDC VR
March 14-15, 2016
TRIBECA FILM FESTIVAL (STORYSCAPES)
April 13-24, 2016
https://tribecafilm.com/festival/submissions
SXSW
March 11-18, 2016
E3
June 14-16, 2016
https://www.e3expo.com/
SIGGRAPH
July 24-28, 2016
http://s2016.siggraph.org/
NEW YORK FILM FESTIVAL (CONVERGENCE)
September 23- …, 2016
September 23- …, 2016
https://www.filmlinc.org
DEVELOPER CONFERENCES
MS BUILD
GOOGLE I/O
OCULUS CONNECT
CONTESTS / HACKATHONS / OPEN SOURCE COLLABORATIONS
MEETUPS
NY
…
LA
http://www.meetup.com/find/?allMeetups=false&keywords=virtual+reality&radius=50&userFreeform=los+angeles&gcResults=Los+Angeles%2C+CA%2C+USA%3AUS%3ACalifornia%3ALos+Angeles+County%3ALos+Angeles%3Anull%3Anull%3A34.0522342%3A-118.2436849&change=yes&sort=default&eventFilter=mysugg
…
SF
http://www.meetup.com/find/?allMeetups=false&keywords=virtual+reality&radius=50&userFreeform=san+francisco&gcResults=San+Francisco%2C+CA%2C+USA%3AUS%3ACalifornia%3ASan+Francisco+County%3ASan+Francisco%3Anull%3Anull%3A37.7749295%3A-122.41941550000001&change=yes&sort=default&eventFilter=mysugg …
…
JOBS, INVESTORS, INCUBATORS …
JOBS
TYPES OF JOBS
Coding
Unity
3D Artists
Experts in 3D and panoramic imaging
Project Producers
User Interface Designers
Game Designers
3D Panoramic Filmmakers, editors, cinematographers, etc
Hardware Engineers
PR, Marketing and Sales
Trainers and Evangelists
Indie/Entrepreneurs/Self-employed
Legal & Accounting
Startup business support
Actors and models (improv, green-screen, or motion capture experience a +)
Lots of similar and related jobs also in IoT, AI, etc
…
COMPANIES THAT MAY BE HIRING
Most companies with active XR development projects
Most industries “experimenting” with XR
Hollywood studios
Hollywood FX studios
Game companies
Medicine
Space, Astronomy
Science research
Advertising
Fashion
Travel
Cars and other vehicles
Real Estate
News
Publishing
…
RECRUITERS
…
FUNDING
CROWDFUNDING
Kickstarter
IndieGogo
INVESTORS
Rothenberg Ventures
http://rothenbergventures.com/
Formation 8
http://formation8.com//
Comcast Ventures
http://www.comcastventures.com/
Time Warner Investments
http://www.timewarner.com/company/time-warner-investments
Andreesson Horowitz
…
COMPANIES WITH SIGNIFICANT REPORTED VENTURE FUNDING
Magic Leap
NextVR
…
INCUBATORS & ACCELERATORS
Rothenberg Ventures River
http://rothenbergventures.com/riverriver.com/
COURSES AND RESEARCH
ACADEMIC
Stanford Virtual Human Interaction Lab
NYU
INDUSTRIAL
GOOGLE ATAP
OCULUS
IBM RESEARCH
http://www.research.ibm.com/
MS HOLOLENS
SAMSUNG
http://thinktankteam.info/
http://samsungaccelerator.com/
http://samsungaccelerator.com/
IDEAL TRAINING FOR NEW XR HARDWARE & SOFTWARE
LIVE & LOCAL!
Conferences & hands-on demos
Classes & tutorials
Meetups with experts, developers, and users
ONLINE
VIDEO
Classes
Tutorials
Demos and samples in action
CODE
Samples
NEWS
Blogs
Reviews
Previews
INTERVIEWS
With users, developers and other experts
PRINTED (& ONLINE) EXPLANATORY & TUTORIAL
Well written specs
Well written reference manuals
Well written user manuals
Well written best practices manuals
Well written books
HELP & SUPPORT
Chat line
Phone desk
Monitored forums
Known bug list
Evangelist visits
RELATED TOPICS
AI - ARTIFICIAL INTELLIGENCE
LARGE DATA ANALYSIS
IMAGE AND SPEECH RECOGNITION AND ANALYSIS
EMERGENT BEHAVIOR
AL - ARTIFICIAL LIFE
ROBOTICS
IoT – INTERNET OF THINGS
AND ALSO
GAME DESIGN AND DEVELOPMENT
FILM AND THEATER TECH, INCLUDING DIRECTING AND ACTING
JOURNALISM
SCIENCE RESEARCH AND SCIENTIFIC VISUALIZATION
ALMOST EVERYTHING ELSE
…
***
ENJOY THE HOLIDAYS!
GR
** GIFTING REALITY **
HOLIDAY GIFTS
VIDEOS
Lawnmower Man is an early film about a Virtual Reality project that has inspired many people to become interested in Virtual Reality, and it is remarkably prescient.
Inception and Avatar suggest Virtual Reality experiences, and Caprica is a fascinating TV show in which a Virtual Reality social space is very important.
HER is a terrific film about Virtual Personalities becoming part of our lives.
I think Avatar, (but only) in IMAX and 3D, is one of the greatest movies ever made and one of the most important. I would love to see it rendered to be viewed in panoramic 3D VR on an Oculus Rift!
The Lawnmower Man
Inception (Blu-ray)
CAPRICA
Caprica: Season 1.0
AVATAR
Avatar
HER
Her
BOOKS
Here are three good books to learn how to create Virtual Reality projects.
Since the development software (Unity or WebVR) is free (though eventually if you make lot$ of $$ with Unity there is a cost) and Google Cardboard can be very inexpensive in cost (but it is recommended to get one of the better models!), there is almost no $$ barrier to starting development of real applications. Gear VR is less than $100.00 (a penny less not counting tax or shipping, but if you do not have a late model Samsung phone that is an extra cost). Gear VR is a step up from the best Cardboard HMD's (Head Mounted Displays), but an easy path is to start on Cardboard. Then the evolution to Gear VR is simple.
The Oculus Rift is several levels up from Gear VR, but it will not be available until 2016. But it is wise, if you wish to develop for the Rift, to start now developing for Gear VR..
Other HMD's including Microsoft's HoloLens, which combines generated text and images with the real world, will also begin to ship -- at least to developers -- in 2016, and they recommend Unity as a preferred development system. So beginning development now with Cardboard and Gear VR as a first target is a good way to develop skills.
LEARNING VIRTUAL REALITY
Learning Virtual Reality: Developing Immersive Experiences and Applications for Desktop, Web, and Mobile
UNITY VIRTUAL REALITY PROJECTS
Unity Virtual Reality Projects
OCULUS RIFT IN ACTION
Oculus Rift in Action
"Ready Player One" is a novel that, like the movie Lawnmower Man, has inspired many people to develop an interest in Virtual Reality.
Ready Player One: A Novel
HARDWARE
GEAR VR
To experience VR as either a user or developer, it is necessary, of course, to get some hardware. The best hardware available now is Gear VR.
The new GEAR VR HMD (Head Mounted Display) is now available -- for orders for a penny less than $100.00.
UPDATE: As of Nov 28 2015 orders already taken are shipping from Amazon. For new orders, they are currently "out of stock". According to Samsung (http://www.samsung.com/us/explore/gear-vr/)
an order on the Samsung site "Ships in 3 - 4 weeks. NOTE: Product may NOT arrive by 12/25/15".
.
Compared to the initial version, the new unit is:
- Lighter weight
- More comfortable
- With improved fit, including room for most eyeglasses and improved padding for extra comfort and durability
- With an upgraded touch pad that’s easy to use
- And cheaper.
NOTE: It is compatible (only) with
- Galaxy Note5
- Galaxy S6 edge+
- Galaxy S6
- Galaxy S6 edge
(also, a software update from your carrier is required for S6 edge+ and Note5)
The GEAR VR is actually the only real VR device currently available. (More about Google CARDBOARD below.) Other devices will debut next year.
I have found the first version of GEAR VR to be comfortable and rather fun and pleasant to use. This new version should be even better.
There is interesting content available now. including 3D full sphere dramas. There is a lot of content experimenting with the possibilities of VR. Movies from NETFLIX are available. Some news and entertainment content is being streamed live by NextVR. Right now, the most stunning content is a collection of 3D 360 photos created by OTOY.
All you need to do is put the GEAR VR on and select content from the OCULUS store, which pops up automatically. Most content is currently free. In addition to content on the OCULUS store, there will be more and more VR apps that can be downloaded from other sites onto your SAMSUNG smartphone and then immediately viewed on the GEAR VR.
More and more content is becoming available for the GEAR VR every day. 3D video is going to get real good very soon with 4096x4096 full sphere 3D videos coming to MILK VR.
Anyone planning to develop for XR should absolutely get the GEAR VR and experiment with it. (Note: XR = VR, AR, MR or any other R. I give the whole bunch the single nickname XR - for eXtended Reality; and X is also often used for the unknown, which I think is appropriate.)
If you are developing for VR, content -- including 3D interactive apps, and 2D, 3D and spherical photos and videos -- can all be directly loaded from your computer to your smartphone and viewed on the GEAR VR. (You may need developer access to your phone to load the content.) App content can be created in either native OCULUS/ANDROID code, or using UNITY. Photos and video can be just standard photos and videos, created by an ordinary camera with an ordinary editor. But with special cameras and editors, it is possible to create full spherical 3D video content yourself.
CARDBOARD
Cardboard is an entry level version of Virtual Reality. There are many possibilities for hardware and for applications. But Cardboard is simply a specification, and it does not specify quality, so there are many implementations of Cardboard HMD's and some -- especially those actually made of cardboard -- are not always very good. I suggest a sturdy plastic version.
CARDBOARD PRODUCTS
ZEISS VR ONE CARDBOARD COMPATIBLE
The Zeiss VR One should be one of the better Cardboard compatible HMD's. The device itself actually may cost more than the Gear VR, but it may be compatible with your current smart phone. Note that it may need a specific tray for your particular smart phone, so be sure to check the specifications for any one you purchase.
ZEISS VR ONE ON TUMBLR
http://zeissvrone.tumblr.com/
***
ENJOY THE HOLIDAYS!
ENJOY THE HOLIDAYS!
Labels: AR, Augmented Reality, Cardboard, eXtended Reality, Gear VR, Hololens, Mixed Reality, MR, Oculus, Virtual Reality, VR, working outline, XR