Friday, June 10, 2016

 

GETTING STARTED WITH THE MICROSOFT HOLOLENS


GETTING STARTED WITH
THE MICROSOFT HOLOLENS



CONTENTS OF THIS ARTICLE:

INTRODUCTION
WHAT IS A HOLOGRAM?
WHAT CAN THE HOLOLENS DO?
SUGGESTIONS
COMPETITION
POTENTIAL APPLICATIONS FOR THE HOLOLENS
LINKS
HOLOLENS SPECS


INTRODUCTION

I have begun using and developing for the HoloLens. In this article I review my impressions of the HoloLens and provide links to extensive resources, both for suggesting the kind of Apps for which the HoloLens is uniquely appropriate, and links to training, documentation, and other aids to developing HoloLens applications.

It is convenient to separate the functions of the HoloLens into five parts:
·        Display,
·        Sensors, data capture, and content creation,
·        A wearable computer,
·        External interactions and
·        Mixing of real world and computer generated images.

Although not usually considered when talking about the HoloLens, these functions are somewhat independent and could be realized, to an extent, separate from each other.

The display allows you to see and hear the real world you are in, while adding computer generated audio, text and images (called “holograms”) that seem to be in the real world with you.

The HoloLens contains distance sensors and video and audio capture devices that can sense, capture and record the real space around you and save videos that include the real world together with the holograms you perceive looking through the HoloLens; it can use that information to create content.

The computer that controls the HoloLens is in the HoloLens. So you wear it! It controls the display, monitors the sensors, and stores data in real time. 

The HoloLens can communicate externally with Bluetooth and Wi-Fi, or via a USB cable to other hardware, and also allows communication between devices, and allows different HoloLenses to share their view of holograms.

It can mix holograms with the real world and allows interaction between the real and computer generated worlds.

It is the fact that the holograms act as if they are in the real world that the system is called Mixed Reality:
Because there is a continuum of applications and hardware devices amongst these types of applications, I use the term eXtended Reality (XR) to refer to all of them, as well as other “Realities”...

XR = VR + AR + MR + ….


The most important features that are most unique to the HoloLens are:

The HoloLens
Photo by Eric Roffman
QPORIT


WHAT IS A HOLOGRAM?

A hologram, as the term is used by Microsoft, has a meaning somewhat different from the “classic” meaning of the term.

The “Classic” definition of a hologram:

A hologram is a photographic recording of a light field, rather than of an image formed by a lens, and it is used to display a fully three-dimensional image of the holographed subject, which is seen without the aid of special glasses or other intermediate optics. 


A (“Microsoftic”) hologram:

A 2D or 3D image seen with special glasses which can be controlled and animated by a computer, so that it seems as if it were in the same actual room as the viewer.






A hologram shot directly by my HoloLens.
Video by Eric Roffman
QPORIT




WHAT CAN THE HOLOLENS DO?


THE HOLOLENS...




SUGGESTIONS

The current version of the HoloLens hardware, and the current software (OS, development software, Apps…) is being provided to developers only.  It is not yet ready for consumers. Before making it available to consumers, the hardware and software needs to be improved.  Here are some possible features that would make it easier, more useful, and a better value for consumers.

It could be improved:

o   Making it softer around the head and the nose
o   Making it easier to adjust
o   Making it easier for people to use eyeglasses, including switching easily between near glasses, computer glasses, distance glasses, and no glasses
o   Changing the location of the USB cable, which tends to get in the way and also fall out;
o   Including, especially, holographic and non-holographic apps that extend the use cases for the on-board computer;
o   Making it 3D,
o   Making it wide angle (even panoramic),
o   Making it able to record holograms ready to use in other applications
o   Making it easy to stream live (3D, panoramic) video,
o   Increasing the target video hosting sites to which it can be easily shared,
o   Making it easy to create “selfies” (for example, by providing software to flip a mirror image)
o   Computing power
o   Visual resolution
o   Audio quality
o   Sensory devices;


At the present time, the HoloLens is a very specialized device. The general value of Microsoft’s contributions and development of XR would be enhanced with a complete vision for developing software and hardware to support the XR industry.

In addition to partnering with other companies to use MS holographic computing (which has already begun), for example, it would be useful to improve the resolution, the size of the memory, and access to the distance measuring features of the HoloLens so it could be used for very accurate and very large scale measuring and recording of physical spaces. (Note: The ability to scan, measure, and record spaces requires high security to prevent inadvertent disclosure of personal or proprietary information embedded in the spatial information.)

Most important, MS should extend the hardware and software offerings from MR to all of XR, including, for example, a tight linkage to VR, IoT, AI and especially machine intelligence, which can power many of the most important applications, including navigation, and any application in which real world vision is linked to computer assisted processing for searching, identifying and providing relevant information.

If, for example, the Mixed Reality Capture was able to capture high-res panoramic 3D videos and audio, and holographic videos (as opposed to 2D, limited FOV, normal res videos), it would put the HoloLens directly in the center of XR capture technology; and if the display could obscure (as well as view) the real world, it would allow VR as well as MR and put the HoloLens directly in the center of XR display technology as well;

I am suggesting that MS put the HoloLens at the very center of an eXtended Reality ecosystem, rather than leaving it as an outlier.


COMPETITION

At the present time, there is really no competition (for Mixed Reality applications) to the HoloLens. Competition is most likely to come in the future from:
Note that integrating real-time video and distance measurements from remote cameras & sensors and integrating that with computer generated content into a VR device is essentially virtual teleportation; add the ability to integrate that with viewing the local environment as well, and it becomes what MS calls “holoportation.”


POTENTIAL APPLICATIONS FOR THE HOLOLENS

First, some comments:

Note that applications may be designed for...


Notice that, because the computer is specialized (and head-mounted), finding generalized uses for the computer (thus amortizing the cost) may be difficult, and many industrial applications of the HoloLens will need to justify the whole cost of the HoloLens for a particular application. 

Types of applications…

The Wikipedia page for HoloLens (see link below) lists some applications that have been announced or demonstrated, including, in particular, a collaboration with NASA.

(And of course I have plans for my own HoloLens applications, particularly in gaming, entertainment, education, advertising and reporting.)

The most needed applications for all of eXtended Reality (not just the HoloLens), might be applications that facilitate content creation, including editing for panoramic and 3D video, and editing and creating XR within an XR application; and applications that facilitate navigation and creating and reading textual content within XR. (To read text, one prerequisite may be higher display resolution.) 


CONCLUSION

The kinds of applications for which the HoloLens is not only uniquely qualified, but is actually currently unique in being able to do them, include those for which:

  • The user needs a powerful onboard computer;
  • The user needs to see the real world, while getting additional information, especially information that is generated by what is in the real world (like identifying information about a building you pass on a trip);
  • The user needs to create an interaction between real people and computer generated objects or characters;
  • The user needs to wear a camera; especially a camera which is capable of filming the real world and the computer generated world together;
  • The user needs to share information about some remote object, while seeing additional information about the object or seeing the real world.


All in all, the bottom line is that there is still work to be done on both HoloLens hardware and holographic software and, because it will be expensive, the applications must be valuable enough to justify the cost of purchase.

But, most important, the HoloLens is potentially a tremendously valuable device, with many important, even vital, use cases and possible applications, and it is REAL: development on the actual HoloLens as well as the HoloLens emulator is now possible.




LINKS


MAIN HOLOLENS SITES

HOLOLENS OFFICIAL SITE

HOLOLENS OFFICIAL DEVELOPER SITE

HOLOLENS DOCUMENTATION

HOLOLENS FORUMS

FAQ

MICROSOFT HOLOLENS CHANNEL ON YOUTUBE

HOLOLENS ON TWITTER:
@HOLOLENS

HOLOLENS ON FACEBOOK

HOLOLENS EVENTS


TO GET THE HOLOLENS AT THE MICROSOFT STORE



HOLOLENS TUTORIALS, TRAINING & EXAMPLES

GETTING STARTED

THE HOLOGRAPHIC ACADEMY

HOLOGRAMS 100 – GETTING STARTED WITH UNITY

HOLOGRAMS 101 – INTRODUCTION TO THE HOLOLENS

HOLOGRAMS 101E – INTRODUCTION TO THE EMULATOR

HOLOGRAMS 210 – GAZE

HOLOGRAMS 211 – GESTURE

HOLOGRAMS 212 – VOICE

HOLOGRAMS 220 – SPATIAL SOUND

HOLOGRAMS 230 – SPATIAL MAPPING

HOLOGRAMS 240 – SHARING HOLOGRAMS


HOLOLENS DEVELOPER DOCUMENTATION

GETTING STARTED

INSTALL

HOLOLENS SHELL OVERVIEW

MAPPING THE ROOM YOU ARE IN

EMULATOR

DEVELOPMENT OVERVIEW

UNITY DEVELOPER OVERVIEW

DIRECT X DEVELOPMENT OVERVIEW


INFORMATION ABOUT THE HOLOLENS …

HOLOLENS ON WIKIPEDIA


… & HOLOGRAMS (classic definition!)

HOLOGRAMS ON WIKIPEDIA


APPS & IDEAS FOR DEVELOPING HOLOLENS APPLICATIONS

HOLOLENS APPS

DESCRIPTION OF THE FIRST APPS

THE GALAXY EXPLORER APP (with code on GitHub)

HOLOPORTATION


HEALTH & SAFETY, LICENSE TERMS

HEALTH AND SAFETY

MICROSOFT STORE TERMS OF USE

WINDOWS INSIDER PROGRAM

MICROSOFT STORE LICENSE TERMS


RELATED STORIES & VIDEO IN QPORIT

EXTENDED REALITY XR = VR + AR + MR + …

QPORIT’S VAMP:ire XRG – OUR NEW XR PRODUCTION GROUP

QPORIT’S DAILY XR NEWSLETTER

VIDEO: MY FIRST HOLOGRAM [*]

[*] Note: this video was shot by the HoloLens through the HoloLens. It shows what I was seeing. The cube is stationary in space, but as I moved – side to side and up and down – I am looking at the hologram from different angles. The video also shows the effect of gestures and voice commands.

The hologram was created in Visual Studio using …

Templates / Visual C# / Windows / Universal / Holographic


THIS STORY...



HOLOLENS HARDWARE SPECIFICATIONS

THE HOLOLENS HARDWARE (at the link below, click on HARDWARE DETAILS)

OPTICS
See-through holographic lenses (waveguides)
2 HD 16:9 light engines
Automatic pupillary distance calibration
Holographic Resolution: 2.3M total light points
Holographic Density: >2.5k radiants (light points per radian) See-through holographic lenses (waveguides)
2 HD 16:9 light engines
Automatic pupillary distance calibration
Holographic Resolution: 2.3M total light points
Holographic Density: >2.5k radiants (light points per radian)

SENSORS
1 IMU
4 environment understanding cameras
1 depth camera
1 2MP photo / HD video camera
Mixed reality capture
4 microphones
1 ambient light sensor

INTERACTIONS AND HUMAN UNDERSTANDING
Spatial sound
Gaze tracking
Gesture input
Voice support

INPUT/OUTPUT CONNECTIVITY
Built-in speakers
Audio 3.5mm jack
Volume up/down
Brightness up/down
Power button
Battery status LEDs
Wi-Fi 802.11ac
Micro-USB 2.0
Cable

MEMORY
64GB Flash
2GB RAM (2GB CPU and 1GB HPU)

POWER
Battery Life
•         2-3 hours of active use
•         Up to 2 weeks of standby time
•         Fully functional when charging
Passively cooled (no fans)

WEIGHT
579g

WHAT YOU NEED TO DEVELOP
Windows 10 PC able to run Visual Studio 2015 and Unity 5.4





Labels: , , , , , , , , ,


This page is powered by Blogger. Isn't yours?

TWITTER UPDATES:

    follow me on Twitter


    QPORIT --
    Quick PREVIEWS Of Random Interesting Things

     
    (c) Copyright 2005-2009 Eric H. Roffman
    All rights reserved