“This morning’s announcements, deep integration with hardware software and services, it’s something only Apple can do. Today we’re going to make some of our biggest announcements at WWDC. As well as introducing some exciting new products.”Tim Cook, CEO, Apple
Over the past two decades, Apple Inc. has released many industry-shifting products: the iPod in 2001, the iPhone in 2007, the iPad in 2010, and the Apple Watch in 2015. Yesterday the company hoped to do it again — with its first mixed-reality headset. While no product will ever live up to the iPhone, the headset has the potential to usher in a new era: It could kick off the shift to a different interface that upends how people work, play games, and entertain themselves.
The headset announcement was part of Apple’s keynote address at its Worldwide Developers Conference, and it was a long presentation — more than two hours — to jam everything in. Beyond the headset, Apple covered various changes to its software platforms, and the company rolled out other new hardware, such as fresh Macs. So, here’s a summary of what was covered, along with some thoughts about how each development might play in healthcare.
Apple unveils 15-inch MacBook Air – Apple just unveiled the 15-inch MacBook Air. It’s 11.5mm thick, just over 3 pounds, and powered by Apple’s own M2 chipset. The video demo showcased a standard headphone jack, two USB-C ports, and Apple’s MagSafe charging dock. The laptop will ship in four colors, including Midnight and Starlight. The laptop features an 18-hour battery with an 8-core CPU and 10-core GPU. It costs $1,299, can be ordered today, and will be available next week.
Apple unveils new $6,999 Mac Pro with M2 Ultra – It’ll feature the M2 Ultra’s 24-core CPU, support up to a 76-core GPU, eight Thunderbolt 4 ports, and up to six of Apple’s high-end Pro Display XDR. Both products can be ordered today for availability next week.
Apple announces iOS 17 with a ton of new features – Apple is unveiling updates to three apps, Phone, Facetime, and Messages. Some highlights:
- Personalized contact posters, either using photos or Memoji, will allow users to choose their contact card for other users. Live transcription, in real-time, will also be coming for phone voicemails.
- Facetime will finally get a voicemail functionality, allowing users to leave a video message for their friends.
- And messaging will be revamped with more powerful search functionality and a catch-up feature for group chats. You can also see the location of your friends while messaging them.
- New stickers live in a “drawer,” with expanded options, including emoji expansion and rotatable optionality.
- Live photos can also be made into an animated sticker.
- A Check-in feature lets you tell a family member or friend when you’re home. Or, if you’re delayed, it’ll tell your friend that, too
- Messaging apps will now have a different layout, expanding to take up the entire screen.
- Users can now bump two iPhones or Apple Watches together to share contacts, music, internet, or other shared activities with each other, a feature called “NameDrop.”
What might this mean for healthcare? – Consider the opportunity to share a person’s health care record using the Name Drop feature. Or the check-in feature being used to make sure that a patient who needs ongoing care is safe at home and doing well.
Apple announces new iPadOS 17 software for iPad – Some highlights:
- Multiple timers for iPad, at last. “We truly live in an age of wonders,” Apple executive Craig Federighi said.
- The Health app has been added to the iPad as well.
- Machine-learning tech for PDFs to identify fields even if they’re not built into the PDF.
- Expanded functionality for iPad’s widgets, including enhanced interactivity for third-party developers. Apple showcased functionality for Quizlet.
- New customization options for the iPad lock screen, similar to how iPhone users can customize their lock screens. Weather, photo shuffle, and emojis are all part of the offering.
- Live Activity, a la iOS, which creates a widget-like functionality for users to track scores, delivery orders, or other developer-customizable, continually updating information.
Apple announces macOS Sonoma for Macs – so, the mystery about the name is finally over. Here are some of the highlights:
- New Gaming functionality, taking advantage of Apple’s more powerful graphics processing units and Apple’s Metal 3 framework.
- They have expanded video-conferencing functionality through FaceTime. New overlay functions allow customers to appear on top or alongside their presentations.
- New AR functionality for full-screen reactions that will also be available in Zoom, Teams, and WebEx.
- Game Mode will prioritize the game’s processing power utilization, allowing more consistent framerates.
- Widgets are coming to the macOS on the desktop. They used to live in the Mac’s Notification Center. Like the iPad, they’ll be interactive and developer-enabled.
- Screensavers, like on tvOS, will be coming to macOS.
- Safari is getting some under-the-hood improvements and enhanced Private Browsing functionality.
Apple announces watchOS 10 for Apple Watch – The software will be available to all users in the Fall. Some highlights:
- Users can turn the digital crown to reveal widgets, like the weather and calendar.
- Users can long press to add a widget to their smart stack.
- Apple introduced new apps like world clock, which features clocks with background colors reflecting the time of day.
- When users rotate the digital crown, it shows new, full-screen displays.
- Two new watch faces are available, including a Snoopy and Woodstock watch face and a palate watch face.
- Users can access new active features, like cycling and hiking. Cycling workouts from the watch will show up on paired iPhone devices, and hikers will have access to new topography elevation details.
- Developers can also use new workout APIs so that users can start a workout from apps like Training Peaks, for example.
- In the health app, users can use screening tools and resources to support and evaluate their mental health.
- WatchOS 10 can measure the amount of time users spend in daylight, as well as how close users’ screens are to their eyes. Apple said these new features could help prevent conditions like myopia.
What might this mean for healthcare? – I find it interesting that the health app features are reviewed during the WatchOS presentation. Apple has announced a range of new health features coming to iOS 17, iPadOS 17, and watchOS 10, focusing on two key areas: mental health and vision health. Recognizing the importance of mental health, Apple has introduced new features that allow users to log their momentary emotions and daily moods and gain insights into their mental well-being. Through the updated Health app on iPhone, iPad, and Apple Watch, users can now engage in an interactive and intuitive reflection on their state of mind. By selecting from a range of multidimensional shapes and describing their emotions, users can develop emotional awareness and resilience. The digital crown lets you scroll through emotions to choose how you’re feeling, and you can even identify triggers that have made you feel this way.
Nearsightedness (Myopia) is the most common cause of visual impairment worldwide. which is estimated to affect more than 30 percent of the current population. And is expected to increase to 50 percent or about 5 billion people in 2050. care providers recommend a few critical behaviors in children. To help reduce the risk of myopia. Two of these behaviors include spending more time outdoors in bright light and increasing distance to see things such as equipment or books. The new screen distance feature uses the same TrueDepth camera that powers Face ID on iPad and iPhones to encourage users to move their devices further away after holding the device closer than 12 inches continuously. The Screen Distance feature can remind young users to practice healthy viewing habits, which can reduce the risk of myopia and offers adult users the opportunity to reduce the visual strain associated with digital devices.
Apple Vision Pro headset – By this point, we were close to two hours into the event, and everyone was holding their breath for the “big announcement.” Tim didn’t disappoint with his “one more thing” transition.
Apple’s headset is finally here, and it’s called Vision Pro. It’ll retail for $3,499 early next year on Apple’s website. Tons of detail was provided, and it was challenging to capture it all. But here are some of my notes:
- It is controlled by your eyes, hands, and voice.
- You are no longer limited by the display.
- The entire interface will feel “present” in your room.
- Shadows are used to help you understand scale.
- Apps can be made any scale in your space (room), just like moving natural objects.
- “Environments” transform your space to extend your room or allow you to immerse within a scenic environment.
- Tapping fingers to click and flicking to scroll looks amazing.
- Siri will work with Vision Pro.
- ‘EyeSight’ innovation allows others to see your eyes and tell if you are using an app or in an experience while the goggles are on.
- When someone is nearby, they will appear in your view.
- It’s in sync via iCloud to your other devices.
- Your entire world is a canvas for apps.
- You can send and receive 3D objects in iMessage and then interact with that object in 3D.
- Magic keyboard and trackpad allow you to type and browse in VR.
- Bring a MAC into vision pro, and then expand your view.
- FaceTime becomes Spatial. People appear in your room, life-sized, with spatial sound.
- You can use apps with other people in Vision Pro.
- Panoramic images appear all around you.
- Vision Pro is Apple’s First 3D camera.
- You can take a spatial video or photo with a button push while wearing the device. It allows you to essentially record real life and rewatch those events later on in VR.
-Spatial Cinema allows you to create a movie screen anywhere in your home and any size. It’s a movie theater with spatial auto anytime you want.
- You can use AirPods with the device so others can’t hear the movie you may be watching.
- 3D movies work on Vision Pro.
- You can play your favorite Apple Arcade games.
- Apple has teamed with Disney for some content on Vision Pro.
- Apple Vision Pro will work for people who wear glasses using special lenses.
- It will unlock via eye recognition. No more facial recognition required, as it will use a much more comprehensive eye recognition.
- The 23-million-pixel panel has more pixels than a 4K television per eye.
In their initial segmentation for the product, Apple presentations focused on work, home, and entertainment. Most of the applications that were said to be available at product delivery were presented in each of those segments.
Disney CEO Bob Iger also appeared to tout a partnership between Disney and Apple: enhancing the viewing experience for Disney’s treasure trove of content through Apple’s VR technology. For example, the demo included a three-dimensional visualization of a basketball court and an immersive National Geographic application that placed the viewer in the ocean.
Working with Unity, “hundreds of thousands” of iPad and iPhone apps will run on visionOS at launch. Microsoft’s Office Productivity suite will also work on visionOS.
What I found especially interesting was that Apple carefully avoided using the terms VR, AR, MR, and Metaverse in the presentation. They carefully described the Vision Pro as a “spatial computing” device in what I guess was an attempt at a product differentiation strategy.
Somewhat frustratingly (but also probably for the best experience), Vision Pro requires magnetically attached corrective lenses, for users with prescription lenses like me. Other headsets use physical spacers to accommodate glasses, but Vision Pro’s advanced eye-tracking capabilities help drive visionOS. That probably means eye glass reflectivity is a deal breaker for Vision Pro, thus the less convenient solution. Vision-correct accessories are sold separately. However, and this is a big but, Apple adds that “not all prescriptions are supported.” What does this mean? Hopefully, we’ll find out soon.
What might this mean for healthcare? – Building on the experience first made available on Apple Watch, the Mindfulness app on Vision Pro will allow users to create private and calm spaces to reflect, focus and center themselves in what is likely to be a game-changer.
I’m also interested in how the Vision Pro might be used in medical education. One of the images shown was an anatomy app that gave a short example of how the system might make studying anatomy a more personal and immersive experience.
You could also imagine the headset being used to train surgeons in complex operations and combining the simulation with expert proctoring by experts across the globe with AI assistants.
There’s no doubt that Vision Pro could be used in physical and occupational therapy too. XRHealth is one of the few companies focused on providing V.R. physical and occupational therapy at home; based in Boston, it is covered by many insurance programs in Massachusetts and nationally by Medicare. The company is working to get more insurance companies to cover its services. Without insurance, people can pay $179 monthly for the headsets and two physical or occupational therapy appointments monthly with a panel of therapists the company provides.
One of the most successful areas in healthcare has been post-traumatic stress disorder (the Veterans Health Administration was an early adopter) and has also been used to reduce pain and anxiety without medication.
Apple announces visionOS software platform for Vision Pro headset – Apple’s Vision Pro headset will run on visionOS, a brand-new platform for the Vision Pro headset. It’s a spatial computing platform that developers can build for, much as they would for iOS on iPhone or macOS for Mac. It will be able to run a multi-app, 3D engine, and Apple said it is the first operating system designed from the ground up for spatial computing.
The immediate aftermath – The crowd assembled at Apple Park was reported to be excited. Still, the reaction to Apple’s new augmented reality headset reveal was somewhat more muted than other announcements earlier in the day, such as the iPad’s ability to set multiple timers or the ability to locate a lost Apple TV Siri Remote. Apple stock dipped on Monday after it announced its highly-anticipated Vision Pro headset, falling into the red momentarily.
I think it’s important to remember that Apple isn’t positioning this first version as a “consumer” device. One thing is still certain, however. Apple clearly understands the shortcomings of AR right now, and the Vision Pro seems to dismantle the issues with the tech piece by piece. It’s insanely expensive, and it’s only for a select few early adopters. But from what we know right now, that group will likely be very pleased. My bet: they’ll sell as many as they can manufacture in 2024. I’ll wait until I can see one at the Apple Store and find out more about the prescription lens options before I consider purchasing one of these. But I’m impressed.