Immersive Design in Real Life

For quite a long time, all my immersive designs were only concepts, not related to any real product. Thanks to my full-time job, I didn’t have to worry about paying my bills and jumping on every gig. It gave me much more freedom to learn and explore useful, exciting, and interesting things. During the first three years, I had a few gigs, but they were mostly about augmented reality interactions, which are less challenging than VR/MR.

Everything changed during spring 2020, when I left my position as lead designer at the company where I spent >4 years and started my journey as a consultant. I know it’s not the best timing :) My laser-targeted focus is helping me to keep myself +100% busy. I’m helping with everything related to interactions, prototyping, and design in general for immersive projects (AR, VR, MR).

Here are my takeaways on how working on concepts differs from real products.

Responsibility and impact

It was a long time ago but I still remember freelancing and running my studio. I’d often receive requests like “Everything is done, just beautify it”; I wasn’t happy with the freedom and trust that I had. This time, everything is different for me (at least for now). Suppose someone is searching for an MR prototyper (I’m still not sure what to call this position); they know what they’re lacking and what expertise they’re looking for. Things get even more interesting when a serious company doesn’t have much experience with immersive fields and they’re looking for someone to help them build their first product for other realities.

It gives me a chance to have an impact on future products, but at the same time I feel a lot of responsibility. So, if something sucks eventually, it’s clear who fucked up.

My advice here is to be as open as possible. If you’re not confident, it’s ok to say “I’m not sure, but I have a few options, let’s test them” or “I don’t know, let me explore” and so on.

Delivery

I noticed that UI/UX designers are quite often confused about what their delivery for MR/VR should look like. And my answer is:

It can be whatever you want.

Yeah, in the regular UI there are clear rules on how you should deliver your assets or present prototypes. But there are no Figmas or even Zeplins for XR.

Once I was sharing design files in PSD format with sorted and renamed layers with a client and then discovered that the devs never opened them and were just measuring the JPGs that I attached as a preview. And now, I understand that there wasn’t an issue with tools, formats, or delivery methods. The issue was with our communication. Sometimes it’s enough to send a few messages on slack or call to agree on the whole delivery strategy.

There are many options to choose from for sharing your ideas, designs, and specs. Most of them are suited to specific situations, while others are more widely used. Starting with the simplest:

1.Sketch. Most time/effort effective, but sometimes it’s enough to make sure that everyone is on the same page as early as possible. Quite often, I do storyboards, sometimes 360 panoramas.

2. Mockups. Yeah, ordinary mockups. Probably you shouldn’t worry about scale and grid, and just use a real photo in the background, if it’s MR or AR, to give a sense of scale. I like building flows and organizing them in a deck so that the whole flow can be clicked through. Sometimes such a storyboard is not enough. If I know that someone will have to figure out what’s going on without a call or any intro, I record a screencast of me talking through all the steps and giving details about interaction. It doesn’t take a lot of time but helps to reduce misunderstandings and false assumptions.

3. Video. Sometimes using AE to make a video that explains a flow is overkill, but it pays off for me most of the time. It’s not always easy to connect slides into a flow, but the video makes everything smooth and clear. Also, it helps uncover UX issues.

4. Unity prototype. It’s by far the best way to test new unique interactions. As the saying goes: it’s better to feel once than see a hundred times.

5. 3D files — I expected to use 3D files more often. Anyway, from time to time, it’s necessary to create a quick model for a prototype or set of assets for final delivery, so it’s good to have one of the 3D modeling tools under your belt.

6. Specs — As usual, I use one of the 2d design tools (Photoshop, Figma, Sketch) to show every detail from different perspectives. It depends on the type of project, but it’s always a good idea to keep some real values for scale.

7. Texts — I write a lot. For each mockup, slide, or prototype, I explain details, thoughts, and potential issues. I’ve got so much into writing docs that I’m a kind of Jira evangelist in my team in Verizon Media. Never saw this coming; I hated documentation, writing, and especially Jira.

Hardware

It’s extremely difficult to design without having access to headsets. Imagine designing for iPhone, if you’d never tested any smartphone, even a touchscreen phone. The best that you have ever seen is your neighbor’s black and white Nokia. It’s the same if you attempt to design for MR headset, but have only tested 3DOF VR headset.

It’s not crucial to buy every headset out there. Sometimes clients will send you hardware, and sometimes it’s enough to borrow to test.

Do your homework

The field is growing impressively fast. Almost every week there is news about some collaborations, hardware improvements, or new releases. It’s important to stay updated, and before diving into work, figure out all the details of the hardware that you’re dealing with.

I often intentionally neglected many technical limitations while I created my concepts. It helped me to experiment more. With real projects, everything should be realistic to implement. Just being aware of how Unity or ARkit works gives you the advantage of designing for a specific platform. Also, if you’re not sure, feel free to ask what’s possible before committing to a particular interaction approach.

Continuous process

Creating concepts is something that you’ve completed, gained insights into, shared, and forgotten. Working as a contractor requires longer involvement, as the first release is only the beginning, :) I always like playing the long game. And quite often, while designing, I keep in mind how this could evolve. It’s a good thing to ask the client what their plans are for the near and distant future. Most probably, it won’t end up like this, but you can be ready.

Experiment

And the most important is not to be afraid of experimenting. The field is still quite new, and there aren’t many established processes, so feel free to experiment, try new ways to build and deliver, and of course, don’t forget to share your findings with the rest of the community.

‍