©2022 All rights reserved
Extracting data from VR is cool. Making it easy record, manage, and view even cooler.
My progress so far:
1. Built a simple 2048 puzzle game and added simple analytics:
- 2048 game on Sidequest
- Google Spreadsheet with data
2. Built quantitate test for finding optimal poke button size.
- To be approved by Sidequest
- Google Spreadsheet with data
When Oculus mentioned how they do quantitive data analysis to inform their UI decisions, I expected soon to do the same with a native or third-party solution, but nothing was coming.
I imagined it as something like SDK, which tracks specific events in VR and helps extract value from this data. It was (and still is) a complex problem to tackle on my own, so I kept it in the back of my mind.
Also, during my career, I've been facing a different problem, which might have a similar solution. It's not easy to measure detailed statistics of VR experience. Stores, as usual, focus more on general numbers, like downloads or time spent. But there is no way to track specific actions and, of course, no A/B tests.
Google Analytics and other tools for the web became monsters. They gather all necessary, unnecessary and useless data, leaving users unaware of how it will be used. In VR, it's even riskier. It can get creepy quite fast. So, I decided to make this research entirely public, including my assumptions, tests, results and even all data I collect.
Worse for bad research is unfinished research. I've planned to spend only a couple of weekends and evenings here and there. I knew that to complete the project in a meaningful timeframe, I had to keep the scope lean.
Also, I was sure that for A/B tests and quantitative analysis, I don't need to have many metrics; as usual, just a couple is more than enough to build data-backed statements.
I'm using Unity for prototypes, but everything that I found was a way too complex. So I decided to avoid any overblown APIs and craft something lightweight.
Also, I wanted to easily view my data, so storing it in the database and building a viewer sounded feasible but not the most flexible solution.
So, I came back to basics. After playing with Airtable, I found a solution on how to send data from VR experiences using Google Forms straight into Google Spreadsheets. It didn't only solve my problem of storing data but also helped me to visualize it optimally using graphs from Spreadsheets.
As the first step, I wanted to find out if it works at all, but I needed users. Obviously, I decided to procrastinate and build an addictive game. Over the weekend, I had a copy of the 2048 puzzle game, but with blackjack and hookers more tactile feeling of rotating the game board instead of swiping.
I decided to don't even try to save any user's personal data, even under encrypted identifiers, and simply save score, number of game, time of game and number of moves at the end of every game. In this way, I could see how many people engaged with the game.
BTW you can download this game from Sidequest.
After you win, lose or reset the game, you will find your results on this public Spreadsheet.
In the next step, I want to build a couple of alternative interactions to control the game and run AB tests to see which is better.
For now, the plan is to add hand tracking as a less restrictive option and joystick as a more solid option. Let me know if there is anything else you would want to improve or test (Actually, I know that there is a lot to improve, but don't forget it's just a solution to test the hypothesis :))
In this step, I want to build some simple experience that will help find optimal properties of UI elements. For example, to find out the optimal size and distance to the button to be selected with raycast, I would build an experience where buttons of various sizes would appear on various distances, and with enough data points, it should become clear when the error rate starts rising and time between presses doesn't improve. Of course, it's an oversimplification. But it should be enough to determine if this solution makes sense, and it could be used to test the hypothesis quantitatively, not only by Oculus but also smaller studios and indie devs.
I don't really know where it would lead me. If it works out, the most probably I'll try to reuse some of the learnings in the projects I'm working on.
Even when you throw away all technical difficulties, there are a lot of ethical questions to be answered. Like, what records are ok to export and save or what the consent form should look like.
By doing it in public, I expect, in addition to practical experience, to get feedback from the community and maybe contribute to the public perception of VR analytics tools.
Let me know what do you think about this exploration over email. Follow me on Twitter or sign up for my email list to keep getting updates.
©2022 All rights reserved