Measuring Indie MEGABOOTH’s Success & Impact – Part 1

Academic1
/

After each showcase, we send out a post-mortem survey to the teams that were featured. We use the responses to identify our weaknesses, and to help us develop methods to improve upon them. When a team of academics offered to run a study on Indie MEGABOOTH, our events, and our alumni, we jumped at the chance to gain a new perspective on our work to learn if -and how- it is effecting change within the industry at large. 

In this first part of a two-part introduction, we learn which countries teams traveled from for PAX Prime 2015, their goals for the show, and other demographics.– Christopher Floyd

An independent academic study’s preliminary findings

As you already know, Indie MEGABOOTH was formed in 2012. Its mission was simple: to create a critical mass of indie devs that could band together to collectively bargain for prime expo floor space at events such as PAX, Gamescom and GDC – space that previously was the sole domain of affluent corporate players such as Activision and EA.

In a few short years, the MEGABOOTH has widely expanded its mandate. It’s a community hub where indie developers come together, sharing knowledge, resources, war stories and networks. While presenting a ‘vertical slice’ of the best of the game industry, there are parallel motivations to showcase devs who represent the diversity of indies. So, how has the Indie MEGABOOTH done so far?

In terms of raw growth, the Indie MEGABOOTH is a success – the number of fans going through PAX Prime, (approximately 80k), the number of Indie MEGACAST episodes showcasing the best of indie games (over 100 at time of press), the number of events that Indie MEGABOOTH hosts (6 this past year, across 3 continents), and the number of Indie MEGABOOTH alumni (over 300). But Kelly and Christopher wanted to measure something more complex: Is the Indie MEGABOOTH making the game industry a better place to work? Is it promoting diversity in games? Does being part of the Indie MEGABOOTH alumni family measurably create more sustainable careers, leading to creative and economic success?

That’s where we come in. Kelly and Christopher asked me, Dr. Jen Whitson (a sociologist at the University of Waterloo), Dr. Bart Simon (a sociologist at Concordia University) and Dr. Felan Parker (a postdoctoral fellow at Concordia University), to start empirically answering these questions. Felan and I began collecting some baseline data at PAX Prime 2015 in August. We surveyed developers about who they were, what they get out of the Indie MEGABOOTH, and how well their games have done. 61 of the teams (85%) volunteered to contribute anonymous data in our survey. To provide some further detail and context, we conducted in-depth interviews with 19 devs, volunteers, and Indie MEGABOOTH staff, and chatted with many others including PAX enforcers and attendees.

It’s clear that devs are just as interested in finding out these answers as Kelly and Christopher. Online post-mortems may give us a sense of what’s happening in the industry, but it’s hard to tell whether these are special cases or representative. So we’re writing a few blog posts sharing what we’ve found so far. Caution: this is only preliminary, baseline data. Our real goal is to chart long-term trends in support networks, economic sustainability, and diversity in the indie space. The most interesting findings will only come from comparing data from multiple events, but what we found at PAX Prime 2015 is still worth sharing. Today, I’ll outline some demographic data. In the next post, Felan will talk about economics and sustainability, linking our findings to current claims and counter-claims about the so-called ‘indiepocalypse’. We’ve also brought a statistician on board who will explain some of the more complex patterns (for example, How does where you live and your education, race, and gender, correlate to your success? How does government funding correlate to success?). So, on to the numbers!

Who exhibited at PAX Prime 2015?

Thirty-five respondents (57%) were presenting at Indie MEGABOOTH for the first time, and for nineteen (31%), this was their first ever PAX. This reflects the fact that the MEGABOOTH is not a closed network; each year a significant number of new developers are being added to the MEGABOOTH alumni network.

In the follow-up question (represented in the above pie-chart), a matching 31.1% were exhibiting their first ever game ever. The remainder were showing an already-launched game (36.1%) or were showing a pre-launch game, but had launched games in the past (32.8%).

All respondents  were either full-time developers (78.7%) or aspiring to be. While we cannot make claims as to why amateur, ‘moonlighting’ or hobbyist developers aren’t presenting at PAX Prime, we suspect the substantial financial cost of exhibiting and the degree of professionalism needed to put together an application package filter out all but the most committed applicants.

There were a fair number of far-travelling exhibitors, including teams from Australasia (5), Europe and the UK (6), and elsewhere  – e.g. Japan and South America (2). However most exhibitors (74%) were North American. Not surprisingly, 39% were ‘local’ from the western United States. The average age of respondents was 30, with a range of 22-49 and a median age of 28.

Demographically, those who filled out the survey mirror the demographics of the game industry at large. Of those that responded to this question (59/61 response rate), 93% respondents were male and 7% were female, meaning that women and non-binary individuals were under-represented. In terms of respondents’ reported race (56/61 response rate), 82% self-identify as white, while the remaining 18% identify as Mixed, Asian, Filipino, Black, Indian, or Hispanic.

These numbers come with a caveat. Only one member from each team, usually in a leadership role, filled out the survey, resulting in a potential selection bias. Programmers and bizdev may be over-represented at PAX compared to other roles such as artists. Thus these numbers may reflect the demographics of those roles, but not the composition of the larger teams. To improve data for future years, we will be collecting anonymous demographic data for whole teams. In general, though, our data seems to reflect the demographics of the game industry at large, and it’s clear that more concerted efforts (such as travel support for under-represented groups) need to be made to represent a wider demographic of developers.

It is well-timed that, for PAX Prime 2015, Indie MEGABOOTH partnered with Intel to cover the wholesale showfloor costs of a select number of teams who came from underrepresented demographics, an initiative that remained heretofore unpublicized to avoid the risk of any such teams receiving unwelcome attention. It is worth mentioning that this initiative will remain in operation for the coming year.

What do Exhibitors hope to get from the Indie MEGABOOTH?

JennTable2

As expected, exhibitors applied to the MEGABOOTH for a variety of reasons, but promotion and exposure are paramount (97%). Building connections within the indie community network is another key motivation (74%), as is engaging directly with players: building community (77%) and playtesting (64%).

In an open-ended question we asked respondents what their ideal outcome of PAX Prime was. For most, it was related to building larger networks with fans, influential YouTubers and streamers, press, other indie devs, as well as the mainstream industry. This means rather than coming to PAX with a singular goal or focus (e.g. talking to press only), exhibitors were running a gauntlet for 4 straight days, chasing several different forms of exposure, and wearing multiple hats to pack in as much as possible. Unsurprisingly, ‘not getting sick’ was also a common goal.

These are some representative answers to the question ‘What is the ideal outcome of PAX Prime 2015 for you?’:

Foot in the door as a ‘real’ indie dev (respect in the community, dev contacts, mainstream game contacts)‘ (respondent 18)

Gain new fans. Make more industry + media contacts. Get an article featuring the game. Playtest the build to find design flaws.‘ (respondent 61)

Ideally we’d get 5-10 pieces of good press. Meet some cool devs which we form long-term friendships with. Sell a bunch of games or get installs because people liked what they played. Eat some good food. Don’t get sick. Get some swag. Have FUN!!!‘ (respondent 13)

The wide range of different answers we received for this question points to the increasingly complex role played by the Indie MEGABOOTH. Rather than simply securing floor space for cheap, MEGABOOTH is valued for creating and maintaining a carefully balanced ecosystem of devs, business contacts (investors, publishers, platform holders, service providers, etc.), media contacts (press, YouTubers and streamers) and, of course, fans.

In the next blogpost, Felan will talk about money. How are MEGABOOTH exhibitors funding their game’s production? Where are they going for investment? How are they doing in terms of copies sold, revenues, and recouping development costs?  How satisfied are they with their revenue numbers?  And, as a teaser, what is the annual household income for the average Indie MEGABOOTH exhibitor?

Stay tuned, as I suspect some of these answers will surprise you.

About the Author

Jennifer R. Whitson is an Assistant Professor in the Department of Sociology & Legal Studies at the University of Waterloo. She mostly works with game devs, is on the board of the university's Games Institute, and is a Research Advisor for Execution Labs, an accelerator for game studios. Her research centres on the shifting production models of the global game industry, and tracing how risk management practices, datamining, and digital distribution shape developers' creative work and the larger cultural role of games (i.e. how do indies manage to survive ?!?). A very occasional expert blogger for Gamasutra, you can dig up most of her writing and talks online.

  • David Van Brink

    Interesting summary, looking forward to next part. (And of course keenly interested since I just recently applied for the first time; if accepted I’ll widen your age range a smidge. :-) )

    • Jen Whitson

      I’m guessing the age range will widen once we start to collect data on all the other events IMB runs. Good luck with the application. :)

  • Tim Cullings

    Really good read. I’ve been informally polling devs I know who have either been in the IMB or are aspiring to get in on these very topics and I’m interested to see what a formal study turns up though I think I can make some educated guesses at this point.

    • Jen Whitson

      Thanks Tim! We’d actually be very interested to to hear your informal results. You may have questions we should add the survey. And, I’d personally love to see if your educated guesses match up with what other people have noted in the in-depth interviews we’ve also done.

  • Min Tsai

    Looking forward to to the follow up articles. I’ve attended a few festivals and most don’t actually collect postmortem results. It would be interesting if the researchers can also provide a guide to how to collect and analyze this information. This would be useful to help organizer of other events to improve in the future.