Click Nothing

Finally Caved In...

Finally, Jonathan Morin managed to find the time to put up his blog.

For those who don`t know the name, he was the Lead Level Designer on Far Cry 2 and we worked very closely together on that title for three and a half years. Maybe you know him from solid talks he gave at GDC 08 (which I can`t find on the GDC site, but info and a video is here) and GDC 09 (discussed by Sirlin, and Burch (who I am sure you heard a lot about today already)).

Anyway, I highly recommend you click your way over there and put some pressure on him to keep blogging his thoughts to get him over the 18 month hump I figure it takes to become a habitual blogger.

I don`t see his slides and presentation materials posted up there yet, but I can only assume he is planning on getting that up in the near future.

Welcome aboard Jon.

GDC09 - Part 3 - Wines and Rants

The last set of slides I have to put up are those for my five minute talk about game rating systems that I gave as part of Richard Lemarchand's Microtalks session on Thursday morning.

Overall I really enjoyed the session - though the prep was a nightmare. I guess it's pretty easy to enjoy a session where you only have to do 10% of the work and then you get to listen to smart people explain to you what exactly it is that you do for a living.

Anyway, the five minute format is colossally difficult. Especially if you are trying to make a point and support it with arguments instead of being all hand-wavy and theoretical. I wish I could have brought to bear all the arguments I have as for why we should switch from 10, 20, 100 or even 1000 point rating systems to a simple five point rating system, but I just didn't have time.

I guess that's what blogs are for.

Here are the arguments I made in the talk:

100 point rating systems are so analog that the human brain cannot make sense of the granularity (what is the difference between an 86 and an 87 in real perceptible, measurable terms?). These systems nobly attempt to make a threshold-less system - but when the brain is overloaded we add thresholds back in... we say '61-70 is above average' and '71-80 is good' or whatever. This is effectively adding back in the thresholds that a 100 point system endeavors to remove... but it is adding them back in in a less rigorous way leading to a mysticism created around certain improbable thresholds at the far reaches of the curve. I called this mysticism the Cult of 90+ and talked about it specifically in the context of wine rating.

I talked also about the relationship between the review scores, the reviewer, the publishers of the review and the developers and publishers making the games. There are pressures in these relationships that push reviewers to give higher scores. This is very easy to do in a 100 point system (because of the bracketed question in the previous paragraph), meaning a reviewer under pressure can relieve that pressure incrementally by inflating a score ever so slightly to make everyone involved a little bit happier. This systemic, distributed pressure release into the system of review scores is inflationary and increases game review scores over time.

In terms of inflation, I present the evidence of it by looking at the aggregated game ratings taken from gamerankings.com and I discuss why it is bad: principally because it prevents cross-generational comparison of games (in fact, this need for cross generational comparison means we should impose and maintain a bell curve in ratings even if games are getting empirically better over time).

As an aside: Adam Sessler ranted against a different set of problems inherent with the way aggregators work at this years Rant session. Adam is also right, in my opinion, and a global switch to a five star system would also work alleviate his complaint to some extent.

In any case, if you want the discussion on the above points, you can download the slides and the text of the talk here.

Also of note is that I used Kent Hudson's 'How To Pick a Lock' talk from GDC 2008 as a point in the discussion and I have had a couple people ask me about it. For those interested in that discussion, Kent's slides and video support can be found here. Thanks Kent.

Now - over and above the points raised in the talk there are a couple other arguments for why we ought to undertake a global switch to a five star rating system for games that I did not have time to cover in the talk itself.

Firstly, 100 point rating systems leverage the 'fallacy of precision' to lend authority to the accuracy of the rating (difference between accuracy and precision is here). Essentially the argument here is that if I claim a game is an 82, then most readers are likely to think 'well, it must be somewhere between an 80 and an 85'. In fact, this is a fundamentally invalid assumption. If I were to use a 10 point system and give the game an 8, readers would assume it to be 'probably between a 7 and a 9'. If I were to use a five star system and give a 4, they would assume it 'probably between a 3 and a 5'.

The problem is that the precision afforded by an 82 does not make the accuracy of the rating higher. It is as likely to be a 60 or a 100, just as a 4 star rating is probably somewhere between a 3 and a 5. Accuracy and Precision are orthogonal concepts, but the fallacy of precision leverages the fact that our brains tend to correlate them. In a the highly subjective field of rating games, it is much more important that we be accurate than that we be precise and someone who plays a lot of games can be accurate in a five star system.

To go one step further with this argument, I would point out that in a world full of aggregators, we still have 100 and 1000 point systems and having these systems aggregating five star ratings into percentages is useful and good (provided everyone uses a five star system and their aggregation policies are standardized, fair and well understood - cf Sessler). Having them aggregating ratings that are already claiming 100 points of precision makes the aggregators susceptible to the effects of the aforementioned inflation (in fact it is in the aggregators that we are able to measure the inflation).

For the record - this point was misquoted in IGN's article on the talk wherein they wrote:

According to Hocking ... 35 of the 100 current top-rated games (as tracked by an aggregate site) were released after 2001. That presents a skewed view of the industry's history and points toward a recent inflation of sorts in game reviews.

Note that an astute commenter then went on to reply this:

To say 35 of the top 100 rated games of all time came out since 2001 means that scores have been inflated since them is not consistent. Since 2001, there have been 8 years of games released. We can assume that almost all games with ratings have come out since around 1985, or 24 years ago. The past 8 years represents 1/3 of this time, and 35/100 is about 1/3 of the total game volume. It is consistent that ratings have NOT been inflated as 1/3 of the top rated games have come out in 1/3 of the time period of game releases.

The commenter is exactly right if 35 of the Top 100 rated games of all time were released since 2001. But the real quote was that 35 of the Top 41 highest rated games of all time were released since 2001. That`s not the 35% we would predict, that is 85%. Get it right IGN, you kind of invalidated the entire talk by erroneously reporting the most critical and central piece of data. Maybe it was the precision that threw you off?

I won't go into the other horribly broken arguments in the comments thread... I should know better... but here's my personal favorite:

Clint is a joke. Far Cry was overrated trash. The 100 point system is fine. It allows a more accurate score. Period.

That's like a straw-man multiplied by the denial of the fallacy of precision. Awesome stuff.

As a follow-on from the fallacy of precision point is the argument that the kinds of precision implied in 100 point systems feel elitist to 'ordinary' people. At a time when the industry is experiencing explosive growth - when new players of all demographics are starting to take up gaming - it serves our interest to give them reviews and rating scores they can understand. An 89.3 implies a level of refinement in perception among gamers and reviewers that not only does not exist, but is also intimidating for the mass market. It is the equivalent of terms like 'mature oak finish' and 'fruity nose' in wine discussion. These concepts are alienating for the market and make people feel excluded when - in the end - the entire purpose of game reviews and particularly of giving numerical scores at all is to help inform people about the product and invite them to play.

The final point I wanted to make was that - as members of a press with increasingly important social responsibilities - the reviewers, writers and editors giving the review scores should be the ones championing the push to a five star system. For all the reasons mentioned above, finer granularity systems are broken and compromise the integrity fo their work (and the utility of their work to developers). Chris Hecker also gave a mini rant this year and he called for the gaming press to do their job well. I believe the first step in doing that lies in radically reducing the importance and weight given to numerical scores and emphasizing the importance placed on quality reviews, well written.

I am pretty sure no matter what I say, this call is going to go unheeded... but I am also pretty sure that those who don't make the transition are going to find themselves very rapdily serving a smaller and smaller niche of an explosively growing market. I look very much forward to continuing to work with those who survive the transition.

GDC09 - Part 2 - Improvisation presentation materials

So I promised a lot of people I would have the slides from my talk up by Monday, but then I got lazy. But finally, a whole 24 hours late, here they are. you want a quick overview of what the talk was about, Chris Remo recapped it pretty clearly here.

Here's some usability tips for those who aren't aware:

Download the big zip, and open and print the word document (or have it on a second monitor) then run the powerpoint in presentation mode (or have it on your main monitor), then - every time you see a bold word in the text of the word doc, you advance the slides. This will keep all the slides and text in synch and give you the full effect of the presentation.

Just reading the presentation alone will be confusing without the slides, and the slides themselves will be totally meaningless without the text - it's really the synchronization of the two that you need if you hope to follow it.

Also - for reference, you might want the following:

My GDC 2006 Intentionality talk is here.
Harvey Smith's Orthogonal Unit Differentiation talk is here.
Harvey Smith and Randy Smith's talk on emergent gameplay design is here.

Thanks - obviously - to Harvey and Randy for continuing to host the slides and making all this great stuff available for freeloaders like me (and hopefully now you) to cash in on.

GDC09 - Part 1 - GDW and Family Dynamics

Here begins my long slow recap of GDC wherein I will ultimately get around to posting all of my various slides and presentation materials, as well as different commentary on the sessions I saw and what I thought was valuable about GDC this year.

First up is my quick recap of the Game Design Workshop tutorial. The GDW this year was the best yet, even eclipsing last years. It seemed a little smaller, but that might just be because we (ie: Mahk) were better organized and there were some structural changes that made it run more smoothly (I think I foolishly resisted making those changes last year... my bad).

It's clear to me (and not just from GDC) that the quality of students, new grads and recently-employed game developers is going up rapidly. I am pretty sure the rest of the GDW 'faculty' agrees with me on this. It is humbling - even a little bit intimidating - to see these smart new people rolling in and showing us that they have already learned much (or all) of the theory that underpins the workshop... good thing it's not a talky thing - but rather a practical, hands-on thing, or we'd be outta business (whether we all end up out of business in the broader sense remains to be seen).

 

Speaking of practical and hands-on, above is one of the groups that signed up for my elective 'Family Dynamics'. Family Dynamics is an exercise that asks each team to take a random graph which represents the relationship between two unnamed variables (ie: a mechanic) and a random word from a heavily loaded list of things related to families (ie: an aesthetic) and attempt to join the two using limited materials and their own design sensibilities.

The above team made a game about 'impotence' using a two-part graph that plotted (what they defined as) fertility and a progression through three-stages of emotional and physical arousal to try and explore how it might feel to struggle to conceive a child under the pressures that can build up in a family relationship over time.

It was an excruciatingly difficult exercise, I think, and not all of the teams made it through. The point is not really to succeed, though - it's to learn and I hope everybody did. Either way, whatever I come up with next year will likely be a bit less insane.

If you have any interest in reviewing the exercise materials, or in replicating the exercise yourself, feel free to download the parts I am able to make available, over on the left.

My GDC Schedule

Honestly, as much as this is gonna look like a plug for all these interesting talks - mostly what this is, is easy access for me so I can see what talks I am going to and when becuase this is the easiest way to make this stuff quickly accessible from my phone.

Those of you who consequently get an endorsement - hey lucky you :)

(Parties, lunches and other stuff not listed for my own protection against the hordes of hot chicks who stalk me relentlessly, but also so I can crash your party without you knowing it in advance... that's right - I'm talkin' to you Microsoft!)

Monday, March 23rd, 2009

Game Design Workshop
Full Day Tutorial – (Two Days)
Date/Time: Monday (March 23, 2009), 10:00am — 6:00pm
Location (room): 2020, 2022, 2024

Tuesday, March 24th, 2009

Game Design Workshop
Full Day Tutorial – (Two Days)
Date/Time:  Tuesday (March 24, 2009)   10:00am — 6:00pm
Location (room): 2020, 2022, 2024

IGDA Members Party
Date: Tuesday, March 24th
Time: 8pm to 1am
Location: Westin Hotel, Metropolitan ballroom, 50 Third Street, San Francisco

Wednesday, March 25th, 2009

Fault Tolerance: From Intentionality to Improvisation
Speaker: Clint Hocking (Creative Director, Ubisoft)
Date/Time: Wednesday (March 25, 2009)   10:30am — 11:30am
Location (room): Room 2002, West Hall

Breathing LIFE into an Open World
Speaker: Scott Phillips (Design Architect/Lead Designer, THQ\Volition)
Date/Time: Wednesday (March 25, 2009)   12:00pm — 1:00pm
Location (room): Room 2020, West Hall

IGDA VIP Luncheon
Date: Wednesday, March 25th
Time: 1:15pm to 2:15pm
Location: Westin Hotel (aka The Argent), Metropolitan Ballroom III, 50 Third Street

Making Friends is Hard: Social Mechanics in Contemporary Design
Speaker: Alex Hutchinson (Creative Director, Electronic Arts Montreal)
Date/Time: Wednesday (March 25, 2009)   2:30pm — 3:30pm
Location (room): Room 131, North Hall

alternate
The Game Design Challenge: My First Time
Speaker: Panel
Date/Time: Wednesday (March 25, 2009) 2:30pm — 3:30pm
Location (room): Room 135, North Hall

Meaning, Aesthetics, and User-Generated Content
Speaker: Chris Hecker (Technology Fellow, Maxis/Electronic Arts)
Date/Time: Wednesday (March 25, 2009)   4:00pm — 5:00pm
Location (room): Room 135, North Hall

Nominee Reception
Date/Time: Wednesday, March 25, 2009, 5:30pm
Locations: Esplanade Room, South Hall, Moscone Center, San Francisco

9th Annual Game Developers Choice Awards
Date/Time: Wednesday (March 25, 2009)   6:30pm — 8:30pm
Location (room): Esplanade Room

 
Thursday, March 26th, 2009

GDC Microtalks - One Hour, Ten Speakers, Unlimited Ideas
Speaker: Panel
Date/Time: Thursday (March 26, 2009)   9:00am — 10:00am
Location (room): Room 2007, West Hall

From CS to L4D: Creating Replayable Coop Experiences
Speaker: Michael Booth (Game Designer/Developer, Valve)
Date/Time: Thursday (March 26, 2009)   1:30pm — 2:30pm
Location (room): Room 2007, West Hall

alternate
Physical Play: for Interaction, Collaboration and Creativity
Speaker: David Merrill (MIT Media Lab)
Date/Time: Thursday (March 26, 2009)   1:30pm — 2:30pm
Location (room): Room 130, North Hall

Helping Your Players Feel Smart: Puzzles as User Interface
Speaker: Randy Smith (Game Designer, EA)
Date/Time: Thursday (March 26, 2009)   3:00pm — 4:00pm
Location (room): Room 2014, West Hall

Post Mortem: Mission Architect for City of Heroes
Speaker: Joe Morrissey (Senior Game Designer, NCsoft)
Date/Time: Thursday (March 26, 2009)   4:30pm — 5:30pm
Location (room): Room 3007, West Hall

 
Friday, March 27th, 2009

Game Writers' Round Table: Techniques, Tips, and Concerns
Speaker: Richard Dansky (Manager of Design/Writer, Red Storm Entertainment)
Date/Time: Friday (March 27, 2009)   9:00am — 10:00am
Location (room): Room 110, North Hall

Beyond Balancing: Using Five Elements of Failure Design to Enhance Player Experiences
Speaker: Jesper Juul (Lecturer / Researcher, Singapore-MIT GAMBIT Game Lab)
Date/Time: Friday (March 27, 2009)   10:30am — 10:50am
Location (room): Room 2022, West Hall

Paper Prototypes of SPORE
Speaker: Stone Librande (Game Designer, EA/Maxis)
Date/Time: Friday (March 27, 2009)   11:10am — 11:30am
Location (room): Room 2022, West Hall

The Human Play Machine
Speaker: Chaim Gingold (Designer, Levity lab)
Date/Time: Friday (March 27, 2009)   12:00pm — 1:00pm
Location (room): Room 2020, West Hall

alternate
Vast Narratives and Open Worlds, Part Deux -- Big Huge Problems
Speaker: Ken Rolston (Lead Designer, Big Huge Games), Mark Nelson (Lead Designer, Big Huge Games)
Date/Time: Friday (March 27, 2009)   12:00pm — 1:00pm
Location (room): Room 2007, West Hall

Destruction of Design
Speaker: Luke Schneider (Designer, Volition Inc)
Date/Time: Friday (March 27, 2009)   4:00pm — 5:00pm
Location (room): Room 2004, West Hall

alternate
Games Have Feelings Too!
Speaker: E. Daniel Arey (Chief Creative Officer, VisionArey Entertainment)
Date/Time: Friday (March 27, 2009)   4:00pm — 5:00pm
Location (room): Room 2022, West Hall

zoom zoom zoom, blah blah blah

Okay, so once again I am absent for a month. Once again I have been prepping presentations and globetrotting and talking about stuff. Last week I was in Vancouver giving the keynote at VFS's Game Design Expo. The talk is about the changing demographics of gamers and game developers. I'm not going to host the slides just yet because I will be giving the talk a few more times in the next few weeks and will likely be making some tweaks to it as I go.

I'll be doing the same talk at the local IGDA Chapter meeting on February 25th, and then I'll be doing it again down at the Savannah College of Art and Design in Georgia at GDx. Never been to Georgia - so I'm looking forward to that.

Anyway - the slides for that talk (and the massive pile of data I aggregated for it) will go up on the blog after I get back from Savannah.

Aside from that I'm prepping my GDC talk about what I will call 'improvisational play'- which will be a 'sequel' to my 2006 talk about Intentionality. I'm also doing the Game Design Workshop again this year, and I'm going to be giving a Pecha Kucha style microtalk in a crazy session put together by Richard Lemarchand.

On top of all of that I am also, of course, working on a game.