I Feel Your Pain, Dude

David Jaffe blogged about the review Gamespot gave his newly released PS3 downloadable game Calling All Cars.

I have to say that I know exactly how he feels because there was a pretty common consensus that Gamespot always lowballed the scores for Splinter Cell titles. Kasavin gave the original Splinter Cell an 8.6 when the average was a 9.2 and he then gave Chaos Theory another 8.6 when it averaged a 9.4. It was even more painful because those lower scores came in the first week after launch, and usually the overall scores right after launch are higher (I have since observed that an average game score falls approximately 0.5 on 10 from the average of the first 5-6 scores to the final average on Gamerankings a year after launch).

So dude, I feel for you. It hurts to get lowballed like that especially as your 3rd or 4th review. What hurts even more in the end is that even though the average overall will come down by half a point or so, this review is so much lower than the average that it could possibly still be at the very bottom of the list.

In the end, though, and in all fairness, I had to concede that both of the Splinter reviews were pretty honest. Furthermore since it seems to me that there are just too many games getting 8.5 or higher - I have to say that maybe Gamespot reviews are actually really accurate. Obviously sometimes a bias seems to creep in from nowhere, but that's what gamerankings is for.

If I said I would only play games rated 9.0 or higher, I still would not have time to play those games... and why don't more games get a 1 or a 2 or a 3... god knows there are a boatload of those and many of them still get a 5/10. All that does is crowd up the middle with shit games and make it impossible to give a mediocre boring uninspired game a 5 and instead it gets a 7 and people waste their time wondering of they should play it or not.

The solution - well, we need more game criticism to counterbalance the overabundance of game reviews. The reality is most reviews are just spouting off the same old crap that is being repeated on 100 other sites anyway. If some of these sites would switch over to providing in depth analytical criticism instead of reviews, guys like us wouldn't need to get our hair in a knot over a review at all.

Comments

[Allen Varney]

(((If some of these sites would switch over to providing in depth analytical criticism instead of reviews, guys like us wouldn't need to get our hair in a knot over a review at all.)))

Uhhh... Or you could, you know, have confidence in your own judgment of your work and in the recognition of your peers. Seems more constructive than investing emotional energy in some stranger whose tastes and judgment you don't know.

Now, if this stranger's review affects sales, okay, that's important. But in your meticulous numerical analysis, where is the data about a Gamespot review's effect on sales of an established franchise like Splinter Cell?

[Xu Xiaojun]

Gamespot also gave Ghost Recon Advanced Warfighter (XBOX) a 5.9, which was also not fair. They compared GRAW XB with GRAW XBOX 360，but these two games just shared the story and nothing more. GRAW XB is definately not a terrific game, but doesn't deserve a 5.9 just because his 360 cousin is so good.

[Walter]

"and why don't more games get a 1 or a 2 or a 3..."

My guess is that this is because scores out of 10 and 100 are interpreted in the same way as the grades you got in school. Which means a 6 or 60 automatically denotes failure. It's likely most people using such a scoring system would be reluctant to score a really bad game down in the 1, 2, and 3s unless it was completely broken at a software level.

That's why I prefer the 5-star system, where 3 stars gets interpreted as "average" or "decent", freeing up the lower end of the spectrum to denote genuine crapness. But this ends up getting recast as a score out of 10 or 100 on GameRankings and Metafilter anyway, which totally defeats the point.

[Clint]

Allen:

I don't really give a shit about sales - at least not to the extent that they are affected by this review or that one. Of course I would prefer my games to sell a lot, but I don't at all believe the path to doing that is by pandering to reviewers.

And I have plenty of confidence in my work and the recognition of my peers. My point is mostly to say "yeah it's not fun, but you'll get over it". I pay a lot more attention to coverage of my games get from sites like The Escapist than I do to Gamespot for exactly the reasons you suggest. Unfortunately, I get two thoughtful articles about my game for every 100 banal reviews.

I think game designers are hungry for meaningful feedback on what we actually do, and it's hard to get it, and it's even harder to not try to get it from reviews. Again, if there was more real criticism, we wouldn't have to pull our hair out reading reviews hoping to learn something.

[dlockblok]

I think Clint just nailed it. We're "hungry for meaningful feedback." It's part of the feedback loop. We need to take in the cross section of opinions, including reviewers who may or may not be part of our target audience. When a review seems off the mark, you wonder why. When you are misunderstood, you try to determine if the message was poorly constructed or if you're simply dealing with a poor listener. Real criticism would help us answer that question, and help us make better games. Glib reviews are just useless for both the gamers and the game makers.

[Patrick]

I've been ragging on this issue for a while. We need more criticism. Unfortunately, it seems the people doing it best (or most, at least) are game developers/theorists, like myself, Ian Bogost, and Borut Pfiefer. Kieron Gillen is one of a few game journalists that do criticism. Jerry Holkins is probably the best from a pop access point of view.

The thing is, criticism is not only useful for specific AAA titles, its essential to tipping the situation of indie art games to the point where the already completed work is culturally elevated and alternative financing sees it as something worth encouraging. A lot of progressive games like flOw are not suited to a conventional review. We need our journalism to grow up with the medium!

[Clint]

Patrick

Yeah, there are a few out there doing it, and doing it well. Maybe a bunch of these developers/theorists need to collaborate on a site that is JUST that, and see how popular/useful it becomes. If it flies, other sites will spring up to copy the idea.

[Walter]

I'm seriously frustrated that there are no significant outlets for in-depth game criticism. For a while now I've wanted to set up the sort of peer critique journal that Warren Spector called for so many years ago. Designofplay.com was an attempt to fill that role, albeit short-lived, as life got in the way. (Don't bother checking the URL as I've since taken the site down. There was only a (rather lengthy) critique of Fahrenheit/Indigo Prophecy on there anyway, which I'll probably repost to my blog.)

I'd happily work to get something going again, however, if we can get a bunch of other people on board. It would, ideally, be a bona fide business, non-profit or otherwise, because it becomes really difficult getting regular, high-quality contributions otherwise. But just having any sort of sustainable, in-depth criticism site would be worthwhile.

Yeah, let's get something started.

[Sean Barrett]

Tim Rogers' site http://www.actionbutton.net/ is doing reviews that delve into criticism. Lots of good stuff there.