Wednesday, August 29, 2012

#36: Game Reviews and the Gaming Press: Why Do They Suck?

The gaming press is a pretty powerful entity in this industry. The consumers go to them for the latest in gaming news, including what is about to come out, what companies are partaking in less than consumer-friendly business practices, and other news a gamer might want to know. The publishers rely on them to get information to their customers and effectively spread the word on projects they are working on. However, the press also has one critical function to serve: To review video games. Game reviews are supposed to be an important part of the gaming ecosystem. It is, theoretically, immensely helpful when contemplating the purchase of a game to have a collection of opinions regarding it. They should help to distinguish between bad games, average games, and good games, but how effective are they in that capacity and why do they sometimes get called into question? This week, I attempt to figure out the answer.

Before we get to that, we need to understand the process behind most reviews on major game review websites. More often than not, major review sites receive copies of the games they are to review a couple of days to a week prior to the scheduled release date. The reviewer in question then has to use their free time (I am not aware of a review outlet that actually allots time for its staff to play games on the job since they have other duties to attend to, though I am sure they exist.) to play far enough through the game to get a good understanding of it (since it is rare that they play through to the end) yet leave enough time to gather their thoughts and write a review about it. The pressure will be on them to finish the review as quickly as possible because the faster it gets uploaded to the site, the more page views it will get and more advertising money the site recieves.

Obviously, this style of reviewing, which is commonplace in the industry, is not very condusive to writing reviews of the highest quality. One of the biggest issues in this is time. Reviewers are rarely given an adequate time to write very good reviews. First off all, games nowadays usually take around 10-12 hours to beat (30 or more for an RPG). For someone who has a job, a social network of people they communicate with regularly, and many of the responsibilities life throws at us, playing through a game that long in so short a time while trying to analyze it critically is asking for a lot. It is not likely that the person in question will be able to throughly explore a game and look for pros and cons beyond what is immediately obvious. Any extra moment they spend playing the game reduces the amount of time left over to gather their thoughts and write the actual review, which is already lacking enough. In the remaining days before the deadline, they need to structure their thoughts on the game and all of its aspects. Then they need to plan out and compose a written review that encapsulates all of those thoughts as best as it can. Considering the timeframe often required to write these reviews, it is a miracle that they are anywhere near coherent. The amount of time to write and proofread those reviews is nowhere near enough to do anything more than list-off what the game does and whether they think it is good or bad. Going into any real detail is almost completely out-of-the-question.

The less obvious, yet more critical problem is the window in which the review is released. Many people would, quite logically I might add, say that the best time to release a review of a game is somewhere around 3-4 days before/after it is released. However, I disagree with this for a couple of very important reasons. The first reason for this is that a review published within the release window for a game will not sway people who are on the fence about buying the game. When a game is released, the only people who buy it on the very first day it comes out will be people who were already eagerly anticipating the game. Regardless of how positive or negative the review will be, they will buy the game. All the review will do is reinforce the decision in their head (even a negative review will be seen as a personal attack and will not be recognized as a legitimate opinion). The same can be said about negative reviews and people who will not buy the game. Anyone who is on the fence regarding the purchase of a video game will not be immediately swayed. They will often wait for anywhere from a few weeks to a month because of my second point regarding the window of time: Real good conversation on a video game only arises in the weeks after its release. It is only then that people have bought, fully-played, and digested a game and all of its parts. This is when people can critically analyze the minutiae of the game in question and figure out exactly why the game is as good or bad as it is. We saw this with the Mass Effect 3 ending controversy. Most reviewers, in a rush to get their reviews out on release date, did not get the chance to play through the entirety of the game and reach the ending. A few weeks later, after the game was out, many people actually had the chance to experience the finale of Mass Effect 3 and... react to it. It was only then that people were able to look into it and think about what worked/did not work about it. People do not form opinions in a vaccum. Since we are social creatures, we think and form opinions by talking with other people. By gathering information from different viewpoints and perspectives, we gain intelligence and form better opinions. This is why I do not think releasing reviews on the same day a game comes out is a good idea. It is difficult for a reviewer to form detailed, well-informed opinions when they have not had the time or even the ability to discuss the game and gain the necessary prespectives of other people nor have they the time to fully comprehend and analyze their own prespective. It is a recipe for disaster.

Another serious issue, though one I suspect is blown out of proportion, is the fact that game reviews are supported by advertisers within the games industry. Everybody knows that major game publishers like Activision or EA buy ad space on review hubs like IGN or Gamespot. Because of this, there is a perception that these reviews are being bought by publishers in order to make their games look better. It is not that hard to suspect when sites like IGN release an article titled “Why Do People Hate EA?” and only ask Peter Moore, Chief Operating Officer of EA, the reasons behind it instead of asking the aforementioned people. Again, I do not believe this is as serious an issue as people make it out to be. I think that the people who write these reviews generally stand behind them. However, it is still a issue worth bringing up and discussing as it does have the potential to impact reviews and how the gaming press thinks with regards to the games they talk about. There are well-known cases of reviewers being forced into embargos as a result of receiving review copies of certain games, like how Konami forbid mention of the cut-scene length and install times of Metal Gear Solid 4. It is not an easy problem to solve and there is no simple solution beyond not showing game advertisements, which would cut into profits.

One last problem that I am far from the first person to make note of is the over-inflation of review scores. A scale of 1 to 10 is often used to indicate the overall quality of a game in comparison to other games. On this kind of scale, 5 is often denoted as average simply because it is in the middle. Anything above a 5 is above average and below a 5 is below average. This makes sense and is intuitive for the most part. However, this is not necessarily how review scores (which are a terrible way to handle reviews, but have been accepted as commonplace simply because of the fast-paced nature of today's society) work. If you were to go to review-aggregate site Metacritic, you would find that there are many more average or positively-reviewed games than there are negative ones. This is simply not possible mathematically. If there are more positive reviews than average or low reviews, then the praise becomes the new accepted standard for average and the review scores should be placed back in equilibrium. We are not seeing this and I think I know the reason why. The reason for this inflation of review scores (and resulting decrease in credibility and weight that reviews carry) is that the fans of franchises cannot tolerate reviews that are neither perfect nor near perfect. We saw this when fans of the Uncharted series lashed out against reviewers for giving the upcomming-at-the-time (as in, they had not played it yet) third game an 8 out of 10, which is well-above average on a 1 to 10 scale. The game was not ever released, yet people claimed that the reviewers had “no clue what they are talking about.” It is a huge testament to the source of many of the problems with game reviews.

In fact, upon very close scrutiny, many of the other problems that plague game reviewers and the reviews they write come down to the fans who read them. The reason they are rushed to finish these reviews before a game is even out is because fans do not want to wait. They want to see opinions on upcomming games as soon as possible. Fans cannot wait until popular consensus has arisen and critical analysis can be had because fans do not want that. It is a desire for instant gratification and someone to support their opinions of the serieses they love and hate that drives them. A critical eye and valuable insight into the fine details of a game is not wanted by the masses. The request is for a “You are right” or an “I think you are wrong” (which will be preceived as a “selling out”). The only problem that does not come down to the fans is the issue of advertisers, and that is complex enough as it is. This is frustrating for people like myself who want insight and critique of the medium. To improve the quality of reviews from major gaming journalists, we need to fundamentally alter the culture driving them. This is not an easy thing to do as it would require much work and be met with much resistance. I do not even know if the effort would be worth it. But at the very least, this is discussion worth having. This is something that needs to be said.


Aldowyn said...

Writing a review of a game you haven't finished is just... bad. I'm not against release-day reviews, and I don't think they're pointless, but if you don't get a review copy a week or so in advance, there's no way you can do a legitimate review of most games.

As for publishers supporting the outlets, that can be a problem, especially on big sites, but for the most part I agree that that's not a big an issue as the others we've mentioned.

Finally, on the issue of the review scale, I essentially view it on a 6-10 4 point scale, with 8 being average. It tends to work quite well, honestly. And I wouldn't call Uncharted 3 average. Not that giving scores to a game isn't inherently flawed, but I see the reasons for it.

newdarkcloud said...

Yeah, but a 4 point scale from 6-10 is absurd. What's the point of the other numbers then? You might as well just go down to a 0-4 scale at that point. You're doing nothing but artificially inflating scores.

And while I don't like them, I've grown to accept scores as a part of reviews. It just feels like they could be done better than they are. (I also hate what scores and sites like Metacritic, through no fault of their own, have led to in the industry.)

As for the first part of that, I am in complete agreement. If you have not finished the game, you really shouldn't write a review. I can't blame reviewers for this, though. After all, a deadline is a deadline.

Indy said...

I disagree with finishing the game when the review itself has nothing to do with the story. They intentionally avoid spoilers and highlighting plot twists, something the ending of games (and other mediums) usually have. The reviews should focus on gameplay and mechanical interactions with the game world. If they do mention the story, than however, I expect them to have finished the story like you and Aldowyn said.

With regards to the scores, I prefer the method Yatzhee outlined in one of his columns. Give separate values to things like Context, Challenge and Gratification, 3 separate elements of fun that people enjoy differently and keep them separate.

Anonymous said...

It is absurd. And frankly I'm surprised you haven't mentioned the 11/10 nonsense.

And even if you didn't have to subtract a magical constant from every review score, they're useless anyway. As with any part of the review they are entirely subjective. Except the subjectivity is hidden behind an opaque number.

newdarkcloud said...

The 11/10 nonsense? I don't think I've heard of that. Can you tell me about or link me to it. It would be an excellent supplement to this post.

And I don't necessarily agree with the second part of that comment. Yes. Reviews are extremely subjective, by the nature of them being opinions. However, I do think they have use. By reading another person's thoughts on something, we can help to better form opinions in our heads as to what we think of the subject. Like I said, people don't form thoughts and opinions in isolation.

SougoXIII said...

Really it is like what you said: The entire review process is extremely broken and inefficient. (Some of that would also apply to gaming journalism as a whole but I digress...) Why waste time writing a glorified preview of something that you are playing out of obligation just so that fanboys can ignore it and lineup for launch-day anyways?

The thing that annoyed me the most about the 8-10 rating scale is when developers themselves are starting to insist that an 8/10 is not good enough for their games. It just make me face-palm at the current state of the industry.

One tangential gripe with 'edgy' reviewer/critics like Yatzhee: I know he exaggerated flaws for humorous effect and does raise good points when playing a genre he's familiar with but some of my friends seems to think a 4 mins video can list out pros and cons of a game and insisting that I'm wrong even though I'm the one that spend hundreds of hours on it. Grrrrr...

newdarkcloud said...

I wish I could dig up the link for you, but I know for a fact that Yahtzee's gone on record to say that he only talks about the bad without really touching the good. He tried a positive review with Psychonauts and people didn't like that.

It does sadden me when people forget that he does it mostly for laughs, with criticism a secondary (but still important) concern. That's why he started Extra Punctuation.

Anonymous said...

Well, to my knowledge nobody has ever rated a game 11/10, but I did read a few reviews where the game in question was given a 10/10 and the reviewer said he'd give it 11/10 if it was possible.

As for the second part, I wasn't talking about the reviews themselves, they are useful. It's the score that bugs me. With a review you can clearly see the though process behind what the reviewer is writing. You can see what he considers important, you may disagree with it and just not take that paragraph into consideration.

That does not work with a score though. Here, have a number is supposed to represent how good the game is. Except it doesn't, it's just a TL;DR version of the review with the same biases. So you need to read the actual review to learn the whole truth. Which is fine until you take that number out of context and put it on Metacritic for example.