We don't know about you, but we've never been very impressed with the review system on Xbox. Look closely enough, and you'll find far too many reviews that are full of inappropriate language — they're just not moderated very well.
That's what the developer of Calico has been stating this week, after the game was "review bombed" following its inclusion as part of August 2022's Xbox Games with Gold selection. Here's the original complaint:
https://twitter.com/CalicoGame/status/1554591882927149056?s=20&t=-zu2MU0pvv6I6z7MRBOipQ
As you can imagine, this drummed up a lot of attention and ultimately led to a response, with confirmation from the Calico team that the Xbox team were "handling the reviews that include hate speech and targeted harassment".
However, they also called on "better moderation" for reviews on the Xbox Store moving forward, which we definitely agree with here at Pure Xbox. It can't be easy keeping track of everything, but there's undoubtedly work to be done.
Here's what the Calico team had to say in their follow-up:
"We got through and Xbox is handling the reviews that include hate speech and targeted harassment. Thanks for the support! I'm hoping long term they implement better moderation for store reviews when they have a free event."
"Reviews are so valuable to both devs and players. We're a small team working on our skills, and reviews help us figure out what issues we should focus on! We have valuable feedback to gather from there, and it makes it much easier when we dont have to wade through fake reviews."
What do you think of the review system on Xbox? Let us know down in the comments below.
Comments 70
The review system just needs to go altogether. Filled with absolute nonsense for almost every game on the store.
About time a dev called this out.
@Clankylad I believe all the offending reviews on the Calico page have since been removed.
I don't really pay attention to reviews on the Xbox and PS Stores. They are generally not a great indication and often don't follow the industry standard. That kind of goes for all user scores really, they differ from critic scores greatly. A 5/10 user score is probably equal to a 8/10 critic score.
Especially progressive ones 🤣 those tweets are cringeworthy.
After looking the game up it doesn’t look that great anyway.
The only thing that's really wrong imo is if people are able to get in more than one review (or if they are being hateful on the review text).
If most people think your game sucks, for whatever reason, why is it a "review bomb". People just don't like it, grow up
"don't like our game? you're a bigot"
nah fam, your game looks like garbage
It’s pretty tricky… what we don’t want is anything deemed as being a bad review as a ‘fake review’ to appease the devs/publishers.
They could drop it all together.
They could implement a scoring system out of say 5 stars that is only available after you’ve played a game you’ve paid for.
Lock the ability to review a game till you’ve actually completed that game. Let’s be honest, the majority is feedback not reviews…most haven’t completed what they’re reviewing…most are simply saying it’s broken for them in some way.
And feedback, letting us know if a game is broken, lack support, multiplayer dead etc…should definitely be kept as part of the Xbox service. This needs to be implemented better. Blatantly offensive posts should be pulled. But then you start walking a tricky line with what’s deemed offensive…
If you don’t want to deal with negative/b.s. reviews, don’t release your product for public consumption.
@Xeno_Aura I usually check out a few user reviews that have more than a line of text. And take in any common baseline.
I find they often give a more realistic view in regards to bugs and performance…
majority of mainstream sites reviews I take on board opinions in story…but little else.
Majority of these sites/YouTubers end up swaying their opinion a certain way for various reasons…be it cause they’re sent games, and want to continue to be sent games. Or they need to pander to a certain audience otherwise they’ll lose subs/clicks. Or they have promotions going on with certain publishers…etc
Or sometimes, which we’ve seen more recent, it’s through no fault of their own and they end up reviewing versions of games that don’t reflect the version that came to market. They don’t often update their reviews to reflect this.
@ChromeMud I agree.
Hate to break it to these devs but no one's going out of their way to 'review bomb' an obscure indie game about owning a cat cafe.
People using the xbox review system to spew hateful speech however is a whole other issue and Microsoft should absolutely look into this.
The review system on Xbox is an absolute joke, every time I look at a game's reviews on the store to try and gauge if a game is worth buying in the sales the "reviews" are absolutely nonsensical and give me no useful information at all.
Granted there are only 5 reviews for the PC version and 13 for the Switch post but both get a 57 on Metacritic.
That's not to say it's without an audience as @ChromeMud said his daughter is enjoying it. Review scores are only so useful. But a shame to see these being used to bring a game down and spread hate.
100% needs better moderation. From Bad language to bigotry. Most ofit is prob young teenagers being Edgelords but a few I've reported as been disgusted what was written and I'm a 42 year old man.
Personally, I nearly always take a look at reviews before buying a game. What I would like to see though is a system similar to Steam's wherein not only do you get an overall rating from the time of release, but also a 'recent reviews' element so that you can see how the game is now, and not necessarily how it performed upon release. A game such as Cyberpunk 2077 was rightly criticised upon release, but it is in much better shape now, and to be able to see this at a glance would be much more helpful than the overarching reviews that included those given upon release.
One thing I would say though is that many reviews are either irrelevant or even a complete mystery whereby the review probably only makes sense to the author, and I would emphasise the 'probably', as even that is of no certainty...
@ChromeMud yes and it's even more worse when its a gwg or gamepass title. I took a look at the screen shots and decided, I'll prob not give this a go, attached it to my account and went about my day. Not once did I think about saying rubbish for th3 sake of it. Just don't play it
Never noticed Xbox has a player review system... Whenever I want to know something more from a game I just google it. But it appears that you do not have to at least play the game for a few hours before posting and that should be quite a big no.
Absolutely agree, @SplooshDmg. Steam does do it much, much better. I'm not sure people would have the patience to write some of the very detailed and lengthy reviews of the likes to be found on Steam when using a controller to input their comment though. There is also an upper limit of characters that you can use on the Xbox, which actually prevents you writing a properly detailed review, and which I would like to see removed. After all, if people cannot be bothered to read it, they won't! I agreed though, I would much prefer to see reviews done the way that Steam does (and including recent reviews, as I mentioned above)...
The Xbox store is laughably bad, its quite clearly poorly maintained and full of half ideas that were never followed through, the only worse bit of software is the PC gamepass app, which just doesnt work most of the time.
Its sad that Steam is so clearly the way to correctly do this stuff, but despite having such a clear leader to follow, we are still left with ms apps. There are many reasons I dont like playing on pc, but the apps store and pc gp are a large part of it. When I do play pc I keep it steam.
Whilst im keen on comments being moderated effectively, if you make a poor game, your gonna get mauled and can only cry fowl if its genuinely wrong. Get rid of the idiots by all means, but you need a better game if you expect better reviews..
I actually do check the Xbox store reviews and find them often useful. I usually will stumble on some obscure game on sale and the reviews can give a quick insight into whether it's worth looking closer at the game. As an example there was one small game that looked interesting until I noticed over half the reviews mentioned the game was broken at a certain point.
With that said, I'm not using those reviews as the end all be all. Just a quick glance direct from the console.
I also find the store reviews helpfull as they often make you aware if a game is still a buggy mess or has since been fixed
@Dusk_Actual There's a difference between getting objective and fair reviews that negatively criticise your product, and getting reviews that spew bigotry and intolerance.
No one should have to endure the latter, whether they're in the public eye or not.
Reviews and opinions are like aceholes, everyone has one and it stinks. If I personally knew the reviewer, I'd still take the review with a grain of salt, let alone taking a stranger's opinion as gospel.
Maybe a user should be required to download (or stream), launch, and spend some time interacting with the game before they can leave a review. Some attempt at proving that the people leaving reviews have actually played the game.
maybe make a better game? never heard of the game but that screen shoot looks like a free to play mobile game. That might be one of the reasons its getting hit hard
Ah yes of course. It should be totally illegal to criticize anything woke. I'm sure it soon will be. This is an extremely important article.
The Xbox Store has reviews?!
I actually never noticed. Then again, the only time I ever use the store is to buy a specific game I want. The only thing I ever look at is the Buy button.
I use the software management area to browse Game Pass games.
IMO politics don't belong in games but there are nicer ways to say it than in many of these reviews.
'Better Moderation'.... Right, control what people think, say and do, as long as it benefits you, right? GTFO. I don't think I'll be supporting this dev in the future.
@Markatron Definitions of 'bigotry and intolerance' vary wildly. Most of the time, if someone is not an avowed communist they are labeled as 'alt right'. So, unfortunately the ability for people to fairly enforce correct-speech cannot be trusted.
I absolutely love Game pass , however it has pissed me off on how it's allowed a wave of idiot biased and many times unjustified reviews that have affected the selection included in game pass. Many of these games are actually much better and should be value tested by no one else but yourself for yourself.
@Grail_Quest How about just keep discussions/critiques/reviews on point and don't devolve into irrelevant, personal attacks? A crazy concept for some, I know. 🙄
@Grail_Quest hmmm yes, but let's face it: we're taking about people on the internet posting "reviews" about Xbox games, not a political forum arguing about snowflake culture and the right to free speech, and as a result I think that we could both probably hazard a guess at what kinds of things were being posted in these "reviews". Pick a game on Xbox, go read the user reviews on the store and you'll see that they're invariably a cesspool of homophobia, transphobia, xenophobia, racism, you name it.
There's no wildly varying definitions at work here. Just c**ts.
I only bother reviewing a game if I think it really good but I don't write anything just give it 5 stars like the ori games
Calico is getting review bombed because it's a horrible game. Some of what I played of it was fun and amusing, but overall the graphics and gameplay are just bad. I couldn't believe they released this, expecting people to pay money for it, and then Microsoft thought it was a good idea to release with Gold.
Considering that the demographic for this is probably little girls, adding in witchcraft and lesbianism is inappropriate. It's no wonder people are leaving reviews just about that.
Outside of how bad the game is, I have seen inappropriate reviews and reported them, but more often than not the reviews have been at least somewhat helpful. Lets say I was thinking about buying Calico: looking through the reviews and seeing that it has lgbtq+ content is helpful.
Removed - inappropriate
@Markatron couldn’t agree more. Reviews should be clean and free of the trash you mentioned. Sadly that just isn’t the way the internet works. Call me cynical I suppose.
The review section is like this comment section. Very intolerable.
@Xeno_Aura there is no industry standard.
"Intolerable" is a sign of weakness.
"Review bombing" is a loose term with several contexts. Specify. One context suggests coordination amongst a subset of users - that is likely not the case, for various reasons (laziness being one of them).
It is incorrect and deficient to down-rate something to lowest score because it's not aesthetically, thematically/politically catered to you; something that you've fundamentally disagreed with. Overall quality of a game experience has next to nothing to do with the message in its medium (and the medium is the message, in case you haven't heard); today's writers brush on this when they talk about the "cultural impact" of a game — I think that's fine to talk about, but not fine for anyone to base their opinion of something.
Thus, if the constituent design elements (e.g. gameplay in particular) sucks, take the effort to explain why. If it IS the writing, as described above, still be tactful, explicit, pertinent - what it means to be on point. You'll find that fundamental disagreement doesn't translate to "zero" or the closest substitute to that.
If 1 star reviews are 'review bombing' then 5 star reviews are 'review boosting'. It will all balance out in the wash
@Dusk_Actual Developers and gamers need to realize that the opinions of those that rate things a 1/5 don't matter. Those opinions are worthless. There are few games that are that bad and yet you see that score given out to 90% of games out there just because somebody is mad. Whenever I see a score like that I just think that person has personal issues they have to deal with.
@nrXic
Do you feel the same about 5/5 scores? Very few games come to mind that deserve a perfect 5/5. Calico certainly isn't one of them (nor would I rate it 1/5)
The review should be a 1-5 star point for: Graphics, Audio, Gameplay, and Story, no word needed. And gave xbox players some points (like nintendo points) so gamers like to review the games.
So we love feedback on how we are doing with our development. The feedback has been amazing. There is a problem, though. Some people don't review the game well. They use swear words, so they are troll reviews. Nobody 8s upset about spending money on this game or spending storage space on this game. We love feedback, but please refrain from leaving bad feedback. Positive feedback is the only honest and accepted feedback, if you don't like the game then you are a troll and please don't leave a comment.
@ExtremePat this is an extremely jaded viewpoint on what’s actually going on in these situations.
First off money was most likely not spent as this was a game with gold game. Free with online membership. Not even Gamepass. So that argument is null.
Storage space is a ridiculous issue as you can delete anything.
Finally, if the game at least functions and doesn’t have glitches the score should be higher than a zero.
And yes actually you should be able to communicate your point without a cuss word. If you can’t, then you have no right to contact or use a support channel.
Which by the way support channels are allowed to hang up once the person starts cussing. It’s considered hostile.
@mousieone it's an extremely jaded view to assume everyone has game pass. This is a game that has a price tag, this is a game people spent money to buy because they didn't have it on subscription. You can always delete games, but some people have data caps and can't constantly delete and download games. If you spend $15 a month to have a subscription service then you pay for and have a right to review and judge games. Money spent is money spent, and a bad game is a bad game. Your points render completely invalid when you look at the issue through the scope of an average consumer and not someone who has every subscription service and every next gen console/graphics card and processor
@wiiware But then how would you get across if a game has bugs? Is still being supported? Still has an active player base? If the game has potentially offensive language or messaging for its target audience?
@Bleachedsmiles I think that full reviews should be on games forums, not on the game reviews pages. And about the player base, microsoft should copy steam, where gamers can see how many players each game has in the store website/digital store.
I'm surprised the store is yet to implement a playtime feature similar to the one on Steam. The stat is obviously tracked, you can check it in the Xbox app and all. Not only would having a bunch of 1-star, 0.1 hour reviews make it easy to filter out less defined opinions, it would make the job of community moderation easier particularly in the case of a game like Calico that obviously isn't for everybody and a lot of people are gonna bounce off of.
That said, perhaps the easiest way to moderate things and to "fix" store reviews is for Xbox to actually give out community bans for the people who are using the feature to be homophobic, sexist, racist etc. and that might actually disincentivise the behaviour of that loud minority.
This sounds like the equivalent of YouTube removing dislikes. Microsoft will probably get rid of reviews at some point at the request of developers. Steam is much better for user interaction.
I would prefer a 90s style review system.
Gameplay 1 to 5 stars
Sound 1 to 5 stars
Graphics 1 to 5 stars
Replayabilty 1 to 5 stars
Polish 1 to 5 stars
I don't need to see ppl write nonsense.
@wiiware Yeah that would be a good update if you could see how many active players a game still has via the Xbox itself.
Just a quick question: is there anyone here that DOESN'T check the Metacritic score of a game before buying something?
@ExtremePat thing is it’s not a Gamepass game. It’s games with gold. You know the service people need to play online. It’s sub service sure. But most people don’t have gold for the monthly games but to be able to play online with friends. It’s also not 15 dollars a month. I mean neither is GP. But my point here is that people have Gold because they have to. Not for the games like as in GP. But clearly you don’t know that and are just trying to argue a game is bad because it of course must be bad. But you don’t really know do you? You are just assuming that’s it okay to negative review a game because it’s not your style.
So then if I go around giving first person shooters like Wolfenstein bad reviews that’s okay right? I didn’t like it, so I can give it it a zero right? See where your logic falls apart?
@mousieone If your opinion of a game is a 1/5 who am I to say you're wrong? Also game pass ultimate is $15. Either way you are saying if someone doesn't like a game then it's just not their style. That's proving my point. Not everyone will like a very stylistic game. It doesn't make anyone's opinion invalid. You could say this game is a 10/5, that doesn't mean that someone who rates it 1/5 is wrong... Different strokes for different folks. Also my bad for confusing their two game services that typically fall under the same payment
Honestly I am conflicted when it comes to this issue. While it's stupid and immature to turn reviews into a political battleground, I can see how this developer was sorta baiting the audience into having a really divisive reaction with some of their... uhh... design and direction decisions they made. I mean I'm sure it would have been easy for them to just make a cat cafe game without any politics or sexuality on display, but they went in the direction they went with for whatever reason. My personal take is that they probably wanted the controversy.
@Bleachedsmiles Problem with your mentality is that nobody completely 100% completes a game if they didn't like it, only people who like and enjoy games do something like that, so therefore your imposed requirements would make it so only fans and people who really enjoyed the games could review them, which would defeat the entire purpose of user feedback. It isn't meant to be an extension to fan praise, it's meant to be a means for people to simply give their opinion about something. It's like you don't need to watch every single episode of The Twilight Zone in order to form an opinion on if you enjoy the show or not, in fact if you watched every episode that would imply that you must be a big fan of it.
I think the approach that Steam has is the best. A simple thumbs up or thumbs down review, with people just having had played the game for a half hour or something before they can write a review. Asking anything more would be to eliminate criticism.
What is being labeled "hate speech" and "bigotry" these days is absolutely ludicrous and has a double standard applied to.
It sounds like the game sucks anyway so I won't bother looking at it.
@SplooshDmg Oh definitely, it helps you understand where the reviewer is coming from, and depending upon the type of feedback you're looking for, it can really help you narrow things down. For example I am looking for some casual criticism, I would know to avoid glowing reviews from people with a ton of hours racked up because they are most likely fans who are really into that game. Like it can be interesting to hear from people like that at times, but I find that I can have a lot more to learn from people who aren't big fans of things, at least from a standpoint of indicating potential flaws that I might have an issue with.
That and it pretty much makes it impossible for people to be effective at trolling with reviews for games they barely played, because you can see if they only played it for a few minutes or something. If someone has an opinion worth listening to, they should at least have SOME experience with it.
@50PlusGamer That's probably because those are often examples of ad-hominem attacks, which is a logical fallacy. It's an attempt at a personal attack on somebody/thing as opposed to really addressing whatever was said, an easy form of dismissal. A cheap label to apply to an argument.
Keep in mind how this works by misapplying words with valid negative notions towards critical/contrary thought. Like racists and bigots exist, but that doesn't mean everyone who gets called one legitimately meets the description, nor does it excuse the accuser from any such similarities. That's when you get into projections, "no I'm not, you are" sort of arguments, which is often used to steer that attention away from themselves.
@JayJ See you’ve just highlighted one of the main problems…confusing reviews and feedback. Majority of ‘reviews’ on Xbox consist of feedback… “This game is broke” “graphics suck” “this game crashed”. My ‘imposed’ requirements (suggestion) of having people who have completed the game be able to leave reviews would give a chance to offer more in-depth opinions on the actual game…with the added value that you know they’ve not just spent 10 minutes on it.
A feedback section would be separate.
Also, people generally possess an ability to form more of an opinion than ‘this is good’ after they’ve completed a game. Most can function enough thought to put into words why they thought it was good - what they liked about it.
Most people also don’t come away from every game they complete with the mindset “this is a 10/10”:.. maybe you do? But there’s plenty of games I’ve completed that would fall into the 7/8/9 bracket. Just because I complete a game doesn’t mean I’m a blind fanatic to that game and can’t recognise what I liked and what I didn’t like. What a very odd thing to suggest.
Last game I completed was As Dusk falls. Played to completion. Enjoyed my time with it. It’s a solid 7 for me…because, though I enjoyed it, I can recognise its faults. Recognise what I would have liked it to do better. You’re telling me you’ve never had the ability to do this with any game you’ve completed? They’ve all been 10’s??!
A review of a game that’s only played for 30mins holds more value to you than reading one off somebody who’s played the game to completion? Bet countless gaming sites wished that held true for all of us.
@Bleachedsmiles I would ask for something more balanced. Feedback from people who enjoyed a game enough that they played through it's entirety are not the kind of people I would be wanting a review from. They might be a good source of advice on a fan forum or something, but not the kind of person that I would think represents a good average in terms of how a regular person with casual interest might feel. Heck I know only a small percentage of the games I own have been beaten, heck there's some games that can't really be beaten, or they were never really intended to be truly completed. Like you really think everybody has the time to spend 100's of hours on an epic RPG before they could have an opinion about if they enjoyed it or not, or if they would recommend it to others? Not everyone lives and breathes video games you know, some people just want to play a game in their spare time and they don't want all the feedback to revolve around the type of people who obsess over games.
All these demands, it just sounds like the kinda stuff you would hear from people who take video games way too seriously. Remember, this is supposed to be entertainment, not serious business.
@JayJ tbh what you write makes very little sense.
For one…how are you to know if a game lasts 100’s of hours if the reviewer has only invested 30mins in it? In your preferred scenario you, as a casual gamer, then ends up buying a game that isn’t at all suited to your needs.
Why do you have this weird point of view that those that complete a game must be a fanatic about that game? So there for their opinion can’t be trusted…yet the kid who hit the start screen and messed around for a few minutes can tell you it’s a 10/10 and have you spend your money?
It’s all very odd.
But yes, gaming is supposed to be about entertainment and enjoyment. Your time is precious. The point of reviews are to be informative so you don’t waste that time. And you gain the most enjoyment out of a game that’s right for you.
@Bleachedsmiles I am simply pointing out the flaws in your thinking, the fact that you are now resorting to gaslighting my argument tells me that we are beyond a reasonable exchange. I can tell you must take a lot of pride in your concept of valid reviews, which I suppose is great for you in your own way, but that's not something that I feel would result in a healthy environment for the general public, nor would it be fair or reasonable as a standard.
@JayJ heh that’s pretty interesting…you’re fine pointing flaws in my thinking…yet I’m ‘gaslighting’ when I point out flaws in yours? Things sudden make more sense now.
My ‘concept of a valid review’…I dunno man, I know what I expect to get from reviews…when I read a review on here it’s not a novel outlandish expectation that the reviewers played more than 30mins of the game and is capable of giving a critical opinion on it. I reckon we live in different worlds.
And I’ve no idea why you still think reviews with some validity would mean no option for feedback in general…I’ve never suggested such.
@Bleachedsmiles Well you're clearly not a reasonable person.
@JayJ excellent 👍
am i the only one who is going to say this, maybe just maybe a game about a cat is not a good game? from the videos i have seen this game doesnt even look interesting to me...
A lot of ppl leaving bad reviews are just doing it because they are sick of seeing the quality of games added to GwG go down month after month....I personally just ignore them but I would rather see them gone and be given a discount on XBL Gold as an option.PS:As a comparison PS+ had Yakuza:Like a Dragon and Tony Hawk ProSkater 2 this month while we got a game about a cat nobody cares about,not even my 11 y.o girl wanted to play it ....
Show Comments
Leave A Comment
Hold on there, you need to login to post a comment...