True Skill 2 Is a Potential Problem

Posted by Steve

Wednesday, June 8, 2022 9:09 AM

True Skill 2 Is a Potential Problem

The current Valorant ranking system was a product of where I and many friends came from.
I personally come from the OG days of online gaming from "the golden days" of Halo 3 on Xbox 360. I have more hours in competitive Halo than most players have combined in all the games they play. The matchmaking system was brilliant at the time, but had room for improvement. That being said, there's plentiful memories of playing matchmaking for hours and having that special feeling of needing to keep playing (one more game then I'll go to bed type beat). That feeling is lost in most games I play, especially Valorant.

In those days, that was the origin and godfather of online matchmaking. Rather than an independent elo system (developed from chess), TrueSkill (aka TS1) was developed to determine balanced ranks with more than 2 people facing off against each other. This turned out to be a revolutionary system that was a baseline for most competitive games back then with a few variations depending on what game you were playing. It was a reliable, mathematical based system that evened out the random pool of players thrown into a game.

To keep it simple for now, one of Microsoft's employees under 343i working for Halo 5 developed the next generation of skill based matchmaking. It was an evolved version of TrueSkill (TrueSkill2). He recently moved onto work for Riot, which implemented a massive ranking system update awhile ago. This employee is undoubtedly a skilled, intelligent person with huge goals in mind. I can't argue with his visions as he truly has paved the way for modern online gaming and part of his developments have become a baseline in online gaming.

Although this has been a widely accepted norm, there are flaws of the system designed to balance matches with at least 8 players involved in a match. This link summarizes the improvements of the matchmaking experience with the math backing it up. If you want to read further into the changes and math, the link is provided.

Essentially, the system grabs the most balanced lobby at any given time to improve matchmaking time based off an algorithm that pulls a sample of players within a relative skill range as quickly as possible in order to balance teams. This is a good idea, but the flaw is that the time saved from matchmaking comes from the expense of the players. I can bet my entire life savings that a good amount of the games you've played you've felt that either one player was carrying hard or that you had to play absolutely perfect in order to win. This may be a rare occasion for those in the higher end ranks. A good analogy to keep in mind is you are closer to TenZ than the average diamond is to you (you are still are dogshit even when pushing those higher ranks!) This is because of the fact that TS2 was designed to harbor a 50% win rate for either team no matter the skill disparity of each team. In fact, its a perfect idea. Why not have balanced, randomly selected 8-10 players battle it out to the death to determine a match? The problem: in execution, it doesn't matter to the system if the top player from each team has to drop a 40 bomb as long as the game is close in score.

Back to Halo. before TS2 was released in 2018, matches were fairly balanced with top heavy stats relative to the top 2 or 3 players in a pool of 8 players total (TS1). This might sound familiar, but it was a TOUGH carry in those matches. Those top stats were always comparative. It used to be rare where 1-2 players were obviously the best in the lobby unless you were in the top 1-2% of players in the entire game. True Skill entirely relies on a large player base. In Halo 5, the player base shrunk every year, which challenged the matchmaking system even after TS2 was released. A smaller population meant that it was harder for the system to determine an accurate assessment of a fair match. Essentially, the system is a mathematical calculation of all players in any given sample torn down to the raw numbers players produce, relative to the total population of players available in a given skill range before being spit out as the best team available.

Think of it like this: You have LeBron James putting up historic NBA stats with 4 of the most average college players at any given time. You take that team VS the average college roster in the US. The math says that it's a 50% win rate and that it could go either way. Yet, LeBron's team blows out the best matched team by 20 points, with LeBron leading the way. This is the inherent problem with TS2. The mathematics behind it are sound and make perfect sense. The issue truly comes down to the skill disparity. Basically, it assumes that ONE LeBron James is worth your entire starting college lineup, which isn't the case. It is inherently flawed.

Yes, there are plenty of us who play Valorant who are your average diamond-master player where this is a meme worthy complaint. I'm not complaining that it's impossible to rank up. I understand its a grind and there's a skill gap at a foundational level of this game. The current data that I've seen is that there's a slight negative distribution of players below the gold level of play, which is boggling to me. I have the same exact complaint as a better than average Halo player as. Infinite has the same problem. As a stat nerd it bothers me that the bell curve isn't perfect given the massive player base Valorant has.

My main point is that the MM system is in this weird state where you rely on either carrying your team so hard that you need premium medical insurance to pay for your back surgery or you hope that there's one other person on your team to help bear the weight of your team in order to win.

A good video to watch is snipedown's experience playing Halo Infinite, which is a variation (possibly could be the same) of TS2, I'm not entirely sure right now. It's basically a roll of the dice when you're queuing.

References

  • https://www.reddit.com/r/VALORANT/comments/v6pbtx/true_skill_2_is_a_potential_problem/
  • https://reddit.com/v6pbtx

More Like This