(Co-authored by Kevin Bailey & Colin O’Brien)
We propose to adjust the current ranking system to a more balanced and fair method.
100% Gonzalez (75% carry over)
Our Proposed System:
70/30 Hybrid: 70% Gonzalez (still 75% carry over) / 30% NHL
Listed below is the most in depth ‘rationale’ section in the history of NCDA rule/policy proposals:
Written by: Kevin Bailey
The National Collegiate Dodgeball Association’s current ranking system (Gonzalez) has some great aspects, but as a stand-alone system, it is not what we need as a league right now.
We need a ranking system that achieves balance. One where teams can’t go 3-2 in the regular season and claim the #1 seed (that is exactly what would’ve happened if we used Gonzalez back in 2012. Shoutout to CMU). We also do not want a system where a team can schedule a bunch of games against cupcake opponents, and coast to the #1 seed (This is exactly what Kent State did in 2012, and at the time the NCDA used the NHL System where each win counts for 2 points, regardless of how tough the opponent is).
In other words, we need a balanced system that ranks teams based on merit (teams are given the seeding they have EARNED).
Win lots of games, especially ones against good opponents, and you will get a high seed. Don’t win many games, especially the ones against good opponents, and you won’t get a high seed. Makes a lot of @%$#ING sense… right?
Why is this new 70/30 system better than what we have now?
The intention of the Gonzalez System is to predict the winner of each matchup. A “predictive system”. By the way, it is very good at the whole prediction thing…
But, a prediction-based system is not what should be used to produce “standings”. When it matters most (seeding teams for Nationals), we need a system that puts teams in the spot they DESERVE. We need a merit-based system. That is clearly the fair way to seed teams, and it is a shame that we won’t even have this in place until next year.
By switching from 100% Gonzalez, to a hybrid system that is 30% NHL, we will have a well-adjusted system that rewards teams that win a lot of games, while also rewarding teams that play very tough schedules. It’s a happy medium between our current system (that favors historically strong teams), and a pure NHL system (where teams could cheat the system by scheduling a bunch of easy opponents).
Bottom line, if you are in fact going to vote based on what is best for the league, adjusting our ranking system from Gonzalez to a Hybrid system is a NO-BRAINER!
Now, let’s dig in.
Background: Gonzalez System
Written by: Kevin Bailey
Here are the basics. We are proposing a 70-30 hybrid between two systems. Gonzalez, and NHL.
We can start with the Gonzalez System (much of this info was taken from the initial post on the NCDA website announcing the league’s move to this system back in Dec. 2016). The Gonzalez System is a computer ranking model similar to Elo and is a rating exchange system based on research performed by World Rugby.
The Rating Exchange is a scale from 0 to 2 rating points (for each individual match). The exchange directly corresponds to a Ratings Gap of 0 to 10 rating points. There are two types of exchanges.
A predicted result will have an exchange from 0 to 1 rating points.
A technical upset will have an exchange from 1 to 2 rating points.
Below is the formula for Gonzalez:
Short Formula: (1* ((1) * ((((45 + 0) – (49 + 1)) * 0.1) + 1))) = 0.500
*Note: since this image was created, the NCDA has switched to a 1 point home court modifier, rather than 3.
Now you know how the Gonzalez system works.
The Gonzalez rating for a team does not just disappear at the end of a season either. Instead, the current system calls for a 75% carryover from the previous season.
For the remaining portion of the rating, add 25% of the League Mean Rating at the end of the season is to the Team’s initial rating for the upcoming season.
Why is there a 75% carryover? A few reasons actually, but here is the logic that was described in the league’s formal description of the system: “Teams keep 75% of their rating from the previous season. This matches the data reported by Member Teams in the Summer 2016 off-season Proposal vote. On average, teams retained about 75% of their roster. This causes a minor hit to predictability (about 1-2%), but better reflects real world “losses” on rosters between seasons.”
Another reason for this carryover is simply to protect teams with high strength of schedule. As Colin will point out, our league is geographically bound. Schools don’t have the means to travel the entire country to play games, thus teams tend to play schools within their region more frequently than other regions.
Due to this, Michigan schools play each other very often, East Coast schools play each other very often, etc.
If each Gonzalez rating was reset at the start of a new season (reset is a 40.0 rating), then matches against the defending champ would be equal to matches against a new school at the start of the year. For a Michigan school forced (geographically) to go up against top tier teams more often, it would be unfair to them that teams from Georgia can rack up wins over much less skilled teams, and still accumulate significant Gonzalez points. The carryover is in place to protect from inconsistencies in that regard, and it does the job fairly well.
I am probably boring some people with the specifics behind this system, but here is the whole point: the Gonzalez System is a very well thought out system, that tends to be very accurate for its intended purpose of predicting the winner of a match.
Bottom line, the Gonzalez System can predict the winner of an NCDA match as good as any system we have been able to find. The issue with all of this is that our method of seeding teams should not be strictly based off a predictive system, but rather should be a system that rewards performance throughout the NCDA season, and places teams in the order in which they deserve based off merit. Much, much more on that below. Anyway, that is the Gonzalez System.
Background: NHL System
Written by: Kevin Bailey
The NHL System (aka the Champ System in the NCDA), is much more simple than what I just described with the Gonzalez System. Here is how the NHL System ranks teams:
Teams are ranked based on their point total. Points are earned a few ways:
Win = 2 points
Overtime Win = 2 points
Overtime Loss = 1 point
Loss = 0 points
That’s it! It is that simple.
Well, also here are the tie breakers if multiple teams finish with the same point total:
Tiebreaker – If two Member Teams have the same number of points, the following is used:
- Greater number of Champ Points
- Greater Win Percentage (W%) (wins over total games played)
- Greater number of Matches played
- Date the Team joined the NCDA (reflected as the lower Team ID)
Obviously there are issues with this system, but also some positives. Here are my takes on the NHL System as a ranking system for this beautiful thing we call college dodgeball.
PRO: It incentivizes scheduling more games. You only get points if you win games (or lose in OT), so this clearly gives teams the incentive to play as much as possible. The league is better when teams are traveling and playing tournaments frequently. The more dodgeball the better.
PRO: This system is really simple, so even dumb people will understand how it works. Hooray!
CON: First off, the most obvious argument against this system is that it doesn’t factor in strength of opponent at all. In this system, every opponent is created equal, and thus, every win is viewed as equal. Not exactly what we want, considering there is a huge discrepancy in strength of schedule throughout our league. Ideally we get to a place where every team plays an equal number of games, and each team’s slate is compatible from a strength of schedule standpoint. Sadly, we are not there yet as a league. Until we get to that place, the NHL System’s inability to factor in strength of opponent will be a flaw. Not good for this system’s resume when applying for the position of “NCDA RANKING SYSTEM”.
CON: The NHL season is 82 games. That means every team plays 82 regular season games. End of story. How many games is the NCDA season? Exactly. There is no regulation for number of games played in the NCDA, so this system clearly will lose accuracy compared to in a league where every team plays the same amount of games, and that variable is controlled. Once again, in an ideal world this issue won’t be an issue anymore. In an ideal NCDA, each team plays the same number of games, so this becomes a non-factor. We aren’t there yet as a league, but when that time comes, I’ll make sure to delete this section of my description…
Now let’s move on to a few sections written by Colin O’Brien that discuss these systems even further, before taking a deeper dive into our proposed hybrid system that is 70% Gonzalez / 30% NHL.
Defense of the Gonzalez System
Written by: Colin O’Brien
I will begin my portion of this proposal with a defense of the Gonzalez system as it currently stands.
The ultimate goal of the standings in its current existence is to fairly and accurately seed every team for the Sunday bracket at Nationals. That is the only time the rankings actually have any direct effect on the league.
Some may want standings to reflect the exact placement and ranking of teams throughout the season, but due to the current makeup of the league regarding number of games played and diversity of teams played, that is not a reality for any objective ranking system. I argue that the Gonzalez system does the job that it is designed to do.
The way the system works is simple and intuitive. If you beat a good team, you are rewarded more than beating a bad team. If you lose to a bad team, you are hurt more than if you lose to a good team. Throughout the course of the season, the proper adjustments are made to each team’s ranking, but it takes the full season for the proper corrections to be made. I will cite specific examples to help demonstrate such.
In the 2014-15 season, CMU began ranked 8th with a 42.301 ranking. They demonstrated throughout the season they were an elite team, and by the end of Saturday of Nationals they were ranked 2nd with a 53.434 ranking.
So far this season, we have seen two teams go in opposite directions. Ohio State started the season ranked 4th with a 47.706 ranking. They had a fairly lackluster season and now are ranked 10th with a 42.779 ranking. JMU started with a 43.827 ranking and have won consistently and now are ranked 3rd with a 49.161 ranking.
The 75% carry over makes sense because for the most part teams perform similarly to their past season. When they don’t, as in the three cases cited above, the proper adjustments take place by the end of the season.
Some people proclaim, “But Colin, in what other sport does the previous season have an impact on the rankings of the next season???!?!?! There should be no carry over!!!!!” To that, I have several responses.
- In other sports the schedule is regulated and everyone plays the same number of games.
- Not only do we not play the same number of games, but there is such an incredible variance in the strength of each team’s schedule.
- In our league more than any other good teams tend to remain good, while poor teams tend to remain poor. Each season some teams rise or fall, but it is generally a gradual process.
The carry over protects against several situations that are unavoidable in the current makeup of the league, which are directly related to the issues stated above. For instance, it protects against teams that play very few games. Here’s a very real, potential scenario. Two isolated teams exist in the league. They only can play each other all year due to travel constraints. One team is significantly better than the other and wins every time. Say the teams only play 6 times the whole year. Ignoring home court advantage for this specific example (which if each team alternated hosting it should relatively even out anyway), the 6-0 team would have a 43.689 ranking. That would be good for 9th place as the standings are right now. A carry over helps protect against this, as the losing team’s ranking would be lower each season and thus the better team would gain less for each win.
The carry over also protects against teams that play historically strong teams or historically weak. In general, good programs remain good year to year. With a carry over, teams such as the Michigan schools aren’t severely punished for losing to the four-time reigning national champion year after year. The converse is also true. It protects against overly rewarding teams that play historically weak teams all season long. The Gonzalez system naturally tends to correct itself in these cases, but the early matchups nonetheless would unfairly punish or reward certain teams based on their situation. Taking only 75% of the Gonzalez ranking each season protects the relative ranking of each team while also accounting for the year to year rollover of rosters.
If for some reason you don’t believe the statement that good teams tend to stay good, then just look at the Final Four the last five seasons.
2012: Kent State, GVSU, UK, SVSU
2013: Kent State, UK, MSU, GVSU
2014: JMU, MSU, SVSU, GVSU
2015: JMU, SVSU, CMU, GVSU
2016: OSU, MSU, CMU, GVSU
There are 20 Final Four spots above and only 8 teams represented, with every team represented multiple times except Ohio State. Upon closer examination, UK and Kent State appeared the first two years and then not since while CMU appeared the last two seasons while missing the first three. JMU also has a two season stretch in the middle. In our league seemingly more than any other, good programs tend to perform similarly to how they performed the previous season. This means carrying over 75% of a team’s ranking inherently makes sense.
Rebuttal Against “Point Differential”
Written by: Colin O’Brien
Another change to the Gonzalez system I’ve heard many times is an incorporation of point differential. Again, I am against such a change for several reasons
First, as Felix pointed out when this topic was brought up on the Captain’s Page a while back, accurate point stats have really only been kept the last two seasons or so. The Gonzalez system incorporates the years of accurate win-loss data and an incorporation of point differential would either result in an arbitrary start date or a complete reset of the system, which as I outlined above would not be good (that would be a year without carry over).
Second, the only two sports I can think of where point differential comes into effect is for tiebreaking procedures in soccer and the NHL, and that’s for the cumulative differential for the season. In soccer, very few goals are scored per game and ties are a regular occurrence, so goal differential makes sense in that regard. Furthermore, in the English Premiere League every team plays 38 games (each team twice), so there is a uniformity to the schedule that results in point differential being a fair statistic.
In the NHL, goal differential is the fourth tie breaking procedure. Again, teams play a uniform schedule of 82 games, with slight variations based on division. 82 games is a huge sample size and it once again results in goal differential being a more fair stat, but even then it’s only the fourth tiebreaker.
The NCDA this season had the highest team play 34 games and several teams play a number of matches in the single digits. There is no uniformity in the schedule, as some teams play strong teams almost all the time while others play weaker teams for the most part. Point differential would vary far too much across the league for it to be a fair variable in the ranking system.
Third, I will relate point differential directly to the NCDA in a practical manner and why it should never be used for a ranking system in the foreseeable future. My first defense would be I always want a team to try to win a game, not prevent losing by a larger margin. How many times do you see teams losing late in the game go all-out to try to tie the score. They put their bodies on the line and dare the other team to throw from close range in the hopes of the miracle catch. Those efforts would largely disappear with the incorporation of point differential into the ranking system. In all likelihood, the risk of likely giving up another point would outweigh the small chance of a comeback. So you know what you would see at the end of games? Stall ball by the losing team to minimize the loss. I don’t even want to imagine a world where that is the norm. When there is a possibility of victory, I want all teams to try and win, not try to lose by the lowest margin.
Furthermore, the most common argument I hear in favor of point differential is that beating a team by three is more convincing than beating a team by one. While that is true, too often points are won or lost at the end of games when the result is already decided. I’m not worried about teams running up the score late, as the common proposal is that there would be a cap where a point differential larger than 3 wouldn’t count for any more than a 3-point victory. I’m more concerned with the meaningless points scored at the end of games. Once the win-loss result of the game is effectively decided based on the combination of point margin and time remaining, teams should play to have fun and not worry about who takes a meaningless point at the end of a game.
Back in the Blast from the Past tournament, GVSU went up 3-0 on CMU halfway through the second half. GVSU rested their top players to let their subs get some playing time, the only time they saw the court on the day. If point differential mattered, those subs never see the court because GV couldn’t afford dropping a point. In fact, this event occurred as CMU ended up taking the last meaningless point to end the game 3-1. Teams would not be able to rest with the score until up four points. No team is coming back from 3 points down, at least not in any game I’ve ever heard of. So why would we create a scenario where that team has to keep trying their best to not give up a point when they are in no risk of losing the game?
Once the win-loss result of a game is determined, teams should use the rest of the game to have fun and allow their subs to get some game time.
Hybrid 70-30 Ratio Explanation
Written by: Colin O’Brien
Now that I have defended the Gonzalez system as a whole, it is now time to explain why I feel a 70% Gonzalez – 30% NHL rankings split would be good for the league.
Two things make up a large percentage of why people act the way they do: motivations and incentives. The split of the two system helps create incentives that help promote the what is desired in the league in regard to team scheduling.
Back when the NCDA exclusively used the NHL system, Kent State cruised to a 1 seed largely by playing and beating bad teams all season long. To be fair to Kent, they played 28 matches that season, 10(!) more than the next highest team. This example shows both the good and the bad incentives created by the NHL system.
We want teams to play a lot of matches throughout the season. We don’t want to overly reward a team simply for playing and beating bad teams over and over. The incentives created by the NHL system is to play many matches, regardless of quality. Thus, we need to pair the NHL system with one that controls for quality of opponent.
Enter the Gonzalez system. Under the Gonzalez system, a good team could beat a bad team over and over and receive very little rankings benefit. Under this system, it would make no sense purely from a rankings standpoint for an “elite” team to schedule a “bad” team, because they have nothing to gain and everything to lose
This situation happened just last weekend when recent success by GVSU and recent slumping by MSU meant GVSU would receive the minimum exchange for a win and maximum for a loss. If the second-place team was closer to Grand Valley, they might have avoided scheduling MSU do the huge risk and little reward under the Gonzalez system.
We don’t ever want a team to not play a match or tournament purely out of fear of the ranking system. We want to encourage teams to play as often as possible. Again, this example shows the good and bad incentives of the Gonzalez system: it protects against merely beating up on bad teams, but it offers little incentive to play more matches especially when ranked significantly above a potential opponent.
A pairing of the two systems creates a balanced incentive system that also ranks teams fairly, both on quality and merit. Scheduling constraints are a real thing. The Michigan schools should not be overly punished merely because the quality of play among each’s three closest opponents is high.
Conversely, other regions should not be punished merely because of a lack of several top teams. The pairing of the systems protects against quality of opponent, and rewards teams for beating high quality opponents, while simultaneously also rewarding teams for scheduling, playing, and winning matches.
I’m going to finish by addressing a concern brought up on the Captain’s Page by our Director of Officiating, Zigmas Maloni, that seemingly directly addressed this proposal a week before it was even announced.
“Don’t dilute the Gonzalez ratings by saying “Gonzalez standings = 75% and Champ standings = 25%”. Diluting the Gonzalez ratings to pure integers is a horrible waste of the system, which is meant to evaluate a Team’s strength. We might not be able to tell the difference between a 1 seed, 2 seed, and 5 seed, but we can tell the difference between a 56.593 rating, a 49.717 rating, a 48.708 rating.”
I agree 100% with the last sentence. However, we already dilute the Gonzalez system to pure integers when we seed the teams on Sunday of Nationals. In fact, just as I said as I opened my rationale to this proposal, diluting a rankings system to pure integers is the exact purpose of any ranking system for our league. I don’t believe creating a Gonzalez-NHL split dilutes the ranking system. Instead, I think it provides better incentives for the current makeup of the NCDA. I don’t mean to claim this is a perfect system, but I believe it’s better and a step in the right direction from what we have currently. With that being said, there’s always room for improvement and I will continue to look for ways to tweak and add to the ranking system so that it best suits the needs and aspects of our league. I stand behind the Gonzalez system as is, but I think adding a NHL system component into the rankings equation better suits the NCDA as it currently stands. As our league continues to expand and develop, additional change will likely prove necessary. But until then, I think this new proposed system offers positive change.
Ranking System Examples / Comparisons
Written by: Kevin Bailey
After hours of testing different ratios, the system that Colin and I concluded was the perfect balance for our league right now was a 70-30 ratio. As mentioned above, this is a happy medium between the two different systems (Gonzalez and NHL).
Below, are a few actual rankings examples that will help prove our point. To start, I will use the 2012 season as a case study for this.
After that, I will give you a 2017 rankings example. I will provide a (side by side by side) comparison of our hybrid system next to the Gonzalez and NHL systems. This should help justify our reasoning, and prove that our system is in fact, a more balanced method for seeding teams.
2012 is an interesting example because there are observations to be made on both ends of the spectrum. On one end, Central Michigan was coming off a national title, and with an unbelievably talented team, they proceeded to play only 5 regular season games before Nationals. On the other hand, Kent State was not a GREAT team, but they understood that the NCDA was currently using an NHL System, so they went ahead and scheduled plenty of games against weak opponents.
The results are easy to see when examining the rankings for this season under both the Gonzalez System and the NHL System. If you are not yet convinced that we need to find a middle ground between these two methods, the following rankings charts will help you realize this.
Gonzalez gives historically strong teams the ability to not play many games and grab a seeding they don’t deserve. Don’t believe me? Here is what the top 10 of the standings would’ve been in 2012 under a 100% Gonzalez system… I’ll put them side by side with the team’s regular season record so you can easily see the issue.
Yeah, a 3-2 team should not be able to take the #1 seed, and any system that doesn’t protect against that is clearly flawed.
Now let’s take a look at the top 10 under a 100% NHL system:
Now, these seedings might not look as bad at first… until you consider a few things. First of all, Kent took advantage of the NHL system and scheduled a bunch of weak opponents to easily boost their NHL point total. Really smart move by Kent, but a system that doesn’t protect against this is not a good standalone system.
Here is the list of teams Kent State beat that regular season:
Only one of these opponents was even a top 5 level team in the NCDA that year (UK). Keep in mind, this is back when we barely had double-digit teams actively competing throughout the season. So Kent State essentially grabbed all their wins from teams in the bottom half of the league. Good for them for getting significantly more wins than every other team. That should be rewarded, but not to the extent of a #1 overall ranking, unless you earn it by getting some QUALITY wins.
The NHL System is based solely on the number of wins you get, so Kent State understood that and executed their strategy well.
I am all for a system that incentivizes scheduling more matches. The 70-30 system will still do this, in fact, it will be better than pure NHL in a way, because it incentivizes scheduling lots of games, with more weight given to wins over top level teams. Teams won’t be able to just schedule cupcakes. They will have to branch out and play tougher opponents, if they intend to get a very high seed for Nationals. The mix of Gonzalez and NHL gives us balance here.
Now, let’s move on to a year of dodgeball that all of you are more familiar with: this year!
Let’s talk about this year:
Anyone who has paid any attention to the standings throughout the year noticed that under the Gonzalez System, CMU was able to claim the #2 spot with a poor 3-6 record. A lot has changed since then, with CMU getting a lot more games under their belt, including some quality wins over teams like MSU (twice in a month), and Kent State.
Yet, the issue was easy to see earlier in the year. CMU made it to the title game each of the past two seasons, so it is clear that their Gonzalez rating would be high. But, it was disappointing to see our ranking system put a 3-6 team at #2, above so many other deserving clubs (those three wins were against BSGU, Kent, and an SVSU team down like 6 starters).
With all of that said, I will let you examine the ranking for yourself. Listed below are three separate rankings. On the far left is the Gonzalez, in the middle is our proposed hybrid 70-30 system, an on the right is the NHL system. Without a doubt the middle ranking is the most fair. Take a look:
2017 Rankings System Comparisons:
You know what? Screw it. We showed you 2012 and 2017. Before we move on to our conclusion section, we may as well complete the circle by showing rankings comparisons for 2013-2016. Listed below, in order of more recent, are the Gonzalez/Hybrid/NHL rankings comparisons for each of those seasons… for your viewing pleasure.
Not as many severe differences in these years, but that is because there were less outlier teams that were far on either end of the spectrum, like what we saw in 2012 with CMU and Kent State. Sometimes the Gonzalez system or NHL system can be close to accurate for seeding teams, but there are situations where each system is extremely flawed. The hybrid system protects us from both extremes. Take a look at years 2016, 2015, 2014, and 2013 below:
2016 Rankings System Comparisons:
2015 Rankings System Comparisons:
2014 Rankings System Comparisons:
2013 Rankings System Comparisons:
Written by: Kevin Bailey
Listen folks, we need to put a better ranking system in place. One that is fair in its placement of teams for the Nationals seeding. The Hybrid 70-30 system that Colin and I have worked very hard to propose, is in our minds, the best option for our league currently.
We all want what is best for this league, and moving to a more balanced ranking system will be a huge step in the right direction. This will help our league grow, and improve, and become more competitive going forward. This will help make College Dodgeball better, and that’s what it’s all about.
Colin and I have a strong feeling that almost everyone in the league will be on board with this proposal of an adjusted ranking system that seeds teams in the position they deserve, rather than a predictive system.
With that said, if you do have an issue with our system of choice, I will relay a well known saying to you: “Don’t oppose unless you can propose”. In other words, don’t oppose our system unless you have a better idea to propose.
Truth be told, Colin and I are not married to this 70-30 hybrid. But we are 100% committed to finding a better system. If someone proposes a different way to rank teams that achieves this balance better than our 70-30 system, we will be all for it. But as of now, this is far and away the best option we have come across.
We hope you do the right thing this year, and vote for our Hybrid 70-30 Ranking System. It will greatly benefit College Dodgeball now and in the future!
4 thoughts on “Hybrid Ranking System Proposal”
100 percent on board.
I’d like it better if you were more 70% / 30%
I’ve changed my position on the topic due to Kevin being a straight up SAVAGE*
*The Offiicial Apparel Partner of the National Collegiate Dodgeball Association*
“You get more points for playing more, better teams.”
“Point differentials are the devil”