Rating System

Overview

The rating system I have created is based on a system devised by Arpad Elo to rank Chess players.  I am not the first person to create such a system for Snooker, and I’m sure that I won’t be the last either.

The ratings are designed to reflect the strength of a player relative to all other players, so if Player A (rating = 1,000) plays Player B (rating = 500) then you would expect Player A to win twice as many frames as Player B.

The ratings are updated at the end of every tournament, with the adjustment to a player’s rating based on the actual number of frames they won, compared with the number of frames you would expect them to win given the strength of the opposition they faced.  The intention is therefore to update the estimation of the relative difference between the players based on this new evidence; the system does not automatically reward players for reaching a particular stage of the competition.

This type of rating system is in many ways the purest way to rank players according to their ability and I hope that it will be of interest to followers of the game.  I believe that this alternative system highlights the current bias against lower ranked players, and offers a more accurate picture of a player’s ability.  It can also be incorporated into models simulating changes to the Tour and I believe that it will be a valuable tool for assessing any alternative structures that are proposed.

 

What information has been used in order to create these ratings?

To create the ratings I have used the results of matches played in every ranking event from 2004/05 onwards.  Note that I have not included pre-qualifying rounds for the World Open, World Championship or PTC events – I’ve just counted results from the first round in which professionals entered the tournament.

I have also incorporated results from The Masters tournament (including qualifying matches), which is the most prestigious of the invitation events.  I have not included results from any other non-ranking event, although these could be added at a later date.

[2015/16 update – I have now included the Champion of Champions tournament (introduced in 2013/14) and the World Grand Prix (introduced in 2014/15)]

 

How are the ratings updated?

At the end of each tournament each player’s rating is updated using the following formula, where the expected frames won is based on the player’s current rating compared with their opposition’s rating.

New rating = Current rating + [actual frames won – expected frames won] x weight

For example, Player A (rating = 400) plays 7 frames against Player B (rating = 300) and 10 frames against Player C (rating = 600).  You would expect Player A to win a total of 8 frames (see calculation below), so if he actually won more than 8 his rating would increase, while if he won less than 8 his rating would decrease.  The magnitude of the change is bigger the further away from the expected result.

Player A (400) v Player B (300)

Expected frames won by Player A = Frames played x Player A rating / (Player A rating + Player B rating)

Expected frames won by Player A = 7 x 400 / (400 + 300) = 2800 / 700 = 4

Player A (400) v Player C (600)

Expected frames won by Player A = frames played x Player A rating / (Player A rating + Player C rating)

Expected frames won by Player A = 10 x 400 / (400 + 600) = 4000 / 1000 = 4

[2015/16 update – For Version 3 of the rating system I changed the way that I calculated the expected frames won over the course of a tournament.  In this version the expected frames won are calculated separately for each match and then summed up, so in the above example, the expected number of frames won by Player A = 4 + 4 = 8.

In Version 2, the calculation I was performing was slightly different:

Expected frames won by Player A = frames played x Player A rating / (Player A rating + weighted average rating of Player B and Player C)  (where the weighting takes into account the number of frames played against each opponent).

In the above example, Expected frames won by Player A = 17 x 400 / (400 + (7 x 300  + 10 x 600) / 17) = 7.76.

 

What is the weight and how is it used?

The system allows you to apply different weights to the calculation – the higher the weight the greater the emphasis on recent results.  The weight can be altered depending on the type of tournament or could be based on the current rating of the player.

For this version of the rating system I have simply weighted all results the same, with a factor of 10.  In the above example this means that Player A’s rating would increase by 10pts if he actually won 9 out of 17 frames, by 20pts if he won 10 out of 17 frames, and so on.  Similarly, Player A’s rating would decrease by 10pts if he only won 7 out of 17, or by 20pts if he just won 6 frames in the two matches.

New rating for Player A = Current Rating + [actual frames won – expected frames won] x weight

New rating for Player A = 400 + [f – 8] x 10

(where f = actual number of frames won)

[2012/13 Update:  For Version 2 of the rating system I have added an extra step at the end of each tournament to re-scale the results so that the average rating remains constant.  Without this adjustment there is a tendency for the ratings to gradually increase over a time (roughly a point a tournament) making any comparison across years less informative.]

 

What rating is given to new players?

Every player who has competed in a professional tournament since 2004/5 has been included in this rating system.  An initial rating is assigned to all new players (350 for professionals, 300 for amateurs).  Although it is adjusted after every event, this rating is considered to be provisional until the player has played at least 100 frames; at which point you would expect the rating to provide a reasonable guide to the player’s true ability.

[2012/13 Update:  For Version 2 I have revised the initial rating awarded to players to 300 for new professionals and 200 for amateurs.  This partly reflects the fact that a larger number of amateurs are now able to enter professional tournaments, representing a wider range of abilities.  Analysis of results over the last few years indicates that setting the initial ratings at this level is a fair reflection of the ability of these player relative to their more experienced competitors.]
What happens if a player doesn’t compete in a tournament?

A player’s rating is not updated if they don’t participate in an event; we have no information to suggest that their rating has changed.  It would be counter-productive to penalise a player for missing an event as their rating is likely to simply re-adjust the next time they do play.  There may, however, be an argument for reducing a player’s rating if they miss a number of events in succession – in theory you may expect their inactivity to affect their subsequent performances.

 

Will the ratings be updated after the qualifying rounds for a tournament have been completed?

For this version of the system I have taken the decision to only update the ratings once the final stages of a tournament have been completed – so there will be a time lag before results of qualifying matches will be taken into account.  This is consistent with the way that the official world rankings are updated.

 

Should PTC events be given a lower weight than full ranking events?

There is an argument for applying a lower weight for the PTC events –these are essentially minor ranking events and some players have admitted that they take these less seriously.  On the other hand, these events are particularly valuable for lower ranked players and I am reluctant to model them as being less important.

There is actually some variation by tournament anyway due to the number of frames played (PTC events are best of 7 frames, whereas the World Championship are at least best of 19).  The more frames played in a competition, the greater potential there is for a player’s rating to change.

It is worth noting, however, that the PTC events are not evenly distributed throughout the season, so at certain points of the year they will have a relatively large influence over a player’s rating.

 

Could this system replace the current system of ranking players?

I don’t think that this system would be suitable for ranking top players.  At this level of the game, the main objective for players is to compete in the latter stages of tournaments, and ultimately win them.  The ranking system therefore needs to be set up to reward this.

Lower down the rankings the emphasis is slightly different and it is more important to ensure that players are accurately ranked relative to one another.  This is where the current structure of the Tour and the ranking system are relatively weak – the inherent biases in the system mean that the world rankings do not accurately reflect current ability.  This alternative rating system is more powerful, taking into account every frame played and the relative strength of opposition players.  There would still be strong reasons against adopting it as the main ranking system – particularly given the current tiered structure of the Tour – but it certainly has more value at this end of the professional game.

The type of system could be used most effectively in the amateur game.  It isn’t something that I’ve looked into yet, but I would welcome thoughts and suggestions about this.

 

How do you plan to use these ratings?

It is possible to use these ratings to help preview tournaments – identifying players with relatively easy / tough draws and identifying potential upsets.  They can also be used to help assess the fairness of different Tour structures.

 

James Collingwood – 1st November 2011

Updated on 13th June 2015

 

Leave a Reply

Your email address will not be published. Required fields are marked *