Add Reply Create a new Topic Start a Poll

 *Preliminary NSFL ELO Ratings
Beaver
 Posted: Nov 15 2017, 02:19 PM
Quote

N/A

Group:
Updater
Member ID:
487
Member Since:
26-September 17
Total Post:
209

N/A



I'm posting this in the hopes of getting some feedback to improve the methodology so if you have any suggestions or questions I'd love to hear them and if you see anything that looks glaringly wrong point it out because there is a non-zero chance that I fucked up somewhere along the line. These are far from being finished (see the last section for some of the flaws) so don't get too butthurt if you think your team is lower than it should be, etc.

Alright so I've been working through historical ELO Ratings for the NSFL and I've settled on these parameters for now:
  • 1500 ELO will be average and each team starts out there, including expansion teams
  • Beating a better team will increase your rating by more than beating a worse team (ex: in Week 14 last season ARI beat LVL 45-0 but only gained 3 ELO points, in the first playoff round they beat OCO 27-7 and gained 16)
  • Margin of victory matters but has diminishing returns (ex: in Week 6 last season BAL was 104 ELO points better than SJS, beat them 20-17, and gained 7 ELO points; 2 weeks later PHI was 102 ELO points better than COL, beat them 40-20, and gained 26 ELO points)
  • Preseason games don't count and playoff games are weighted the same as regular season games
  • All games are zero-sum: if one team goes up 20 points, the other team goes down 20. This means that each week that every team plays the average ELO is 1500 and doesn't fluctuate.
  • 25% season-to-season regression (that is, the start of season ELO Rating is 75% of your previous end of season ELO Rating and 25% average)
user posted image

The line graph looked ugly as sin when I had bye weeks as breaks in the lines so the flat parts of the lines are typically when teams aren't playing. (PS: I hate whoever's idea it was to have bye weeks in Season 2)
S1W1 = Season 1 Week 1
S1P1 = Season 1 Playoff Round 1
Considering this is just preliminary I didn't fancy up the graph too much (x axis labels are uggo and I'm going to emphasize the demarcation between seasons more in the final version).

Current ELO Ratings:
1. Arizona - 1771
2. Orange County - 1628
3. Baltimore - 1571
4. Yellowknife - 1499
5. San Jose - 1490
6. Philadelphia - 1472
7. Las Vegas - 1329
8. Colorado - 1238

Some fun facts (keep in mind this is based on preliminary numbers):
  • Biggest ELO gain to date: Season 3 Week 5 when San Jose gained 80 points after beating Colorado 40-0
  • Biggest underdog to win: Season 3 Week 12 when Balitmore beat Arizona 26-23 with a ~19% win probability
  • Baltimore has the most wins as an underdog with 16
  • Arizona has won their only 2 games played as an underdog: Season 1 Week 9 against Orange County 19-7 and Season 1 Week 12 against Orange County 23-3
  • Las Vegas has been a favorite in just 5 of their 33 games played (15.15%), San Jose is just ahead with 7 in 46 (15.22%)
  • Yellowknife has lost as a favorite 16 times, most in the NSFL
  • Favorites overall are 115-71 (61.83%)
  • Yellowknife has been the steadiest team, never dipping below 1470 ELO (2nd highest floor behind ARI's 1501) and maxing out at 1576 ELO
  • Colorado has fluctuated the most, recording the 3rd highest peak at 1593 ELO (behind ARI's 1828 and OCO's 1670) as well as the lowest valley at 1238 ELO
  • San Jose has the lowest peak ELO of any team, maxing out at 1503 ELO in Week 2 of Season 1 and nearly matching that with 1501 in Week 4 of Season 2
Going forward some things I'm going to look at in an effort to improve these are:
  • Introduce home field advantage: this completely slipped my mind until I was typing this up so I'm posting all of this without taking home field advantage into account which is a glaring oversight, luckily Home/Away shouldn't be hard to pull from the data so it's just a matter of figuring out exactly how much home field is worth
  • Test the variables so make the ratings as predictive as possible (specifically how swingy the ratings are from game to game - don't want them too static but also don't want them to overreact)
  • Perhaps weight playoff games more than regular season games (I'm hesitant to do this because a team that makes the Ultimus and loses shouldn't have their rating crater below teams that didn't make the playoffs)
  • Whatever other ideas y'all may have


GRADED

This post has been edited by jparks98: Nov 18 2017, 12:14 PM

--------------------
Blackford Oakes - FS
Player Page || Updates
^
7hawk77
 Posted: Nov 15 2017, 02:22 PM
Quote

N/A

Group:
Grader
Member ID:
158
Member Since:
25-May 17
Total Post:
1834

N/A



Excellent write up.

Well done!

--------------------
user posted image
user posted imageuser posted imageuser posted imageuser posted image

^
ErMurazor
 Posted: Nov 15 2017, 02:36 PM
Quote

N/A

Group:
Members
Member ID:
76
Member Since:
24-May 17
Total Post:
3029





Nice. I like this. You should include Arizona' preseason S1 ELO of -9000.

This post has been edited by ErMurazor: Nov 15 2017, 02:36 PM

--------------------





Player Updates Profile

user posted image
^
bovovovo
 Posted: Nov 15 2017, 02:40 PM
Quote

N/A

Group:
NSFL GM
Member ID:
266
Member Since:
26-June 17
Total Post:
1091

N/A



This is very awesome. I love the graph, I'm a sucker for data visualization //files.jcink.net/html/emoticons/biggrin.gif

I do think it's important to incorporate home field advantage because we know the sim puts a lot of value in it.


QUOTE (Beaver @ Nov 15 2017, 04:19 PM)
Yellowknife has lost as a favorite 16 times, most in the NSFL


This confirms my world view


--------------------


Player Page ... Update Page
^
manicmav36
 Posted: Nov 15 2017, 03:03 PM
Quote

N/A

Group:
Updater
Member ID:
295
Member Since:
7-July 17
Total Post:
1069

N/A



This is awesome stuff! I'd love to see this through the end of the season. I imagine things will change quite a bit when you include home-field advantage.

--------------------
user posted image

user posted imageuser posted image
^
AdamS
 Posted: Nov 15 2017, 04:00 PM
Quote

N/A

Group:
NSFL GM
Member ID:
452
Member Since:
1-September 17
Total Post:
425





hot damn that's good work

--------------------
This space for rent.

user posted image

^
37thchamber
 Posted: Nov 15 2017, 11:39 PM
Quote

N/A

Group:
Banker
Member ID:
125
Member Since:
25-May 17
Total Post:
684

N/A



I suspect your k factor is a little on the high side. An 80 pt gain is a 160 pt swing. That's way too much variance for one result IMO and a possible cause for rank inflation. This would also allow you to weight playoff games more. I'd probably go for 24 in the regular season and 32 in the playoffs. Just need to make sure your margin of victory adjustments don't allow it to extend beyond those limits probably.

Home advantage is also a must. In any implementation of the Elo rating system where home field is a factor, it is used (typically by adjusting rankings in the calculation) because it hurts the rating system to exclude it (and you get rank inflation). I'd suggest a rank advantage of about 150 pts as a test. I think it was JuOSu who did it before but can't remember what adjustment was made.


Solid stuff though. You beat me to it with the margin of victory implementation (I was planning to use a variant of the system used for soccer) which was one of the things I noted as being missing from the last implementation we saw of Elo here.




(Also why does everyone call these ELO ratings? It's Elo. Elo is a dude's name.)

This post has been edited by 37thchamber: Nov 15 2017, 11:43 PM

--------------------


user posted image

user posted imageuser posted image
^
ADwyer87
 Posted: Nov 16 2017, 12:35 AM
Quote

The Heckin Bamboozler

Group:
Head Office
Member ID:
19
Member Since:
12-May 17
Total Post:
2131





holy jamoly

--------------------
user posted image





^
JuOSu
 Posted: Nov 16 2017, 01:15 AM
Quote

N/A

Group:
Members
Member ID:
408
Member Since:
21-August 17
Total Post:
353





Excellent stuff, though I disagree with some of your choices. You should go back and read my first post of my ELO ratings to see some of the different numbers I used. Otherwise great work.

--------------------
user posted image
^
Beaver
 Posted: Nov 16 2017, 10:01 AM
Quote

N/A

Group:
Updater
Member ID:
487
Member Since:
26-September 17
Total Post:
209

N/A



Thanks everybody, this is definitely a work in progress but it's a fun little project to play around with.

QUOTE (37thchamber @ Nov 16 2017, 01:39 AM)
I suspect your k factor is a little on the high side. An 80 pt gain is a 160 pt swing.  That's way too much variance for one result IMO and a possible cause for rank inflation. This would also allow you to weight playoff games more. I'd probably go for 24 in the regular season and 32 in the playoffs. Just need to make sure your margin of victory adjustments don't allow it to extend beyond those limits probably.

Yeah that makes sense. I went with a k factor of 20 as a starting value since that's about what real life sports tend to be around so I think that 160 point swing was either just a massive outlier or my MOV adjustment is too lenient - bumping the k factor up to 24 turns that into a 192 point swing. The mean swing with these numbers is 42 points with a standard deviation of 32 so that 160 point swing was 3 and a half standard deviations above average. That standard deviation is a bit high for my liking but I'll revisit this after I make the adjustments, especially home field, as making the k value more precise is one of my priorities going forward.

QUOTE (37thchamber @ Nov 16 2017, 01:39 AM)
Home advantage is also a must. In any implementation of the Elo rating system where home field is a factor, it is used (typically by adjusting rankings in the calculation) because it hurts the rating system to exclude it (and you get rank inflation). I'd suggest a rank advantage of about 150 pts as a test. I think it was JuOSu who did it before but can't remember what adjustment was made.

Yeah, this started out as an SHL project where I don't think there's a home advantage and I just completely blanked on including it. Unfortunately I'm a fairly casual NSFL member so I'm not super familiar with the sim we use. It looks like in an evenly matched game having home field would make win probability increase from 50% to 70% with a 150 point adjustment. Does that sound about right with the way the sim works? Time to do some testing.

QUOTE (37thchamber @ Nov 16 2017, 01:39 AM)
Solid stuff though. You beat me to it with the margin of victory implementation (I was planning to use a variant of the system used for soccer) which was one of the things I noted as being missing from the last implementation we saw of Elo here.

I leaned heavily on this article (among others) in developing this and based my margin of victory adjustment on theirs (specifically footnote 2). This was fortunate for me because I had originally planned on going with something similar to the soccer system but I didn't really want a piecewise function out of my own laziness.

QUOTE (37thchamber @ Nov 16 2017, 01:39 AM)
(Also why does everyone call these ELO ratings? It's Elo. Elo is a dude's name.)

That's a very good question. Habit, I suppose //files.jcink.net/html/emoticons/wink.gif

QUOTE (JuOSu @ Nov 16 2017, 03:15 AM)
Excellent stuff, though I disagree with some of your choices. You should go back and read my first post of my ELO ratings to see some of the different numbers I used. Otherwise great work.

As usual you beat me to it lmao. I'll definitely dig that up, appreciate the heads up.

--------------------
Blackford Oakes - FS
Player Page || Updates
^
0 User(s) are reading this topic (0 Guests and 0 Anonymous Users)
0 Members:

View Topic Options Add Reply Fast Reply Create a new Topic Start a Poll



 


 


Mobile Version Last Visit: --