The End of RPI?

[Edrick]

The Sagarin Ratings are already what they seemingly are trying to accomplish.

Here is it's explanation...

The PREDICTOR, is such that the score is the only thing that matters.
PREDICTOR is also known as PURE_POINTS, BALLANTINE, RHEINGOLD, WHITE OWL
and is a very good PREDICTOR of future games.
----------------------------------------------------------------------------------------------------------------
GOLDEN_MEAN also utilizes the actual SCORES of the games in a different way but is also completely SCORE BASED
and thus should be a good match for the PURE POINTS in terms of predictive accuracy for upcoming games.
----------------------------------------------------------------------------------------------------------------
The RECENT, is score-based and weights RECENT play more heavily than earlier games. Its effect will become
more pronounced the longer a season goes if a given team happens to have an upward or downward trend.
----------------------------------------------------------------------------------------------------------------
The overall RATING is a synthesis of the three different SCORE-BASED methods, PREDICTOR(PURE_POINTS), GOLDEN_MEAN,
and RECENT and thus should be a good predictor in its own right.

If they wanted to do this, they've had something that has done it for years.

I doubt much of anything will come out of the talks and RPI is still the 'official' metric but every Committee member continues to use whatever mix of analysis they deem right.

[Fieldhouse Flyer]

Not sure I like this. Advanced analytics are cool discuss and look at, but I don't think they should play a large role in determining at-large bids - that should strictly be about results.

The NCAA.com article linked in post # 1 addressed that, and is well worth a detailed read.

http://www.ncaa.com/news/basketball-men ... s-involves

Another hurdle is making sure metrics are a valuable resource, but not the end-all.

“I think always the most critical things in terms of determining where teams are placed is always going to be who you play and who you beat and where you played those games." said Jim Schaus, the Ohio University athletic director who will be representing the Division I Men’s Basketball Committee.

I also found this excerpt instructive:

There’s a reason for this meeting of the math minds. The NABC, representing the coaches, has sent a message that it would like to look into the use of more advanced metrics in the selection and seeding process. An even more powerful microscope to go with the time-honored RPI. The NCAA listened and agreed. A group of coaches and committee members is now at work, and this get-together is for everyone to hear the possibilities for the future — from those who know.

Dan Gavitt, the NCAA senior vice president of basketball, said: "The committee has some pretty strong thoughts on what a composite metric could look like and maybe should look like. I also think they recognize their fallibility in advanced mathematical and analytical areas, so they feel that it’s important to engage experts in the field.”

So are advanced metrics coming to an NCAA tournament bracket near you? Yep, and soon. Gavitt can give several reasons why.

Jim Schaus (who will be representing the Division I Men’s Basketball Committee) is a Mid-American Conference AD, and NOT a Power 5 conference guy. He won't be pushing anyone to make changes that will benefit the P5 conferences.

NCAA Tournament Selection Committee - Members and Committee's Role - NCAA.com

Joseph R. Castiglione - Committee chair and director of athletics at University of Oklahoma

Other committee members include:

Joe Alleva, director of athletics, Louisiana State University
Janet Cone, director of athletics, University of North Carolina at Asheville
Tom Holmoe, director of athletics, Brigham Young University
Mark Hollis, director of athletics, Michigan State University
Bernard Muir, director of athletics, Stanford University
Bruce Rasmussen, director of athletics, Creighton University
Peter Roby, director of athletics, Northeastern University
Jim Schaus, director of athletics, Ohio University
Kevin White, director of athletics, Duke University

[sciencejay]

I don't love the RPI, but unless they transition to a meta-rankings system where they take multiple, independent systems (RPI, Sagarin, KP, BPI, etc), I fear that the Football 5 will figure out how to use a newer, single system to keep the rest of us out. And yes, I include the BEast in that group of 'those left out'. We can likely agree that there is so much money in the system (BB, Football), that everyone wants to control/obtain as much money/power as possible. And while BB is significant, it pales in comparison to football, so the football 5 remain at the top of the food chain.

Here's where I think they can 'game' the system (if a single metrics is settled upon): each of the rankings systems has its own biases. If they are all sufficiently different (obviously, there is a lot of overlap with SOS, W-L %, etc), then averaging multiple systems in a meta-system should prevent conferences/teams from gaming the system. However, if the F5 decide that they will only play each other in the non-con, everyone else gets screwed. One of the inherent problems with all of the computer systems is that they all start off with a guesstimation of who is good and who isn't before the season even starts. You beat Duke and that's a 'great' win because Duke is good. So if the F5 don't play anyone outsiders in the non-con, they insulate themselves from interlopers who might steal 'good' wins. I can't remember the exact year, but around 2005 (maybe it was 2006 the year Missouri St got screwed), MVC teams had a huge number of great non-con wins. So then all their conference games came with higher RPI considerations (much like it is for the BEast this year--not comparing actual talent/competitiveness nationally between the two situations, just the quality non-con wins and the RPI benefit), and more teams went into selection Sunday with high RPIs. Jay Bilas even said something like MVC teams were 'gaming' the RPI like the whiny biatch that he is. Point being, that year the MVC went on the road in the non-con and earned good wins (seems like WSU winning at Syracuse was one of them). After that, I think it was even harder for MVC teams to schedule games with the big boys. Unless they play them in a tourney, they don't get to play them.

I'm not saying that the BEast will get locked out in the cold, but you can't deny that big money will make people do less than honest things. The way the NCAA tourney money is paid out, they could ensure themselves more money by simply locking everyone else out of the party. Winning a non-F5 conference may then be the only way to get into the big dance. Is this apocalyptic sounding, yes, but not that far-fetched in my mind.

I am totally in favor of taking multiple systems and coming up with an averaging equation that everyone could see clearly and understand what the committee is using for metrics input during the decision-making process. There will still be human bias--committee member X overvalues a win at Kentucky (in a year when UK is good, but not great), just because of the name, and undervalues a win over someone else who is typically mediocre, but it really good that year. But if we fans have a clearer idea of what metrics are used and how they are used, we would all be happier.

The MVC in 05-06 didn't have a number of great OOC wins. They had 2 top 25 and 6 top 50 wins. 15-15 vs top 100 teams.

compare to this year's Big East OOC just top 4....
Nova 1 top 25 win, 3 top 50 wins, 8 top 100 wins
Creighton 5 top 50 wins, 5 top 100 wins
Butler 2 top 25 wins, 3 top 50 wins, 7 top 100 wins
Xavier 2 top 50 wins, 3 top 100 wins

so just top 4 have 3 top 25, 13 top 50, and 23 top 100 wins. Teams 5-10 have 1, 4, and 8 wins btw- so as a conference it's 4 top 25 wins, 17 top 50 wins, and 31 top 100 wins. So Big East this year has more top 100 wins than the MVC back then had top 100 games.

Also, you are confusing years. The Wichita win @ Syracuse was in 2006-07 season, not 2005-06 season.

Where the MVC did game the system was they had 7 games vs non D1 schools. So yes, the Valley did game the system(just like the MWC did a few years ago).

looking the year that Wichita did beat Syracuse-
MVC had 1 top 25 win, 8 top 50 wins, and 21 top 100 wins. Went 21-21 vs top 100. So even there, no where close to the Big East this year.

Stever: here we go again. The only way to 'game' the RPI or any system is to win games! You gave a lot of stats trying to poke holes in the portions of my argument where I already stated either uncertainty (the year the MVC was accused of gaming the RPI) or stated that the analogy with the current BEast wasn't meant to be entirely direct. Reasonable people get that, but you can't help 'correcting' anyone with a bunch of stats. I often appreciate the way you dig in on stats--I also think it's pathological that you think of those stats as absolute arbiters of being right. People use analogies to emphasize commonality between two scenarios (as I did in my post), not to indicate that they are identical situations in every respect. Somehow you are not wired to recognize subtleties like that. You're like Sheldon Cooper.

Let me be clear: this year's BEast is WWWAAAAAAAAAYYYYYYY better than the MVC of the 2000s. When conference teams have successful non-con runs, that improves their individual RPIs, which in turn improves the RPI ramifications of every conference game. But conferences that win big OOC games get RPI boosts that help them come selection Sunday. That is confusing to no one. The BEast will get lots of bids this year, in part (but not only lest you start mining more stats to argue against the periphery of my statement) because so many teams have strong RPIs.

Again, the point of this thread is that the RPI may go by the wayside due to its shortcomings that have been described at length. My comments were twofold: they were directed at whiners who were upset that their team (with faulty resumes) didn't get into the big dance. Stop whining and win games. RPI or not, that's how you get into the tourney. Second, I think a combine metrics system (a la Bracketmatrix, which I'm a huge fan of) solves a lot of problems due to the inclusion of so many algorithms that all have biases, but those biases are likely weighted differently each in algorithm, so they wash out in the end).

[Edrick]

A disproportionate part of RPI is SOS. You can effectively game RPI by simply removing low RPI opponents (which the MVC did by stipulating as much). It mostly isn't about winning, its about playing those who win. That is the game.

Everyone came to understand it, and that's when the Committee started being liberal with its use of it.

[stever20]

A disproportionate part of RPI is SOS. You can effectively game RPI by simply removing low RPI opponents (which the MVC did by stipulating as much). It mostly isn't about winning, its about playing those who win. That is the game.

Everyone came to understand it, and that's when the Committee started being liberal with its use of it.

Yeah and not only that- but also the fact that the non d-1 games don't count means playing say Texas A&M Commerce is better than playing Texas A&M Corpus Christi because Non D1 schools don't count and so they can't hurt your RPI. Why do you think Creighton played Non D1 Truman St rather than a D1 school? That's why.

[HoosierPal]

A disproportionate part of RPI is SOS. You can effectively game RPI by simply removing low RPI opponents (which the MVC did by stipulating as much). It mostly isn't about winning, its about playing those who win. That is the game.

That is what Xavier did this year. They have played only one game with a +200 RPI team, Missouri currently #274. And who would have thought the Missouri would be so bad. Next 'lowest' team in North IA at 185. This has kept X's RPI high, even with only 2 Top 50 wins. Really, kudos to X for 'figuring it out'.

[DudeAnon]

A disproportionate part of RPI is SOS. You can effectively game RPI by simply removing low RPI opponents (which the MVC did by stipulating as much). It mostly isn't about winning, its about playing those who win. That is the game.

That is what Xavier did this year. They have played only one game with a +200 RPI team, Missouri currently #274. And who would have thought the Missouri would be so bad. Next 'lowest' team in North IA at 185. This has kept X's RPI high, even with only 2 Top 50 wins. Really, kudos to X for 'figuring it out'.

X deciding not to play trash team is them "figuring it out"?

[JohnW22]

A disproportionate part of RPI is SOS. You can effectively game RPI by simply removing low RPI opponents (which the MVC did by stipulating as much). It mostly isn't about winning, its about playing those who win. That is the game.

That is what Xavier did this year. They have played only one game with a +200 RPI team, Missouri currently #274. And who would have thought the Missouri would be so bad. Next 'lowest' team in North IA at 185. This has kept X's RPI high, even with only 2 Top 50 wins. Really, kudos to X for 'figuring it out'.

Is this a compliment or an insult?

[Fieldhouse Flyer]

Listen. There is always going to be flaws with the metrics. But the concern shouldn't be whether the RPI can ALWAYS get the best 68 teams. The concern should be whether the metric is consistent and easy for the public to understand and hard to be manipulated by committees. RPI meets both. My biggest concern here is predictability and that the decision makers cannot pick and choose which metric to use to give certain teams an advantage. We are already seeing a larger share of Power-5 schools getting at-large bids than ever before. That's largely in a way due to the committee picking-and-choosing when to use RPI and when to use something else (KP, S, Etc).

I wouldn't be opposed against using other metrics if the committee precisely indicated how much of each metric is going to be used to decide who is in and who is out. A composite based on 25/25/25/25 weights wouldn't be a problem. The problem I see here is that the committee does 25/5/70/0 for one team and then does 50/10/15/25 for another for the convenience of the power-5 teams.

Excellent post ecasadoSBU.

Another big problem is the ‘black box’ nature of many or most of the advanced metrics. The absence of public disclosure of details of their proprietary formulas and/or algorithms shields their methodology from public scrutiny. I have never read that the NCAA selection Committee has access to the proprietary formulas and/or algorithms referred to in the New York Times article referenced below:

Previously posted:

Officially, the Selection Committee will retain the RPI Rankings as its primary evaluation tool because RPI does not reward coaches for running up the score, and the NCAA does not want to be seen to encourage college basketball coaches to run up the score against overmatched opponents.

On Selection Day, There Are Metrics, and Then There Are Advanced Metrics – Marc Tracy, New York Times - March 11, 2016

According to several current and former members of the men’s basketball selection committee, the 10-member panel has increasingly relied on more sophisticated metrics to guide its decisions.

“The common metrics most of us use are KenPom, Sagarin, L.R.M.C., B.P.I., and K.P.I.,” the committee chairman, Oklahoma Athletic Director Joe Castiglione, told reporters.

Let’s have a look at ESPN’s Basketball Power Index (B.P.I.) as an example:

The College Basketball Power Index (BPI) is a measure of team strength that is meant to be the best predictor of performance going forward. BPI represents how many points above or below average a team is. Strength of Record (SOR) is a measure of team accomplishment based on how difficult a team's W-L record is to achieve. Game predictions account for opponent strength, pace of play, site, travel distance, day's rest and altitude, and are used to simulate the season 10,000 times to produce season projections.

GLOSSARY

BPI RK: Rank in the Basketball Power Index (BPI) among all Division I teams.

SOS RK: Strength of Schedule (SOS) rank among all Division I teams based on how a typical top 25 team would do against each team’s schedule to date. SOS accounts for the game location, day’s rest, travel distance, and high altitude in addition to opponent strength.

SOR RK: Rank of Strength of Record (SOR) among all Division I teams. SOR reflects the chance a typical Top 25 team would have team’s record or better, given the schedule on a 0 to 100 scale, where 100 is best.

RPI RK: Team’s rank in official NCAA Ratings Percentage Index (RPI).

ESPN tells us what they want us to believe their proprietary formulas and/or algorithms do – “meant to be the best predictor of performance” – but not how it is done. The strengths, weaknesses, and built-in systemic biasness (if any) of the BPI is shielded from independent scrutiny. The truth is, we don’t know exactly what the BPI measures or how it is measured, and it is unlikely that the NCAA Selection Committee (comprised of the 10 Athletic Directors) do either. The only thing we know with certainty is that the BPI produces an ‘advanced metric’ ranking of teams. Here are today’s ESPN B.P.I. Rankings:

1 - Villanova
21 - Butler
24 - Creighton
27 - Xavier
33 - Marquette
55 - Seton Hall

The next logical question is: "How much confidence does ESPN’s Joe Lunardi have in ESPN’s B.P.I. ? "

Bracketology – January 19, 2017

# 1 - Villanova
# 3 - Butler
# 3 – Creighton
# 6 – Xavier
# 12 – Seton Hall (play-in game)

The obvious answer is: "very little." If ESPN’s Joe Lunardi has no faith in ESPN’s BPI, then why should the NCAA Selection Committee?