A deep-dive feature from The Wall Street Journal recently explored the merits of the S2 cognitive test, which is now infamous for failing to project Houston Texans quarterback CJ Stroud as an instant superstar.
Having successfully projected awesome QBs like Patrick Mahomes, Joe Burrow and Brock Purdy, the S2 test was an emerging, buzzy evaluation tool. Half the teams in the NFL work with S2 nowadays. Quite a rapid rise from its inception in 2015.
However, Stroud was the odds-on favorite to be the No. 1 overall pick over Bryce Young — ya boii got Young at +275, NBD — at least for a little while prior to the 2023 NFL Draft. Nobody really knows how or why it happened, but the alleged flunking of the test by Stroud leaked to the masses. Proud to say I wrote about this, got a lot of pageviews off it, and had this doozy of a closing line:
“We’ll know for sure in about a week whether this impacts where Stroud lands. I still maintain all the Will Levis hype is a smokescreen and Stroud will go no lower than the Indianapolis Colts at No. 4 overall.”
But that’s neither here nor there. Young had reportedly passed the S2 test with flying colors. The S2 thingy is designed to project how quarterbacks can process information and make split-second decisions at the professional level. Given that Stroud came from a long line of non-pro-ready QBs at Ohio State, he wasn’t given the benefit of the doubt that he’d be different. Everyone already raved about Young’s schematic aptitude, decision-making and processing.
Not a surprise to see this clip of Stroud making the rounds ahead of the Texans’ playoff showdown with the Ravens on Saturday, as he wrote off the S2 result once it surfaced:
The Carolina Panthers proceeded to draft Young No. 1 overall, while the Texans “settled” for Stroud at No. 2. It’s looking like a decision that could haunt the Panthers franchise forever, as they sacrificed this year’s first overall pick among other assets to move up for the rights to Young. Against all odds and his implicit lack of S2 smarts, Stroud has led Houston to a division title and a playoff victory in Year 1. Might’ve had the best rookie quarterback season in NFL history.
As is only natural, S2 founders Brandon Ally and Scott Wylie are taking a lot of heat. Ally spoke to WSJ about it and tried to explain away the Stroud mishap as a small piece to the larger whole of evaluating a draft prospect, an inexact science, and also tried to validate prior successes with a percentages argument. To save time, let’s just get all his relevant quotes down in one dot dot dot skip-ahead blurb:
“There’s nothing on the planet that’s going to be a crystal ball…We can’t predict success. […] If you scored the 99th percentile on a cognitive metric, does that mean you’re going to make it? No, it does not. […] Let’s say we miss 20% of the time.
If our standard has to be we can’t miss ever, or we can’t miss on one player, man, that’s tough…I don’t know anybody in sports who’s that good.”
Additionally, it’s mentioned in the article that “a dozen or two” tests out of a thousand or so get flagged each year as potentially unreliable. You’re never gonna believe this, but Stroud’s test got that marking. Excuses about the various rigors players go through in the pre-draft process are cited as reason for a possibly faulty result.
I’m not here to totally pile on. Some of the points made by Ally make sense. If everyone had a 1.000 batting average on evaluating QBs, there wouldn’t be any drama. Every team would have a franchise guy, and the element of human error that makes the draft so glorious would be absent. That said, Stroud is such a catastrophic misfire for S2 that it threatens to cancel out most of the credibility the test had accumulated over nearly a decade. Some of the hypothetical “20%” misses count a lot more than others when they’re this definitive.
At the very least, Stroud has proven that the S2 exam shouldn’t hold much more weight than its only standing competitor, the Wonderlic test, which is more like an IQ test and is available to practice for free online.
What stands out most is the quote in the headline from the article I wrote last year on S2: “We’ve Never Had Somebody Grade Low And Play Well.” Welp, now you have! His name is Coleridge Bernard Stroud IV. And he has the Houston Texans on the brink of their first AFC Championship Game.
