PERS-8, who sponsors statutory boards, is very limited in what they can provide in the way of a post mortem due to the requirements for secrecy in board proceedings. There is no legitimate way for anyone who was in the board spaces during the selection process to report on exactly what were "make or break" features of a given record.
In my experience, there are obvious show stoppers in some records that get everyone's thumbs down, and some records that would fly through any selection board. That leaves the majority of records that leave the board members digging for subtle differences to break out a particular officer. The particular board member who briefs a record makes a huge difference in how that record gets voted. My suspicion is that a lot of the observed variability in the board results is more highly correlated to what's not on the FITREP rather than what is. And there is no way to get to that data under the current rules of statutory boards.
Remember also, that every selectee goes through a detailed vetting process before the board results are released. PERS-8 scrubs the records of every potential selectee looking for anything that might suggest that individual doesn't meet the "best and most fully qualified" standard. For the purposes of discussion, we'll assume that none of the non-selects had any bad paper anywhere in their non-FITREP lives that could hurt them. Rarely, selectees can get taken off the list at that point, but not many people fall into that category who aren't on somebody's radar before the board. It does happen though.
Remember, PERS-4 cannot be present in the board spaces for statutory boards. Once the board results are out, PERS-4's only recourse is to try and read the tea leaves of groups of records looking for obvious patterns or anomalies, or looking at an individual record of either a select or non-select to see if there is anything there that stands out as toxic. Failing that, you are left with the response quoted above as "We don't have a clue why he didn't select". As mentioned, detailers have neither the time nor the expertise to do that sort of analysis on anything other than a very superficial level on data that may or may not be relevant.
If there were any big Navy interest in a more detailed parsing the results of boards, there are any number of avenues that could be pursued. The fact that this hasn't been done tells me that there really isn't the desire on the part of senior officers to scratch that itch.
R/
