Tuesday, October 11, 2016

We'll Be the Judge of That: 2014 Sochi Olympics - Results

So this was a much more massive undertaking than I originally thought.  I'm not the Excel wiz I wish I were otherwise this might not have taken so long.  Regardless, it was still a fun project.  I'm not surprised with the outcome but I am surprised with some of the scores we came up with.

Let's see if our judges agreed with the standings

The Judges

I originally planned for 9 judges for both segments but we ended up with 7 judges for the SP and 6 for the FS.  As the high and low scores are thrown out, that meant we had 5 scores that counted in the SP and only 4 for the FS.  I know having fewer scores may have skewed the numbers one way or the other, but rather than just say eff it and not do the FS, I decided to work with the scores I had.

Our judges were as follows:
  • Judge 1: Naked Ice 
  • Judge 2: DNP (Did Not Participate)
  • Judge 3: InflationMuch
  • Judge 4: Justin
  • Judge 5: Claire Cloutier
  • Judge 6: WhatTheWhat? 
  • Judge 7: alchemy void
  • Judge 8: mrusskie93
  • Judge 9: DNP (Did Not Participate)

A Few Notes...

The protocols are in a link below the scores.  All of our judges were instructed to comment on each element and component score they gave.  My ambition for this project far exceeded my ability to work all of that data into a coherent post, so instead I've linked a page to all of the comments for each skater's performance.  Click on the bolded name of each skater in the recap below to see the detailed comments from our judges. Unfortunately all of the GOE and PCS were summarized into percentages on that page, so you'll have to look at the protocol to see who scored what.  However, the comments for each element are numbered according to the judge who wrote them (Note: only the numbered comments were used in the results).

BONUS:  As an additional treat Judge #5 (Claire Cloutier) decided to give everyone an insider's perspective of what it was like to re-watch these performances and score them as a judge would rather than just a spectator.  Please make sure to check out her post, Walking in the Judge's Shoes: The Sochi Judging Project, on her blog A Divine Sport.  (Thanks again Claire!)

Okay.  Big breath.  You ready?  Here we go!  The first ever Naked Ice "We'll Be the Judge of That" project...the 2014 Sochi Olympics.

Short Program

Here are the results after the SP:

Click here to enlarge

And of course, here are the WBTJOT protocols.  Remember scores are in numerical order and correspond to our judges listed above.  I've also included the actual Sochi protocols for comparison. 

Our scores had Yu-Na Kim ahead of the field by 1 point (73.97), whereas in Sochi the top 3 were separated by only tenths of a point.  It's still a very close race but the separation is a bit wider.  Interesting to note that Yu-Na placed 2nd in both TES and PCS in this segment.  Yu-Na's jumps and step sequence are what gave her the edge here.  Those elements received high GOE (+2s and +3s) and accounted for the majority of the +GOE in her score.  Yu-Na's spins earned +GOE but not nearly as much as her jumps. 

Our judges had Carolina Kostner in 2nd place with 72.97 (versus 3rd in Sochi), only 1 point behind Yu-Na after the SP.  Just like in Sochi our judges had Carolina 1st in PCS by a narrow margin.  Carolina's combo and solo triple as well as her step sequence earned her the most GOE.  Her axel and spins received +GOE but not as much as the other top ladies.

Adelina Sotnikova places 3rd with 71.03, not far behind the top two, but far enough behind for there to be the separation most people expected to see.  She places 1st in TES and earned the most +GOE in the field (+7.76; Yu-Na: +6.50, Carolina: +6.08).  Adelina earned 1.00+ in GOE for every element except her step sequence which helped to boost her TES above the rest of the field.

Like in Sochi Gracie Gold places 4th.  Judging by the score she received in Sochi (68.63) and our score (67.12), our judges were in agreement with the Sochi panel.  Gracie's elements were solid but did not have as much flow and ease as some of the others which earned her less GOE.

Our judges had Ashley Wagner in 5th place after the SP with 64.40.  She lost points on her 3-3 combo but gained 1.40 on her solo triple and earned a solid amount of GOE with her other elements which was enough to keep her in the top 5.

Julia Lipnitskaya finishes the SP in 6th place with 62.84 instead of the 65.23 she received in Sochi.  The less generous PCS made the difference here. Compared to our judges' scores Julia received padded PCS and that held her up despite her major error in the SP.

Akiko Suzuki places 7th with 61.51.  We came up with the exact same TES as the Sochi panel 28.71); however, we had her slightly higher in PCS (32.80 vs 32.26 in Sochi).

Polina Edmunds finishes the SP in 8th place with 60.77.  Her scores are more or less in line with what she received in Sochi (61.04).

Mao Asada earns 57.00 for her SP versus 55.51 in Sochi.  We didn't score all of the other competitors but if we use their given scores then Mao would have placed 13th behind Valentina Marchei (57.02).  Our judges had Mao a few tenths higher in TES (23.16 vs 22.63 in Sochi) but nearly 1 point higher in PCS (34.84 vs 33.88).

Free Skate

Remember we lost one judge in the FS so instead averaging 5 judges, we dropped down to only 4 judges whose scores were calculated to come up with the FS totals.

Here are the FS results:

Click here to enlarge

Here are the WBTJOT protocols and the Sochi protocols.

I will say I was surprised by how close the scores were in this segment of the competition.

Mao comes out on top by a sizable margin winning the FS by more than 8 points.  Our judges gave her 148.23 vs the 142.71 she received in Sochi.  Fact is even with the under-rotations on two of the combinations, Mao's base value of 66.34 was nearly 5 points higher than Adelina who had a base value of 61.43 (2nd highest).  Our judges also had Mao tied for 2nd in PCS (72.02) which wasn't the case in Sochi (69.68).  She is often lowballed in PCS relative to the other top names but with our judges' scores Mao was on the same level which allowed her TES to be the deciding factor in the FS.

I was super-surprised to see that Carolina finished 2nd in the FS.  She beat out Yu-Na, who finished 3rd, by only 0.02.  With our results Carolina and Yu-Na were basically in a dead heat.  Yu-Na had the slightest lead in the TES (67.88 to 67.30); Carolina had the slightest lead in PCS (72.62 to 72.02).  Looking at it on paper they had very similar base values (Carolina: 58.45, Yu-Na: 57.49), both received roughly 10 points in +GOE from our judges, and both had similar strengths and weaknesses in PCS as well.  I thought this was a very interesting outcome.  In Sochi Yu-Na earned 1.58 points more overall in the FS, which is a rather small margin when you think about it.  It's possible that having fewer judges lessened the margin; if we had a full panel we may have had a slightly bigger gap or the placements may have flip-flopped.

Adelina finishes 4th in the FS.  The surprising result here is that Adelina's score ended up being only 1.62 points behind Yu-Na's score (138.28), but nowhere near what she earned in Sochi (149.95). Adelina was 2nd in base value and 2nd in TES overall.  Other than the stumble on the three-jump combo (which cost her -1.40 points from our judges), nearly everything else she did earned positive GOEs.  Most of her jumps were solid and her spins were excellent.  Our judges rewarded her for those parts of her skating; however, we were not nearly as impressed with her in terms of PCS as the Sochi judges were (66.82 vs 74.41 in Sochi).  That's nearly an 8 point difference in PCS...

Like in Sochi Gracie finishes 5th in the FS.  Our judges came up with roughly the same TES she earned (69.74 vs 69.57 in Sochi) but the biggest difference (like with most of these) comes down the the PCS.  Our judges had Gracie significantly lower in PCS (64.70 vs 68.33).

Ashley ends up with roughly the same score (128.82 vs 127.99 in Sochi) but finishes 6th in the FS instead of 7th.  Overall I found that Ashley's scores were pretty much on the money in Sochi; however, it was the way other skaters (in particular Julia) were scored relative to her that raised some eyebrows.

Our judges came up with a significantly different score for Julia (127.97) versus what she received in Sochi (135.34).  The major difference of course is in PCS.  Our judges gave Julia a 64.40 rather than the 70.06 she received in Sochi.  That's nearly a 6 point difference in PCS...

Akiko's score came out to be within a point of what she actually received (126.58 vs 125.35 in Sochi).  We had her slightly lower in TES but higher in PCS...

Same thing for Polina.  Our score was less than a point away from what she earned in Sochi (122.03 vs 122.21 in Sochi).

So...drum roll please...

Final Results

Note: With the margin being so small it is very possible that 1st and 2nd place may have come out differently if we had a full panel of scores to calculate.  Oh well, I worked with what I had and this is what we came up with!

Our judges' podium: Yu-Na Kim (gold), Carolina Kostner (silver), Adelina Sotnikova (bronze)

Yu-Na Kim wins her second Olympic gold medal and Carolina Kostner takes home an Olympic silver medal.  Adelina Sotnikova does very well at her first Olympics and walks away with a well-earned bronze medal (and significantly less hate and vitriol from the figure skating community).

Mao Asada jumps from 13th in the SP to finish a relatively close 4th overall.  Though the SP was a bit of a disaster, Mao's high base value and strong PCS allowed her to make up ground in the FS and finish just off of the podium.

Gracie Gold places 5th overall.  Gracie had two solid albeit lackluster performances in Sochi.  Still, she got the job done and our judges placed her top 5 at her first Olympics.

Ashley Wagner finishes in 6th place behind Gracie.  Despite Ashley's under-rotation issues she did complete both programs without any major errors and our judges rewarded her for that.

Julia Lipnitskaya places 7th overall.  Julia had major errors in both programs.  Thanks to a boost in PCS she was able to stay in the top 5 in Sochi; however, without the PCS our judges placed her lower.

The only skaters who retain their placements from Sochi are Akiko Suzuki and Polina Edmunds, who finish 8th and 9th respectively.

This was a lot of fun and a lot of work.  I don't think I really understood how much calculation, data crunching and time this would take.  Still, I enjoyed all of it.  I'd definitely be down for doing it again.  I'll let you guys decide if you want to score another event (leave me a comment below).  If we get enough judges who want to participate, I'll make it happen.

A big THANK YOU!! to all of the judges who participated! You guys rock!!

Well, it's almost time for the 2016-2017 Grand Prix season to begin.  I'll be posting my season preview next week.  Bye now!


  1. Great recap of the judging, Kwanatic!! You know I'm in for another edition... 😊

  2. Good job to everyone who participated!

    Request: 2014 Worlds or 2016 Worlds

  3. This was fantastic! This was really really great

    I'm especially pleased with the rankings of Carolina and Yuna, and with Mao's 4th place finish.

  4. Could you possibly do a comparison of the ice dance competitions?

  5. Really appreciate this and it kind of proves my argument in ways other than the obvious. I'm a Yuna fan but I have argued that it was always Yuna's destiny to never have a controversial WIN; if she had won there would have been some controversy as your re-assessment places Carolina a sliver just below her. I have argued that very thing and so many of Yuna's fans (of which I am one) wanted to pretend the controversy only ever was about Adelina and Yuna; discounting and dismissing Carolina's remarkable and out of character Olympic performances. But besides that, all the placements you and your fellows came up with has a righteous feel to it. Congratulations and well done.

  6. Hello thanks for your efforts in rejudging the competition. While I agree with the placement in the freeskate, there are a couple of points I wanted to make. First of all, this heat map of the top 4 skaters clearly shows that Mao had the greatest and riskiest rink coverage as her jumps were placed in the far corners of the rink which not only shows superior speed but also displays great rink awareness by not colliding with the boards, and it also factors into performances execution and even artistry since it brings her performance elements much closer to the audience.

    Hello thanks for your efforts in rejudging the competition. While I agree with the placement in the freeskate, there are a couple of points I wanted to make. First of all, this heat map of the top 4 skaters clearly shows that Mao had the greatest and riskiest rink coverage as her jumps were placed in the far corners of the rink which not only shows superior speed but also displays great rink awareness by not colliding with the boards. This clearly shows that Mao should have the highest score in skating skills, especially considering that she had more transitions, fewer crossovers, the best posture and more one legged and swaying movements in her skating. It also factors into performances execution and even artistry since it brings her performance elements much closer to the audience One thing that is never mentioned is that Mao was a sprinter as a child and intended at one point to pursue track along with ballet. the problem with the erroneous ISU system is that skating skills are either totally ignored and based on corridor scoring for favorites, tied to tech or they ignore more effortless speed as displayed by Mao. I noticed this phenomena when I watched Usain Bolt with the 100 meteres against Tyson Gay. It appeared that Tyson Gay was going faster in a sense because of the amount of effort her exerted in appearing to pump his arms and legs harder, but in actuality Usain Bolt's fluid motions provided less air resistance and so he was faster. Videos of Mao show she sprinted in a similar way.

    Mao's rink coverage which is clearly the most and should translate into the highest PCS:



    Carolina Kostner's rink coverage which was second:


    Adelina Sotnikova's rink coverage:


    Yuna Kim's rink coverage:


    I also find Mao's artistic expression to be the most realistic and emotionally moving as her huge response shows with her most watched video removed by the IOC in an evident attempt to muzzle this feeling and to manipulate and deflate her view count like the ISU did with Mao's scores.

    As for the ur calls, given urs and edges that were missed in the first group Mao's ur calls were totally unjustified since they were actually more rotated than some calls missed in the final group. This is consistent with the inversion of PCS scores. And Mao's GOE were also deflated so that all jump elements received below +1 GOE despite excellent flow, posture and symmetrical revolution with an aesthetically pleasing leg wrap and magnificent expression with the entire body on exit to the jumps. Step sequence should have clearly been the highest as well with the most intricate and acrobatic footwork that matched the music best, spread out over the entire length of the rink and a fan spiral and arabesque mixed in which no other competitor included. In short, Mao's Sochi LP was a world record performance that should stand to this day. Furthermore, while I think an overall gold should be rewarded, a gold should also be given for the greatest LP performance as is given to the athlete who runs the fastest race in track or swimming or the one who does the longest or highest jump or pole vault.

  7. Finally a word about Mao's SP, the JSF forced Mao to train in a sandy and bitterly cold rink at high elevation with less oxygen in Armenia, a poor and unfamiliar country for skaters like Mao. This forced her to readjust her blades and to adjust to the lack of oxygen and the bad conditions compelled her to return to Sochi a couple of days early with little training. Furthermore, her requests to train in Japan and in Russia as offered by Tarasova were denied by the JSF and Mao was forced to enter the team competition even though she had suffered from a back injury that required time off the ice through early January.

  8. I’m confused. Adelina’s 3Lz3T had obviously wrong edge and was clearly underrotated, yet she received here 0.00 points? It should have been downgraded. Yuna’s 3Lz was underrotated less than Adelina’s 3T and received -0.36. That’s really unfair. Also Yuna’s StSq in FS was level 4, not 3 and Adelina’s level 3. Here is detailed analysis of the step sequences of both skaters: https://www.goldenskate.com/forum/showthread.php?49801-Analyzing-Sotnikova-and-Kim-s-footwork-in-the-FS .