How to Apply Artificial Intelligence and Computer Vision to Arm Wrestling

Note: I have embedded six videos using real-time arm wrestling analytics below.

Why Would Someone Use AI for Arm Wrestling?

If you’re reading this article, you might be wondering why an attorney would be writing about how anyone can apply AI and computer vision to arm wrestling videos. While it is certainly true that they don’t teach AI in law school (or, at least, they certainly didn’t when I was in law school), it is a fun and emerging area with lots and lots of uses – including uses that would be immensely helpful to law firm clients. I may go into some of those at a later date (along with what I have been developing to service this need), but, for now, I share the information below to show just how powerful AI and computer vision are – even in the hands of an attorney and very novice programmer.

Real-Time Arm Wrestling Statistics?

For those who don’t know, competitive arm wrestling is booming in popularity. Like all sports, people want actionable information about the sport. The uses for such information come in two primary varieties:

Information used to determine relative skill and ability of an arm wrestler – This type of information could be used, for instance, to try to predict who would win a hypothetical or future arm wrestling match, or, more generally, to try to validate an argument about, e.g., who is the best arm wrestler of all-time. To do any of the foregoing, one needs to have access to concrete statistics — and such stats must go well beyond “simple” stats, like how much someone can dumbbell curl or their overall wins-and-losses record.
Information used to enhance performance – Just as a “little leaguer” receives instruction from his hitting coach about how to swing a baseball bat in a more effective manner, an aspiring arm wrestler could use meaningful statistics about successful (and unsuccessful) arm wrestling techniques as a way to improve his own performance in the sport.

Unfortunately, there exists no database or set of statistics collecting or measuring such statistics for the sport of armwrestling. … So I decided to create one. And I did it using AI and computer vision, as set forth below.

What Statistics Matter in Arm Wrestling?

Before we dive into the AI, we should first outline some general truths about arm wrestling. Arm wrestling is, at its core, about leverage and strength. Regarding leverage, the following are almost universally true almost all of the time:

Distance between shoulder and hand – All else equal, the closer your hand is to your own shoulder, and the further away your opponent’s hand is from his shoulder, the better your position is relative to your opponent. This is also a very good proxy for the percentage of your overall strength that you can apply to your opponent’s arm. For instance, if your hand is close to your body, you can apply nearly 100% of the maximum force you can generate to your opponent’s arm. By contrast, if your hand is nearly fully extended and two feet away from your shoulder, you can likely apply only a small percentage of your maximum potential force to your opponent’s arm. (There are exceptions to this rule, but those are beyond the scope of this article.)
Distance between elbow and body – Again all else equal, the closer your elbow is to your body, the better your overall position on the table. This is again only a general rule, and there are specific arm wrestling positions (e.g., a “posting toproll” or “King’s move”) where this is not the case. But unless you are an advanced arm wrestler specializing in a posting toproll, it is better to keep your elbow close to your body rather than let it be pulled away.
Hand position relative to the middle of the table – It is almost universally better to pull your opponent’s hand toward your side of the table than vice versa. By doing so, you can close the distance between your own arm, hand, and body and thereby put your opponent in a disadvantageous position.

How Can These Key Metrics Be Measured?

Using only computer vision and AI, we are able to train the computer to measure and report all relevant data in real-time. For purposes of this demonstration, we measure the following statistics:

Elbow angle – Measures the angle of the elbow using the forearm and the upper arm as the two “legs” forming the angle.
Upper arm angle – Measures the angle formed by the upper arm and body as the two legs forming the angle (essentially measures the angle at the arm pit).
Distance between the shoulder and hand in inches (or cm) – Using only computer vision, we are able to estimate, with a high degree of accuracy, the distance between the center of each competitor’s hand and shoulder with a high degree of accuracy.
Location of the opponents’ interlocked hands relative to the center of the table – We are able to monitor and report the exact location of the competitors’ hands relative to the middle of the center of the table, which shows which competitor is winning the “backpressure” battle at any moment in the match. Further, by tracking this information over time, we are able to see who generally controlled the match and how changes in backpressure lead to subsequent positional advantages in the match.
Distance of opponents’ interlocked hands from the center of the table – Not only can we measure who is winning the backpressure battle, but we can also quantify such measurement. We can again tell, with a high degree of precision, how far a competitor is able to pull his opponent’s hand toward him.

We can also show these statistics in both “snapshot” form (i.e., at any particular moment in the match) and over the full match (i.e., combining all such snapshot measurements into an area chart that updates as the match progresses.

As an aside, the AI can also recognize the identity of the arm wrestler and the move that they are performing at any time. However, these are beyond the scope of this article and so will not be addressed further.

So What Do Real-Time Arm Wrestling Statistics Look Like?

Taking inspiration from Second Spectrum and how they overlay real-time stats and probabilities on live video of sporting events, I have created an AI that overlays stats and probabilities and also populates historical stats for each arm wrestling match on the left and right sides of the screen. See below for a demonstration of what I’ve created.

What the Statistics Mean

Check out the screenshot below, which is from the match linked to above. As I’ll describe below the video, using only computer vision and TensorFlow, we are able to generate numerous real-time statistics and track how they change over the course of the match. And after these statistics are compiled for enough competitors over enough matches, we can run the data through TensorFlow again and determine real-time win probabilities of both actual and hypothetical matches.

Visual Chart Indicators

As can be seen above, we track all major statistics for each competitor as both real-time snapshots and over the entire duration of the match. The real-time statistics are on the upper left and right sides of the video, while the duration-of-the-match stats are shown on the bottom portion of the video. The stats are also color coded.

Real-Time, Color-Changing Indicators

I’ve also added another visually interesting feature that assists someone watching the match to make a judgment about the relative positioning of each opponent at any time during the match … without having to look away from the match itself. This is done through the color-changing triangle indicators overlayed over each competitor’s arm.

A red triangle indicates that a competitor’s elbow angle (or angle between upper arm and body) is in a highly disadvantageous position.
A yellow triangle indicates a neutral position.
A green triangle indicates a highly favorable position.

Note, also, that the AI “grades” elbow angle and arm pit angle separately, such that a competitor could have a green triangle (indicating an ideal positioning) re: the angle between his upper arm and body while also having a yellow or red triangle (indicating neutral or poor positioning) re: his elbow angle. In fact, that is what we see in the still shot above for the competitor on the right side of the screen. His upper arm is in tight to his body, and so that triangle is green. His elbow angle is shaded yellow, indicating that it is a neutral position.

These angles, and the shading applied to them, are measured in every single frame of the video and are updated in real-time based on the positioning shown therein.

The AI Training Results

Though I won’t go into boring details here, the AI was trained on a relatively limited data set. Yet, it still does a pretty good job even when applied to similar videos outside of the data set. See, for example, the short video below. This is an arm wrestling match between Larry Wheels, a popular YouTube personality / strongman / bodybuilder / powerlifter / aspiring armwrestler, and Thor Bjornsson, a winner of the World’s Strongest Man competition.

Note that the AI is able to do a good job of measuring, recording, and reporting statistics for this brief arm wrestling match as well, even though it was outside of the training set.

My next steps, if I ever get around to them, would be, first, using the AI to analyze many additional videos. I would also use a portion of such videos to further train and improve the model. Then, once a sufficient database of information for a wide range of arm wrestlers was generated, we could train a new predictive model to generate probabilities on outcomes of potential or hypothetical matches. While that would certainly be fun, I’m not sure it is worth the time that I’d have to invest.

If you want to discuss the training set or the nitty gritty of the code needed to generate statistics like this, please contact me and I’ll be happy to help out. And don’t worry about the complexity – if a lawyer like me can figure it out, then so can you!

Here are links to my github and stackoverflow pages.

About Brad Grounds

Brad Grounds’ bio is here.