FUT Scripting, Handicapping & Momentum: FIFA Survey Results

Getting a better understanding of the FUT community’s perception

Earlier this week we invited the FUT community to participate in a survey on scripting, momentum and handicapping. For the sake of ease, we will refer to these phenomena as ‘manipulation’ from now on.

The subject has caused lots of controversy, and the FIFA community is divided on the subject.

The proponents generally believe that the matches are manipulated in order to serve some kind of commercial goal. Although this theory exists in different flavors, the general perception is that the game levels the playing field to the advantage of less skilled players.

The skeptics do not believe that the matches are subject to deliberate manipulation. Instead, they argue that events described as unusual in fact are natural. According to the skeptics, match results in general can be explained by differences in skill between the involved players.

In this article, we will use the results from the survey and other sources to test some of the core preconditions of both theories.

The focus of this analysis will be on the following questions:

  1. Is manipulation a blatantly clear fact?
  2. Do better players experience more manipulation?
  3. Are losing streaks happening more often than they should?
  4. Do people experience more manipulation within certain game modes?
  5. Are our shopping habits affected by various events in the game?
  6. Do late goals happen more often than they ought to?

A few general facts & thoughts on the survey

486 respondents have participated in the survey, although some of them didn’t respond to all questions. As one could expect, countries with English-speaking populations are more represented than others. Almost 50% of the respondents are UK-based, and some 16% are based in the USA.

Although we do not know this for sure, there is a good chance that people who have strong feelings about the subject are over represented in the survey.

Another bias comes from the fact that this survey was presented on a community website, which increases the chance of getting responses from community members, and as such more responses from people who play the game more often and have more experience than the average FIFA player.

The entire population of FIFA players is normal distributed, meaning that the number of players below average in terms of skill equals the number of players above average. When comparing this to the sample, 8 out of 10 respondents claim they are above average, 2 out of 10 think they are average, and only 2 % think that they are below average.

This picture may not be completely correct, though. As an example, 3 out of 10 respondents claim that they lose despite being the better player in 9 out of 10 matches.

The data from the survey is available for download, meaning that everyone has the option to challenge the conclusions below.

Analysis

In the following section, we will look into some questions of particular interest. Each section starts with a few notes on the reason why I do what I do, followed by some facts (observations) and finally an interpretation, where I link the facts with the questions.

1. Is manipulation a blatantly clear fact?

Whether you believe in manipulation or not is a question of interpretation: Some people interpret the events they observe as an expression of the game being unfair, whereas others interpret the exact same events as an expression of differences in skill. Quite clearly, one of them is wrong. Either, the skeptics are ignoring clear facts or the proponents are misinterpreting things.

Although this will neither prove nor disprove that manipulation exists, I think it’s important to find out whether the respondents who believe in FIFA scripting actually agree on what they see, where the see it and how often they see it.

If people are observing the same things, this will increase the likelihood that the skeptics simply are ignoring facts. If people are observing very different things, this will support the claim that the observations are a product of misinterpretation rather than facts. Neither conclusion will prove anything, though.

Observations

8 out of 10 respondents say that it’s likely that the game manipulates certain matches, but there is significant variation in the perceptions of what kinds of manipulation is believed to be present in the game, and in which game modes and to what extend it can be found:

  • The four most likely kinds of manipulation each appear to be present in the game to 8 out of 10 respondents, but only 5 out of 10 think that all four kinds of manipulation are present in the game. Obviously, this also implies that 5 out of 10 finds it unlikely that all four kinds of manipulation are present.
  • When looking across all kinds of manipulation mentioned in the questionnaire, the most widespread combined belief is shared by only 5 % (that all proposed kinds of manipulation are likely except for deliberate matching against expert opponents and deliberately injuring star players).

Moving on to the presence of manipulation within various game modes, the picture is just as blurry. For example, roughly 1/3 think that Online Friendlies are manipulated at least every second game or more. Another 1/3 says it happens in approximately 2 out of 10 games, whereas the final 1/3 thinks it happens rarer than 1 out of 10 games or never.

Interpretation

To understand why this is relevant, think of the Lock Ness Monster as an example. Plenty of eye witnesses claim they have seen a monster, but their descriptions vary greatly in terms of shape and size. Although we can’t rule out that some of the eye witnesses actually saw a monster, we can definitely rule out that all of them saw it due to the inconsistency in their descriptions.

This leaves us with some eye witnesses who may have seen a monster rather than many eye witnesses who probably saw the same animal. I’m not saying that this proves that Nessie is a product of imagination or ‘visual misinterpretation’, but it certainly raises the odds.

The same obviously goes for manipulation within FUT: If manipulation was a crystal clear fact rather than a product of misinterpretation, people would most likely observe the same things to a much larger extent than they do.

2. Do better players experience more manipulation?

Most manipulation proponents believe that EA’s rationale for manipulating the game can be explained by the following statements:

  • EA believes that giving easy victories to less experienced players will make them more likely to keep playing, buy more FUT packs etc.
  • By adding extra drama to the game (make you lose occasionally), EA makes you want to buy more packs.

In the survey, both rationales receive support from 7 out of 10 respondents and 8 out of 10 believes in at least one of these explanations.

Both explanations share the assumption that manipulation will disadvantage experienced players and benefit less experienced players, or in essence that the game ‘levels the playing field’.

A precondition for this assumption has to be that better players will experience more manipulation than less good players, and hence that the ‘experienced intensity of manipulation’ increases with the skill level of the player. We can actually test this precondition by measuring (a) the skill level and (b) the experienced intensity of manipulation for each respondent.

The survey provides three different ways of assessing the skill level, and we will use all three to give the conclusion extra credibility:

  • The self-assessed skill level
  • The XP
  • Their win/loss ratio

We can estimate the experienced intensity of manipulation via the data from question 4a (people’s personal experiences with ten different kinds of manipulation).

Observations

The tables below contain an averaged overview of the observations.

For each of the data pairs, I have calculated the correlation coefficient as a measure of the strength of the linear relationship between the two variables, in other words the likelihood that v1 [the experienced manipulation intensity] is related to v2 [the skill level].

In all three cases, the conclusion is extremely clear: There is no correlation between the experienced manipulation intensity and the skill level. The experienced level of manipulation does not depend on the skill level of the player.

Interpretation

The observations show that the more skilled players aren’t experiencing manipulation to a higher extent than the less skilled players. This fact strongly suggests that we can reject the general hypothesis that the game is levelling the playing field.

A reasonable objection to this conclusion is that the sample probably contains few responses from less skilled players. Thus, it will be natural to ask whether the absolute beginners are helped by the game, and whether this could explain why so many players think the game is unfair. We could call this the ‘easy start’ hypothesis.

Although this hypothesis can’t be ruled out based on the data presented above, it is still very unlikely. The respondents generally claim that they experience manipulation as often as in 5 out of 10 matches. For this claim to fit with the ‘easy start’ hypothesis, people would need to be matched with absolute beginners in an extremely high percentage of their games.

No matter which FUT game modes you play, this is highly unlikely. If you look at your recent match history, you will see that hardly any of your opponents actually are beginners in terms of XP level. A quick sample of the match history for two of my latest opponents and myself (57 matches in total) contains only two opponents with an XP below 5, and only 1 opponent with an XP of 1.

3. Are losing streaks happening more often than the should?

Many people complain about losing streaks in FIFA. Approximately 8 out of 10 respondents say that they believe the game imposes losing streaks on them from time to time. Almost 5 out of 10 claim that they suffer from losing streaks in at least half of their games.

At a first glance, this suggests that an incredible amount of people are suffering from losing streaks of great length and to an extent, which definitely indicates that the game actually does intervene with nature in this case.

However, there are obvious reasons why we should question this claim. According to psychology, all humans are subject to different kinds of cognitive bias, including something called clustering illusion. Clustering illusion is the tendency to overestimate the importance of small runs, streaks, or clusters (that is, seeing phantom patterns).

Besides, we can actually test whether the game imposes unnatural streaks on the players or not.

Observations

I haven’t defined losing streak in the questionnaire, but I believe that most people will agree that a losing streak consists of an uninterrupted row of at least three consecutive losses.

How common are various lengths of losing streaks in FIFA? To find out, I simply recorded the match history of 20 randomly chosen players (my 20 latest opponents) – in total 400 matches. Everyone can do this: Simply open ‘My profile’ via the web app and browse your opponents. For each opponent, I counted the number of occurrences of losing streak of various lengths:

LengthOccurrences
315
47
53
63
71
81
 
131

Next, we need to decide on the number of occurrences in an un-manipulated gaming environment. To do this, I think it makes sense to look at FUT as a coin tossing experiment because what we actually need to measure is the chance of an average player being matched with a better opponent 3 times in a row, 4 times in a row and so on.

Due to possibility of getting a draw in FUT seasons, the actual probability of 3 consecutive defeats is slightly lower than the probability of flipping 3 consecutive heads.

What you see below is the actual number of occurrences compared to the expected number of occurrences. As it turns out, the numbers come pretty close, and the deviations definitely aren’t statistically significant.

LengthActual occurrencesExpected occurrences
31513
476
533
631
71<1
81<1
  
131<1

Interpretation

We have observed that the number of losing streaks in the sample are similar to what we would expect in a natural, un-manipulated gaming environment.

Because of the size of sample, I’m hesitant to conclude anything definitive, but presuming that the sample is representative, this definitely suggests that the claims about frequent and unnatural losing streaks are a product of cognitive bias rather than manipulation.

4. Are people experiencing more manipulation within certain game modes?

Is there a connection between game mode and the observed level of manipulation?

Manipulation proponents generally believe that EA levels the playing field in order to make people buy more packs. According to this explanation, it would make sense to assume that the level of manipulation within different FUT game modes would be pretty much the same.

On the other hand, manipulation skeptics argue that the game isn’t manipulated, and that people simply are being matched up with better players and lose. According to that explanation, it would make sense to assume that the reported level of manipulation would follow the difficulty of the various game modes.

Observation

As I already described earlier, people aren’t experiencing manipulation to the same degree across all game modes. On the contrary, manipulation is reported to have the largest effect in FUT Online seasons, FUT Online Tournaments and FUT Online World Cup followed by Online Friendlies, then the offline game modes and finally game modes outside FUT.

Interpretation

Please consider the observations above in the light of the previously mentioned theories about why EA – allegedly – would manipulate the game:

  • If the field is being levelled in order to get someone to buy more packs, then how come it doesn’t happen to the same degree in all game modes?
  • If EA uses match manipulation to make us buy more packs, then why does 1/3 of the respondents say that they observe manipulation within game modes which aren’t even played with players found in FUT packs?

I can’t come up with any reasonable answers to those questions.

On the other hand, I find it likely that the observed level of manipulation actually corresponds with the level of difficulty across various game modes. Although there is no way to measure the level of difficulty, the observations above are fully in line with my personal experiences of difficulty within various game modes.

Assuming I’m right about that, people’s tendency to report manipulation within a game mode is directly related to the probability of getting matched with a stronger opponent within a game mode.

5. Are our shopping habits affected by various events in the game?

Since the vast majority of the respondents believe that manipulation is real and that it has a commercial purpose, it make sense to study how the respondents think they are affected by the events which allegedly should lead to more packs being bought.

Observation

In question 15, I asked how different results would affect the FUT pack shopping inclination of the respondent.

The majority of the respondents answer that their own inclination to buy packs is largely unaffected by match results, tournament results and so on.

If we move on to the net impact (the difference between the number of people reacting positive and negative), it’s approximately neutral for the majority of the events, including winning and losing matches etc.

Four events have an overly positive impact on the likelihood of buying packs: Experiencing pack luck, seeing others experience pack luck, happy hour and special tournaments.

Another interesting observation is that the respondents say that they react negatively to both winning despite the opponent being better and losing despite being better than the opponent is.

How does the responses of the respondents fit with their thoughts on how others will react to the same events? To find out, we can use the responses from question 6:

Interestingly, there is a huge gap between the reactions of our respondents and their expectations regarding the reactions of other FIFA players. This stands in stark contrast to that almost 8 out of 10 respondents believe that others will react to FUT handicapping in the same way they do.

Interpretation

There is no apparent reason to assume that the sample isn’t representative to the majority of Ultimate Team players or for that matter all humans in terms of the inclination to buy packs based on various types of events.

And the matter of fact is that the majority of the respondents don’t react to winning or losing, and that they react negatively to matches being decided by other forces than skill, no matter whether they are taking benefit from it or not.

If EA make usage statistics, they should be able to measure our inclination to buy packs upon various events in the game as well, and they should be able to reach the same result we just saw here. In other words, they probably know this already.

Although approximately half of respondents claim that match results doesn’t affect the pack shopping inclination, there is a residual who says their inclination to buy packs either increases or decreases. This raises the question of whether EA could target specific users and impose certain results on them.

To answer this, I simply looked at the relationship between the experienced manipulation intensity and the degree of inclination to buy more packs upon certain match results. The results shows that there is absolutely no dependency between those variables, which clearly indicates that EA aren’t targeting specific users.

6. Do late goals happen more often than they ought to?

Background

After I published the first version of this article, I decided to add this section on late goals as a general response to some of the common questions raised in the debate below.

A lot of people complain about late goals apparently happening more often than they ought to. We don’t know the actual goal distribution within FIFA, but we can still compare the reported observations against the expected occurrence of late goals in a natural gaming environment in order to determine whether the reported numbers exceed what we should expect to see due to natural reasons.

Observations

According to the survey, 3 out of 10 say that late goals happen in 9 out of 10 matches, 4 out of 10 say that they happen in 5 out of 10 matches and 2 out of 10 say they occur in 2 out of 10 matches.

These results may be slightly exaggerated as people tend to remember ‘unusual’ events better. Besides we cannot rule out that some of the respondents base their opinions on a broader definition on ‘late goals’.

The reported late goal frequency varies quite a bit from respondent to respondent. This could reflect the reality to some degree, as the number of late goals naturally increases with the total number of goals.

Before I move on, I will define a ‘late goal’. In general, ‘late goals’ could be all goals scored towards the end of the match, goals scored from 89:00 and onwards or perhaps goals scored during stoppage time. Based on my understanding of the complaints about late goals, I will look at stoppage time goals alone, i.e. goals scored while the clock stays at 45:00 or 90:00.

Obviously, the number of expected goals will increase if we extend the scope to include the 45th and the 90th minute, so this is a somewhat conservative definition.

How can we determine the expected number of stoppage time goals in a natural gaming environment? The easiest approach would be to determine (a) the average, natural amount of stoppage time and hence the percentage of effective playing time occurring during stoppage time, (b) the average, natural distribution of goals throughout a match and (c) the average number of goals per match in FUT.

Continued Observations…

Regarding the average amount of added time, we can use data from Barclays Premier League. In the BPL, the average stoppage time is approximately 7 minutes. As 83 effective minutes are played between 00:00 and 89:59, 8 % of the total effective playing time is played during stoppage time.

If the chances of scoring were equal for all minutes throughout the match, we would expect 8 % of the goals to be scored during stoppage time. This is however not the case. In the BPL, the number of goals scored during the last 10 minutes of a half is 70 % higher.

In fact, this percentage increases minute by minute towards the end of the match, but for the sake of ease, we will use the 70 % increase, which leads to the assumption that 14 % of all goals should be scored during stoppage time.

To determine the expected number of stoppage time goals in FUT, we need to know the average total number of goals per match in FUT. In a sample of 380 matches played by my 19 latest opponents, 4.4 goals were scored on average (varying between 3.5 and 5.9). At a rate of 4.4 goals per match, we should expect

(.14 x 4.4 = .62) stoppage time goals (STG’s) per match or approximately 1 STG per 1.6 matches. The average of 4.4 will include some 3-0 results due to disconnects and similar, which could indicate that the actual average is marginally higher.

Since the average number of goals varies from player to player, it may be helpful to calculate the expected occurrence of STG’s at various goal rates:

  • 3.5 goals per game: STG in 4.9/10 matches
  • 4.0 goals per game: STG in 5.6/10 matches
  • 4.4 goals per game: STG in 6.2/10 matches
  • 5.0 goals per game: STG in 7.0/10 matches
  • 5.5 goals per game: STG in 7.7/10 matches

Interpretation

Given the high number of late goals due to the natural circumstances, it appears likely that many of the complaints about late goals in fact are complaints about late goals occurring naturally.

4 out of 10 respondents claim that they experience late goals in approximately 5 out of 10 matches, which roughly corresponds with the expected number of late goals in a comparable, un-manipulated gaming environment at an average goal rate corresponding to the sample.

Another 3 out of 10 respondents claim that they experience late goals as often as in 9 out of 10 matches. To achieve a late goal in 9 out of 10 matches, the player would need a goal average of 6.4 goals per match.

It is less likely that 1/3 of the respondents actually do have a goal average of 6.4 when considering that the sample covering my 19 latest opponents didn’t contain any players with an average above 5.9.

It should however be noted that at an average of 5.9, the player would see late goals in 8 out of 10 matches. On top of that, it has to be considered that my estimates regarding the amount of added time and the increase in the goal scoring ratio towards the end of the match are somewhat conservative.

With this in mind, it doesn’t appear unlikely that 1/3 of the respondents experience late goals in approximately 9 out of 10 games.

Quite a few players have been complaining about decisive matches being decided by late goals. What we can conclude is that for someone who mostly plays tight matches (matches being decided by 1 goal), a significant portion of those matches will be decided by late goals due to the high percentage of late goals in general.

Conclusions

1. Is manipulation a blatantly clear fact?

No. On contrary, the respondents have very different experiences in terms of what they observe, where they observe it and to what extent. This indicates that the experience of being manipulated is a product of the observer’s interpretation rather than what he actually observed.

2. Do better players experience more manipulation?

They don’t. In fact, there is no connection between skill level and the experienced level of manipulation. This directly contradicts the claim that EA levels the field to serve some commercial goal.

3. Are losing streaks happening more often than they should?

They are not. In fact, the number and length of losing streaks correspond to what you would expect to find in an un-manipulated gaming environment.

4. Are people experiencing more manipulation within certain game modes?

Yes, they are. They experience more manipulation within – apparently – more difficult game modes than within easier game modes. This corresponds with the claim that people are losing, because they are matched with better opponents rather than because they are subject to manipulation.

5. Are our shopping habits affected by various events in the game?

People buy more packs using FUT Coins or FIFA Points during Happy Hour when they experience pack luck, but they don’t react to match results and similar. If EA manipulates match results, it’s highly unlikely that this will affect our pack buying inclination.

6. Do late goals happen more often than they ought to?

Most likely no. We have found that the expected number of late goals is quite high due to natural circumstances, and that the expected number of late goals corresponds well with the reported number of late goals.

Although we can’t conclude that late goals only happen for natural reasons, we can definitely conclude that a significant proportion of the complaints regarding late goals can be attributed to natural circumstances.

Avatar
About David Cotton

David has been a passionate video gamer his entire life and has owned and played most gaming consoles since his early childhood. He has over nine years experience writing about video games and is a well known FIFA Ultimate Team expert and 'FUT Founder'. As well as providing insight on all the latest FIFA Ultimate Team news, David has a history of creating extremely effective in-depth guides on how to make FUT coins. He founded and created UltimateTeamUK in 2011 and has previously worked as the 'Head of FIFA' at the popular esports company, Dexerto.

Leave a Comment