Shooters!

Never Trust the Client

POSTED ON APRIL 25, 2016 BY GLENN FIEDLER

 

Never Trust the Client

A lot of people are complaining online about cheating in The Division lately, especially on PC. So much so that it’s starting to hit mainstream media.

But how bad are these hacks, and more importantly, can they be fixed?

I don’t own a copy of The Division so I watched a few videos and dug around to see what’s going on. As a game networking expert (I’ve shipped a few titles) I can usually look at exploit videos and see what’s going on fairly quickly.

The first videos I saw were the usual overly smug glitchers out there, super proud to have discovered things the QA department should have probably picked up when testing the game:

 

 

Not a great thing to see in a shipped game, but also not the end of the world. These glitches can be fixed.

 

But then, I saw this video:

 

 

And this is super bad news

 

Because here we have a client-side cheat program that is poking memory locations and giving players infinite health, infinite ammo, and teleporting players around the level.

 

This indicates that The Division is most likely using a trusted client network model.

 

I sincerely hope this is not the case, because if it is true, my opinion of can this be fixed is basically no. Not on PC. Not without a complete rewrite. Possibly on consoles provided they fix all lag switch timing exploits and disable players moving and shooting while lag switch usage is detected (trusted client on console exclusive games is actually more common than you would think…), but not on PC unless they completely rewrite most of their netcode and game code around a server-authoritative network model.

 

When I present such a bleak outlook people often people ask: why can’t they just implement more server-side checks? In fact the developers prior to release seemed to say something along the lines of: Oh, don’t worry. We haven’t implemented server side checks yet. Everything will be fine. Don’t worry. We’ve got this! Hmm. <network spider senses tingling>

 

To me this displays a fundamental misunderstanding of how FPS games are networked. It’s actually a fairly common misunderstanding so I’m going to spend some time in this post debunking this myth.

 

Top-tier competitive FPS games don’t work by having clients send a bunch of actions to the server for validation such as: I moved over here <position>, I reloaded my weapon, I fired my gun, I hit this guy and did X damage. The server does not sit there and run checks over the stream of actions sent to them by each client, validating each action, checking to make sure each action is safe, magically detecting cheaters and banning them from the game.

 

What top-tier competitive FPS games actually do is actually quite different…

 

For a competitive first person shooter there is a pretty standard approach to networking pioneered by Quake and later on perfected by Counterstrike. This is the same network model used today by top tier FPS games like Call of Duty, Overwatch and Titanfall.

 

This networking model has two main features you’ve probably heard of:

 

  • Client side prediction so players don’t feel lag on their own actions (movement, shooting etc…)
  • Lag compensation so when you shoot another player and bullets hit on your machine, you generally get credit for that hit as you saw it (so you don’t have to lead the target according to lag). But, critically, this decision of bullets hitting other players is decided on the server, not on the client.

Behind all of this, the key idea behind this network model is that the server is THE REAL GAME. What happens on the server is all that counts and the server never trusts what the client says they’re doing:

 

  • The server does not trust where the client says they are (position)
  • The server does not trust when the client says they fire a bullet or when the client says they hit another player with that bullet.
  • The server does not trust the fire rate of the weapon or the ammo count on the client.
  • The server does not trust what the client says they have in their inventory, and especially not the weapon the client says is in their hands…

What happens instead is that the server takes the inputs from the client and runs those player inputs in THE REAL GAME (on the server). What happens as a result of those inputs on the server is what really happens and is seen and experienced by other players in the game. Not because those actions were sent from the client to the server and magically validated, but because those actions are what actually happened on the server in response to the player’s inputs.

 

For example, if the input sent from a player is that they are holding down the fire button, then the server runs the same player code on those inputs (fire button pressed) which results in bullets being fired on the server according to the the current weapon the player is holding, the fire rate of the weapon, the amount of ammo in that weapon, and so on.

 

And those bullets emitted on the server are the ones that actually hit and do damage to other players.

 

In the standard FPS network model, what the client sees if they hack their client is just a ghost…

 

  • If they increase ammo counts or fire rate of their weapon, it doesn’t matter because they are just firing ghost bullets on the client that don’t actually do any damage.
  • If they hack their inventory and give themselves a weapon they don’t really have, it doesn’t matter because the bullets that actually do damage are fired on the server out of the weapon that the server thinks they have.
  • If they poke memory to warp around it doesn’t matter because bullets are fired from the server position not the hacked client position, and bullets fired by other players hit or miss them based on their server position.

If a competitive FPS was networked the other way, with client trusted positions, client side evaluation of bullet hits and “I shot you” events sent from client to server, it’s really difficult for me to see how this could ever be made completely secure on PC.

 

I’m rooting for the dev team on this one and I sincerely hope really they can turn this one around.

 

I hope they’re not using a trusted client networking model. I hope they have something up their sleeves. I hope they have a valid networking approach based around server-side checks that can address this issue in some way…

 

But unfortunately, so far, all signs point to no

 
About the Weary Warriors

The Weary Warriors Gaming Guild has the single purpose to forge and strengthen camaraderie between a group of mature game hobbyists. The company of good people becomes the best part of all of the games we play. While we strive to excel in the games we play with or against each other, excellence of character will always be more important than gaming skill.

 

No FB Yes FB Hand (smaller) Lap 40.063em Desk 64.063em Wall 90.063em