使用全局变量或者静态变量是导致多线程编程中非线程安全的常见原因。在多线程程序中，保障非线程安全的常用手段之一是使用互斥锁来做保护，这种方法带来了并发性能下降，同时也只能有一个线程对数据进行读写。

如果程序中能避免使用全局变量或静态变量，那么这些程序就是线程安全的，性能也可以得到很大的提升。

如果有些数据只能有一个线程可以访问，那么这一类数据就可以使用线程局部存储机制来处理，虽然使用这种机制会给程序执行效率上带来一定的影响，但对于使用锁机制来说，这些性能影响将可以忽略。

还有一种大致相当的编程技术就是使用 线程特有数据(没 线程局部存储 易用, 也没 线程局部存储 高效) ，这将在 线程特有数据 中讨论。

. . .

在 Linux 系统中使用 C/C++ 进行多线程编程时，我们遇到最多的就是对同一变量的多线程读写问题，大多情况下遇到这类问题都是通过锁机制来处理，但这对程序的性能带来了很大的影响，

当然对于那些系统原生支持原子操作的数据类型来说，我们可以使用原子操作来处理，这能对程序的性能会得到一定的提高。那么对于那些系统不支持原子操作的自定义数据类型，

在不使用锁的情况下如何做到线程安全呢？本文将从线程特有数据方面，简单讲解处理这一类线程安全问题的方法。

如果有些数据只能有一个线程可以访问，那么这一类数据就可以使用线程特有数据机制来处理，虽然使用这种机制会给程序执行效率上带来一定的影响，但对于使用锁机制来说，这些性能影响将可以忽略。

还有一种大致相当的编程技术就是使用 __thread (比 线程特有数据 易用, 也比 线程特有数据 高效), 它是 GCC 内置的线程局部存储设施 ，这将在 线程局部存储 中讨论。

数据类型

在 C/C++ 程序中常存在全局变量、函数内定义的静态变量以及局部变量，对于局部变量来说，其不存在线程安全问题，因此不在本文讨论的范围之内。

全局变量和函数内定义的静态变量，是同一进程中各个线程都可以访问的共享变量，因此它们存在多线程读写问题。

在一个线程中修改了变量中的内容，其他线程都能感知并且能读取已更改过的内容，这对数据交换来说是非常快捷的，但是由于多线程的存在，对于同一个变量可能存在两个或两个以上的线程同时修改变量所在的内存内容，同时又存在多个线程在变量在修改的时去读取该内存值，如果没有使用相应的同步机制来保护该内存的话，那么所读取到的数据将是不可预知的，甚至可能导致程序崩溃。

如果需要在一个线程内部的各个函数调用都能访问、但其它线程不能访问的变量，这就需要新的机制来实现，我们称之为 Static memory local to a thread (线程局部静态变量)，同时也可称之为线程特有数据（TSD: Thread-Specific Data）

这一类型的数据，在程序中每个线程都会分别维护一份变量的副本 (copy)，并且长期存在于该线程中，对此类变量的操作不影响其他线程。
如下图：

一次性初始化

在讲解线程特有数据之前，先让我们来了解一下一次性初始化。

多线程程序有时有这样的需求：不管创建多少个线程，有些数据的初始化只能发生一次。

列如：在 C++ 程序中某个类在整个进程的生命周期内只能存在一个实例对象，在多线程的情况下，为了能让该对象能够安全的初始化，一次性初始化机制就显得尤为重要了。

——在设计模式中这种实现常常被称之为单例模式（Singleton）。
Linux 中提供了如下函数来实现一次性初始化：

1
2
3
4
5
6
7
8
9
10
11
12

#include <pthread.h>

// Returns 0 on success, or a positive error number on error
int pthread_once (pthread_once_t *once_control, void (*init) (void));
// 利用参数once_control的状态，函数pthread_once()可以确保无论有多少个线程调用多少次该函数，
// 也只会执行一次由init所指向的由调用者定义的函数。

// init所指向的函数没有任何参数，形式如下：
void init (void)
{
   // some variables initializtion in here
}

另外，参数 once_control 必须是 pthread_once_t 类型变量的指针，指向初始化为 PTHRAD_ONCE_INIT 的静态变量。
在 C++0x 以后提供了类似功能的函数 std::call_once ()，用法与该函数类似。
使用实例请参考 :
https://github.com/ApusApp/Swift/blob/master/swift/base/singleton.hpp
实现。

线程特有数据API介绍

在 Linux 中提供了如下函数来对线程特有数据进行操作

1
2
3
4
5
6
7
8
9
10
11
12
13

#include <pthread.h>

// Returns 0 on success, or a positive error number on error
int pthread_key_create (pthread_key_t *key, void (*destructor)(void *));

// Returns 0 on success, or a positive error number on error
int pthread_key_delete (pthread_key_t key);

// Returns 0 on success, or a positive error number on error
int pthread_setspecific (pthread_key_t key, const void *value);

// Returns pointer, or NULL if no thread-specific data is associated with key
void *pthread_getspecific (pthread_key_t key);

pthread_key_create

// Returns 0 on success, or a positive error number on error
int pthread_key_create (pthread_key_t *key, void (*destructor)(void *));

函数 pthread_key_create() 为线程特有数据创建一个新键，并通过 key 指向新创建的键缓冲区。

因为所有线程都可以使用返回的新键，所以参数 key 可以是一个全局变量（在 C++ 多线程编程中一般不使用全局变量，而是使用单独的类对线程特有数据进行封装，每个变量使用一个独立的 pthread_key_t）。

destructor 所指向的是一个自定义的函数，其格式如下：

void Dest (void *value)
{
    // Release storage pointed to by 'value'
}

只要线程终止时与 key 关联的值不为 NULL，则 destructor 所指的函数将会自动被调用。

如果一个线程中有多个线程特有数据变量，那么对各个变量所对应的 destructor 函数的调用顺序是不确定的，因此，每个变量的 destructor 函数的设计应该相互独立。

pthread_key_delete

// Returns 0 on success, or a positive error number on error
int pthread_key_delete (pthread_key_t key);

函数 pthread_key_delete() 并不检查当前是否有线程正在使用该线程特有数据变量，也不会调用清理函数 destructor，而只是将其释放以供下一次调用 pthread_key_create() 使用。

在 Linux 线程中，它还会将与之相关的线程数据项设置为 NULL。

由于系统对每个进程中 pthread_key_t 类型的个数是有限制的，所以进程中并不能创建无限个的 pthread_key_t 变量。Linux 中可以通过 PTHREAD_KEY_MAX（定义于 limits.h 文件中）或者系统调用 sysconf(_SC_THREAD_KEYS_MAX) 来确定当前系统最多支持多少个键。Linux 中默认是 1024 个键，这对于大多数程序来说已经足够了。如果一个线程中有多个线程特有数据变量，通常可以将这些变量封装到一个数据结构中，然后使封装后的数据结构与一个线程局部变量相关联，这样就能减少对键值的使用。

pthread_setspecific

// Returns 0 on success, or a positive error number on error
int pthread_setspecific (pthread_key_t key, const void *value);

函数 pthread_setspecific() 用于将 value 的副本存储于一数据结构中，并将其与调用线程以及 key 相关联。

参数 value 通常指向由调用者分配的一块内存，当线程终止时，会将该指针作为参数传递给与 key 相关联的 destructor 函数。

pthread_getspecific

// Returns pointer, or NULL if no thread-specific data is associated with key
void *pthread_getspecific (pthread_key_t key);

当线程被创建时，会将所有的线程特有数据变量初始化为 NULL，因此第一次使用此类变量前必须先调用 pthread_getspecific() 函数来确认是否已经于对应的 key 相关联，如果没有，那么 pthread_getspecific() 会分配一块内存并通过 pthread_setspecific() 函数保存指向该内存块的指针。

参数 value 的值也可以不是一个指向调用者分配的内存区域，而是任何可以强制转换为 void * 的变量值，在这种情况下，先前的 pthread_keycreate() 函数应将参数 _destructor 设置为 NULL

函数 pthread_getspecific() 正好与 pthread_setspecific() 相反，其是将 pthread_setspecific() 设置的 value 取出。在使用取出的值前最好是将 void * 转换成原始数据类型的指针。

使用线程特有数据API

我们先讨论一下非线程安全的 stderror() 的实现, 接着说明如何使用线程特有数据来实现该函数的线程安全.

非线程安全的stderror()

An implementation of strerror() that is not thread-safe.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

/*************************************************************************\
*                  Copyright (C) Michael Kerrisk, 2017.                   *
*                                                                         *
* This program is free software. You may use, modify, and redistribute it *
* under the terms of the GNU General Public License as published by the   *
* Free Software Foundation, either version 3 or (at your option) any      *
* later version. This program is distributed without any warranty.  See   *
* the file COPYING.gpl-v3 for details.                                    *
\*************************************************************************/

/* Listing 31-1 */

/* strerror.c

   An implementation of strerror() that is not thread-safe.
*/
#define _GNU_SOURCE                 /* Get '_sys_nerr' and '_sys_errlist'
                                       declarations from <stdio.h> */
#include <stdio.h>
#include <string.h>                 /* Get declaration of strerror() */

#define MAX_ERROR_LEN 256           /* Maximum length of string
                                       returned by strerror() */

static char buf[MAX_ERROR_LEN];     /* Statically allocated return buffer */

char *
strerror(int err)
{
    if (err < 0 || err >= _sys_nerr || _sys_errlist[err] == NULL) {
        snprintf(buf, MAX_ERROR_LEN, "Unknown error %d", err);
    } else {
        strncpy(buf, _sys_errlist[err], MAX_ERROR_LEN - 1);
        buf[MAX_ERROR_LEN - 1] = '\0';          /* Ensure null termination */
    }

    return buf;
}

线程安全的stderror()

这是使用线程特有数据技术实现的线程安全的stderror().

如果对使用线程局部存储技术实现的线程安全的stderror()感兴趣,
请转 线程局部存储

An implementation of strerror() that is made thread-safe through the use of thread-specific data.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84

/*************************************************************************\
*                  Copyright (C) Michael Kerrisk, 2017.                   *
*                                                                         *
* This program is free software. You may use, modify, and redistribute it *
* under the terms of the GNU General Public License as published by the   *
* Free Software Foundation, either version 3 or (at your option) any      *
* later version. This program is distributed without any warranty.  See   *
* the file COPYING.gpl-v3 for details.                                    *
\*************************************************************************/

/* Listing 31-3 */

/* strerror_tsd.c

   An implementation of strerror() that is made thread-safe through
   the use of thread-specific data.

   See also strerror_tls.c.
*/
#define _GNU_SOURCE                 /* Get '_sys_nerr' and '_sys_errlist'
                                       declarations from <stdio.h> */
#include <stdio.h>
#include <string.h>                 /* Get declaration of strerror() */
#include <pthread.h>
#include "tlpi_hdr.h"

static pthread_once_t once = PTHREAD_ONCE_INIT;
static pthread_key_t strerrorKey;

#define MAX_ERROR_LEN 256           /* Maximum length of string in per-thread
                                       buffer returned by strerror() */

static void                         /* Free thread-specific data buffer */
destructor(void *buf)
{
    free(buf);
}

static void                         /* One-time key creation function */
createKey(void)
{
    int s;

    /* Allocate a unique thread-specific data key and save the address
       of the destructor for thread-specific data buffers */

    s = pthread_key_create(&strerrorKey, destructor);
    if (s != 0)
        errExitEN(s, "pthread_key_create");
}

char *
strerror(int err)
{
    int s;
    char *buf;

    /* Make first caller allocate key for thread-specific data */

    s = pthread_once(&once, createKey);
    if (s != 0)
        errExitEN(s, "pthread_once");

    buf = pthread_getspecific(strerrorKey);
    if (buf == NULL) {          /* If first call from this thread, allocate
                                   buffer for thread, and save its location */
        buf = malloc(MAX_ERROR_LEN);
        if (buf == NULL)
            errExit("malloc");

        s = pthread_setspecific(strerrorKey, buf);
        if (s != 0)
            errExitEN(s, "pthread_setspecific");
    }

    if (err < 0 || err >= _sys_nerr || _sys_errlist[err] == NULL) {
        snprintf(buf, MAX_ERROR_LEN, "Unknown error %d", err);
    } else {
        strncpy(buf, _sys_errlist[err], MAX_ERROR_LEN - 1);
        buf[MAX_ERROR_LEN - 1] = '\0';          /* Ensure null termination */
    }

    return buf;
}

深入理解线程特有数据机制

深入理解线程特有数据的实现有助于对其 API 的使用。

在典型的实现中包含以下数组：

• 一个全局（进程级别）的数组，用于存放线程特有数据的键值信息
  pthread_key_create() 返回的 pthread_key_t 类型值只是对全局数组的索引，该全局数组标记为 pthread_keys，其格式大概如下：
  

  数组的每个元素都是一个包含两个字段的结构，第一个字段标记该数组元素是否在用，第二个字段用于存放针对此键、线程特有数据变的解构函数的一个副本，即 destructor 函数。

• 每个线程还包含一个数组，存有为每个线程分配的线程特有数据块的指针（通过调用 pthread_setspecific() 函数来存储的指针，即参数中的 value）

在常见的存储 pthread_setspecific()函数参数 value 的实现中，大多数都类似于下图的实现。

图中假设 pthread_keys[1]分配给 func1()函数，pthread API 为每个函数维护指向线程特有数据数据块的一个指针数组，
其中每个数组元素都与图线程特有数据键的实现 (上图) 中的全局 pthread_keys 中元素一一对应。

参考

• [1] Linux/UNIX 系统编程手册（上）
• [2] http://www.groad.net/bbs/thread-2182-1-1.html
• [3] http://baike.baidu.com/view/598128.htm

之前有一份私人git笔记老长老长了, 今天得空, 把它浓缩成5分钟版本.
感觉纯基础性的东西整理成博客差也差不多了, 还有很多凌乱的工作笔记慢慢在一点一点整理放上来吧,
估计下面几篇博客就开始游戏服务器的开发心得之类的了.

. . .

原文

原文出处

原文标题 : Source Multiplayer Networking

Multiplayer games based on the Source Engine use a Client-Server networking architecture. Usually a server is a dedicated host that runs the game and is authoritative about world simulation, game rules, and player input processing. A client is a player’s computer connected to a game server. The client and server communicate with each other by sending small data packets at a high frequency (usually 20 to 30 packets per second). A client receives the current world state from the server and generates video and audio output based on these updates. The client also samples data from input devices (keyboard, mouse, microphone, etc.) and sends these input samples back to the server for further processing. Clients only communicate with the game server and not between each other (like in a peer-to-peer application). In contrast with a single player game, a multiplayer game has to deal with a variety of new problems caused by packet-based communication.

Network bandwidth is limited, so the server can’t send a new update packet to all clients for every single world change. Instead, the server takes snapshots of the current world state at a constant rate and broadcasts these snapshots to the clients. Network packets take a certain amount of time to travel between the client and the server (i.e. half the ping time). This means that the client time is always a little bit behind the server time. Furthermore, client input packets are also delayed on their way back, so the server is processing temporally delayed user commands. In addition, each client has a different network delay which varies over time due to other background traffic and the client’s framerate. These time differences between server and client causes logical problems, becoming worse with increasing network latencies. In fast-paced action games, even a delay of a few milliseconds can cause a laggy gameplay feeling and make it hard to hit other players or interact with moving objects. Besides bandwidth limitations and network latencies, information can get lost due to network packet loss.

To cope with the issues introduced by network communication, the Source engine server employs techniques such as data compression and lag compensation which are invisible to the client. The client then performs prediction and interpolation to further improve the experience.

Basic networking

The server simulates the game in discrete time steps called ticks. By default, the timestep is 15ms, so 66.666… ticks per second are simulated, but mods can specify their own tickrate. During each tick, the server processes incoming user commands, runs a physical simulation step, checks the game rules, and updates all object states. After simulating a tick, the server decides if any client needs a world update and takes a snapshot of the current world state if necessary. A higher tickrate increases the simulation precision, but also requires more CPU power and available bandwidth on both server and client. The server admin may override the default tickrate with the -tickrate command line parameter, though tickrate changes done this way are not recommended because the mod may not work as designed if its tickrate is changed.

Clients usually have only a limited amount of available bandwidth. In the worst case, players with a modem connection can’t receive more than 5 to 7 KB/sec. If the server tried to send them updates with a higher data rate, packet loss would be unavoidable. Therefore, the client has to tell the server its incoming bandwidth capacity by setting the console variable rate (in bytes/second). This is the most important network variable for clients and it has to be set correctly for an optimal gameplay experience. The client can request a certain snapshot rate by changing cl_updaterate (default 20), but the server will never send more updates than simulated ticks or exceed the requested client rate limit. Server admins can limit data rate values requested by clients with sv_minrate and sv_maxrate (both in bytes/second). Also the snapshot rate can be restricted with sv_minupdaterate and sv_maxupdaterate (both in snapshots/second).

The client creates user commands from sampling input devices with the same tick rate that the server is running with. A user command is basically a snapshot of the current keyboard and mouse state. But instead of sending a new packet to the server for each user command, the client sends command packets at a certain rate of packets per second (usually 30). This means two or more user commands are transmitted within the same packet. Clients can increase the command rate with cl_cmdrate. This will increase responsiveness but requires more outgoing bandwidth, too.

Game data is compressed using delta compression to reduce network load. That means the server doesn’t send a full world snapshot each time, but rather only changes (a delta snapshot) that happened since the last acknowledged update. With each packet sent between the client and server, acknowledge numbers are attached to keep track of their data flow. Usually full (non-delta) snapshots are only sent when a game starts or a client suffers from heavy packet loss for a couple of seconds. Clients can request a full snapshot manually with the cl_fullupdate command.

Responsiveness, or the time between user input and its visible feedback in the game world, are determined by lots of factors, including the server/client CPU load, simulation tickrate, data rate and snapshot update settings, but mostly by the network packet traveling time. The time between the client sending a user command, the server responding to it, and the client receiving the server’s response is called the latency or ping (or round trip time). Low latency is a significant advantage when playing a multiplayer online game. Techniques like prediction and lag compensation try to minimize that advantage and allow a fair game for players with slower connections. Tweaking networking setting can help to gain a better experience if the necessary bandwidth and CPU power is available. We recommend keeping the default settings, since improper changes may cause more negative side effects than actual benefits.

Servers that Support Tickrate

The tickrate can be altered by using the -tickrate parameter

• Counter Strike: Global Offensive
  


• Half-Life 2: Deathmatch
  

The following servers tickrate cannot be altered as changing this causes server timing issues.

Tickrate 66

• Counter Strike: Source
  


• Day of Defeat: Source
  


• Team Fortress 2
  

Tickrate 30

• Left 4 Dead
  


• Left 4 Dead 2
  

Entity interpolation

By default, the client receives about 20 snapshot per second. If the objects (entities) in the world were only rendered at the positions received by the server, moving objects and animation would look choppy and jittery. Dropped packets would also cause noticeable glitches. The trick to solve this problem is to go back in time for rendering, so positions and animations can be continuously interpolated between two recently received snapshots. With 20 snapshots per second, a new update arrives about every 50 milliseconds. If the client render time is shifted back by 50 milliseconds, entities can be always interpolated between the last received snapshot and the snapshot before that.

Source defaults to an interpolation period (‘lerp’) of 100-milliseconds (cl_interp 0.1); this way, even if one snapshot is lost, there are always two valid snapshots to interpolate between. Take a look at the following figure showing the arrival times of incoming world snapshots:

The last snapshot received on the client was at tick 344 or 10.30 seconds. The client time continues to increase based on this snapshot and the client frame rate. If a new video frame is rendered, the rendering time is the current client time 10.32 minus the view interpolation delay of 0.1 seconds. This would be 10.22 in our example and all entities and their animations are interpolated using the correct fraction between snapshot 340 and 342.

Since we have an interpolation delay of 100 milliseconds, the interpolation would even work if snapshot 342 were missing due to packet loss. Then the interpolation could use snapshots 340 and 344. If more than one snapshot in a row is dropped, interpolation can’t work perfectly because it runs out of snapshots in the history buffer. In that case the renderer uses extrapolation (cl_extrapolate 1) and tries a simple linear extrapolation of entities based on their known history so far. The extrapolation is done only for 0.25 seconds of packet loss (cl_extrapolate_amount), since the prediction errors would become too big after that.

Entity interpolation causes a constant view “lag” of 100 milliseconds by default (cl_interp 0.1), even if you’re playing on a listenserver (server and client on the same machine). This doesn’t mean you have to lead your aiming when shooting at other players since the server-side lag compensation knows about client entity interpolation and corrects this error.

Input prediction

Lets assume a player has a network latency of 150 milliseconds and starts to move forward. The information that the +FORWARD key is pressed is stored in a user command and send to the server. There the user command is processed by the movement code and the player’s character is moved forward in the game world. This world state change is transmitted to all clients with the next snapshot update. So the player would see his own change of movement with a 150 milliseconds delay after he started walking. This delay applies to all players actions like movement, shooting weapons, etc. and becomes worse with higher latencies.

A delay between player input and corresponding visual feedback creates a strange, unnatural feeling and makes it hard to move or aim precisely. Client-side input prediction (cl_predict 1) is a way to remove this delay and let the player’s actions feel more instant. Instead of waiting for the server to update your own position, the local client just predicts the results of its own user commands. Therefore, the client runs exactly the same code and rules the server will use to process the user commands. After the prediction is finished, the local player will move instantly to the new location while the server still sees him at the old place.

After 150 milliseconds, the client will receive the server snapshot that contains the changes based on the user command he predicted earlier. Then the client compares the server position with his predicted position. If they are different, a prediction error has occurred. This indicates that the client didn’t have the correct information about other entities and the environment when it processed the user command. Then the client has to correct its own position, since the server has final authority over client-side prediction. If cl_showerror 1 is turned on, clients can see when prediction errors happen. Prediction error correction can be quite noticeable and may cause the client’s view to jump erratically. By gradually correcting this error over a short amount of time (cl_smoothtime), errors can be smoothly corrected. Prediction error smoothing can be turned off with cl_smooth 0.

Prediction is only possible for the local player and entities affected only by him, since prediction works by using the client’s keypresses to make a “best guess” of where the player will end up. Predicting other players would require literally predicting the future with no data, since there’s no way to instantaneously get keypresses from them.

Lag compensation

All source code for lag compensation and view interpolation is available in the Source SDK. See


for implementation details.

Let’s say a player shoots at a target at client time 10.5. The firing information is packed into a user command and sent to the server. While the packet is on its way through the network, the server continues to simulate the world, and the target might have moved to a different position. The user command arrives at server time 10.6 and the server wouldn’t detect the hit, even though the player has aimed exactly at the target. This error is corrected by the server-side lag compensation.

The lag compensation system keeps a history of all recent player positions for one second. If a user command is executed, the server estimates at what time the command was created as follows:

Command Execution Time = Current Server Time - Packet Latency - Client View Interpolation

Then the server moves all other players - only players - back to where they were at the command execution time. The user command is executed and the hit is detected correctly. After the user command has been processed, the players revert to their original positions.

On a listen server you can enable sv_showimpacts 1 to see the different server and client hitboxes:

This screenshot was taken on a listen server with 200 milliseconds of lag (using net_fakelag), right after the server confirmed the hit. The red hitbox shows the target position on the client where it was 100ms + interp period ago. Since then, the target continued to move to the left while the user command was travelling to the server. After the user command arrived, the server restored the target position (blue hitbox) based on the estimated command execution time. The server traces the shot and confirms the hit (the client sees blood effects).

Client and server hitboxes don’t exactly match because of small precision errors in time measurement. Even a small difference of a few milliseconds can cause an error of several inches for fast-moving objects. Multiplayer hit detection is not pixel perfect and has known precision limitations based on the tickrate and the speed of moving objects.

The question arises, why is hit detection so complicated on the server? Doing the back tracking of player positions and dealing with precision errors while hit detection could be done client-side way easier and with pixel precision. The client would just tell the server with a “hit” message what player has been hit and where. We can’t allow that simply because a game server can’t trust the clients on such important decisions. Even if the client is “clean” and protected by Valve Anti-Cheat, the packets could be still modified on a 3rd machine while routed to the game server. These “cheat proxies” could inject “hit” messages into the network packet without being detected by VAC (a “man-in-the-middle” attack).

Network latencies and lag compensation can create paradoxes that seem illogical compared to the real world. For example, you can be hit by an attacker you can’t even see anymore because you already took cover. What happened is that the server moved your player hitboxes back in time, where you were still exposed to your attacker. This inconsistency problem can’t be solved in general because of the relatively slow packet speeds. In the real world, you don’t notice this problem because light (the packets) travels so fast and you and everybody around you sees the same world as it is right now.

Net graph

The Source engine offers a couple of tools to check your client connection speed and quality. The most popular one is the net graph, which can be enabled with net_graph 2 (or +graph). Incoming packets are represented by small lines moving from right to left. The height of each line reflects size of a packet. If a gap appears between lines, a packet was lost or arrived out of order. The lines are color-coded depending on what kind of data they contain.

Under the net graph, the first line shows your current rendered frames per second, your average latency, and the current value of cl_updaterate. The second line shows the size in bytes of the last incoming packet (snapshots), the average incoming bandwidth, and received packets per second. The third line shows the same data just for outgoing packets (user commands).

Optimizations

The default networking settings are designed for playing on dedicated server on the Internet. The settings are balanced to work well for most client/server hardware and network configurations. For Internet games the only console variable that should be adjusted on the client is “rate”, which defines your available bytes/second bandwidth of your network connection. Good values for “rate” is 4500 for modems, 6000 for ISDN, 10000 DSL and above.

In an high-performance network environment, where the server and all clients have the necessary hardware resources available, it’s possible to tweak bandwidth and tickrate settings to gain more gameplay precision. Increasing the server tickrate generally improves movement and shooting precision but comes with a higher CPU cost. A Source server running with tickrate 100 generates about 1.5x more CPU load than a default tickrate 66. That can cause serious calculation lags, especially when lots of people are shooting at the same time. It’s not suggested to run a game server with a higher tickrate than 66 to reserve necessary CPU resources for critical situations.

If the game server is running with a higher tickrate, clients can increase their snapshot update rate (cl_updaterate) and user command rate (cl_cmdrate), if the necessary bandwidth (rate) is available. The snapshot update rate is limited by the server tickrate, a server can’t send more then one update per tick. So for a tickrate 66 server, the highest client value for cl_updaterate would be 66. If you increase the snapshot rate and encounter packet loss or choke, you have to turn it down again. With an increased cl_updaterate you can also lower the view interpolation delay (cl_interp). The default interpolation delay is 0.1 seconds, which derives from the default cl_updaterate 20. View interpolation delay gives a moving player a small advantage over a stationary player since the moving player can see his target a split second earlier. This effect is unavoidable, but it can be reduced by decreasing the view interpolation delay. If both players are moving, the view lag delay is affecting both players and nobody has an advantage.

This is the relation between snapshot rate and view interpolation delay is the following:

interpolation period = max( cl_interp, cl_interp_ratio / cl_updaterate )

“Max(x,y)” means “whichever of these is higher”. You can set cl_interp to 0 and still have a safe amount of interp. You can then increase cl_updaterate to decrease your interp period further, but don’t exceed tickrate (66) or flood your connection with more data than it can handle.

Tips

Don’t change console settings unless you are 100% sure what you are doing

Most "high-performance" setting cause exactly the opposite effect, if the server or network can't handle the load.

Don’t turn off view interpolation and/or lag compensation

It will not improve movement or shooting precision.

Optimized setting for one client may not work for other clients

Do not just use settings from other clients without verifing them for your system.

If you follow a player in “First-Person” as a spectator in a game or SourceTV, you don’t exactly see what the player sees

Spectators see the game world without lag compensation.

译文

自我总结

注 : V社这篇文章相当有价值, 所以会有尽可能详细的注解以及对原译文各种翻译纰漏的修正.

广义的延迟补偿主要包括两个方面 :

• A 如何显示目标
  • a. 对于本玩家自己
    • 客户端预表现(本文翻译为”客户端预测”) : 比如对于玩家移动的预测, 可以把服务器确认过的movement信息作为开始, 然后使用自己本地的movement input来进行预表现, 服务器跟客户端共享同一套move代码, 当服务器的确认信息过来之后就直接使用服务器发过来的from state进行修正并以from state为基础执行之后的预测.
  • b. 对于其他玩家
    • i. 外推法 : 即航位推测法, 外推法把其它玩家/物体看作一个点，这个点开始的位置、方向、速度已知，沿着自己的弹道向前移动。因此，假设延时是100ms，最新的协议通知客户端这个玩家奔跑速度是500单位每秒，方向垂直于玩家视线，客户端就可以假设事实上这个玩家当前实际的位置已经向前移动了50个单位。客户端可以在这个外推的位置渲染这个玩家. 这个方法不适用于FPS游戏, 因为大部分FPS游戏采用非现实的玩家系统，玩家可以随时转弯，可以在任意角度作用不现实的加速度，因此外推法得到的结果经常是错误地
    • ii. 内推法 : 即影子跟随法, 这种方法是用延时来换取平滑, 客户端物体实际移动位置总是滞后一段时间。举个例子，如果服务器每秒同步10次世界信息，客户端渲染的时候会有100ms滞后。这样，每一帧渲染的时候，我们通过最新收到的位置信息和前100ms的位置信息（或者上一帧渲染位置）进行差值得到结果.
      • 如果一个更新包没有收到，有2种处理方法 :
        • 用上面介绍的外推法（有可能产生较大误差）；
        • 保持玩家位于当前位置直到收到下一个更新包(会导致玩家移动顿挫)

• B. 延迟补偿, 步骤如下 :

  1. 为玩家计算一个相当精确的延迟时间
  2. 对每个玩家，从服务器历史信息中找一个已发送给这个玩家并且这个玩家已收到的的world update, 这个world update是在这个玩家将要执行这个movement command之前的world update
  3. 对于每一个玩家，将其从上述的world update处拉回到这个玩家生成此user command的更新时间中执行用户命令。这个回退时间需要考虑到命令执行的时候的网络延时和插值量
  4. 执行玩家命令（包括武器开火等。）
  5. 将所有移动的、错位的玩家移动到他们当前正确位置

原文

原文出处

原文标题 : Latency Compensating Methods in Client/Server In-game Protocol Design and Optimization

Overview

Dsigning first-person action games for Internet play is a challenging process. Having robust on-line gameplay in your action title, however, is becoming essential to the success and longevity of the title. In addition, the PC space is well known for requiring developers to support a wide variety of customer setups. Often, customers are running on less than state-of-the-art hardware. The same holds true for their network connections.

While broadband has been held out as a panacea for all of the current woes of on-line gaming, broadband is not a simple solution allowing developers to ignore the implications of latency and other network factors in game designs. It will be some time before broadband truly becomes adopted in the United States, and much longer before it can be assumed to exist for your clients in the rest of the world. In addition, there are a lot of poor broadband solutions, where users may occasionally have high bandwidth, but more often than not also have significant latency and packet loss in their connections.

Your game must behave well in this world. This discussion will give you a sense of some of the tradeoffs required to deliver a cutting-edge action experience on the Internet. The discussion will provide some background on how client / server architectures work in many on-line action games. In addition, the discussion will show how predictive modeling can be used to mask the effects of latency. Finally, the discussion will describe a specific mechanism, lag compensation, for allowing the game to compensate for connection quality.

. . .