Architecture Diagram

This architecture diagram is automatically generated by writing PlantUML.

Key Features

Business layer development based on ECS framework: You can inherit from the base entity class and base component class.
Integrated service mechanism based on etcd: It includes service registration, TTL, service discovery, load balancing, load reporting, and Watch mechanism.
RPC framework based on msgpack: It supports direct calling via IP address and direct calling of remote virtual entities/components in combination with ECS.
Coroutine business layer support based on asyncio asynchronous IO: You can achieve the effect of directly calling RPC and getting the return value like result = await rpc_call().
- A coroutine pool is implemented and encapsulated into a concise decorator for easy calling in the business layer.
Support for TCP and RUDP
Asynchronous HTTP microservice framework based on Sanic: It is convenient for developing various public services.
- An authentication module based on JWT.
- A data persistence module based on Redis.
- An ODM module based on Umongo.
Hot - update reload module:
- Full - scale update: Stable.
- Incremental update: Faster, convenient for daily development.
Asynchronous TimedRotating log module support:
- Automatically rolls over log files according to date and time.
- Supports callbacks of coroutine objects.
- Changes colors according to log levels for easy querying.
- Prints the stack and locals when reporting traces.
- Provides file jump support in PyCharm for log levels above warning.
Timer module supporting 1:N model: It avoids the error - prone issue of overwriting the same key.
- You can reuse a key without overwriting the previous timers associated with the key. However, when calling cancel_timer, all timers with the same key will be cancelled at once.
Enhanced JSON parser: It supports comments, automatic removal of commas, and variable macros.
Data persistence module based on MongoDB
Client - side simulation and automated testing support
Pre - gateway gate server for the lobby server: It is responsible for data compression/decompression, encryption/decryption, and authentication.

Python Version

Python 3.8.8

Q&A

Here are the answers to common questions. Please don’t spam the group with these questions:

Where can I find the old version?
- Do you mean the CPP version in the following demo image? It’s in the master branch.
Why choose Python for the new version instead of C++? Or why not make C++ the bottom layer and Python the upper layer?
- It’s possible to use C++ as the bottom layer and Python as the upper layer for calling. However, the current vision is to encourage more developers to contribute. The threshold of C++ is significantly higher than that of Python for other contributors.
- If there are enough users in the future, we will consider making the bottom layer in C++ and packaging it into .pyd or .so for the upper layer to call.
- For most games, the performance of Python is sufficient.
- The vision of this server engine is to target mass developers for rapid development. Python has a wide audience and is easy to learn.
What’s the significance of lobby_gate? Why doesn’t the client connect directly to the lobby?
- Generally, when latency is not a critical issue, the client accesses the lobby through the gate. The gate is responsible for proxying and forwarding network communication data between the client and the game. It also handles data encryption, decryption, compression, and decompression, reducing the performance pressure on the lobby.
- The logic of message processing in the lobby can be simpler without dealing with I/O multiplexing, as all messages come from a single TCP connection (single gateway) or a fixed number of TCP connections (multiple gateways).
- After the client starts, it connects to the gateway, which then connects to the scene server and forwards all message packets. When switching scenes, the connection between the client and the gateway remains intact. The gateway is responsible for connecting to the new scene. Since the gateway and the game server are usually in the same local area network, the problem of disconnection is greatly improved.
- In a multi - process online game architecture, the game server not only processes client messages but also messages from other processes such as accounts and GM tools. For security reasons, it is necessary to isolate and restrict client messages.
- Game servers often need to start new servers, merge servers, or migrate due to hardware failures. If the game server directly provides services to the client, the client usually needs to log out and log in again when the service changes. If a gateway is used, when the server address changes internally, only the gateway needs to be updated and reconnect, and the client may not even notice.
What is RUDP?
- Its full name is reliable UDP.
- Currently, this server engine is implemented in Python based on the standard KCP algorithm.
- There are plans to improve KCP in the future, and the performance will be further enhanced. Stay tuned:
  - Add a dynamic redundant packet mechanism based on SRTT monitoring.
  - Simplify the packet header (for example, some fields in the packet header do not need 32 bits and can be changed to 16 bits).
  - Introduce the dupack mechanism, not just sending IKCP_CMD_ACK packets.
    - Make full use of the fields in the packet header. For example, when the number of ACKs to be sent in the ACK list array is less than or equal to 3, use the len, rdc_len, and sn in the packet header to represent the sequence numbers of the received packets.
    - Redundant ACK mechanism: When the number of ACKs to be sent in the ACK list array is greater than 3, the body will contain all the ACKs in the merged ACK list and the previously sent ACKs until the packet size reaches the MSS.
  - …
What about etcd?
- It implements service registration, service discovery, load balancing, and load reporting through its TTL and Watch mechanisms.
- It is convenient for implementing distributed locks.
- The V2 version of the HTTP API is used for interacting with etcd. Since the interaction is not sensitive to latency, gRPC is not used to keep it simple.
- Then why choose etcd instead of ZooKeeper?
  - The purposes of the two applications are different. The purpose of etcd is to be a highly available Key/Value storage system, mainly used for sharing configurations and service discovery; the purpose of ZooKeeper is to be a highly effective and reliable collaborative working system.
  - The interface call methods are different. etcd is based on an HTTP + JSON API, which can be easily used with just curl, making it convenient for each host in the cluster to access; ZooKeeper is based on TCP and requires a dedicated client.
  - The functions are quite similar. Both etcd and ZooKeeper provide key - value storage services, cluster queue synchronization services, and the ability to observe changes in the value of a key.
  - The deployment methods are also similar: They both use a cluster mode and can support thousands of nodes. etcd is written in Go and can be deployed by directly compiling the binary files; ZooKeeper is written in Java and depends on JDK, so JDK needs to be deployed first.
  - Implementation languages: Go has almost the same efficiency as C, especially since Go is a language designed for multi - threading and process communication, and it performs very well in small - scale clusters; Java requires more code to implement, and its performance is average in small - scale clusters. However, after optimizing multi - threading in large - scale scenarios, its performance is not much different from that of Go.

. . .

To - Do List

Introduce Python in Unity based on Python.NET for hot updates or business writing to unify front - end and back - end code.
Develop supporting tools for flame graphs.
Create tools for importing and exporting configuration tables.
…

一个轻量级的游戏服务器引擎

要点

业务层基于ECS框架来做开发, 继承实体基类与组件基类即可
基于etcd的服务注册 / TTL / 服务发现 / 负载均衡 / 上报负载 / Watch机制一体化
基于msgpack的RPC框架, 支持 ip地址直接call以及配合ECS的remote虚拟实体/组件直接call
基于asyncio异步IO的协程业务层支持, 可实现类似 result = await rpc_call() 的直接调RPC拿返回的效果
- 实现了协程池, 封装成了简洁的装饰器便于业务层调用
支持TCP与RUDP
基于sanic开发的异步HTTP微服务框架供方便开发各类公共服务
- 基于jwt的auth模块
- 基于redis的数据落地模块
- 基于umongo的ODM模块
热更新reload模块
- 全量式, 安全保障
- 增量式, 速度更快, 方便平时开发
支持异步的TimedRotating日志模块
- 根据日期时间自动滚动切换日志文件
- 支持协程对象的callback
- 根据日志level改变颜色, 方便查询
- 报trace可打印堆栈与locals
- 对于 warning 以上的日志级别直接对Pycharm提供文件跳转支持
支持1:N模型的定时器模块, 避免覆盖同一个key的易错点
- 可以重复使用一个key, 并不会冲掉之前key的timer, 但是当调用cancel_timer的时候, 会一次性全部cancel掉所有
制作了增强型json解析器, 支持注释/自动去除逗号/变量宏
基于MongoDB的数据落地模块
client端的模拟与自动化测试配套
大厅服务器的前置网关gate服务器, 负责压缩/解压, 加密/解密数据以及鉴权

架构图

本架构图根据 PlantUML 自动生成