Kafka 详解

引言

深度使用kafka协议做了两年的产品，想把kafka里面的一些东西记录一下。

kafka协议

Kafka 使用基于 TCP 的二进制协议，所有的Apis都是由[request，reponse]对来构成。

kafka鉴权链

sequenceDiagram
    participant C as Client
    participant B as Broker
    C ->> B: ApiVersions Request
    B -->> C: ApiVersions Response
    C ->> B: SaslHandshake Request
    B -->> C: SaslHandshake Response
    C ->> B: SaslAuthenticate Request
    B -->> C: SaslAuthenticate Response

kafka消费者

sequenceDiagram
    participant C as Consumer
    participant B as Broker
    C ->> B: ApiVersions Request
    B -->> C: ApiVersions Response
    C ->> B: Metadata Request
    B -->> C: Metadata Response
    C ->> B: FindCoordinator Request
    B -->> C: FindCoordinator Response

sequenceDiagram
    participant C as Consumer
    participant B as Broker
    C ->> B: JoinGroup Request
    B -->> C: JoinGroup Response
    C ->> B: SyncGroup Request
    B -->> C: SyncGroup Response
    loop
        C ->> B: Heartbeat Request
        B -->> C: Heartbeat Response
    end
    C ->> B: OffsetFetch Request
    B -->> C: OffsetFetch Response
    C ->> B: OffsetCommit Request
    B -->> C: OffsetCommit Response
    C ->> B: LeaveGroup Request
    B -->> C: LeaveGroup Response

sequenceDiagram
    participant C as Consumer
    participant B as Broker
    C ->> B: ListOffset Request
    B -->> C: ListOffset Response
    C ->> B: Fetch Request
    B -->> C: Fetch Response

kafka生产者

sequenceDiagram
    participant P as Producer
    participant B as Broker
    P ->> B: ApiVersions Request
    B -->> P: ApiVersions Response
    P ->> B: Metadata Request
    B -->> P: Metadata Response

sequenceDiagram
    participant P as Producer
    participant B as Broker
    P ->> B: Produce Request
    B -->> P: Produce Response