Handling Blockchain Reorgs in Kafka Streams

Bitquery Kafka topics publish blocks as soon as the pipeline sees them, before the network has settled on them. On Solana a message can leave the stream while the block is still being assembled from shreds. A block that is later abandoned by the network is therefore already in your consumer. Bitquery does not send a "this block was reverted" message. Instead every message carries the block hash and the parent hash, which is enough for the consumer to build the block tree, follow the longest branch and roll back the blocks that fell off it.

This page documents what Bitquery sends, the algorithm, and a reference consumer.

What a reorg looks like in the stream

Block A has two children. Block B arrived first and became the tip. Block C has the same parent and the same or a later slot. When Block D arrives with ParentHash = C, the branch A → C → D is longer than A → B. B is orphaned: anything the consumer derived from B has to be rolled back, and D becomes the new tip.

Two rules follow:

1. Block identity is the hash, never the slot or block number. On Solana two different blocks can claim the same slot. On EVM chains two blocks can carry the same number during a reorg.
2. A reorg is decided by branch length, not by arrival. A competing block of equal length is stored but does not replace the tip. Only a strictly longer branch triggers a rollback.

What Bitquery sends

Every message is one protobuf block (or, on Solana, one part of a block). The block identity travels in two places: the protobuf Header and the Kafka record headers. Both were read live on 9 November 2026 from solana.transactions.proto, eth.transactions.proto and tron.transactions.proto.

Protobuf block header

Field	Solana	EVM (eth, bsc, base, ...)	Tron
Block hash	Header.Hash (bytes)	Header.Hash (bytes)	Header.Hash (bytes)
Parent hash	Header.ParentHash (bytes)	Header.ParentHash (bytes)	Header.ParentHash (bytes)
Block number	Header.Slot	Header.Number (bytes, big-endian)	Header.Number
Parent number	Header.ParentSlot	not in the header; use Number - 1	Header.ParentNumber
Height	Header.Height	Header.Number	Header.Number
Time	Header.Timestamp	Header.Time	Header.Timestamp

Schemas: solana/block_message.proto, evm/block_message.proto, tron/block_message.proto. The same BlockHeader is used by the transactions, tokens, dextrades and raw topics of each chain.

Kafka record key and headers

The producer sets the Kafka key and a set of string headers on every record, so a consumer can route or group messages without decoding the protobuf.

	Solana	EVM	Tron
Kafka key	slot as a decimal string, e.g. 467271470	block hash, 0x...	block hash, hex
BlockHash	base58 block hash	0x hex	hex
ParentHash	base58 parent hash	0x hex	hex
BlockNumber	slot	block number	block number
ParentNumber	parent slot	0 (not set)	parent block number
Height	block height	0 (not set)	0 (not set)
TransactionsExpectedCount	total transactions in the block, see below	-	-
TransactionsIndexes	indexes of the transactions in this part, e.g. [64 65 ... 127]	-	-
TransactionsCount	-	transactions in the block	transactions in the block
ChainId, MinerReward, UncleRewardsCount	-	set	set

Because the key is the slot (Solana) or the block hash (EVM, Tron), all parts of one block land on the same partition and keep their relative order there. Different blocks spread across the six partitions, so order across blocks is not guaranteed: in the 7-minute Solana sample 11% of consecutive messages went to a lower slot than the previous one. EVM and Tron blocks arrived in order in the sample, but the contract is the same, so do not depend on it.

Solana: blocks arrive in parts, and early parts have no hash

A Solana block is published as 1 to 5 messages (most often 3 or 4), each carrying a slice of the transactions. The parts that leave before the block meta is known carry only Header.Slot and Header.Timestamp; their Hash, ParentHash, ParentSlot and Height are empty, and so are the BlockHash and ParentHash Kafka headers. In the sample, 34% of messages were such hash-less parts. The later parts of the same block carry the full header and the TransactionsExpectedCount header.

This means a Solana consumer has to assemble a block before it can place it in the tree:

• stage parts by Slot until a part with Hash arrives, then key the assembly by that hash;
• deduplicate transactions by Signature (duplicates happen, see Kafka Streaming Concepts);
• treat the block as complete when the union of TransactionsIndexes across its parts reaches TransactionsExpectedCount (378 of 379 slots in a 90-second sample completed this way).

EVM and Tron topics publish one message per block with the full header, so no assembly is needed.

What Bitquery does not send

• No revert, "dropped" or "finalized" message. Finality is the consumer's decision, based on tree depth.
• No ordering guarantee across blocks, and no guarantee that a block is delivered once.
• Nothing about blocks that never reached Kafka. If a slot is skipped on the kept chain (ParentSlot is more than one below Slot), the skipped slots were abandoned, whether or not you received parts of them. In the Solana sample one such case appeared: slot 467273400 with ParentSlot 467273395.

How to detect and roll back a reorg

The consumer keeps a small tree of recent blocks keyed by hash, with the parent hash and the branch depth of each. For every assembled block:

1. Extend. If ParentHash equals the current tip, the block is the new tip, depth = tip depth + 1. No reorg.
2. Fork. Otherwise walk back from the tip through parent hashes and collect the visited set, then walk back from the incoming block's parent until a visited hash appears. That is the fork point. If none is found the parent is unknown (a gap or a block older than the buffer); store the block and keep the tip.
3. Compare. Incoming branch length = depth(parent) + 1. If it is not greater than the tip depth, store the block as a side branch and keep the tip.
4. Roll back. If it is greater, every block from the tip back to (but excluding) the fork point is orphaned. Delete or mark what you derived from those hashes, then make the incoming block the tip.

Because messages arrive out of order, hold a block until its parent has been processed. A block whose parent is not yet known is not a fork; it is early. Release it when the parent arrives. On Solana a buffer of about 30 slots at the head is enough, since a block is rooted within about a second and competing branches are a few slots long. On EVM chains a buffer of a few blocks covers the reorgs seen in practice.

Finality

Consumers that cannot roll back should instead delay: act on a block only when it is N blocks below the tip of the longest branch (for example 32 slots on Solana, 2 to 6 blocks on EVM depending on the chain). The same tree gives you that depth.

Reference consumer

The complete Python consumer is in bitquery/Solana-Kafka-Stream-Reorg. It has three modules:

• buffer.py assembles Solana parts into blocks, holds blocks until their parent was released, and hands over batches;
• computations.py holds the tree and the reorg decision;
• consumer.py reads Kafka with confluent_kafka and wires the two together.

The tree logic, trimmed to the essentials:

from collections import namedtuple

BlockInfo = namedtuple("BlockInfo", ("slot", "parent_hash", "depth"))

def find_fork_point(tip, incoming_parent, chain):
    visited, h = set(), tip
    while h and h in chain:
        visited.add(h)
        h = chain[h].parent_hash
    h = incoming_parent
    while h and h in chain:
        if h in visited:
            return h
        h = chain[h].parent_hash
    return None

def apply_block(block_hash, parent_hash, slot, chain, tip):
    """Returns (new_tip, orphaned_hashes_or_None)."""
    if tip is None or parent_hash == tip:              # first block or plain extension
        depth = chain[parent_hash].depth + 1 if parent_hash in chain else 1
        chain[block_hash] = BlockInfo(slot, parent_hash, depth)
        return block_hash, None

    fork = find_fork_point(tip, parent_hash, chain)
    incoming_len = chain[parent_hash].depth + 1 if parent_hash in chain else 0
    chain[block_hash] = BlockInfo(slot, parent_hash, incoming_len)
    if fork is None or incoming_len <= chain[tip].depth:
        return tip, None                               # unknown parent, or not longer: keep tip

    orphaned, h = [], tip
    while h and h != fork:                             # tip -> fork point, exclusive
        orphaned.append(h)
        h = chain[h].parent_hash
    for h in orphaned:
        chain.pop(h, None)
    return block_hash, orphaned

Reading the identity from a Solana message:

from solana import parsed_idl_block_message_pb2

block = parsed_idl_block_message_pb2.ParsedIdlBlockMessage()
block.ParseFromString(msg.value())
h = block.Header
slot, block_hash, parent_hash = h.Slot, bytes(h.Hash), bytes(h.ParentHash)
if not block_hash:
    stage_part_by_slot(slot, block)      # early part, wait for the hash-bearing part
else:
    add_part_by_hash(block_hash, parent_hash, slot, block)

For EVM topics the same code reads ParsedAbiBlockMessage from evm.parsed_abi_block_message_pb2, with int.from_bytes(h.Number, "big") as the block number and no staging step. For Tron, tron.parsed_abi_block_message_pb2 with h.Number and h.ParentNumber.

Rolling back is application-specific. If you write to a database, store the block hash on every row and delete by hash:

DELETE FROM trades WHERE block_hash IN (...orphaned hashes...)

Run one consumer process for the tree. Several consumers in one group each see a different subset of partitions, and a tree built from a subset reports forks that do not exist. Scale the heavy work (parsing, writes) in worker threads behind the single tree owner.

What was measured

Read-only consumers on 9 November 2026, latest offsets, 7 minutes per topic.

	solana.transactions.proto	eth.transactions.proto	tron.transactions.proto
Messages	9,263	35	137
Distinct blocks (by hash)	1,907	35	137
Messages per block	1 to 5, mode 4	1	1
Messages with Header.Hash set	66%	100%	100%
Consecutive messages with a lower block number than the previous	11%	0	0
Two hashes for one slot or number	0	0	0
Skipped slots on the kept chain	1 (4 slots)	0	0
Blocks whose parent was also in the window	99.7%	all but the first	all but the first

Reorgs are rare at this depth. A consumer that is not prepared for them will run for hours without a problem and then double-count or keep a reverted trade. Build the tree from the start.

Pitfalls

• Comparing slots or block numbers instead of hashes. Two blocks with the same slot are two different blocks. A consumer keyed by slot silently merges or overwrites them.
• Treating a block with an unknown parent as a fork. Out-of-order delivery is normal (11% of Solana messages went backwards in the sample). Hold the block until its parent arrives.
• Reorging on equal length. A side block at the same depth as the tip must not replace it. Wait for the next block to decide.
• Using Header.Timestamp to detect the last part of a Solana block. In the sample the timestamp was set on every part. Use Header.Hash (set on the later parts) and the TransactionsExpectedCount header to know a block is complete.
• Multiple consumers in one group with a shared tree. Each consumer sees only its partitions; the tree is wrong. One process owns the tree.
• Expecting a revert message. There is none. The parent hash is the signal.
• Looking for ParentNumber or Height in EVM Kafka headers. They are 0. Use Header.Number from the protobuf.

Frequently Asked Questions

Does Bitquery send a message when a block is reverted?

No. Every message carries the block hash and parent hash. The consumer builds the block tree from those and rolls back the blocks that fall off the longest branch.

Why can't I key my data by slot or block number?

During a reorg two different blocks claim the same slot or number. Only the block hash identifies a block. Store the hash on every derived row and delete by hash on rollback.

Some Solana messages have an empty Header.Hash. Is the data broken?

No. The early parts of a Solana block leave the stream before the block meta is known and carry only Slot and Timestamp. Stage them by slot and merge them when the part with the hash arrives.

How many blocks should I keep in the tree?

About 30 slots at the head on Solana, a few blocks on EVM chains. Prune everything deeper; it will not be reorganized.

How do I know a Solana block is complete?

Union the TransactionsIndexes Kafka headers of its parts and compare with the TransactionsExpectedCount header, which arrives with the later parts.

Can I run several consumers for throughput and still detect reorgs?

Run one process for the tree. Several consumers in one group each see a subset of partitions and report false forks. Put the heavy work in worker threads behind the single tree owner.

Related

• Kafka Streaming Concepts: topics, duplicates, partitioning by key
• Kafka Operations Cookbook: offsets, timestamps, retention
• Solana Shred Stream: why Solana messages arrive before the block closes
• Blockchain Reorg Tree in GraphQL: select_blocks: trunk and branches for the query API
• bitquery/Solana-Kafka-Stream-Reorg: the reference consumer