Add dozer hydration

MUD initial data hydration, and therefore filtering, comes in two flavors: Dozer and generic. Note that this is for the initial hydration, currently limits on on-going synchronization are limited to the generic method.

	Dozer	Generic
Filtering	Can filter on most SQL functions	Can only filter on tables and the first two key fields (limited by `eth_getLogs` (opens in a new tab) filters)
Availability	Redstone (opens in a new tab), Garnet (opens in a new tab), or elsewhere if you run your own instance	Any EVM chain
Security assumptions	The Dozer instance returns accurate information	The endpoint returns accurate information (same assumption as any other blockchain app)

In this tutorial you learn how to add dozer hydration to an existing MUD application, such as the ones created by the template. To avoid running dozer locally, we use a World on Garnet at address 0x95F5d049B014114E2fEeB5d8d994358Ce4FFd06e (opens in a new tab) that runs a slightly modified version of the React template (opens in a new tab). You can see the data schema for the World in the block explorer (opens in a new tab).

Create a client to access the `World`

These are the steps to create a client that can access the World.

Create and run a react template application.

pnpm create mud@latest tasks --template react
cd tasks
pnpm dev

Browse to the application (opens in a new tab). The URL specifies the chainId and worldAddress for the World.
In MUD DevTools see your account address and fund it on Garnet (opens in a new tab). You may need to get test ETH for your own address, and then transfer it to the account address the application uses.
You can now create, complete, and delete tasks.

To see the content of the app__Creator table, edit packages/contracts/mud.config.ts to add the Creator table definition.

mud.config.ts

import { defineWorld } from "@latticexyz/world";
 
export default defineWorld({
  namespace: "app",
  tables: {
    Tasks: {
      schema: {
        id: "bytes32",
        createdAt: "uint256",
        completedAt: "uint256",
        description: "string",
      },
      key: ["id"],
    },
    Creator: {
      schema: {
        id: "bytes32",
        taskCreator: "address",
      },
      key: ["id"],
    },
  },
});

Updating the client to use dozer

The main purpose of dozer is to allow MUD clients to specify the subset of table records that a client needs, instead of synchronizing whole tables.

To update the client, you change packages/client/src/mud/setupNetwork.ts to:

setupNetwork.ts

/*
 * The MUD client code is built on top of viem
 * (https://viem.sh/docs/getting-started.html).
 * This line imports the functions we need from it.
 */
import {
  createPublicClient,
  fallback,
  webSocket,
  http,
  createWalletClient,
  Hex,
  ClientConfig,
  getContract,
} from "viem";
 
import { DozerSyncFilter, getSnapshot, selectFrom } from "@latticexyz/store-sync/dozer";
 
import { syncToZustand } from "@latticexyz/store-sync/zustand";
import { getNetworkConfig } from "./getNetworkConfig";
import IWorldAbi from "contracts/out/IWorld.sol/IWorld.abi.json";
import { createBurnerAccount, transportObserver, ContractWrite } from "@latticexyz/common";
import { transactionQueue, writeObserver } from "@latticexyz/common/actions";
import { Subject, share } from "rxjs";
 
/*
 * Import our MUD config, which includes strong types for
 * our tables and other config options. We use this to generate
 * things like RECS components and get back strong types for them.
 *
 * See https://mud.dev/templates/typescript/contracts#mudconfigts
 * for the source of this information.
 */
import mudConfig from "contracts/mud.config";
 
export type SetupNetworkResult = Awaited<ReturnType<typeof setupNetwork>>;
 
export async function setupNetwork() {
  const networkConfig = await getNetworkConfig();
 
  /*
   * Create a viem public (read only) client
   * (https://viem.sh/docs/clients/public.html)
   */
  const clientOptions = {
    chain: networkConfig.chain,
    transport: transportObserver(fallback([webSocket(), http()])),
    pollingInterval: 1000,
  } as const satisfies ClientConfig;
 
  const publicClient = createPublicClient(clientOptions);
 
  /*
   * Create an observable for contract writes that we can
   * pass into MUD dev tools for transaction observability.
   */
  const write$ = new Subject<ContractWrite>();
 
  /*
   * Create a temporary wallet and a viem client for it
   * (see https://viem.sh/docs/clients/wallet.html).
   */
  const burnerAccount = createBurnerAccount(networkConfig.privateKey as Hex);
  const burnerWalletClient = createWalletClient({
    ...clientOptions,
    account: burnerAccount,
  })
    .extend(transactionQueue())
    .extend(writeObserver({ onWrite: (write) => write$.next(write) }));
 
  /*
   * Create an object for communicating with the deployed World.
   */
  const worldContract = getContract({
    address: networkConfig.worldAddress as Hex,
    abi: IWorldAbi,
    client: { public: publicClient, wallet: burnerWalletClient },
  });
 
  const dozerUrl = "https://dozer.mud.garnetchain.com/q";
  const yesterday = Date.now() / 1000 - 24 * 60 * 60;
  const filters: DozerSyncFilter[] = [
    selectFrom({
      table: mudConfig.tables.app__Tasks,
      where: `"createdAt" > ${yesterday}`,
    }),
    { table: mudConfig.tables.app__Creator },
  ];
  const { initialBlockLogs } = await getSnapshot({
    dozerUrl,
    storeAddress: networkConfig.worldAddress as Hex,
    filters,
    chainId: networkConfig.chainId,
  });
  const liveSyncFilters = filters.map((filter) => ({
    tableId: filter.table.tableId,
  }));
 
  /*
   * Sync on-chain state into RECS and keeps our client in sync.
   * Uses the MUD indexer if available, otherwise falls back
   * to the viem publicClient to make RPC calls to fetch MUD
   * events from the chain.
   */
  const { tables, useStore, latestBlock$, storedBlockLogs$, waitForTransaction } = await syncToZustand({
    initialBlockLogs,
    filters: liveSyncFilters,
    config: mudConfig,
    address: networkConfig.worldAddress as Hex,
    publicClient,
    startBlock: BigInt(networkConfig.initialBlockNumber),
  });
 
  return {
    tables,
    useStore,
    publicClient,
    walletClient: burnerWalletClient,
    latestBlock$,
    storedBlockLogs$,
    waitForTransaction,
    worldContract,
    write$: write$.asObservable().pipe(share()),
  };
}

Explanation

import { DozerSyncFilter, getSnapshot, selectFrom } from "@latticexyz/store-sync/dozer";

Import the dozer definitions we need.

const dozerUrl = "https://dozer.mud.garnetchain.com/q";

The URL for the dozer service. This is simplified testing code, on a production system this will probably be a lookup table based on the chainId.

const yesterday = Date.now() / 1000 - 24 * 60 * 60;

In JavaScript (and therefore TypeScript), time is stored as milliseconds since the beginning of the epoch (opens in a new tab). In UNIX, and therefore in Ethereum, time is stored as seconds since that same point. This is the timestamp 24 hours ago.

  const filters: DozerSyncFilter[] = [

We create the dozer filter for the tables we're interested in. This is the dozer filter, so it is only used for the initial hydration of the client.

    selectFrom({
      table: mudConfig.tables.app__Tasks,
      where: `"createdAt" > ${yesterday}`,
    }),

From the app__Tasks table we only want entries created in the last 24 hours. To verify that the filter works as expected you can later change the code to only look for entries older than 24 hours.

    { table: mudConfig.tables.app__Creator },
  ];

We also want the entire app__Counter table.

const { initialBlockLogs } = await getSnapshot({
  dozerUrl,
  storeAddress: networkConfig.worldAddress as Hex,
  filters,
  chainId: networkConfig.chainId,
});

Get the initial snapshot to hydrate (fill with initial information) the data store. Note that this snapshot does not have the actual data, but the events that created it.

const liveSyncFilters = filters.map((filter) => ({
  tableId: filter.table.tableId,
}));

The live synchronization filters are used after the initial hydration, and keep up with changes on the blockchain. These synchronization filters are a lot more limited, you can read the description of these filters here.

  const { ... } = await syncToZustand({
    initialBlockLogs,
    filters: liveSyncFilters,
      ...
  });

Finally, we provide initialBlockLogs for the hydration and filters for the updates to the synchronization function (either syncToRecs or syncToZustand).

Filter data synchronization Add a system

Add dozer hydration

Create a client to access the World

Updating the client to use dozer

Create a client to access the `World`