Skip to main content

Overview

The Broker is a service that runs within the Bento proving stack. It is responsible for market interactions including bidding on jobs, locking them, issuing job requests to the Bento proving cluster, and submitting proof fulfillments onchain.

Broker Configuration

Broker will live-reload the broker.toml when it changes. In most cases, you will not need to restart the Broker for the configuration to take effect.
Broker configuration is primarily managed through the broker.toml file in the Boundless directory. This file is mounted into the Broker container and it is used to configure the Broker daemon.

Deposit / Balance

The Boundless market requires funds (ZKC) deposited as collateral before a prover can bid on requests. Brokers must first deposit some ZKC into the market contract to fund their account. These funds cover collateral during lock-in. It is recommend that a broker keep a balance on the market >= max_collateral (configured via broker.toml).

Deposit Collateral to the Market

You will need the Boundless CLI installed to deposit/check your balance. Please see Installing the Boundless CLI for instructions.
Terminal
export RPC_URL=<TARGET_CHAIN_RPC_URL>
export PRIVATE_KEY=<BROKER_PRIVATE_KEY>

# Example: 'prover deposit-collateral 100'
boundless prover deposit-collateral <ZKC_TO_DEPOSIT>

Check Current Collateral Balance

Terminal
export RPC_URL=<TARGET_CHAIN_RPC_URL>
export PRIVATE_KEY=<BROKER_PRIVATE_KEY>

boundless prover balance-collateral [wallet_address]
You can omit the PRIVATE_KEY environment variable here and specify your wallet_address as a optional parameter to the balance command, i.e., account balance 0x000....

Settings in Broker.toml

Quotation marks matter in TOML so please pay particular attention to the quotation marks for config values.
Below are all broker.toml settings organized by section:

[market] Settings

SettingDescription
min_mcycle_priceThe minimum price per mega-cycle (i.e. million RISC-V cycles) for the broker to attempt to lock an order. Accepts "<value> ETH" (e.g., "0.00002 ETH") or "<value> USD" (e.g., "0.02 USD"); USD values are converted to ETH at runtime via the price oracle. Plain numbers without a suffix still work and default to ETH for backward compatibility. This value is used for both primary orders (compared against the order’s ETH price per mcycle) and secondary fulfillment orders (converted to ZKC via price oracle for collateral-based comparisons).
expected_probability_win_secondary_fulfillmentExpected probability (as a percentage) of winning a secondary fulfillment proof race. When another prover fails to fulfill an order and is slashed, multiple provers may race to fulfill the order for the collateral reward. This setting scales the expected reward of those lock-expired orders by the configured percentage before comparing against min_mcycle_price. Default: 50. Values below 100 discount the reward (conservative — fewer secondary orders picked up), 100 means no discount, values above 100 boost the reward (aggressive — more secondary orders picked up).
priority_requestor_addressesOptional priority requestor addresses that can bypass the mcycle limit and max input size limit. If enabled, the order will be preflighted without constraints.
priority_requestor_listsURLs to fetch requestor priority lists from. Requestor priority lists specify requestors that the broker should prioritize for proving. Requestors on these lists are considered more likely to request useful work with profitable pricing. Priority requestors are prioritized when there is a surplus of demand: their requests will be preflighted first, locked first (in situations where multiple orders exceed minimum lock pricing), and configuration for max_mcycle_limit and max_file_size will be skipped. These lists will be periodically refreshed and merged with priority_requestor_addresses.
peak_prove_khzEstimated peak performance of the proving cluster, in kHz. Used to estimate proving capacity and accept only as much work as your prover cluster can handle. Estimates can be derived from benchmarking using the Boundless CLI. See Benchmarking Bento.
max_mcycle_limitOptional max cycles (in mcycles). Orders over this max_cycles will be skipped after preflight.
max_journal_bytesMax journal size in bytes. Orders that produce a journal larger than this size in preflight will be skipped. Since journals must be posted onchain to complete an order, an excessively large journal may prevent completion of a request.
min_deadlineMin seconds left before the deadline to consider bidding on a request. If there is not enough time left before the deadline, the prover may not be able to complete proving of the request and finalize the batch for publishing before expiration.
lookback_blocksOn startup, the number of blocks to look back for possible open orders.
max_collateralMaximum collateral amount that the broker will use to lock orders. Accepts "<value> ZKC" (e.g., "200 ZKC") or "<value> USD" (e.g., "100 USD"); USD values are converted to ZKC at runtime via the price oracle. Requests that require a higher collateral amount than this will be skipped.
max_file_sizeMax input / image file size allowed for downloading from request URLs.
max_fetch_retriesMax retries for fetching input / image contents from URLs.
assessor_default_image_urlDefault URL for assessor image. This URL will be tried first before falling back to the image URL from the boundless market contract.
set_builder_default_image_urlDefault URL for set builder image. This URL will be tried first before falling back to the image URL from the set verifier contract.
max_concurrent_proofsMaximum number of concurrent proofs that can be processed at once. Used to limit proof tasks spawned to prevent overwhelming the system.
max_concurrent_preflightsMaximum number of orders to concurrently preflight. Used to limit preflight tasks spawned to prevent overwhelming the system. Recommended default: 8 to be able to follow all market orders.
order_pricing_priorityDetermines how orders are prioritized for pricing. Options: “random” (default, process orders in random order), “observation_time” (prioritize orders in the order they were observed, FIFO), “shortest_expiry” (prioritize orders with the earliest deadline first).
order_commitment_priorityDetermines how orders are prioritized when committing to prove them. Options: “cycle_price” (default, prioritize orders with the highest ETH price per cycle), “random” (process orders in random order), “shortest_expiry” (prioritize orders with the earliest deadline first), “tightest_deadline” (prioritize orders whose deadline is closest, giving preference to the most time-urgent work), “price” (prioritize orders with the highest ETH payment regardless of cycle count).
max_critical_task_retriesMax critical task retries on recoverable failures. The broker service has a number of subtasks. Some are considered critical. If a task fails, it will be retried, but after this number of retries, the process will exit.
allow_client_addressesOptional allow list for customer address. If enabled, all requests from clients not in the allow list are skipped.
deny_requestor_addressesOptional deny list for requestor address. If enabled, all requests from clients in the deny list are skipped.
lockin_priority_gasOptional additional gas to add to the transaction for lockinRequest, good for increasing the priority if competing with multiple provers during the same block.
balance_warn_thresholdOptional balance warning threshold (in native token). If the submitter balance drops below this the broker will issue warning logs.
balance_error_thresholdOptional balance error threshold (in native token). If the submitter balance drops below this the broker will issue error logs.
collateral_balance_warn_thresholdOptional collateral balance warning threshold (in collateral tokens). If the collateral balance drops below this the broker will issue warning logs.
collateral_balance_error_thresholdOptional collateral balance error threshold (in collateral tokens). If the collateral balance drops below this the broker will issue error logs.
cache_dirOptional cache directory for storing downloaded images and inputs. If not set, files will be re-downloaded every time.
lockin_gas_estimateGas estimate for lockin call. Used for estimating the gas costs associated with an order during pricing. If not set a conservative default will be used.
fulfill_gas_estimateGas estimate for fulfill call. Used for estimating the gas costs associated with an order during pricing. If not set a conservative default will be used.
groth16_verify_gas_estimateGas estimate for proof verification using the RiscZeroGroth16Verifier. Used for estimating the gas costs associated with an order during pricing. If not set a conservative default will be used.
max_order_expiry_secsMaximum order expiry duration in seconds (current time to order expiry time). Orders exceeding this duration are skipped. Prevents orders with long deadlines from tying up resources or storing inputs beyond MinIO’s TTL.
min_mcycle_price_overridesOptional per-requestor and per-selector overrides for min_mcycle_price. Allows fine-grained pricing control so you can set different minimum prices for specific requestors or program selectors. See example below.
telemetry_modeControls broker telemetry reporting. "full" (default) sends anonymized health and performance metrics to help the Boundless team diagnose issues across the network. No sensitive data (private keys, wallet balances) is collected. Set to "logsonly" to disable remote reporting and only emit local debug logs.

[prover] Settings

SettingDescription
status_poll_retry_countNumber of retries to poll for proving status. Provides a little durability for transient failures.
status_poll_msPolling interval to monitor proving status (in millisecs).
req_retry_countNumber of retries to query a prover backend for on failures. Used for API requests to a prover backend, creating sessions, preflighting, uploading images, etc. Provides a little durability for transient failures.
req_retry_sleep_msNumber of milliseconds to sleep between retries.
proof_retry_countNumber of retries for running the entire proof generation process. This is separate from the request retry count, as the proving process is a multi-step process involving multiple API calls to create a proof job and then polling for the proof job to complete.
proof_retry_sleep_msNumber of milliseconds to sleep between proof retries.
set_builder_guest_pathSet builder guest program (ELF) path. When using a durable deploy, set this to the published current SOT guest program path on the system.
assessor_set_guest_pathAssessor ELF path.
reaper_interval_secsInterval for checking expired committed orders (in seconds). This is the interval at which the ReaperTask will check for expired orders and mark them as failed. If not set, it defaults to 60 seconds.
reaper_grace_period_secsGrace period before marking expired orders as failed (in seconds). This provides a buffer time after an order expires before the reaper marks it as failed. This helps prevent race conditions with the aggregator that might be processing the order. If not set, it defaults to 10800 seconds (3 hours).

[batcher] Settings

SettingDescription
batch_max_timeMax batch duration before publishing (in seconds).
min_batch_sizeBatch size (in proofs) before publishing.
block_deadline_buffer_secsBatch blocktime buffer. Number of seconds before the lowest block deadline in the order batch to flush the batch. This should be approximately snark_proving_time * 2.
txn_timeoutTimeout, in seconds for transaction confirmations.
single_txn_fulfillUse the single TXN submission that batches submit_merkle / fulfill_batch into a single transaction. Requires the submitRootAndFulfill method be present on the deployed contract.
withdrawWhether to withdraw from the prover balance when fulfilling.
batch_poll_time_msPolling time, in milliseconds. The time between polls for new orders to aggregate and how often to check for batch finalize conditions.
batch_max_journal_bytesMax combined journal size (in bytes) that once exceeded will trigger a publish.
batch_max_feesMax batch fees (in ETH) before publishing.
max_submission_attemptsNumber of attempts to make to submit a batch before abandoning.

[price_oracle] Settings

The price oracle fetches live ETH/USD and ZKC/USD exchange rates, enabling broker config values to be specified in USD. Chainlink (on-chain, ETH/USD only) and CoinGecko (off-chain, both pairs) are enabled by default; CoinMarketCap is optionally available with an API key. Prices from multiple sources are aggregated using the configured aggregation_mode.
SettingDescription
eth_usdPrice source for ETH/USD. "auto" (default) uses live oracle prices. Set to a static number (e.g., "2500.00") to fix the price and disable live fetching.
zkc_usdPrice source for ZKC/USD. "auto" (default) uses live oracle prices. Set to a static number (e.g., "1.00") to fix the price and disable live fetching.
refresh_interval_secsHow often to refresh prices from sources, in seconds (default: 60).
aggregation_modeHow to combine prices from multiple sources: "median" (default), "priority" (use first available source), or "average".
max_secs_without_price_updateMaximum seconds without a successful price update before the broker logs an error (default: 43200 / 12h). Set to 0 to disable this check.
Example configuration with USD-denominated pricing:
[market]
min_mcycle_price = "0.02 USD"
max_collateral = "100 USD"
expected_probability_win_secondary_fulfillment = 50

[price_oracle]
eth_usd = "auto"
zkc_usd = "auto"
Example per-requestor and per-selector pricing overrides:
[market]
min_mcycle_price = "0.02 USD"

# Override min_mcycle_price for a specific requestor address
[[market.min_mcycle_price_overrides]]
requestor = "0x1234...abcd"
min_mcycle_price = "0.01 USD"

# Override min_mcycle_price for a specific program selector
[[market.min_mcycle_price_overrides]]
selector = "aabbccdd"
min_mcycle_price = "0.03 USD"

Experimental RPC Optimizations

This feature was introduced in PR #1715 and is experimental. Behavior and flags may change in future releases.
The --experimental-rpc flag replaces the standard two-service monitoring architecture (ChainMonitorService + MarketMonitor) with a single unified ChainMonitorV2. Instead of polling eth_getLogs for market events on every tick, the ChainMonitorV2 uses eth_getBlockReceipts to fetch all receipts per block in a single call, then filters for market events locally. This significantly reduces the number of RPC calls — especially on chains like Base where eth_getLogs can be expensive or rate-limited. In steady state, the experimental monitor requires roughly 2 RPC requests per block:
  • eth_getBlockByNumber to follow the chain head and get the base fee
  • eth_getBlockReceipts to get all receipts and transaction fee data
The ChainMonitorV2 also performs local EIP-1559 gas estimation from receipt data, removing the need for separate eth_feeHistory calls. On startup, the ChainMonitorV2 uses adaptive log retrieval with binary-search chunking to auto-discover the maximum block range accepted by the RPC provider, catching up on any missed events for open orders. This means the broker won’t miss events if it restarts or experiences downtime.

Enabling Experimental RPC

To enable, set the BROKER_EXTRA_ARGS environment variable before starting the broker:
Terminal
export BROKER_EXTRA_ARGS="--experimental-rpc"
just broker
You can also configure the RPC request timeout (default 15s) to cut off hanging requests so the retry and fallback layers can activate:
Terminal
export BROKER_EXTRA_ARGS="--experimental-rpc --rpc-request-timeout 15"
just broker
If no RPC URLs are provided, the experimental mode defaults to https://base.gateway.tenderly.co.

Sequential Fallback Transport

The experimental RPC mode introduces a SequentialFallbackTransport that tries RPC providers in priority order (rather than in parallel), minimizing calls to paid or metered fallback endpoints. When multiple RPC URLs are configured, the transport includes health tracking: it skips providers after consecutive failures and periodically retries them to detect recovery. The retry and fallback layers are stacked so that on a single RPC failure, the sequential fallback immediately tries the next URL. The outer retry layer only kicks in once all URLs have been exhausted.

Analyzing RPC Usage

To analyze RPC call patterns, enable debug logging for the relevant modules and pipe the output to a log file. An analysis script is available to parse these logs:
Terminal
RISC0_DEV_MODE=1 \
  RUST_LOG=INFO,broker::chain_monitor_v2=debug,broker::rpcmetrics=debug \
  BROKER_EXTRA_ARGS="--listen-only --experimental-rpc" \
  just broker 2>&1 | tee rpc_metrics.log
This feature is experimental. The eth_getBlockReceipts RPC method may not be supported by all RPC providers. While it is possible to run with a free public RPC, these can be unreliable — a dedicated RPC provider is recommended for production use.

Broker Operation

Terminal
2024-10-23T14:37:37.364844Z  INFO bento_cli: image_id: a0dfc25e54ebde808e4fd8c34b6549bbb91b4928edeea90ceb7d1d8e7e9096c7 | input_id: eccc8f06-488a-426c-ae3d-e5acada9ae22
2024-10-23T14:37:37.368613Z  INFO bento_cli: STARK job_id: 0d89e2ca-a1e3-478f-b89d-8ab23b89f51e
2024-10-23T14:37:37.369346Z  INFO bento_cli: STARK Job running....
2024-10-23T14:37:39.371331Z  INFO bento_cli: STARK Job running....
2024-10-23T14:37:41.373508Z  INFO bento_cli: STARK Job running....
2024-10-23T14:37:43.375780Z  INFO bento_cli: Job done!

Benchmarking Bento

Start a bento cluster:
Terminal
just bento
Set the RPC_URL environment variable to the network the order is on:
Terminal
export RPC_URL=<TARGET_CHAIN_RPC_URL>
Then, run the benchmark:
Terminal
boundless prover benchmark --request-ids <IDS>
where IDS is a comma-separated list of request IDs from the network or order stream configured. It is recommended to pick a few requests of varying sizes and programs, biased towards larger proofs for a more representative benchmark. To run programs manually, and for performance optimizations, see performance optimizations.

Running the Broker service with bento

Running a broker with just will also start the Bento cluster through docker compose.
just installation instructions can be found here.
Terminal
just broker

Make sure Bento is running

A Broker needs a Bento instance to operate. Please follow the quick start guide to get Bento up and running.
To check Bento is running correctly, you can send a sample proof workload:
Before running this, install Bento CLI
Terminal
# In the bento directory
RUST_LOG=info bento_cli -c 32

Running a standalone broker

To run broker with an already initialized Bento cluster or with a different prover, you can build and run a broker directly with the following:
Terminal
cargo build --bin broker --release
# Run with flags or environment variables based on network/configuration
./target/release/broker

Stopping The Broker Service

Terminal
just broker down
If running the broker on a network, there may be locked proofs that have not been fulfilled yet. Follow the Safe Upgrade Steps to ensure shutdown and/or restart without loss of stake.

Safe Upgrade Steps

There can be subtle breaking changes between releases that may affect your broker’s state. Following these upgrade steps helps minimize issues from state breaking changes.
When upgrading your Boundless broker to a new version, follow these steps to ensure a safe migration:
1

Stop the broker and optionally clean the database

Terminal
just broker clean

# Or stop the broker without clearing volumes
just broker down
This will wait for any committed orders to finalize before shutting down. Avoid sending kill signals to the broker process and ensure either through the broker logs or through indexer that your broker does not have any incomplete locked orders before proceeding.
While it is generally not necessary to clear volumes unless specifically noted in release, it is recommended to avoid any potential state breaking changes.
2

Update to the new version

See releases for latest tag to use.
Terminal
git checkout <new_version_tag>
# Example: git checkout v0.9.0
3

Start the broker with the new version

Terminal
just broker

Running Multiple Brokers

You can run multiple broker instances simultaneously to serve different networks at the same time while sharing the same Bento cluster. The Docker compose setup supports this through the broker2 service example.

Multi-Broker Configuration

Each broker instance requires:
  1. Separate configuration file: Create different broker.toml files (e.g., broker.toml, broker2.toml, etc.)
  2. Different RPC URL: Use different chain endpoints via setting respective RPC_URL environment variables, or modifying the compose.yml manually.
  3. Optional separate private key: Use different PRIVATE_KEY variables if desired for different accounts on different networks.

Environment Variables for Multi-Broker Setup

If using the default compose.yml file and uncommenting the second broker config:
[.env]
# Export environment variables for the first broker
export RPC_URL=<URL for network 1>
export PRIVATE_KEY=0x...

# Export environment variables for the second broker
export RPC_URL_2=<URL for network 2>
Then, create the new broker config file that the second broker will use:
Terminal
# Copy from an existing broker config file
cp broker.toml broker2.toml

# Or creating one from a fresh template
cp broker-template.toml broker2.toml
Then, modify configuration values for each network, keeping the following in mind:
  • The peak_prove_khz setting is shared across all brokers
    • For example, if you have benchmarked your broker to be able to prove at 500kHz, the values in each config should not sum up to be more than 500kHz.
  • max_concurrent_preflights defaults to 8 and should be set to a value that the bento cluster can keep up with
    • It is recommended that the max concurrent preflights across all networks is less than the number of exec agents you have specified in your compose.yml.
  • max_concurrent_proofs is a per-broker configuration, and is not shared across brokers
Then, just start the cluster as you normally would with:
Terminal
just broker

Broker Optimization

Increasing Lock-in Rate

Once your broker is running, there are a few methods to optimize the lock-in rate. These methods are aimed at making your broker service more competitive in the market through different means:
  1. Decreasing the min_mcycle_price would tune your Broker to bid at lower prices for proofs. This value can be specified in ETH (e.g., "0.00001 ETH") or USD (e.g., "0.02 USD").
  2. Increasing lockin_priority_gas expedites your market operations by consuming more gas which could help outrun other bidders.

Tuning Service Settings

The [prover] settings in broker.toml are used to configure the prover service and significantly impact the operation of the service. The most important configuration variable to monitor and iteratively tune is txn_timeout. This is the number of seconds to wait for a transaction to be confirmed before timing out. Therefore, if you see timeouts in your logs, txn_timeout can be increased to wait longer for transaction confirmations onchain.