Nodeos Replay

Nodeos provides various options for replaying blockchain blocks. This can be useful if, for example, a node has downloaded a blocks.log file from the internet (as a faster alternative to synchronizing from the p2p network) and the node wants to use it to quickly catch up with the network, or if you want to know the chain state at specific points in a blockchain's life.

Replaying data can be done in two ways:

Snapshot and History archive repositories;

Replay from a snapshot archive

The snapshot contains the chain state for the block referenced block when the snapshot is created. It is recommended to use snapshot files created from blocks that are irreversible. Using a snapshot file to replay allows you to quickly start a nodeos instance which has a full and correct chain state at a specified block number. While you may configure history when using a snapshot you will only have historical data from the referenced block in the snapshot forward and not any before that block.

  1. Remove the existing contents of the data, history, and history_index directories. Make sure you back up any existing contents if you wish to keep them.
cd /var/lib
sudo rm -fr fio/data fio/history fio/history_index
  1. Obtain a copy of the snapshot archive. Several BPs maintain FIO snapshots for both Testnet and Mainnet. It is important that you trust the source of the snapshot before downloading. We will use the Blockpane history archive in our examples.
wget https://snap.blockpane.com/mainnet-latest-snapshot.txz
  1. Extract the snapshot to your data/snapshots directory, and update the permissions on fio directory. The tar extraction will take several minutes.
sudo apt install pixz
sudo tar -xS -I'pixz' -C /var/lib/fio -f mainnet-latest-snapshot.txz
rm -f mainnet-latest-snapshot.txz
cd /var/lib
sudo chown -R fio:fio fio

Replay from a blocks archive

The blocks archive, denoted below, contains both the snapshot file and the blocks.log file. The blocks.log file contains all irreversible transactions on the blockchain up to the point it is created, just as the snapshot contains the state of the blockchain at the point it is created. All instances of nodeos write irreversible blocks to the blocks.log file, which is located at the data/blocks directory relative to the nodeos directory. Using a blocks.log file to replay will allow you to start a nodeos instance, which recreates the entire transaction history of the blockchain locally, without adding unnecessary load to the network.

When replaying from a snapshot and a blocks.log file, the blocks.log file must at least contain blocks up to the referenced block in the snapshot but may contain additional blocks that will be applied as part of startup. If a blocks.log file is included but does not contain blocks up to and/or after the snapshotted block then replaying from a snapshot will create an exception.

  1. Remove the existing contents of the data, history, and history_index directories. Make sure you back up any existing contents if you wish to keep them.
cd /var/lib
sudo rm -fr fio/data fio/history fio/history_index
  1. Obtain a copy of the snapshot and blocks.log files which you wish to replay the blockchain from. Several BPs maintain FIO snapshots for both Testnet and Mainnet. It is important that you trust the source of the snapshot before downloading. We will use the Blockpane history archive in our examples.
wget https://snap.blockpane.com/mainnet-latest-blocks.txz
  1. Extract the snapshot to your data/snapshots directory, your blocks.log file to your data/blocks directory, and update the permissions on fio directory. The tar extraction will take several minutes.
sudo apt install pixz
sudo tar -xS -I'pixz' -C /var/lib/fio -f mainnet-latest-blocks.txz
rm -f mainnet-latest-blocks.txz
cd /var/lib
sudo chown -R fio:fio fio

This will install the following files:

DirectoryFile
fio/data/blocksblocks.log
fio/data/blocksblocks.index
fio/data/blocks/reversibleshared_memory.bin
fio/data/snapshotssnapshot-abc.bin
  1. Enable nodeos daemon at runtime and start nodeos:
sudo systemctl enable fio-nodeos
sudo systemctl start fio-nodeos

Sync status will show in the logs:

tail -f /var/log/fio/nodeos.log
  1. Validate your API node

Replay from a V1 History archive

It is not possible to rebuild state from a snapshot for a V1 History node. But, you can bootstrap a history node by downloading a complete tar file of a working FIO node. The following steps describe how to replay from a V1 history node archive.

  1. Remove the existing contents of the data, history, and history_index directories. Make sure you back up any existing contents if you wish to keep them.
cd /var/lib
sudo rm -fr fio/data fio/history fio/history_index
  1. Otain a copy of the snapshot and blocks.log files which you wish to replay the blockchain from. Several BPs maintain FIO snapshots for both Testnet and Mainnet. It is important that you trust the source of the snapshot before downloading. We will use the Blockpane history archive in our examples.
wget https://snap.blockpane.com/mainnet-latest-history.txz
  1. Extract the snapshot to your data/snapshots directory, your blocks.log file to your data/blocks directory, and update the permissions on fio directory. The tar extraction will take several minutes.
sudo apt install pixz
sudo tar -xS -I'pixz' -C /var/lib/fio -f mainnet-latest-history.txz
rm -f mainnet-latest-history.txz
cd /var/lib
sudo chown -R fio:fio fio

This will install the following files:

DirectoryFile
fio/data/blocksblocks.log
fio/data/blocksblocks.index
fio/data/blocks/reversibleshared_memory.bin
fio/data/snapshotssnapshot-*.bin
fio/data/statefork_db.dat
fio/data/stateshared_memory.bin
fio/historyshared_memory.bin
fio/history_indexshared_memory.bin
  1. Confirm the v1 history node settings in config.ini

  2. Enable nodeos daemon at runtime and start nodeos:

sudo systemctl enable fio-nodeos
sudo systemctl start fio-nodeos

Sync status will show in the logs:

tail -f /var/log/fio/nodeos.log
  1. Validate your API node

Replay with State History

As with the Light/V1 history node, it is not possible to rebuild state from a snapshot for a state history node, however, it is possible to bootstrap a state history node. The following steps describe how to replay from a blocks archive.

  1. Remove the existing contents of the data, history, and history_index directories. Make sure you back up any existing contents if you wish to keep them.
cd /var/lib
sudo rm -fr fio/data fio/history fio/history_index
  1. Otain a copy of blocks archive which you wish to replay the blockchain from. At this time two BPs maintain FIO blocks archives for MainNet and one BP maintains a blocks archive for TestNet. It is important that you trust the source of the snapshot before downloading. We will use the Blockpane history archive in our examples.
wget https://snap.blockpane.com/mainnet-latest-blocks.txz
  1. Extract the snapshot to your data/snapshots directory, your blocks.log file to your data/blocks directory, and update the permissions on fio directory. The tar extraction will take several minutes.
sudo apt install pixz
sudo tar -xS -I'pixz' -C /var/lib/fio -f mainnet-latest-blocks.txz
rm -f mainnet-latest-history.txz
cd /var/lib
sudo chown -R fio:fio fio

This will install the following files:

DirectoryFile
fio/data/blocksblocks.log
fio/data/blocksblocks.index
fio/data/blocks/reversibleshared_memory.bin
fio/data/snapshotssnapshot-*.bin
fio/data/statefork_db.dat
fio/data/stateshared_memory.bin
fio/historyshared_memory.bin
fio/history_indexshared_memory.bin

🚧

Warning

Several files in the above list must be deleted to properly start and maintain a State History node.

sudo rm -f /var/lib/fio/data/blocks/blocks.index
sudo rm -f /var/lib/fio/data/blocks/reversible/*
sudo rm -f /var/lib/fio/data/snapshots/*
sudo rm -f /var/lib/fio/data/state/*
sudo rm -f /var/lib/fio/history/*
sudo rm -f /var/lib/fio/history_index/*
  1. Confirm the state history node settings in config.ini

  2. Edit the parameters that fio-nodeos is started with by adding --disable-replay-opts

vi /usr/local/bin/fio-nodeos-run
#exec /usr/local/bin/fio-nodeos --data-dir /var/lib/fio/data --config-dir /etc/fio/nodeos --genesis-json /etc/fio/nodeos/genesis.json
exec /usr/local/bin/fio-nodeos --data-dir /var/lib/fio/data --config-dir /etc/fio/nodeos --genesis-json /etc/fio/nodeos/genesis.json --disable-replay-opts
  1. Enable nodeos daemon at runtime and start nodeos:
sudo systemctl enable fio-nodeos
sudo systemctl start fio-nodeos

Sync status will show in the logs:

tail -f /var/log/fio/nodeos.log