Snagfactory benchmarks

To give an idea of how Snagfactory can perform in terms of flashing speed and parallelization of tasks, some benchmarks results are provided below. The time taken by Snagfactory to complete a factory flashing task depends on many different factors, including but not limited to:

  • Processing capabilities of the host machine
  • Current system load
  • Processing capabilities of the target boards
  • Speeds of the USB links from host to targets
  • Total amount of image data to write to each target
  • USB bus topology
  • Type of target storage used

The following benchmarks are meant to provide some basic performance indicators for various types of storage devices.

Measurement protocol:

Host machine CPU: Intel i5-13500

Host machine memory: 8.00 GB

Operating system: Windows 11

Snagboot version: 2.1 with a minor patch to raise loglevels to "debug" and increase snagflash USB transfer timeouts to 8 minutes

  1. Snagfactory is used to flash a randomly generated 64MB image to the chosen storage device on the target.

  2. Log timestamps are read from the standard logfiles generated by Snagfactory, and the following durations are computed:

  3. Total Snagfactory runtime: from Start to BoardPhase.FLASHER -> BoardPhase.DONE

  4. Time spent running snagrecover: from Installing firmware tiboot3 to Done installing firmware u-boot
  5. Time spent running snagflash: from Fastboot object to last occurrence of [DEBUG ] fastboot OKAY
  6. Time spent erasing flash blocks (for SPI-NAND and SPI-NOR only): from last occurrence of erasing flash area to following occurence of fastboot OKAY
  7. Time spent transmitting image data over USB: from flashing file random.img to following occurrence of fastboot OKAY

  8. Time spent writing data to storage medium: from preceding log line to following occurence of fastboot OKAY

  9. Throughput values are then derived in the following manner:

  10. transmission throughput: size of flashed image / time spent transmitting over USB

  11. write throughput: size of flashed image / (time spent writing to storage + time spent erasing storage)

For each type of storage, this measurement protocol was performed three times with different boards of the same model.

A series of additional total runtime measurements were also performed for the SPI-NAND case with increasing numbers of boards, to showcase Snagfactory's ability to parallelize tasks.

eMMC

Board: AM62x SKEVM

Storage: Micron MTFC16GAPALBH-IT 16GB eMMC

total runtime (s) snagrecover runtime (s) snagflash runtime (s)
18 9 4
18 9 4
18 9 4

NB: The discrepancy between total Snagfactory runtime, and the sum of the other two runtimes is mostly due to the USB reenumeration delay after U-Boot proper is launched on the target

transmission throughput (MB/s) write throughput (MB/s)
24.701 56.288
24.559 57.971
24.578 56.437

SPI-NAND

Board: AM62x Low-power SKEVM

Storage: Winbond W35N01JWTBAG 128MB OSPI-NAND

total runtime (s) snagrecover runtime (s) snagflash runtime (s)
25 10 11
25 9 11
24 9 10
transmission throughput (MB/s) write throughput, including erase cycles (MB/s)
24.568 8.854
24.530 8.443
24.682 9.100

OSPI-NOR

Board: AM62x SKEVM

Storage: Infineon S28HS512TGABHM010 64MB OSPI-NOR

total runtime (s) snagrecover runtime (s) snagflash runtime (s)
426 9 411
401 9 387
390 9 375
transmission throughput (MB/s) write throughput, including erase cycles (MB/s)
25.257 0.157
24.625 0.167
24.806 0.172

Parallelization

Board: AM62x Low-power SKEVM

Storage: Winbond W35N01JWTBAG 128MB OSPI-NAND

number of boards total runtime (s)
1 25
3 25
4 26
7 27
8 27

NB: these results were obtained with "native" USB ports.