Testing and measuring from C#
An assembly library is tested and measured like any .NET code: with an xunit project and a
BenchmarkDotNet project that call it through its C# API. This page shows both for
Crc32C: samples/Crc32C.Tests
and samples/Crc32C.Bench.
The tests
The test project references the C# wrapper, which references the .asmproj, so building the
tests builds the assembly too, for each of its seven RIDs, under runtimes/<rid>/native/ next
to the test assembly. The same test output runs on any of those platforms, with that
platform's library.
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<OutputType>Exe</OutputType>
<IsTestProject>true</IsTestProject>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="xunit.v3" Version="4.0.3" />
<ProjectReference Include="../Crc32C.Net/Crc32C.Net.csproj" />
</ItemGroup>
Each path is tested on its own. A theory runs once per path the CPU supports, through the wrapper's per-path overload, so the table, the single chain and the 3-way kernel are all checked on every machine that can run them, not only the one the dispatcher picks:
public static TheoryData<Crc32CPath> Paths
{
get
{
var paths = new TheoryData<Crc32CPath>();
foreach (Crc32CPath path in Enum.GetValues<Crc32CPath>().Where(Crc32C.IsSupported))
{
paths.Add(path);
}
return paths;
}
}
The reference shares nothing with the code under test. It is CRC-32C's definition, one bit
at a time, in C#: no table and no crc32 instruction, so a mistake in the assembly's table,
constants or instruction use cannot cancel out.
uint register = ~crc;
foreach (byte b in data)
{
register ^= b;
for (int bit = 0; bit < 8; bit++)
{
register = (register >> 1) ^ (0x82F63B78u & (0u - (register & 1)));
}
}
return ~register;
What the 21 tests check:
- Published check values: "123456789", and the four 32-byte patterns of RFC 3720 (iSCSI), appendix B.4.
- Every length from 0 to 5,000 bytes against the reference: both block sizes of the 3-way kernel several times, and every remainder after them.
- Large inputs: 1 MiB, 1 MiB + 1 and 1 MiB + 1,537 bytes.
- Unaligned data: 4 KiB starting at each of 16 offsets, since the kernels load unaligned quadwords.
- Continuing a checksum: splitting the input anywhere, including at the block boundaries, gives the same result.
- The empty input, from 0 and from a running checksum.
- The dispatcher: the path
cpuidchose matches .NET's view of the CPU, and its results agree on both sides of the 4 KiBSuppressGCTransitionlimit. - Concurrency: 64 inputs checksummed by many threads at once.
- A bad path is refused before any native code runs.
The tests were checked against a broken build. With one merge constant's exponent changed from 8n − 33 to 8n − 32, 6 of the 21 tests fail: every 3-way test whose input reaches a 192-byte block, and the dispatcher and concurrency tests, which run on the 3-way path. The correct build passes all 21.
Running them
An xunit v3 project is an executable, so the tests run wherever the built folder can run:
# On Windows, Linux or macOS, x64 or ARM64: either one
dotnet test --project samples/Crc32C.Tests/Crc32C.Tests.csproj
dotnet samples/Crc32C.Tests/bin/Release/net10.0/Crc32C.Tests.dll
# The Linux build from Windows: run the same output folder in the .NET runtime container
docker run --rm -v <full path of samples/Crc32C.Tests/bin/Release/net10.0>:/app:ro -w /app mcr.microsoft.com/dotnet/runtime:10.0 dotnet Crc32C.Tests.dll
On ARM64 the tests check the crc32cx and PMULL paths. On every CPU they run only the paths
Crc32C.IsSupported says it has; Crc32C.Append(…, path) throws
PlatformNotSupportedException for the others, rather than letting an illegal instruction
end the process.
dotnet test needs "test": { "runner": "Microsoft.Testing.Platform" } in the global.json
next to the project, as samples/global.json has.
The benchmark
The benchmark times the library's three paths and its dispatcher against the CRC-32C a C#
developer writes first, BitOperations.Crc32C over 8-byte words, at 64 B, 1 KiB, 64 KiB and
1 MiB:
[Benchmark(Baseline = true, Description = "C# BitOperations.Crc32C")]
public uint Managed() => ManagedCrc32C(_data);
[Benchmark(Description = "asm crc32c_append (dispatcher)")]
public uint Dispatcher() => Crc32C.HashToUInt32(_data);
- A fair baseline:
BitOperations.Crc32Ccompiles to the same SSE4.2crc32instruction as the single-chain path. What the 3-way kernel adds is the three independent chains, not the instruction. - In process: the job uses BenchmarkDotNet's in-process toolchain, so it needs no SDK at run time and runs in the .NET runtime container.
- A GB/s column, from the input size and the mean time.
- It refuses to time wrong code: before timing, it checks every path against the managed
checksum at every size, and
--checkdoes only that (CI runs it).
# every benchmark, on any of the seven platforms
dotnet run -c Release --project samples/Crc32C.Bench -- --filter '*'
# only the check
dotnet run -c Release --project samples/Crc32C.Bench -- --check
Results
Three complete runs on an Intel Core i9-13900K, in the .NET runtime container on WSL2 (.NET 10.0.15, BenchmarkDotNet 0.15.8, default job), with other work running on the machine. Each figure is the median of the three runs; the results folder has the lowest and highest of each, and BenchmarkDotNet's own reports.
"C#" is the BitOperations.Crc32C loop; the other four columns are the assembly library:
crc32c_append (the dispatcher, which runs the 3-way path here) and each path called directly.
| Time | C# | dispatcher | 3-way | 1 chain | table |
|---|---|---|---|---|---|
| 64 B | 3.35 ns | 13.0 ns | 14.3 ns | 13.7 ns | 101 ns |
| 256 B | 21.2 ns | 18.2 ns | 18.3 ns | 35.0 ns | 550 ns |
| 1 KiB | 144 ns | 46.3 ns | 50.1 ns | 85.5 ns | 2.35 μs |
| 64 KiB | 11.0 μs | 2.93 μs | 2.93 μs | 8.33 μs | 154 μs |
| 1 MiB | 178 μs | 48.5 μs | 48.1 μs | 132 μs | 2.47 ms |
| GB/s | C# | dispatcher | 3-way | 1 chain | table |
|---|---|---|---|---|---|
| 1 KiB | 7.10 | 22.1 | 20.5 | 12.0 | 0.44 |
| 64 KiB | 5.95 | 22.4 | 22.4 | 7.86 | 0.43 |
| 1 MiB | 5.90 | 21.6 | 21.8 | 7.92 | 0.42 |
| One call with an empty input | Median |
|---|---|
| with the GC transition | 9.60 ns |
SuppressGCTransition |
3.38 ns |
What they show:
- Large inputs: 3.7 times the C# loop. At 1 MiB the 3-way kernel runs at 21.8 GB/s, the C# loop at 5.90 GB/s (3.69x) and the single chain at 7.92 GB/s (the kernel is 2.75x that). Three chains do not quite triple the single chain: lane 0 of each block waits for the previous block's merge.
- The single chain beats the C# loop that uses the same instruction, 1.34x. The JIT
zero-extends the
uintchecksum before every 64-bitcrc32(mov eax, eax), and that move is in the chain of dependent steps: 4 cycles per 8 bytes instead of 3, and 4/3 = 1.33. The assembly keeps the checksum inrax, whose upper halfcrc32already clears.DOTNET_JitDisasm=ManagedCrc32Cshows the JIT's loop. - Small inputs: the call costs as much as the work. An empty call takes 3.4 ns even without the GC transition. At 64 bytes the C# loop, inlined, takes 3.35 ns, and successive checksums overlap inside the CPU; the library takes 13 ns. From 256 bytes on, the assembly wins: 1.16x at 256 B, 3.1x at 1 KiB. A wrapper that sees many tiny inputs could branch to managed code below about 200 bytes; this sample always calls the library, since showing that call is its point.
SuppressGCTransitionis worth it below 4 KiB. It saves 6.2 ns on an empty call; with real work part of the transition overlaps the work, and the dispatcher (no transition up to 4 KiB) leads the per-path calls (with it) by 0 to 4 ns.- The table path, 0.42 GB/s, is what a CPU without SSE4.2 would get: about 50 times slower than the 3-way kernel. That is why the dispatcher exists.
- Noise: the hybrid CPU and the virtual machine move the benchmark between core types. The 64-byte cases varied up to 17% between runs, the 64 KiB and 1 MiB cases within 4%.