Benchmarks

Go

The overhead of policy enforcement has been benchmarked in model_b_test.go. The testbed configuration is as follows:

Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz, 2601 Mhz, 4 Core(s), 8 Logical Processor(s)

Following are the benchmarking results obtained by running go test -bench=. -benchmem (op = an Enforce() call, ms = millisecond, KB = kilobytes):

Test case	Rule size	Time overhead (ms/op)	Memory overhead (KB)
ACL	2 rules (2 users)	0.015493	5.649
RBAC	5 rules (2 users, 1 role)	0.021738	7.522
RBAC (small)	1100 rules (1000 users, 100 roles)	0.164309	80.620
RBAC (medium)	11000 rules (10000 users, 1000 roles)	2.258262	765.152
RBAC (large)	110000 rules (100000 users, 10000 roles)	23.916776	7,606
RBAC with resource roles	6 rules (2 users, 2 roles)	0.021146	7.906
RBAC with domains/tenants	6 rules (2 users, 1 role, 2 domains)	0.032696	10.755
ABAC	0 rule (0 user)	0.007510	2.328
RESTful	5 rules (3 users)	0.045398	91.774
Deny-override	6 rules (2 users, 1 role)	0.023281	8.370
Priority	9 rules (2 users, 2 roles)	0.016389	5.313

Python

The overhead of policy enforcement in Pycasbin has been benchmarked in the tests/benchmarks directory. The testbed configuration is as follows:

Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz (Runned by Github actions)
platform linux -- Python 3.11.4, pytest-7.0.1, pluggy-1.2.0

Here are the benchmarking results obtained from executing casbin_benchmark (op = an enforce() call, ms = millisecond):

Test case	Rule size	Time overhead (ms/op)
ACL	2 rules (2 users)	0.067691
RBAC	5 rules (2 users, 1 role)	0.080045
RBAC (small)	1100 rules (1000 users, 100 roles)	0.853590
RBAC (medium)	11000 rules (10000 users, 1000 roles)	6.986668
RBAC (large)	110000 rules (100000 users, 10000 roles)	77.922851
RBAC with resource roles	6 rules (2 users, 2 roles)	0.106090
RBAC with domains/tenants	6 rules (2 users, 1 role, 2 domains)	0.103628
ABAC	0 rule (0 user)	0.053213
RESTful	5 rules (3 users)	NA
Deny-override	6 rules (2 users, 1 role)	NA
Priority	9 rules (2 users, 2 roles)	0.084684

C++

The overhead of policy enforcement in Casbin CPP has been benchmarked in the tests/benchmarks directory using Google's benchmarking tool. The testbed configuration is as follows:

Intel(R) Core(TM) i5-6300HQ CPU @ 2.30GHz, 4 cores, 4 threads

Here are the benchmarking results obtained from executing the casbin_benchmark target built in the Release configuration (op = an enforce() call, ms = millisecond):

Test case	Rule size	Time overhead (ms/op)
ACL	2 rules (2 users)	0.0195
RBAC	5 rules (2 users, 1 role)	0.0288
RBAC (small)	1100 rules (1000 users, 100 roles)	0.300
RBAC (medium)	11000 rules (10000 users, 1000 roles)	2.113
RBAC (large)	110000 rules (100000 users, 10000 roles)	21.450
RBAC with resource roles	6 rules (2 users, 2 roles)	0.03
RBAC with domains/tenants	6 rules (2 users, 1 role, 2 domains)	0.041
ABAC	0 rule (0 user)	NA
RESTful	5 rules (3 users)	NA
Deny-override	6 rules (2 users, 1 role)	0.0246
Priority	9 rules (2 users, 2 roles)	0.035

Lua (JIT)

The overhead of policy enforcement in Lua Casbin has been benchmarked in bench.lua. The testbed configuration is as follows:

AMD Ryzen(TM) 5 4600H CPU @ 3.0GHz, 6 Cores, 12 Threads

Here are the benchmarking results obtained by running luajit bench.lua (op = an enforce() call, ms = millisecond):

Test case	Rule size	Time overhead (ms/op)
ACL	2 rules (2 users)	0.0533
RBAC	5 rules (2 users, 1 role)	0.0972
RBAC (small)	1100 rules (1000 users, 100 roles)	0.8598
RBAC (medium)	11000 rules (10000 users, 1000 roles)	8.6848
RBAC (large)	110000 rules (100000 users, 10000 roles)	90.3217
RBAC with resource roles	6 rules (2 users, 2 roles)	0.1124
RBAC with domains/tenants	6 rules (2 users, 1 role, 2 domains)	0.1978
ABAC	0 rule (0 user)	0.0305
RESTful	5 rules (3 users)	0.1085
Deny-override	6 rules (2 users, 1 role)	0.1934
Priority	9 rules (2 users, 2 roles)	0.1437

Benchmark monitoring

In the embedded web page below, you can see the performance changes of Casbin for each commit.

You can also directly browse it at: https://v1.casbin.org/casbin/benchmark-monitoring