Performance recommendations

Always specify the target database

Specify the target database on all queries, both in Driver.ExecutableQuery() calls or when creating new sessions. If no database is provided, the driver has to send an extra request to the server to figure out what the default database is. The overhead is minimal for a single query, but becomes significant over hundreds of queries.

Good practices

await driver.ExecutableQuery("<QUERY>")
    .WithConfig(new QueryConfig(database: "<database-name>"))
    .ExecuteAsync();
using var session = driver.AsyncSession(conf => conf.WithDatabase("<database-name>"));

Bad practices

await driver.ExecutableQuery("<QUERY>")
    .ExecuteAsync();
using var session = driver.AsyncSession();

Be aware of the cost of transactions

When submitting queries through .ExecutableQuery() or through .ExecuteRead/WriteAsync(), the driver wraps them into a transaction. This behavior ensures that the database always ends up in a consistent state, regardless of what happens during the execution of a transaction (power outages, software crashes, etc). As a further robustness layer, the driver also retries failed transactions with an exponential backoff.

Creating a safe execution context around a query yields an overhead that is small, but that adds up as the number of transactions increases. When each query is sent as a transaction of its own, if one transaction fails and needs to be rolled back, all the other transactions are unaffected. This is the safest mode of execution with respect to failures, but also the slowest due to the overhead of transactions scaling with the number of queries.

Each query as a separate transaction (low throughput)
for (int i=0; i<1000; i++) {
    await driver.ExecutableQuery("<QUERY>")
        .WithConfig(new QueryConfig(database: "neo4j"))
        .ExecuteAsync();
    // or session.ExecuteRead/WriteAsync() calls
}

A more performant approach is to group all queries into a single transaction. In this way, the transaction as a whole is isolated from others, but individual queries in the transaction are not isolated, and failure of one results in a rollback of all queries.

Group queries into one transaction (higher throughput)
using var session = driver.AsyncSession(conf => conf.WithDatabase("neo4j"));
await session.ExecuteWriteAsync(async tx => {
    for (int i=0; i<1000; i++) {
        var result = await tx.RunAsync("<QUERY>");
    }
});

An even faster approach is to skip .ExecuteRead/WriteAsync() and call .RunAsync() directly on the session. The queries run as auto-commit transaction, and are still isolated from other concurrent queries, but if any of them fail, they will not be retried. With this method, you trade some robustness for more throughput, as the queries are shot to the server as fast as it can handle. One upper limit on the client size is given by the size of the connection pool: each call to .RunAsync() borrows a connection, so the amount of parallel work is limited by the number of available connections.

Queries as auto-commit transactions (highest throughput)
using var session = driver.AsyncSession(conf => conf.WithDatabase("neo4j"));
for (int i=0; i<1000; i++) {
    await session.RunAsync("<QUERY>");
}

Don’t fetch large result sets all at once

When submitting queries that may result in a lot of records, don’t retrieve them all at once. The Neo4j server can retrieve records in batches and stream them to the driver as they become available. Lazy-loading a result spreads out network traffic and memory usage (both client- and server-side).

For convenience, .ExecutableQuery() always retrieves all result records at once (eager loading). To lazy-load a result, use .ExecuteRead/WriteAsync() (or other forms of manually-handled transactions) and do not call .ToListAsync() on the IResultCursor object when processing the result; iterate on it instead.

Example 1. Comparison between eager and lazy loading
Eager loading Lazy loading

  • The server has to read all 250 records from the storage before it can send even the first one to the driver (i.e. it takes more time for the client to receive the first record).

  • Before any record is available to the application, the driver has to receive all 250 records.

  • The client has to hold in memory all 250 records.

  • The server reads the first record and sends it to the driver.

  • The application can process records as soon as the first record is transferred.

  • Waiting time and resource consumption for the remaining records is deferred to when the application requests more records.

  • The server’s fetch time can be used for client-side processing.

  • Resource consumption is bounded by the driver’s fetch size.
Time and memory comparison between eager and lazy loading
using Neo4j.Driver;

const string dbUri = "<database-uri>";
const string dbUser = "<username>";
const string dbPassword = "<password>";

// Returns 250 records, each with properties
// - `output` (an expensive computation, to slow down retrieval)
// - `dummyData` (a list of 10000 ints, about 8 KB).
string slowQuery = @"
UNWIND range(1, 250) AS s
RETURN reduce(s=s, x in range(1,1000000) | s + sin(toFloat(x))+cos(toFloat(x))) AS output,
       range(1, 10000) AS dummyData
";
// Delay for each processed record (ms)
int sleepTime = 500;

await using var driver = GraphDatabase.Driver(dbUri, AuthTokens.Basic(dbUser, dbPassword));
await driver.VerifyConnectivityAsync();

var watch = System.Diagnostics.Stopwatch.StartNew();
log("LAZY LOADING (ExecuteReadAsync)");
await lazyLoading(driver);
log($"--- {watch.ElapsedMilliseconds/1000} seconds ---");
Console.WriteLine();

watch.Restart();
log("EAGER LOADING (ExecutableQuery)");
await eagerLoading(driver);
log($"--- {watch.ElapsedMilliseconds/1000} seconds ---");

async Task lazyLoading(IDriver driver) {
    using var session = driver.AsyncSession(conf => conf.WithDatabase("neo4j"));
    await session.ExecuteReadAsync(async tx => {
        log("Submit query");
        var result = await tx.RunAsync(slowQuery);
        while (await result.FetchAsync()) {
            int recordN = result.Current.Get<int>("output");
            log($"Processing record {recordN}");
            System.Threading.Thread.Sleep(sleepTime);  // proxy for some expensive operation
        }
    });
}

async Task eagerLoading(IDriver driver) {
    log("Submit query");
    var result = await driver.ExecutableQuery(slowQuery)
        .WithConfig(new QueryConfig(database: "neo4j"))
        .ExecuteAsync();
    foreach (var record in result.Result) {
        int recordN = record.Get<int>("output");
        log($"Processing record {recordN}");
        System.Threading.Thread.Sleep(sleepTime);  // proxy for some expensive operation
    }
}

void log(string msg) {
    Console.WriteLine($"[{DateTimeOffset.UtcNow.ToUnixTimeSeconds()}] {msg}");
}
Output
[1734336457] LAZY LOADING (ExecuteReadAsync)
[1734336457] Submit query
[1734336457] Processing record 1  (1)
[1734336457] Processing record 2
[1734336458] Processing record 3
...
[1734336581] Processing record 249
[1734336581] Processing record 250
[1734336582] --- 125 seconds ---

[1734336582] EAGER LOADING (ExecutableQuery)
[1734336582] Submit query
[1734336594] Processing record 1   (2)
[1734336594] Processing record 2
[1734336595] Processing record 3
...
[1734336718] Processing record 249
[1734336718] Processing record 250
[1734336719] --- 136 seconds ---  (3)
1 With lazy loading, the first record is available sooner.
2 With eager loading, the first record is available once the result is consumed (i.e. after the server has retrieved all 250 records).
3 The total running time is longer with eager loading because the client waits until it receives the last record, whereas with lazy loading the client can process records while the server fetches the next ones. With lazy loading, the client can also stop requesting records after some condition is met (by calling .ConsumeAsync() on the Result object) to save time and resources.

When using .ExecutableQuery(), you can process records as they arrive using the .WithStreamProcessor() method. This will speed up response by not waiting for all records before processing them; it does not however improve memory footprint, as you can’t control the flux of records. You can reduce the memory footprint of EagerResult using .WithMap() or .WithReduce().

The driver’s fetch size affects the behavior of lazy loading. It instructs the server to stream an amount of records equal to the fetch size, and then wait until the client has caught up before retrieving and sending more.

The fetch size allows to bound memory consumption on the client side. It doesn’t always bound memory consumption on the server side though: that depends on the query. For example, a query with ORDER BY requires the whole result set to be loaded into memory for sorting, before records can be streamed to the client.

The lower the fetch size, the more messages client and server have to exchange. Especially if the server’s latency is high, a low fetch size may deteriorate performance.

Route read queries to cluster readers

In a cluster, route read queries to any reader node. You do this by:

Good practices

await driver.ExecutableQuery("MATCH (p:Person) RETURN p")
    .WithConfig(new QueryConfig(
        database: "neo4j",
        routing: RoutingControl.Readers
    ))
    .ExecuteAsync();
using var session = driver.AsyncSession(conf => conf
    .WithDatabase("neo4j")
    .WithDefaultAccessMode(AccessMode.Read)
);

Bad practices

// defaults to routing = writers
await driver.ExecutableQuery("MATCH (p:Person) RETURN p")
    .WithConfig(new QueryConfig(
        database: "neo4j"
    ))
    .ExecuteAsync();
using var session = driver.AsyncSession(conf => conf.WithDatabase("neo4j"));

// don't ask to write on a read-only operation
await session.ExecuteWriteAsync(async tx => {
    var result = await tx.RunAsync("MATCH (p:Person) RETURN p");
    return result.ToListAsync();
});

Create indexes

Create indexes for properties that you often filter against. For example, if you often look up Person nodes by the name property, it is beneficial to create an index on Person.name. You can create indexes with the CREATE INDEX Cypher clause, for both nodes and relationships.

Create an index on Person.name
await driver.ExecutableQuery("CREATE INDEX person_name FOR (n:Person) ON (n.name)").ExecuteAsync();

For more information, see Indexes for search performance.

Profile queries

Profile your queries to locate queries whose performance can be improved. You can profile queries by prepending them with PROFILE. The server output is available in the Profile property of the IResultSummary object.

var result = await driver.ExecutableQuery("PROFILE MATCH (p {name: $name}) RETURN p")
    .WithParameters(new { name = "Alice" })
    .WithConfig(new QueryConfig(database: "neo4j"))
    .ExecuteAsync();
// Note: `result.Result` is non-empty
var queryPlan = result.Summary.Profile;
Console.WriteLine(queryPlan.Arguments["string-representation"]);
/*
Planner COST
Runtime PIPELINED
Runtime version 5.0
Batch size 128

+-----------------+----------------+----------------+------+---------+----------------+------------------------+-----------+---------------------+
| Operator        | Details        | Estimated Rows | Rows | DB Hits | Memory (Bytes) | Page Cache Hits/Misses | Time (ms) | Pipeline            |
+-----------------+----------------+----------------+------+---------+----------------+------------------------+-----------+---------------------+
| +ProduceResults | p              |              1 |    1 |       3 |                |                        |           |                     |
| |               +----------------+----------------+------+---------+----------------+                        |           |                     |
| +Filter         | p.name = $name |              1 |    1 |       4 |                |                        |           |                     |
| |               +----------------+----------------+------+---------+----------------+                        |           |                     |
| +AllNodesScan   | p              |             10 |    4 |       5 |            120 |                 9160/0 |   108.923 | Fused in Pipeline 0 |
+-----------------+----------------+----------------+------+---------+----------------+------------------------+-----------+---------------------+

Total database accesses: 12, total allocated memory: 184
*/

In case some queries are so slow that you are unable to even run them in reasonable times, you can prepend them with EXPLAIN instead of PROFILE. This will return the plan that the server would use to run the query, but without executing it. The server output is available in the Plan property of the IResultSummary object.

var result = await driver.ExecutableQuery("EXPLAIN MATCH (p {name: $name}) RETURN p")
    .WithParameters(new { name = "Alice" })
    .WithConfig(new QueryConfig(database: "neo4j"))
    .ExecuteAsync();
var queryPlan = result.Summary.Plan;
Console.WriteLine(queryPlan.Arguments["string-representation"]);
/*
Planner COST
Runtime PIPELINED
Runtime version 5.0
Batch size 128

+-----------------+----------------+----------------+---------------------+
| Operator        | Details        | Estimated Rows | Pipeline            |
+-----------------+----------------+----------------+---------------------+
| +ProduceResults | p              |              1 |                     |
| |               +----------------+----------------+                     |
| +Filter         | p.name = $name |              1 |                     |
| |               +----------------+----------------+                     |
| +AllNodesScan   | p              |             10 | Fused in Pipeline 0 |
+-----------------+----------------+----------------+---------------------+

Total database accesses: ?
*/

Specify node labels

Specify node labels in all queries. This allows the query planner to work much more efficiently, and to leverage indexes where available. To learn how to combine labels, see Cypher → Label expressions.

Good practices

var result = await driver.ExecutableQuery("MATCH (p:Person|Animal {name: $name}) RETURN p")
    .WithParameters(new { name = "Alice" })
    .WithConfig(new QueryConfig(database: "neo4j"))
    .ExecuteAsync();
using var session = driver.AsyncSession(conf => conf.WithDatabase("neo4j"));
await session.RunAsync("MATCH (p:Person|Animal {name: $name}) RETURN p", new { name = "Alice" });

Bad practices

var result = await driver.ExecutableQuery("MATCH (p {name: $name}) RETURN p")
    .WithParameters(new { name = "Alice" })
    .WithConfig(new QueryConfig(database: "neo4j"))
    .ExecuteAsync();
using var session = driver.AsyncSession(conf => conf.WithDatabase("neo4j"));
await session.RunAsync("MATCH (p {name: $name}) RETURN p", new { name = "Alice" });

Batch data creation

Batch queries when creating a lot of records using the WITH and UNWIND Cypher clauses.

Good practice

Submit one single queries with all values inside
// Generate a sequence of numbers
int start = 1;
int end = 10000;
var numbers = new List<Dictionary<string, dynamic>>();
for (int i=start; i<=end; i++) {
    numbers.Add(new Dictionary<string, dynamic>() {
        {"value", i},
    });
}

await driver.ExecutableQuery(@"
    UNWIND $numbers AS node
    MERGE (:Number {value: node.value})
    ")
    .WithParameters(new { numbers = numbers })
    .WithConfig(new QueryConfig(database: "neo4j"))
    .ExecuteAsync();

Bad practice

Submit a lot of single queries, one for each value
for (int i=1; i<=10000; i++) {
    await driver.ExecutableQuery("MERGE (:Number {value: $value})")
        .WithParameters(new { value = i })
        .WithConfig(new QueryConfig(database: "neo4j"))
        .ExecuteAsync();
}
The most efficient way of performing a first import of large amounts of data into a new database is the neo4j-admin database import command.

Use query parameters

Always use query parameters instead of hardcoding or concatenating values into queries. Besides protecting from Cypher injections, this allows to better leverage the database query cache.

Good practices

await driver.ExecutableQuery("MATCH (p:Person {name: $name}) RETURN p")
    .WithConfig(new QueryConfig(database: "neo4j"))
    .WithParameters(new { name = "Alice" })
    .ExecuteAsync();
using var session = driver.AsyncSession(conf => conf.WithDatabase("neo4j"));
await session.RunAsync("MATCH (p:Person {name: $name}) RETURN p", new { name = "Alice" });

Bad practices

await driver.ExecutableQuery("MATCH (p:Person {name: 'Alice'}) RETURN p")
    .WithConfig(new QueryConfig(database: "neo4j"))
    .ExecuteAsync();
// or
string name = "Alice";
await driver.ExecutableQuery("MATCH (p:Person {name: '" + name + "'}) RETURN p")
    .WithConfig(new QueryConfig(database: "neo4j"))
    .ExecuteAsync();
using var session = driver.AsyncSession(conf => conf.WithDatabase("neo4j"));
await session.RunAsync("MATCH (p:Person {name: 'Alice'}) RETURN p");
// or
string name = "Alice";
session.RunAsync("MATCH (p:Person {name: '" + name + "'}) RETURN p");

Use MERGE for creation only when needed

The Cypher clause MERGE is convenient for data creation, as it allows to avoid duplicate data when an exact clone of the given pattern exists. However, it requires the database to run two queries: it first needs to MATCH the pattern, and only then can it CREATE it (if needed).

If you know already that the data you are inserting is new, avoid using MERGE and use CREATE directly instead — this practically halves the number of database queries.

Filter notifications

Filter the category and/or severity of notifications the server should raise.

Glossary

LTS

A Long Term Support release is one guaranteed to be supported for a number of years. Neo4j 4.4 is LTS, and Neo4j 5 will also have an LTS version.

Aura

Aura is Neo4j’s fully managed cloud service. It comes with both free and paid plans.

Cypher

Cypher is Neo4j’s graph query language that lets you retrieve data from the database. It is like SQL, but for graphs.

APOC

Awesome Procedures On Cypher (APOC) is a library of (many) functions that can not be easily expressed in Cypher itself.

Bolt

Bolt is the protocol used for interaction between Neo4j instances and drivers. It listens on port 7687 by default.

ACID

Atomicity, Consistency, Isolation, Durability (ACID) are properties guaranteeing that database transactions are processed reliably. An ACID-compliant DBMS ensures that the data in the database remains accurate and consistent despite failures.

eventual consistency

A database is eventually consistent if it provides the guarantee that all cluster members will, at some point in time, store the latest version of the data.

causal consistency

A database is causally consistent if read and write queries are seen by every member of the cluster in the same order. This is stronger than eventual consistency.

NULL

The null marker is not a type but a placeholder for absence of value. For more information, see Cypher → Working with null.

transaction

A transaction is a unit of work that is either committed in its entirety or rolled back on failure. An example is a bank transfer: it involves multiple steps, but they must all succeed or be reverted, to avoid money being subtracted from one account but not added to the other.

backpressure

Backpressure is a force opposing the flow of data. It ensures that the client is not being overwhelmed by data faster than it can handle.

transaction function

A transaction function is a callback executed by an .ExecuteReadAsync() or .ExecuteWriteAsync() call. The driver automatically re-executes the callback in case of server failure.

IDriver

A IDriver object holds the details required to establish connections with a Neo4j database.