GenAI integrations

Neo4j’s Vector indexes and Vector functions allow you to calculate the similarity between node and relationship properties in a graph. A prerequisite for using these features is that vector embeddings have been set as properties of these entities. The GenAI plugin enables the creation of such embeddings using GenAI providers.

To use the GenAI plugin you need an account and API credentials from any of the following GenAI providers: Vertex AI, OpenAI, Azure OpenAI, and Amazon Bedrock.

To learn more about using embeddings in Neo4j, see Vector indexes → Vectors and embeddings in Neo4j.

For a hands-on guide on how to use the GenAI plugin, see GenAI documentation - Embeddings & Vector Indexes Tutorial → Create embeddings with cloud AI providers.

Installation

The GenAI plugin is enabled by default in Neo4j Aura.

The plugin needs to be installed on self-managed instances. This is done by moving the neo4j-genai.jar file from /products to /plugins in the Neo4j home directory, or, if you are using Docker, by starting the Docker container with the extra parameter --env NEO4J_PLUGINS='["genai"]'. For more information, see Operations Manual → Configure plugins.

Example graph

The examples on this page use the Neo4j movie recommendations dataset, focusing on the plot and title properties of Movie nodes.

Example graph connecting person and actor nodes with a movie node via acted in and directed relationships

The graph contains 28863 nodes and 332522 relationships. There are 9083 Movie nodes with a plot and title property. To recreate the graph,

  • For Aura or Enterprise Edition, import this dump file into an empty database.

  • For self-hosted Community Edition, import this dump file into an empty database.

To learn how to import dumps, see Aura → Importing an existing database or Operations manual → Restore a database dump.

The embeddings on this page are generated using OpenAI (model text-embedding-ada-002), producing 1536-dimensional vectors.

Generate a single embedding and store it

Use the genai.vector.encode() function to generate a vector embedding for a single value.

Signature for genai.vector.encode() Function
genai.vector.encode(resource :: STRING, provider :: STRING, configuration :: MAP = {}) :: LIST<FLOAT>

  • The resource (a STRING) is the object to transform into an embedding, such as a chunk text or a node/relationship property.

  • The provider (a STRING) is the case-insensitive identifier of the provider to use. See identifiers under GenAI providers for supported options.

  • The configuration (a MAP) contains provider-specific settings, such as which model to invoke, as well as any required API credentials. See GenAI providers for details of each supported provider. Note that because this argument may contain sensitive data, it is obfuscated in the query.log. However, if the function call is misspelled or the query is otherwise malformed, it may be logged without being obfuscated.

This function sends one API request every time it is called, which may result in a lot of overhead in terms of both network traffic and latency. If you want to generate many embeddings at once, use Generate a batch of embeddings and store them.

genai.vector.encode() returns a LIST<FLOAT>. To convert and store this value as a VECTOR, use the vector() function. Storing VECTOR values on an on-prem instance requires Enterprise Edition and block format.

Create a VECTOR embedding property for the Godfather
MATCH (m:Movie {title:'Godfather, The'})
WHERE m.plot IS NOT NULL AND m.title IS NOT NULL
WITH m, m.title || ' ' || m.plot AS titleAndPlot (1)
WITH m, genai.vector.encode(titleAndPlot, 'OpenAI', { token: $openaiToken }) AS vector (2)
SET m.embedding = vector(vector, 1536, FLOAT32) (3)
RETURN m.embedding AS embedding
1 Concatenate the title and plot of the Movie into a single STRING.
2 Create a 1536 dimensional embedding from the titleAndPlot.
3 Store the propertyVector as a new vector embedding property on The Godfather node.
Result
+----------------------------------------------------------------------------------------------------+
| embedding                                                                                          |
+----------------------------------------------------------------------------------------------------+
| [0.005239539314061403, -0.039358530193567276, -0.0005175105179660022, -0.038706034421920776, ... ] |
+----------------------------------------------------------------------------------------------------+
This result only shows the first 4 of the 1536 numbers in the embedding.

Use the db.create.setNodeVectorProperty procedure to store an embedding as a LIST<FLOAT> value to a node property.

Signature for db.create.setNodeVectorProperty() Procedure
db.create.setNodeVectorProperty(node :: NODE, key :: STRING, vector :: ANY)

Use the db.create.setRelationshipVectorProperty procedure to store an embedding as a LIST<FLOAT> value to a relationship property.

Signature for db.create.setRelationshipVectorProperty() Procedure
db.create.setRelationshipVectorProperty(relationship :: RELATIONSHIP, key :: STRING, vector :: ANY)

  • node or relationship is the entity in which the new property will be stored.

  • key (a STRING) is the name of the new property containing the embedding.

  • vector is the object containing the embedding.

The embeddings are stored as properties on nodes or relationships with the type LIST<INTEGER | FLOAT>.

Create a LIST<FLOAT> embedding property for the Godfather
MATCH (m:Movie {title:'Godfather, The'})
WHERE m.plot IS NOT NULL AND m.title IS NOT NULL
WITH m, m.title || ' ' || m.plot AS titleAndPlot (1)
WITH m, genai.vector.encode(titleAndPlot, 'OpenAI', { token: $openaiToken }) AS vector (2)
CALL db.create.setNodeVectorProperty(m, 'embedding', vector) (3)
RETURN m.embedding AS embedding
1 Concatenate the title and plot of the Movie into a single STRING.
2 Create a 1536 dimensional embedding from the titleAndPlot.
3 Store the propertyVector as a new LIST<FLOAT> `embedding property on The Godfather node.
Result
+----------------------------------------------------------------------------------------------------+
| embedding                                                                                          |
+----------------------------------------------------------------------------------------------------+
| [0.005239539314061403, -0.039358530193567276, -0.0005175105179660022, -0.038706034421920776, ... ] |
+----------------------------------------------------------------------------------------------------+
This result only shows the first 4 of the 1536 numbers in the embedding.

Generate a batch of embeddings and store them

Use the genai.vector.encodeBatch() procedure to generate many vector embeddings with a single API request. This procedure takes a list of resources as an input, and returns the same number of result rows, instead of a single one.

This procedure attempts to generate embeddings for all supplied resources in a single API request. Therefore, it is recommended to see the respective provider’s documentation for details on, for example, the maximum number of embeddings that can be generated per request.
Signature for genai.vector.encodeBatch() Procedure
genai.vector.encodeBatch(resources :: LIST<STRING>, provider :: STRING, configuration :: MAP = {}) :: (index :: INTEGER, resource :: STRING, vector :: LIST<FLOAT>)

  • The resources (a LIST<STRING>) parameter is the list of objects to transform into embeddings, such as chunks of text.

  • The provider (a STRING) is the case-insensitive identifier of the provider to use. See GenAI providers for supported options.

  • The configuration (a MAP) specifies provider-specific settings such as which model to invoke, as well as any required API credentials. See GenAI providers for details of each supported provider. Note that because this argument may contain sensitive data, it is obfuscated in the query.log. However, if the function call is misspelled or the query is otherwise malformed, it may be logged without being obfuscated.

Each returned row contains the following columns:

  • The index (an INTEGER) is the index of the corresponding element in the input list, to aid in correlating results back to inputs.

  • The resource (a STRING) is the name of the input resource.

  • The vector (a LIST<FLOAT>) is the generated vector embedding for this resource.

genai.vector.encode() returns a LIST<FLOAT>. To convert and store this value as a VECTOR, use the vector() function. Storing VECTOR values on an on-prem instance requires Enterprise Edition and block format.

Create embeddings from a limited number of properties and store them as VECTOR properties
MATCH (m:Movie WHERE m.plot IS NOT NULL)
WITH m
LIMIT 20
WITH collect(m) AS moviesList (1)
WITH moviesList, [movie IN moviesList | movie.title || ': ' || movie.plot] AS batch (2)
CALL genai.vector.encodeBatch(batch, 'OpenAI', { token: $openaiToken }) YIELD index, vector
WITH moviesList, index, vector
MATCH (toUpdate:Movie {title: moviesList[index]['title']})
SET toUpdate.embedding = vector(vector, 1536, FLOAT32)  (3)
1 Collect all 20 Movie nodes into a LIST<NODE>.
2 A list comprehension ([]) extracts the title and plot properties of the movies in moviesList into a new LIST<STRING>.
3 db.create.setNodeVectorProperty is run for each vector returned by genai.vector.encodeBatch(), and stores that vector as a property named embedding on the corresponding node.
Create embeddings from a large number properties and store them as VECTOR properties
MATCH (m:Movie WHERE m.plot IS NOT NULL)
WITH collect(m) AS moviesList, (1)
     count(*) AS total,
     100 AS batchSize (2)
UNWIND range(0, total-1, batchSize) AS batchStart (3)
CALL (moviesList, batchStart, batchSize) { (4)
    WITH [movie IN moviesList[batchStart .. batchStart + batchSize] | movie.title || ': ' || movie.plot] AS batch (5)
    CALL genai.vector.encodeBatch(batch, 'OpenAI', { token: $openaiToken }) YIELD index, vector
    MATCH (toUpdate:Movie {title: moviesList[batchStart + index]['title']})
    SET toUpdate.embedding = vector(vector, 1536, FLOAT32) (6)
} IN CONCURRENT TRANSACTIONS OF 1 ROW (7)
1 Collect all returned Movie nodes into a LIST<NODE>.
2 batchSize defines the number of nodes in moviesList to be processed at once. Because vector embeddings can be very large, a larger batch size may require significantly more memory on the Neo4j server. Too large a batch size may also exceed the provider’s threshold.
3 Process Movie nodes in increments of batchSize. The end range total-1 is due to range being inclusive on both ends.
4 A CALL subquery executes a separate transaction for each batch. Note that this CALL subquery uses a variable scope clause.
5 batch is a list of strings, each being the concatenation of title and plot of one movie.
6 The procedure sets vector as value for the property named embedding for the node at position batchStart + index in the moviesList.
7 Set to 1 the amount of batches to be processed at once. For more information on concurrency in transactions, see CALL subqueries → Concurrent transactions).
This example may not scale to larger datasets, as collect(m) requires the whole result set to be loaded in memory. For an alternative method more suitable to processing large amounts of data, see GenAI documentation - Embeddings & Vector Indexes Tutorial → Create embeddings with cloud AI providers.

Use the db.create.setNodeVectorProperty procedure to store an embedding as a LIST<FLOAT> value to a node property.

Signature for db.create.setNodeVectorProperty Procedure
db.create.setNodeVectorProperty(node :: NODE, key :: STRING, vector :: ANY)

Use the db.create.setRelationshipVectorProperty procedure to store an embedding as a LIST<FLOAT> value to a relationship property.

Signature for db.create.setRelationshipVectorProperty Procedure
db.create.setRelationshipVectorProperty(relationship :: RELATIONSHIP, key :: STRING, vector :: ANY)

  • node or relationship is the entity in which the new property will be stored.

  • key (a STRING) is the name of the new property containing the embedding.

  • vector is the object containing the embedding.

The embeddings are stored as properties on nodes or relationships with the type LIST<INTEGER | FLOAT>.

Create embeddings from a limited number of properties and store them as LIST<FLOAT> properties
MATCH (m:Movie WHERE m.plot IS NOT NULL)
WITH m
LIMIT 20
WITH collect(m) AS moviesList (1)
WITH moviesList, [movie IN moviesList | movie.title || ': ' || movie.plot] AS batch (2)
CALL genai.vector.encodeBatch(batch, 'OpenAI', { token: $openaiToken }) YIELD index, vector
WITH moviesList, index, vector
CALL db.create.setNodeVectorProperty(moviesList[index], 'embedding', vector) (3)
1 Collect all 20 Movie nodes into a LIST<NODE>.
2 A list comprehension ([]) extracts the title and plot properties of the movies in moviesList into a new LIST<STRING>.
3 db.create.setNodeVectorProperty is run for each vector returned by genai.vector.encodeBatch(), and stores that vector as a property named embedding on the corresponding node.
Create embeddings from a large number properties and store them as LIST<FLOAT> values
MATCH (m:Movie WHERE m.plot IS NOT NULL)
WITH collect(m) AS moviesList, (1)
     count(*) AS total,
     100 AS batchSize (2)
UNWIND range(0, total-1, batchSize) AS batchStart (3)
CALL (moviesList, batchStart, batchSize) { (4)
    WITH [movie IN moviesList[batchStart .. batchStart + batchSize] | movie.title || ': ' || movie.plot] AS batch (5)
    CALL genai.vector.encodeBatch(batch, 'OpenAI', { token: $openaiToken }) YIELD index, vector
    CALL db.create.setNodeVectorProperty(moviesList[batchStart + index], 'embedding', vector) (6)
} IN CONCURRENT TRANSACTIONS OF 1 ROW (7)
1 Collect all returned Movie nodes into a LIST<NODE>.
2 batchSize defines the number of nodes in moviesList to be processed at once. Because vector embeddings can be very large, a larger batch size may require significantly more memory on the Neo4j server. Too large a batch size may also exceed the provider’s threshold.
3 Process Movie nodes in increments of batchSize. The end range total-1 is due to range being inclusive on both ends.
4 A CALL subquery executes a separate transaction for each batch. Note that this CALL subquery uses a variable scope clause.
5 batch is a list of strings, each being the concatenation of title and plot of one movie.
6 The procedure sets vector as value for the property named embedding for the node at position batchStart + index in the moviesList.
7 Set to 1 the amount of batches to be processed at once. For more information on concurrency in transactions, see CALL subqueries → Concurrent transactions).
This example may not scale to larger datasets, as collect(m) requires the whole result set to be loaded in memory. For an alternative method more suitable to processing large amounts of data, see GenAI documentation - Embeddings & Vector Indexes Tutorial → Create embeddings with cloud AI providers.

GenAI providers

The following GenAI providers are supported for generating vector embeddings. Each provider has its own configuration map that can be passed to genai.vector.encode() or genai.vector.encodeBatch().

Vertex AI

Table 1. Configuration map
Key Type Description Default

token

STRING

API access token.

Required

projectId

STRING

GCP project ID.

Required

model

STRING

The name of the model you want to invoke.

Supported values:

  • "textembedding-gecko@001"

  • "textembedding-gecko@002"

  • "textembedding-gecko@003"

  • "textembedding-gecko-multilingual@001"

"textembedding-gecko@001"

region

STRING

GCP region where to send the API requests.

Supported values:

  • "us-west1"

  • "us-west2"

  • "us-west3"

  • "us-west4"

  • "us-central1"

  • "us-east1"

  • "us-east4"

  • "us-south1"

  • "northamerica-northeast1"

  • "northamerica-northeast2"

  • "southamerica-east1"

  • "southamerica-west1"

  • "europe-west2"

  • "europe-west1"

  • "europe-west4"

  • "europe-west6"

  • "europe-west3"

  • "europe-north1"

  • "europe-central2"

  • "europe-west8"

  • "europe-west9"

  • "europe-southwest1"

  • "asia-south1"

  • "asia-southeast1"

  • "asia-southeast2"

  • "asia-east2"

  • "asia-east1"

  • "asia-northeast1"

  • "asia-northeast2"

  • "australia-southeast1"

  • "australia-southeast2"

  • "asia-northeast3"

  • "me-west1"

"us-central1"

taskType

STRING

The intended downstream application (see provider documentation). The specified taskType will apply to all resources in a batch.

title

STRING

The title of the document that is being encoded (see provider documentation). The specified title will apply to all resources in a batch.

OpenAI

Table 2. Configuration map
Key Type Description Default

token

STRING

API access token.

Required

model

STRING

The name of the model you want to invoke.

"text-embedding-ada-002"

dimensions

INTEGER

The number of dimensions you want to reduce the vector to. Only supported for certain models.

Model-dependent.

Azure OpenAI

Unlike the other providers, the model is configured when creating the deployment on Azure, and is thus not part of the configuration map.
Table 3. Configuration map
Key Type Description Default

token

STRING

API access token.

Required

resource

STRING

The name of the resource to which the model has been deployed.

Required

deployment

STRING

The name of the model deployment.

Required

dimensions

INTEGER

The number of dimensions you want to reduce the vector to. Only supported for certain models.

Model-dependent.

Amazon Bedrock

Table 4. Configuration map
Key Type Description Default

accessKeyId

STRING

AWS access key ID.

Required

secretAccessKey

STRING

AWS secret key.

Required

model

STRING

The name of the model you want to invoke.

Supported values:

  • "amazon.titan-embed-text-v1"

"amazon.titan-embed-text-v1"

region

STRING

AWS region where to send the API requests.

Supported values:

  • "us-east-1"

  • "us-west-2"

  • "ap-southeast-1"

  • "ap-northeast-1"

  • "eu-central-1"

"us-east-1"