Multimodal text and image search
With vectors, we're able to search not just text but images. Unlock features like:
- search for images with text
- image to image search
This is because in vector form, we're able to compare the meaning of these two data formats.
Start by creating vector fields
Vectors are an AI data format which you can create based on your image URL fields. You will want to use our image-text
model.
Learn about how to vectorize your data.
Search your vector fields
When you vectorize a field, such as image_url
, documents in your dataset will have a new field appended with the AI model used to create the vector ([email protected]
) and _vector_
. So in our example: [email protected]_vector_
.
These new image-text
vector fields can be searched with text or with an image URL. Our API will automatically vectorize your search query with the same model so we can compare.
Here's an example of using our SDK to search a dataset of images with text:
// import client and query builder
import { DiscoveryClient, QueryBuilder } from "@relevanceai/sdk";
// initialize client
const client = new DiscoveryClient({
project: process.env.RELEVANCE_PROJECT,
api_key: process.env.RELEVANCE_API_KEY,
});
const dataset = client.dataset("my-dataset");
// search "image_url" vector field with a text query
const query = QueryBuilder().query("dogs in a park").vector("[email protected]_vector_");
const results = await dataset.search(query);