Skip to main content

Deduplication API

Usageโ€‹

import { epilot } from '@epilot/sdk'

epilot.authorize(() => '<token>')
const { data } = await epilot.deduplication.deduplicate(...)

Tree-shakeable importโ€‹

import { getClient, authorize } from '@epilot/sdk/deduplication'

const deduplicationClient = getClient()
authorize(deduplicationClient, () => '<token>')
const { data } = await deduplicationClient.deduplicate(...)

Operationsโ€‹

Schemas

deduplicateโ€‹

Deduplicates Entities

POST /v1/deduplicate

const { data } = await client.deduplicate(
null,
[
{
toKeep: 'string',
toDelete: ['string']
}
],
)
Response
{
"deduplicatedEntities": [
{
"_id": "string",
"_org": "string",
"_schema": "string",
"_created_at": "1970-01-01T00:00:00.000Z",
"_updated_at": "1970-01-01T00:00:00.000Z",
"_created_by": "string",
"created_by": "string",
"_tags": ["string"],
"_acl": {},
"_owners": [
{
"org_id": "string",
"user_id": "string"
}
],
"type": "string"
}
]
}

deduplicateAsyncโ€‹

Submits an async deduplication job. Returns a job ID immediately. Poll GET /v1/deduplicate/jobs/{jobId} for status.

POST /v1/deduplicate/job

const { data } = await client.deduplicateAsync(
null,
[
{
toKeep: 'string',
toDelete: ['string']
}
],
)

getDeduplicationJobโ€‹

Returns the current status of an async deduplication job

GET /v1/deduplicate/jobs/{jobId}

const { data } = await client.getDeduplicationJob({
jobId: 'example',
})
Response
{
"jobId": "string",
"status": "pending",
"message": "string",
"result": [
{
"_id": "string",
"_org": "string",
"_schema": "string",
"_created_at": "1970-01-01T00:00:00.000Z",
"_updated_at": "1970-01-01T00:00:00.000Z",
"_created_by": "string",
"created_by": "string",
"_tags": ["string"],
"_acl": {},
"_owners": [
{
"org_id": "string",
"user_id": "string"
}
],
"type": "string"
}
],
"createdAt": "1970-01-01T00:00:00.000Z",
"updatedAt": "1970-01-01T00:00:00.000Z"
}

detectDuplicatesโ€‹

Detects potential duplicate entities for the given entity using the schema's prioritized uniqueness rules. Returns matches with a confidence score.

POST /v1/detect-duplicates

const { data } = await client.detectDuplicates(
null,
{
schema: 'string',
entity: {}
},
)
Response
{
"matches": [
{
"entity": {
"_id": "string",
"_org": "string",
"_schema": "string",
"_created_at": "1970-01-01T00:00:00.000Z",
"_updated_at": "1970-01-01T00:00:00.000Z",
"_created_by": "string",
"created_by": "string",
"_tags": ["string"],
"_acl": {},
"_owners": [
{
"org_id": "string",
"user_id": "string"
}
],
"type": "string"
},
"confidence": 0,
"matched_attributes": ["string"]
}
]
}

dismissDuplicatesโ€‹

Confirms entities as NOT duplicates: clears the internal duplicate-detection flags (_matching_entities) on each given entity, so they stop appearing as open duplicate sets. The records themselves are

POST /v1/duplicates/dismiss

const { data } = await client.dismissDuplicates(
null,
{
schema: 'string',
entityIds: ['string']
},
)
Response
{
"dismissed": ["string"]
}

listUniquenessCriteriaโ€‹

Lists UniquenessCriteria for the requesting organization. Optionally filtered by schema.

GET /v1/uniqueness-criteria

const { data } = await client.listUniquenessCriteria({
schema: 'example',
})
Response
{
"items": [
{
"id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
"orgId": "string",
"schema": "string",
"matchRules": [
{
"name": "string",
"attributes": [
{
"attribute": "string"
}
],
"confidence": 0
}
],
"createdAt": "1970-01-01T00:00:00.000Z",
"updatedAt": "1970-01-01T00:00:00.000Z"
}
]
}

createUniquenessCriteriaโ€‹

Creates a new UniquenessCriteria record.

POST /v1/uniqueness-criteria

const { data } = await client.createUniquenessCriteria(
null,
{
schema: 'string',
matchRules: [
{
name: 'string',
attributes: [
{
attribute: 'string'
}
],
confidence: 0
}
]
},
)
Response
{
"id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
"orgId": "string",
"schema": "string",
"matchRules": [
{
"name": "string",
"attributes": [
{
"attribute": "string"
}
],
"confidence": 0
}
],
"createdAt": "1970-01-01T00:00:00.000Z",
"updatedAt": "1970-01-01T00:00:00.000Z"
}

getUniquenessCriteriaโ€‹

Fetch a single UniquenessCriteria record.

GET /v1/uniqueness-criteria/{schema}

const { data } = await client.getUniquenessCriteria({
schema: 'example',
})
Response
{
"id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
"orgId": "string",
"schema": "string",
"matchRules": [
{
"name": "string",
"attributes": [
{
"attribute": "string"
}
],
"confidence": 0
}
],
"createdAt": "1970-01-01T00:00:00.000Z",
"updatedAt": "1970-01-01T00:00:00.000Z"
}

updateUniquenessCriteriaโ€‹

Replace the matchRules on an existing UniquenessCriteria record.

PUT /v1/uniqueness-criteria/{schema}

const { data } = await client.updateUniquenessCriteria(
{
schema: 'example',
},
{
matchRules: [
{
name: 'string',
attributes: [
{
attribute: 'string'
}
],
confidence: 0
}
]
},
)
Response
{
"id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
"orgId": "string",
"schema": "string",
"matchRules": [
{
"name": "string",
"attributes": [
{
"attribute": "string"
}
],
"confidence": 0
}
],
"createdAt": "1970-01-01T00:00:00.000Z",
"updatedAt": "1970-01-01T00:00:00.000Z"
}

deleteUniquenessCriteriaโ€‹

Delete a UniquenessCriteria record.

DELETE /v1/uniqueness-criteria/{schema}

const { data } = await client.deleteUniquenessCriteria({
schema: 'example',
})

Schemasโ€‹

DeduplicateRequestBodyโ€‹

type DeduplicateRequestBody = Array<{
toKeep: string
toDelete: string[]
}>

DeduplicateRequestResponseโ€‹

type DeduplicateRequestResponse = {
deduplicatedEntities: Array<{
_id: string
_org?: string
_schema?: string
_created_at?: string // date-time
_updated_at?: string // date-time
_created_by?: string | number
created_by?: string | number
_tags?: string[]
_acl?: Record<string, string[]>
_owners?: Array<{
org_id: { ... }
user_id: { ... }
}>
type?: string
}>
}

Entityโ€‹

Base Entity schema

type Entity = {
_id: string
_org?: string
_schema?: string
_created_at?: string // date-time
_updated_at?: string // date-time
_created_by?: string | number
created_by?: string | number
_tags?: string[]
_acl?: Record<string, string[]>
_owners?: Array<{
org_id: string
user_id: string
}>
type?: string
}

DeduplicateAsyncResponseโ€‹

Response returned immediately when a deduplication job is submitted

type DeduplicateAsyncResponse = {
jobId: string
status: "pending" | "processing" | "completed" | "failed"
message: string
}

JobStatusโ€‹

type JobStatus = "pending" | "processing" | "completed" | "failed"

DeduplicationJobโ€‹

Represents an async deduplication job

type DeduplicationJob = {
jobId: string
status: "pending" | "processing" | "completed" | "failed"
message?: string
result?: Array<{
_id: string
_org?: string
_schema?: string
_created_at?: string // date-time
_updated_at?: string // date-time
_created_by?: string | number
created_by?: string | number
_tags?: string[]
_acl?: Record<string, string[]>
_owners?: Array<{
org_id: { ... }
user_id: { ... }
}>
type?: string
}>
createdAt: string // date-time
updatedAt: string // date-time
}

DetectDuplicatesRequestBodyโ€‹

type DetectDuplicatesRequestBody = {
schema: string
entity: Record<string, unknown>
}

DetectDuplicatesResponseโ€‹

type DetectDuplicatesResponse = {
matches: Array<{
entity: {
_id: { ... }
_org?: { ... }
_schema?: { ... }
_created_at?: { ... }
_updated_at?: { ... }
_created_by?: { ... }
created_by?: { ... }
_tags?: { ... }
_acl?: { ... }
_owners?: { ... }
type?: { ... }
}
confidence: number
matched_attributes: string[]
}>
}

DetectedDuplicateMatchโ€‹

type DetectedDuplicateMatch = {
entity: {
_id: string
_org?: string
_schema?: string
_created_at?: string // date-time
_updated_at?: string // date-time
_created_by?: string | number
created_by?: string | number
_tags?: string[]
_acl?: Record<string, string[]>
_owners?: Array<{
org_id: { ... }
user_id: { ... }
}>
type?: string
}
confidence: number
matched_attributes: string[]
}

DismissDuplicatesRequestBodyโ€‹

type DismissDuplicatesRequestBody = {
schema: string
entityIds: string[]
}

DismissDuplicatesResponseโ€‹

type DismissDuplicatesResponse = {
dismissed: string[]
}

UniquenessCriteriaListResponseโ€‹

type UniquenessCriteriaListResponse = {
items: Array<{
id: string // uuid
orgId: string
schema: string
matchRules: Array<{
name?: { ... }
attributes: { ... }
confidence: { ... }
}>
createdAt: string // date-time
updatedAt: string // date-time
}>
}

UniquenessCriteriaโ€‹

Defines what makes an entity of a given schema unique within an organization.

type UniquenessCriteria = {
id: string // uuid
orgId: string
schema: string
matchRules: Array<{
name?: string
attributes: Array<{
attribute: { ... }
}>
confidence: number
}>
createdAt: string // date-time
updatedAt: string // date-time
}

MatchRuleโ€‹

One way to identify the same entity. Evaluated in order; first rule whose attributes are all available and that returns hits wins.

type MatchRule = {
name?: string
attributes: Array<{
attribute: string
}>
confidence: number
}

MatchAttributeโ€‹

One attribute participating in a match rule. Wrapped as an object so per-attribute options can be added later.

type MatchAttribute = {
attribute: string
}

UniquenessCriteriaCreateBodyโ€‹

type UniquenessCriteriaCreateBody = {
schema: string
matchRules: Array<{
name?: string
attributes: Array<{
attribute: { ... }
}>
confidence: number
}>
}

UniquenessCriteriaUpdateBodyโ€‹

type UniquenessCriteriaUpdateBody = {
matchRules: Array<{
name?: string
attributes: Array<{
attribute: { ... }
}>
confidence: number
}>
}