Mapping

Mapping defines how ERP data transforms into epilot entities. This page covers the configuration structure and available options.

Mapping Configuration

A mapping configuration consists of one or more entity definitions:

Mapping configuration
{
  "entities": [
    {
      "entity_schema": "contact",
      "unique_ids": ["customer_number"],
      "enabled": true,
      "fields": [...]
    }
  ]
}

Entity Properties

Property	Type	Required	Description
`entity_schema`	string	Yes	The epilot entity schema (e.g., `contact`, `contract`, `meter`)
`unique_ids`	string[]	Yes	Fields used to find existing entities
`enabled`	boolean/string	No	Enable/disable this mapping
`mode`	string	No	Operation mode: `upsert` (default), `delete`, `purge`, `upsert-prune-scope-purge`, or `upsert-prune-scope-delete`. See Operation Modes. For meter readings, see also `upsert-prune-scope` in Meter Reading Prune Scope Operations
`scope`	object	No	Required for prune-scope modes. Defines which entities to consider for pruning. See Prune Scope Operations
`jsonataExpression`	string	No	Pre-process the payload before field mapping
`fields`	array	Yes	Field mapping definitions

Field Mapping Types

Direct Field Mapping

Map a source field directly to a target attribute:

{
  "attribute": "first_name",
  "field": "firstName"
}

Nested Field Access (JSONPath)

Access nested data using JSONPath syntax with $ prefix:

{
  "attribute": "street",
  "field": "$.address.street"
}

JSONata Expressions

Use JSONata for complex transformations:

{
  "attribute": "full_name",
  "jsonataExpression": "firstName & ' ' & lastName"
}

Common JSONata patterns:

// Concatenation
"firstName & ' ' & lastName"

// Conditional
"$exists(middleName) ? firstName & ' ' & middleName & ' ' & lastName : firstName & ' ' & lastName"

// Date formatting
"$fromMillis($toMillis(dateField), '[Y]-[M01]-[D01]')"

// Array access
"addresses[0].street"

// Filtering
"items[status = 'active']"

Constant Values

Set a fixed value:

{
  "attribute": "source",
  "constant": "SAP"
}

Enabled Field

Use the enabled property to conditionally map fields:

{
  "attribute": "phone",
  "field": "phoneNumber",
  "enabled": "$exists(phoneNumber) and phoneNumber != ''"
}

Repeatable Fields

Email and phone fields in epilot are stored as arrays. Use _type to specify the field type:

Email Fields

{
  "attribute": "email",
  "field": "emailAddress",
  "_type": "email"
}

This transforms the input value into the epilot format:

// Input
{ "emailAddress": "john@example.com" }

// Output in epilot
{ "email": [{ "email": "john@example.com" }] }

Phone Fields

{
  "attribute": "phone",
  "field": "phoneNumber",
  "_type": "phone"
}

Entity-Level JSONata

Pre-process the entire payload before field mapping:

{
  "entity_schema": "contact",
  "jsonataExpression": "$merge([payload, {'fullAddress': payload.street & ', ' & payload.city}])",
  "unique_ids": ["customer_number"],
  "fields": [
    { "attribute": "address_display", "field": "fullAddress" }
  ]
}

Multiple Entities

Process multiple entities from a single event:

{
  "entities": [
    {
      "entity_schema": "contact",
      "unique_ids": ["customer_number"],
      "fields": [
        { "attribute": "customer_number", "field": "customerId" },
        { "attribute": "first_name", "field": "firstName" },
        { "attribute": "last_name", "field": "lastName" }
      ]
    },
    {
      "entity_schema": "account",
      "unique_ids": ["account_number"],
      "fields": [
        { "attribute": "account_number", "field": "accountId" },
        { "attribute": "name", "field": "companyName" }
      ]
    }
  ]
}

Dynamic Entity Expansion

Create multiple entities from an array in the payload:

{
  "entity_schema": "meter",
  "jsonataExpression": "meters",
  "unique_ids": ["meter_number"],
  "fields": [
    { "attribute": "meter_number", "field": "meterId" },
    { "attribute": "type", "field": "meterType" }
  ]
}

Given this input:

{
  "meters": [
    { "meterId": "M001", "meterType": "electricity" },
    { "meterId": "M002", "meterType": "gas" }
  ]
}

This creates two meter entities.

Conditional Entity Processing

Enable or disable entity processing based on payload conditions:

{
  "entity_schema": "contact",
  "enabled": "customerType = 'individual'",
  "unique_ids": ["customer_number"],
  "fields": [...]
}

Operation Modes

The mode field controls how entities are processed. By default, entities are upserted (created or updated).

Mode Options

Mode	Description
`upsert`	Create or update the entity (default behavior)
`delete`	Soft delete - marks entity as deleted but keeps in Recycle Bin for 30 days
`purge`	Hard delete - permanently removes from the system
`upsert-prune-scope-purge`	Upsert entities from array, then hard delete entities in scope that weren't upserted
`upsert-prune-scope-delete`	Upsert entities from array, then soft delete entities in scope that weren't upserted
`upsert-prune-scope`	(Meter readings only) Upsert readings from array, then delete all other readings for the same meter+counter that weren't upserted

note

The upsert-prune-scope-* modes require a scope configuration. See Prune Scope Operations. The upsert-prune-scope mode is for meter readings only and uses the meter+counter as natural scope. See Meter Reading Prune Scope Operations.

Entity Deletion

To delete entities, use mode: "delete" or mode: "purge":

{
  "entities": [
    {
      "entity_schema": "billing_event",
      "unique_ids": ["external_id"],
      "mode": "purge",
      "fields": [
        {
          "attribute": "external_id",
          "field": "billing_event_id"
        }
      ]
    }
  ]
}

For deletion modes, only the unique_ids fields need to be mapped in fields - other field mappings are ignored since no entity attributes are being updated.

Conditional Deletion

Combine mode with enabled for conditional deletion based on payload data:

{
  "entity_schema": "contract",
  "unique_ids": ["contract_number"],
  "mode": "delete",
  "enabled": "status = 'TERMINATED'",
  "fields": [
    { "attribute": "contract_number", "field": "contract_id" }
  ]
}

Prune Scope Operations

The upsert-prune-scope-purge and upsert-prune-scope-delete modes upsert all entities from an array in the payload, then delete/purge entities within a defined scope that weren't included in the upsert.

This is ideal for synchronizing child entity collections, such as billing events for a billing account.

Scope Configuration

Prune-scope modes require a scope configuration that defines which existing entities are eligible for deletion. The scope resolves against the original event payload (not individual array items).

scope_mode Options

Mode	Description
`relations`	Find scope by looking at all entities related to a specific entity (both direct and reverse relations)
`query`	Find scope entities directly via query parameters

Example: Sync Billing Events for a Billing Account

Sync all billing events for a billing account and remove any that are no longer in the payload:

{
  "entities": [
    {
      "entity_schema": "billing_event",
      "unique_ids": ["external_id"],
      "jsonataExpression": "$map(billingevents[], function($v) { $merge([$v, { \"billingaccountnumber\": billingaccountnumber }]) })",
      "mode": "upsert-prune-scope-purge",
      "scope": {
        "scope_mode": "relations",
        "schema": "billing_account",
        "unique_ids": [
          {
            "attribute": "billing_account_number",
            "field": "billingaccountnumber"
          }
        ]
      },
      "fields": [
        { "attribute": "external_id", "field": "billing_event_number" },
        { "attribute": "billing_account_number", "field": "billingaccountnumber" }
      ]
    }
  ]
}

Input:

{
  "billingaccountnumber": "002800699425",
  "billingevents": [
    { "billing_event_number": "002800699425-2025-08-15" },
    { "billing_event_number": "002800699425-2024-08-05" }
  ]
}

Result:

Upserts 2 billing_event entities with the specified external_ids
Finds all billing_event entities related to billing_account with billing_account_number = "002800699425"
Purges any billing_event entities in that scope that weren't in the upserted list

Example: Query Mode for Direct Matching

Find scope entities directly by query parameters instead of through relations:

{
  "entities": [
    {
      "entity_schema": "billing_event",
      "unique_ids": ["external_id"],
      "jsonataExpression": "$map(billingevents[], function($v) { $merge([$v, { \"billingaccountnumber\": billingaccountnumber }]) })",
      "mode": "upsert-prune-scope-purge",
      "scope": {
        "scope_mode": "query",
        "query": [
          {
            "attribute": "billing_account_number",
            "field": "billingaccountnumber"
          },
          {
            "attribute": "type",
            "constant": "INVOICE"
          }
        ]
      },
      "fields": [
        { "attribute": "external_id", "field": "billing_event_number" },
        { "attribute": "billing_account_number", "field": "billingaccountnumber" }
      ]
    }
  ]
}

warning

If the array yields zero entities (e.g., billingevents: []), this will result in the deletion of all entities in the scope. Ensure your payload always contains the expected data.

Meter Reading Prune Scope Operations

The upsert-prune-scope mode for meter readings upserts all readings from the payload for a given meter/counter, then permanently deletes all other readings for that meter/counter that weren't part of the upsert.

The scope is naturally defined by meter + counter — no explicit scope_mode is needed. An optional scope object can restrict pruning to readings from a specific source.

Scope Configuration (optional)

Property	Required	Description
`source`	No	When set, only readings with this source are eligible for pruning

Example: Basic Meter Reading Prune Scope

{
  "meter_readings": [
    {
      "jsonataExpression": "$.readings",
      "mode": "upsert-prune-scope",
      "meter": {
        "unique_ids": [{ "attribute": "external_id", "field": "meter_id" }]
      },
      "meter_counter": {
        "unique_ids": [{ "attribute": "external_id", "field": "counter_id" }]
      },
      "fields": [
        { "attribute": "external_id", "field": "reading_id" },
        { "attribute": "timestamp", "field": "read_at" },
        { "attribute": "source", "constant": "ERP" },
        { "attribute": "value", "field": "meter_value" }
      ]
    }
  ]
}

Upserts all readings from the readings array, then deletes any other readings for the same meter+counter not in the payload.

Example: With Source Scope

{
  "meter_readings": [
    {
      "jsonataExpression": "$.readings",
      "mode": "upsert-prune-scope",
      "scope": { "source": "ERP" },
      "meter": {
        "unique_ids": [{ "attribute": "external_id", "field": "meter_id" }]
      },
      "meter_counter": {
        "unique_ids": [{ "attribute": "external_id", "field": "counter_id" }]
      },
      "fields": [
        { "attribute": "external_id", "field": "reading_id" },
        { "attribute": "timestamp", "field": "read_at" },
        { "attribute": "source", "constant": "ERP" },
        { "attribute": "value", "field": "meter_value" }
      ]
    }
  ]
}

With source: "ERP", only readings with source: "ERP" are eligible for pruning. Manually entered or ECP readings remain untouched.

Important Notes

Scope: The scope is always all readings for the meter + counter.
Permanent Deletion: Meter reading deletion is always permanent (no soft delete/purge distinction).
Source Filter Recommended: Use source to avoid accidentally deleting readings from other sources (e.g., manually entered readings).
Empty Payload: If the payload yields zero readings, all readings in scope will be deleted (the ERP is treated as source of truth).
External ID: The external_id attribute is used to identify which readings to keep during pruning.

Mixed Operations

You can mix upsert and delete operations in the same use case by using multiple entity entries with different modes:

{
  "entities": [
    {
      "entity_schema": "meter",
      "unique_ids": ["external_id"],
      "mode": "upsert",
      "fields": [
        { "attribute": "external_id", "field": "meter_id" },
        { "attribute": "status", "constant": "DECOMMISSIONED" }
      ]
    },
    {
      "entity_schema": "billing_event",
      "unique_ids": ["external_id"],
      "mode": "purge",
      "jsonataExpression": "billing_events",
      "fields": [
        { "attribute": "external_id", "field": "event_id" }
      ]
    }
  ]
}

Updates the meter status to "DECOMMISSIONED" while purging associated billing events.

Field Mapping Priority

When multiple mapping options are specified, they are evaluated in this order:

info

constant -- Fixed value (highest priority)
jsonataExpression -- Computed value
field -- Direct field mapping (lowest priority)

Array Attribute Operations

Array attributes like _tags support operations to control how values are applied:

Set (`_set`)

Replaces the entire array:

{
  "attribute": "_tags",
  "constant": ["CUSTOMER", "VIP", "ACTIVE"]
}

Append (`_append`)

Adds new unique values to existing ones (with automatic deduplication):

{
  "attribute": "_tags",
  "jsonataExpression": "{ \"_append\": [\"NEW_TAG\", \"IMPORTED\"] }"
}

If ["EXISTING", "CUSTOMER"] already exists on the entity, appending ["CUSTOMER", "NEW_TAG"] results in ["EXISTING", "CUSTOMER", "NEW_TAG"] - the duplicate CUSTOMER is not added again.

Append All (`_append_all`)

Adds all values without deduplication:

{
  "attribute": "_tags",
  "jsonataExpression": "{ \"_append_all\": [\"AUDIT_LOG\", \"PROCESSED\"] }"
}

Best Practices

Use Meaningful Unique IDs

Choose stable identifiers that don't change:

// Good - stable business identifier
"unique_ids": ["customer_number"]

// Avoid - may change over time
"unique_ids": ["email"]

Handle Optional Fields

Use enabled to skip fields when data is missing:

{
  "attribute": "secondary_email",
  "field": "alternateEmail",
  "_type": "email",
  "enabled": "$exists(alternateEmail)"
}

Validate Data with JSONata

Transform and validate in one expression:

{
  "attribute": "status",
  "jsonataExpression": "status in ['active', 'inactive', 'pending'] ? status : 'unknown'"
}

Next Steps

Unique Identifiers - Advanced entity lookup strategies
Relations - Link entities together
Meter Readings - Handle meter reading data

Mapping

Mapping Configuration​

Entity Properties​

Field Mapping Types​

Direct Field Mapping​

Nested Field Access (JSONPath)​

JSONata Expressions​

Constant Values​

Enabled Field​

Repeatable Fields​

Email Fields​

Phone Fields​

Entity-Level JSONata​

Multiple Entities​

Dynamic Entity Expansion​

Conditional Entity Processing​

Operation Modes​

Mode Options​

note

Entity Deletion​

Conditional Deletion​

Prune Scope Operations​

Scope Configuration​

scope_mode Options​

Example: Sync Billing Events for a Billing Account​

Example: Query Mode for Direct Matching​

warning

Meter Reading Prune Scope Operations​

Scope Configuration (optional)​

Example: Basic Meter Reading Prune Scope​

Example: With Source Scope​

Important Notes​

Mixed Operations​

Field Mapping Priority​

info

Array Attribute Operations​

Set (_set)​

Append (_append)​

Append All (_append_all)​

Best Practices​

Use Meaningful Unique IDs​

Handle Optional Fields​

Validate Data with JSONata​

Next Steps​

Mapping Configuration

Entity Properties

Field Mapping Types

Direct Field Mapping

Nested Field Access (JSONPath)

JSONata Expressions

Constant Values

Enabled Field

Repeatable Fields

Email Fields

Phone Fields

Entity-Level JSONata

Multiple Entities

Dynamic Entity Expansion

Conditional Entity Processing

Operation Modes

Mode Options

Entity Deletion

Conditional Deletion

Prune Scope Operations

Scope Configuration

scope_mode Options

Example: Sync Billing Events for a Billing Account

Example: Query Mode for Direct Matching

Meter Reading Prune Scope Operations

Scope Configuration (optional)

Example: Basic Meter Reading Prune Scope

Example: With Source Scope

Important Notes

Mixed Operations

Field Mapping Priority

Array Attribute Operations

Set (`_set`)

Append (`_append`)

Append All (`_append_all`)

Best Practices

Use Meaningful Unique IDs

Handle Optional Fields

Validate Data with JSONata

Next Steps