curl -X POST https://<your-instance>/v1/sdk/import-pipeline \
-H "Authorization: Bearer trvh_sdk_your_key_here" \
-H "Content-Type: application/json" \
-d '{
"name": "ADP - MLS Property Ingestion",
"category": "Real Estate",
"tags": ["adp", "mls"],
"source": {
"connectionId": "D2026A00003",
"type": "api",
"config": {
"endpoint": "/v2/property/snapshot",
"paginated": true,
"paginationKey": "pagetoken",
"rootKey": "property"
}
},
"destination": {
"connectionId": "D2026A00002",
"vendor": "POSTGRESQL",
"schemaName": "mls_data"
},
"steps": [
{
"type": "transform",
"name": "Map & Coerce Fields",
"config": {
"fieldMappings": [
{"source": "address.line1", "target": "address_line1"},
{"source": "summary.yearBuilt", "target": "year_built"}
],
"rules": [
{"field": "year_built", "rule": "to_integer"}
]
}
},
{
"type": "validate",
"name": "Validate Required Fields",
"config": {
"rules": [
{"field": "attom_id", "rule": "not_null"},
{"field": "zip", "rule": "regex", "params": {"pattern": "^\\d{5}$"}}
]
}
},
{
"type": "load",
"name": "Load to Postgres",
"config": { "tableName": "mls_properties_with_address" }
},
{
"type": "reconcile",
"name": "ADP Reconciliation",
"config": {
"mappingSource": "default",
"compareRowCounts": true,
"checkFieldCompleteness": true,
"screenMapEnabled": true
}
}
],
"schedule": {
"isEnabled": true,
"frequency": "daily",
"startTime": "02:00"
}
}'
{
"success": true,
"message": "Pipeline imported successfully.",
"data": {
"pipelineId": "DP-1717000000000",
"name": "ADP - MLS Property Ingestion",
"status": "draft",
"steps": [
{"id": "step-1717000000000-0", "type": "source", "name": "Extract MLS API", "status": "configured"},
{"id": "step-1717000000000-1", "type": "transform", "name": "Map & Coerce Fields", "status": "configured"},
{"id": "step-1717000000000-2", "type": "validate", "name": "Validate Required Fields", "status": "configured"},
{"id": "step-1717000000000-3", "type": "load", "name": "Load to Postgres", "status": "configured"},
{"id": "step-1717000000000-4", "type": "reconcile", "name": "ADP Reconciliation", "status": "configured"}
]
}
}
{
"success": false,
"message": "Pipeline JSON must include a name."
}
{
"error": "API key required"
}
SDK API
Import Data Pipeline
Create a complete data pipeline from a JSON definition (API key auth)
POST
/
v1
/
sdk
/
import-pipeline
curl -X POST https://<your-instance>/v1/sdk/import-pipeline \
-H "Authorization: Bearer trvh_sdk_your_key_here" \
-H "Content-Type: application/json" \
-d '{
"name": "ADP - MLS Property Ingestion",
"category": "Real Estate",
"tags": ["adp", "mls"],
"source": {
"connectionId": "D2026A00003",
"type": "api",
"config": {
"endpoint": "/v2/property/snapshot",
"paginated": true,
"paginationKey": "pagetoken",
"rootKey": "property"
}
},
"destination": {
"connectionId": "D2026A00002",
"vendor": "POSTGRESQL",
"schemaName": "mls_data"
},
"steps": [
{
"type": "transform",
"name": "Map & Coerce Fields",
"config": {
"fieldMappings": [
{"source": "address.line1", "target": "address_line1"},
{"source": "summary.yearBuilt", "target": "year_built"}
],
"rules": [
{"field": "year_built", "rule": "to_integer"}
]
}
},
{
"type": "validate",
"name": "Validate Required Fields",
"config": {
"rules": [
{"field": "attom_id", "rule": "not_null"},
{"field": "zip", "rule": "regex", "params": {"pattern": "^\\d{5}$"}}
]
}
},
{
"type": "load",
"name": "Load to Postgres",
"config": { "tableName": "mls_properties_with_address" }
},
{
"type": "reconcile",
"name": "ADP Reconciliation",
"config": {
"mappingSource": "default",
"compareRowCounts": true,
"checkFieldCompleteness": true,
"screenMapEnabled": true
}
}
],
"schedule": {
"isEnabled": true,
"frequency": "daily",
"startTime": "02:00"
}
}'
{
"success": true,
"message": "Pipeline imported successfully.",
"data": {
"pipelineId": "DP-1717000000000",
"name": "ADP - MLS Property Ingestion",
"status": "draft",
"steps": [
{"id": "step-1717000000000-0", "type": "source", "name": "Extract MLS API", "status": "configured"},
{"id": "step-1717000000000-1", "type": "transform", "name": "Map & Coerce Fields", "status": "configured"},
{"id": "step-1717000000000-2", "type": "validate", "name": "Validate Required Fields", "status": "configured"},
{"id": "step-1717000000000-3", "type": "load", "name": "Load to Postgres", "status": "configured"},
{"id": "step-1717000000000-4", "type": "reconcile", "name": "ADP Reconciliation", "status": "configured"}
]
}
}
{
"success": false,
"message": "Pipeline JSON must include a name."
}
{
"error": "API key required"
}
Single-shot pipeline creation. Accepts a declarative payload describing pipeline steps, schedule, and metadata — returns the created pipeline ID.
This endpoint uses API key authentication. Pass your key via
Authorization: Bearer trvh_sdk_... or x-api-key header.Request Body
Where data comes FROM. This connection is inherited by all
source type steps.Show source properties
Show source properties
Source-level defaults inherited by
source steps. For API sources: endpoint, method, paginated, paginationKey, rootKey. For DB sources: schemaName, tableName.Where data goes TO. This connection is inherited by
load, stage, and reconcile steps.Ordered array of pipeline step definitions. Steps execute sequentially — each step depends on the previous one.
Optional schedule for automatic execution.
Step Type Configs
Source
Extract data from a database, API, file, or S3 bucket. InheritsconnectionId and connection config from the top-level source object. You typically don’t need a source step in the steps array — one is auto-generated from the top-level source definition. Only add explicit source steps when you need multiple sources (e.g., a second API call mid-pipeline for enrichment).
Override connection ID (for additional sources beyond the primary). If omitted, inherits from top-level
source.connectionId.Transform
Field mapping and data transformation rules.Validate
Data validation rules.Quality
Data quality checks.DB Lookup
Read from another database for reference/enrichment.Stage
Write to a staging/temporary table for intermediate processing.Load
Write to the final destination database (Postgres for v1). InheritsconnectionId, vendor, and schemaName from the top-level destination object.
Reconcile
Post-load data reconciliation — compares source vs destination."default" uses the built-in ADP screen map. "custom" expects a mapping file at execution time.Response
curl -X POST https://<your-instance>/v1/sdk/import-pipeline \
-H "Authorization: Bearer trvh_sdk_your_key_here" \
-H "Content-Type: application/json" \
-d '{
"name": "ADP - MLS Property Ingestion",
"category": "Real Estate",
"tags": ["adp", "mls"],
"source": {
"connectionId": "D2026A00003",
"type": "api",
"config": {
"endpoint": "/v2/property/snapshot",
"paginated": true,
"paginationKey": "pagetoken",
"rootKey": "property"
}
},
"destination": {
"connectionId": "D2026A00002",
"vendor": "POSTGRESQL",
"schemaName": "mls_data"
},
"steps": [
{
"type": "transform",
"name": "Map & Coerce Fields",
"config": {
"fieldMappings": [
{"source": "address.line1", "target": "address_line1"},
{"source": "summary.yearBuilt", "target": "year_built"}
],
"rules": [
{"field": "year_built", "rule": "to_integer"}
]
}
},
{
"type": "validate",
"name": "Validate Required Fields",
"config": {
"rules": [
{"field": "attom_id", "rule": "not_null"},
{"field": "zip", "rule": "regex", "params": {"pattern": "^\\d{5}$"}}
]
}
},
{
"type": "load",
"name": "Load to Postgres",
"config": { "tableName": "mls_properties_with_address" }
},
{
"type": "reconcile",
"name": "ADP Reconciliation",
"config": {
"mappingSource": "default",
"compareRowCounts": true,
"checkFieldCompleteness": true,
"screenMapEnabled": true
}
}
],
"schedule": {
"isEnabled": true,
"frequency": "daily",
"startTime": "02:00"
}
}'
{
"success": true,
"message": "Pipeline imported successfully.",
"data": {
"pipelineId": "DP-1717000000000",
"name": "ADP - MLS Property Ingestion",
"status": "draft",
"steps": [
{"id": "step-1717000000000-0", "type": "source", "name": "Extract MLS API", "status": "configured"},
{"id": "step-1717000000000-1", "type": "transform", "name": "Map & Coerce Fields", "status": "configured"},
{"id": "step-1717000000000-2", "type": "validate", "name": "Validate Required Fields", "status": "configured"},
{"id": "step-1717000000000-3", "type": "load", "name": "Load to Postgres", "status": "configured"},
{"id": "step-1717000000000-4", "type": "reconcile", "name": "ADP Reconciliation", "status": "configured"}
]
}
}
{
"success": false,
"message": "Pipeline JSON must include a name."
}
{
"error": "API key required"
}