Loading catalog…
Product briefing
ArcoreConduit
ProductionData & Intelligence
An AI-augmented data discovery, extraction, and normalization engine. ArcoreConduit discovers sources from natural-language prompts, ingests from APIs and the web with multiple crawler strategies, and routes normalized data into dynamically modeled warehouse targets.
API: Available
Endpoints
12
Key features
6
Stack components
11
Controls noted
6
What changed
May 15, 2026: Added governance-aligned copy, compare tooling support, and transparency notes.
Key Features
- 1LLM-driven discovery of new data source templates
- 2Polyglot retrieval via APIs, RSS, BeautifulSoup, Scrapy, and Playwright
- 3Dynamic schema generation and database routing
- 4AI output normalization for unstructured payloads
- 5Celery-based scheduled ingestion and asynchronous execution
- 6Infisical-backed credential injection and schema-drift detection
API Endpoints
| Method | Path | Description |
|---|---|---|
| GET | /api/datasources/ | List all data sources |
| POST | /api/datasources/ | Create new data source |
| POST | /api/datasources/{id}/run_now/ | Trigger immediate extraction |
| GET | /api/datasources/active/ | List active sources only |
| GET | /api/normalized/crypto-prices/ | Cryptocurrency price data |
| GET | /api/normalized/stock-quotes/ | Stock market data |
| GET | /api/normalized/llm-outputs/ | LLM processing results |
| GET | /api/monitoring/job-logs/ | Task execution history |
| GET | /api/monitoring/health-metrics/ | Aggregated health metrics |
| GET | /api/discovery/templates/ | Data source templates |
| POST | /api/discovery/test-template/{id}/ | Test template connectivity |
| GET | /api/health/ | System health check endpoint |
Usage Example
python
import requests
# Example interaction: Create data source
response = requests.post(
url="https://api.arcore.internal/api/datasources/",
headers={"Authorization": "Bearer <token>"},
json={
"name": "CoinGecko BTC Price",
"source_type": "API",
"endpoint": "https://api.coingecko.com/api/v3/simple/price?ids=bitcoin&vs_currencies=usd",
"parser_function": "parsers.crypto_parser",
"schedule_cron": "*/15 * * * *"
}
)
print(response.json())
# Trigger immediate extraction
response = requests.post(
url="https://api.arcore.internal/api/datasources/123/run_now/",
headers={"Authorization": "Bearer <token>"}
)
print(response.json())Recommended next products
Tech Stack
React 18DjangoPostgreSQLRedis/CeleryScrapyBeautifulSoup4PlaywrightGoogle GeminiOpenAI GPTInfisicalDocker
Authentication
- •**Header:** `Authorization: Bearer <token>`
- •**Scopes:** RBAC is enforced at the object level via `ArcoreCodex` policies.
- •**Infisical Integration:** External API keys stored in Infisical vault
Compliance & Security
Compliance
- ✓Secrets Management: API credentials are referenced via Infisical rather than stored raw in the app database
- ✓Data Integrity: Schema drift is detected via hash checks before downstream tables are polluted
- ✓Isolation: Headless browser workers are intended to run in isolated containers
- ✓Audit Logging: Task execution history and normalization flows are retained for review
- ✓Encryption: TLS 1.3 transit, AES-256 rest
Security
- ✓Encryption: TLS 1.3 transit, AES-256 rest
Related Products
Arcore Maestro
Arcore Maestro is an AI workflow orchestration platform that routes tasks through LangGraph-based workflows, local or remote LLMs, and sandboxed worker services. It provides a unified control plane for defining, executing, and monitoring multi-step AI pipelines across the Arcore ecosystem.
Graph-based workflow orchestration with conditional branching
CRUD management for services, models, workflows, graphs, and templates
PythonFastAPIReact
6 endpointsOpen
local-llm-server
A production-ready local model serving platform with an OpenAI-compatible API, orchestration endpoints for model lifecycle management, deep performance telemetry, and an optional Vector-Tester companion UI for staged testing and crash-log capture.
Model registry with load and unload orchestration
OpenAI-compatible and native streaming inference endpoints
PythonFastAPISQLModel
6 endpointsOpen
Chapterize
An intelligent PDF processing platform that converts unstructured books and documents into structured chapters with rich HTML, text, and images. Chapterize combines a Django + React web application with a Python CLI toolkit for batch processing and downstream RAG-friendly exports.
TOC-first chapter detection with regex fallback
Layout-aware extraction of text, images, and tabular regions
PythonDjangoReact
5 endpointsOpen