Discovery

Discovery helps when you have real payloads but no finished data model or routing definition. It captures a bounded sample set, parses candidate structures, and lets you approve the fields you want.

Discovery lifecycle#

1. Open an ingest client and start discovery.
2. Send representative messages to the discovery endpoint or capture samples from the console.
3. Review received, parsed, failed, dropped, and stored counts.
4. Compare generated candidates and their confidence.
5. Select a candidate, model name/key, destinations, and fields.
6. Approve the candidate.
7. ReportPlane creates the model and replays eligible stored samples.

Session states#

A session can be waiting, expired, stopped, or approved. It has configured time, sample-count, and byte limits. When a limit is reached, additional samples can be dropped rather than growing storage without bound.

HTTP discovery payloads#

The /discover runtime accepts one JSON object or an array of objects. The body limit is 10 MB. An empty array, invalid JSON, scalar value, or array containing non-object items is rejected.

Discovery must be active and waiting. Otherwise the runtime returns a conflict response.

Review candidates#

A candidate can include:

• parser and source format;
• match field, operator, and value;
• extractor type and pattern;
• inferred fields and types;
• app name and observed hostname;
• confidence and sample count.

Treat inference as a proposal. Remove unstable fields, correct types, and add descriptions before approval.

Replay#

After approval, ReportPlane attempts to replay stored samples through the created model and reports replay success/failure counts. Replay does not make malformed samples valid.