Skip to main content
observiant.

Traceroutes (MTR)

Agents collect MTR traceroutes automatically for every ICMP target. How often depends on your plan. Each report records the path from one agent to the target, with packet loss and latency for every hop. Observiant adds each hop's reverse DNS name, autonomous system and country when it receives the report.

Each report traces up to 50 hops and sends 10 probes to each hop. The agent uses the target's address family and resolves hostnames itself, so the destination in a report is the address that agent actually reached.

The endpoints on this page let you:

  • list reports for a target
  • pick representative reports for a time window, for charts and timelines
  • find the report closest to a moment, such as a latency spike
  • fetch a full report

Traceroutes are also the input for path changes and correlated events.

List traceroutes​

GET /api/targets/{target_id}/mtr

Returns report summaries for a target, newest first.

ParameterDescription
agent_labelOptional. Only reports from this agent.
startOptional. Earliest report time, Unix milliseconds.
endOptional. Latest report time, Unix milliseconds.
limitOptional. Maximum number of reports, 1–500. Default 50. Larger values are reduced to 500.

Without start and end, the newest reports in the retention period are returned. When you give both, the range can be at most 30 days.

curl --header "Authorization: Bearer $OBSERVIANT_API_KEY" \
  "$OBSERVIANT_URL/api/targets/01M4MZ5TCN74PFRTXDKXYE4XWX/mtr?agent_label=sys-amsterdam&limit=2"
{
  "results": [
    {
      "result_id": "01M4N2XQ8ZKJ5W0D6V3T9R1B4E",
      "target_id": "01M4MZ5TCN74PFRTXDKXYE4XWX",
      "agent_label": "sys-amsterdam",
      "display_name": "Amsterdam",
      "timestamp": 1791706847120,
      "dst": "8.8.8.8",
      "hop_count": 6,
      "destination_reached": true,
      "as_path": ["AS64500", "AS64501", "AS15169"]
    },
    {
      "result_id": "01M4N2M3F1T7Q8H2C5Y0N6X9WA",
      "target_id": "01M4MZ5TCN74PFRTXDKXYE4XWX",
      "agent_label": "sys-amsterdam",
      "display_name": "Amsterdam",
      "timestamp": 1791706472316,
      "dst": "8.8.8.8",
      "hop_count": 6,
      "destination_reached": true,
      "as_path": ["AS64500", "AS64501", "AS15169"]
    }
  ]
}
FieldDescription
result_idReport ID. Use it with Get a traceroute report.
target_id, agent_label, display_nameThe target and the agent that ran the traceroute.
timestampWhen the traceroute started, Unix milliseconds.
dstDestination address the agent traced to.
hop_countNumber of hops in the report.
destination_reachedtrue when the last hop is the destination address.
as_pathAutonomous systems along the path, in order, without repeats and without hops whose AS is unknown.

An unknown target, or a target without reports, returns an empty results array.

Fetch older reports​

To page backwards through history, repeat the request with end set to one millisecond before the oldest timestamp you received. Stop when a response contains fewer reports than limit.

def all_reports(session, base_url, target_id, start, end):
    """Yield every report summary in [start, end], newest first. The range must be 30 days or less."""
    while True:
        response = session.get(
            f"{base_url}/api/targets/{target_id}/mtr",
            params={"start": start, "end": end, "limit": 500},
            timeout=30,
        )
        response.raise_for_status()
        results = response.json()["results"]
        yield from results
        if len(results) < 500:
            return
        end = results[-1]["timestamp"] - 1

Get representative traceroutes​

GET /api/targets/{target_id}/mtr/markers

Returns a bounded selection of reports that covers a whole time window evenly. Use it to annotate a chart or timeline without downloading every report. The window is divided into equal intervals, and at most one report is returned for each. Within an interval, Observiant picks the most informative report, in this order:

  1. a report where a path change was detected
  2. a report with packet loss at the destination, or one that did not reach it
  3. any other report

When several reports qualify equally, the newest one is returned.

ParameterDescription
startRequired. Window start, Unix milliseconds.
endRequired. Window end, Unix milliseconds. The window can be at most 366 days.
agent_labelOptional. Only reports from this agent.
limitOptional. Maximum number of reports, 1–500. Default 300.
END=$(date +%s000)
START=$(( END - 86400000 ))   # last 24 hours

curl --header "Authorization: Bearer $OBSERVIANT_API_KEY" \
  "$OBSERVIANT_URL/api/targets/01M4MZ5TCN74PFRTXDKXYE4XWX/mtr/markers?start=$START&end=$END&limit=100"
{
  "markers": [
    {
      "result_id": "01M4N1G7B2V9K3D8R5W0T6Y4QH",
      "timestamp": 1791702011845,
      "dst": "8.8.8.8",
      "hop_count": 7,
      "destination_reached": true,
      "destination_loss_pct": 20,
      "has_packet_loss": true,
      "has_event": true
    }
  ]
}

Markers are returned oldest first. The marker fields that differ from the list summary are:

FieldDescription
destination_loss_pctPacket loss at the destination, percent. 100 when the destination was not reached.
has_packet_losstrue when the destination had packet loss or was not reached.
has_eventtrue when this report confirmed a path change.

When you zoom into part of the window, request markers again with the narrower start and end to get a denser selection. A missing start or end returns 400 Bad Request.

Get the traceroute closest to a time​

GET /api/targets/{target_id}/mtr/nearest

Returns the full report whose start time is closest to timestamp, before or after it. Combined with latency and loss, this lets you go from a spike in a chart to the path at that moment.

ParameterDescription
timestampRequired. Unix milliseconds.
agent_labelOptional. Only consider reports from this agent. Recommended, because different agents take different paths.
curl --header "Authorization: Bearer $OBSERVIANT_API_KEY" \
  "$OBSERVIANT_URL/api/targets/01M4MZ5TCN74PFRTXDKXYE4XWX/mtr/nearest?timestamp=1791706800000&agent_label=sys-amsterdam"

The response has the same format as Get a traceroute report. If the target, or the agent, has no reports, the response is 404 Not Found with {"error": "No MTR results for this target"}. The closest report can be far from timestamp, so check its timestamp before you rely on it.

Get a traceroute report​

GET /api/mtr/{result_id}

Returns one complete report. result_id comes from the list, markers, nearest, path change and event responses.

curl --header "Authorization: Bearer $OBSERVIANT_API_KEY" \
  "$OBSERVIANT_URL/api/mtr/01M4N2XQ8ZKJ5W0D6V3T9R1B4E"
{
  "result_id": "01M4N2XQ8ZKJ5W0D6V3T9R1B4E",
  "target_id": "01M4MZ5TCN74PFRTXDKXYE4XWX",
  "agent_label": "sys-amsterdam",
  "display_name": "Amsterdam",
  "timestamp": 1791706847120,
  "report": {
    "mtr": {
      "src": "198.51.100.200",
      "dst": "8.8.8.8",
      "tos": 0,
      "tests": 10,
      "psize": "64",
      "bitpattern": "0x00"
    },
    "hubs": [
      {"count": 1, "host": "192.0.2.1", "hostname": "", "host_country": "", "host_asn": "",
       "loss_pct": 0, "snt": 10, "last": 0.41, "avg": 0.44, "best": 0.38, "wrst": 0.61, "stdev": 0.07},
      {"count": 2, "host": "198.51.100.17", "hostname": "ae1-10.ams-core1.example.net", "host_country": "NL", "host_asn": "AS64500",
       "loss_pct": 0, "snt": 10, "last": 1.18, "avg": 1.22, "best": 1.1, "wrst": 1.47, "stdev": 0.11},
      {"count": 3, "host": "198.51.100.66", "hostname": "be20.ams-edge2.example.net", "host_country": "NL", "host_asn": "AS64500",
       "loss_pct": 0, "snt": 10, "last": 1.87, "avg": 1.93, "best": 1.81, "wrst": 2.25, "stdev": 0.13},
      {"count": 4, "host": "???", "hostname": "", "host_country": "", "host_asn": "",
       "loss_pct": 100, "snt": 10, "last": 0, "avg": 0, "best": 0, "wrst": 0, "stdev": 0},
      {"count": 5, "host": "203.0.113.9", "hostname": "", "host_country": "NL", "host_asn": "AS64501",
       "loss_pct": 0, "snt": 10, "last": 2.79, "avg": 2.84, "best": 2.71, "wrst": 3.06, "stdev": 0.1},
      {"count": 6, "host": "8.8.8.8", "hostname": "dns.google", "host_country": "US", "host_asn": "AS15169",
       "loss_pct": 0, "snt": 10, "last": 2.62, "avg": 2.64, "best": 2.55, "wrst": 2.9, "stdev": 0.09}
    ]
  }
}

Report fields​

FieldDescription
result_id, target_id, agent_label, display_name, timestampAs in the list summary.
report.mtr.srcThe agent's address as seen by Observiant. Behind NAT this is the translated public address, not the address of the agent's interface.
report.mtr.dstDestination address.
report.mtr.tosIP type-of-service value of the probes.
report.mtr.testsNumber of probe cycles sent to each hop.
report.mtr.psizeProbe packet size in bytes, as a string.
report.mtr.bitpatternByte used to fill the probe payload, as a hexadecimal string such as 0x00.
report.hubsHops in path order. The first element is hop 1.

Each hop has these fields:

FieldDescription
countHop number, starting at 1.
hostAddress that answered at this hop, or ??? when no reply was received.
hostnameReverse DNS name of host when the report was received, or an empty string.
host_asnAutonomous system of host, such as AS64500, or an empty string.
host_countryISO 3166-1 alpha-2 country code of host, or an empty string.
loss_pctPercentage of probes to this hop that got no reply.
sntNumber of probes sent to this hop.
last, avg, best, wrstLast, average, best and worst round-trip time, milliseconds.
stdevStandard deviation of the round-trip time, milliseconds.

The report follows the JSON format of the mtr tool.

Reading loss in a report​

Loss at an intermediate hop is not always a problem. Many routers rate-limit the ICMP replies they generate themselves while forwarding traffic normally. Treat loss as real forwarding loss when it starts at a hop and continues through every later hop to the destination. A loss figure that appears at one hop and disappears at the next is usually rate limiting. Correlated events apply this rule automatically across targets and agents.