Neon Deploy: Learn how Neon helps developers ship faster with Postgres. Grab your ticket
Docs/Integrations/Datadog Integration

The Neon Datadog integration

new

Send metrics and events from Neon Postgres to Datadog

What you will learn:

  • How to set up the integration

  • The full list of externally-available metrics

Available for Scale and Business Plan users, the Neon Datadog integration lets you monitor Neon database performance, resource utilization, and system health directly from Datadog's observability platform.

Feature coming soon

This feature is available by invitation only for select early-access customers. It is experimental and subject to change.

How it works

The integration leverages a list of metrics that Neon makes available for export to third-party services. By configuring the integration with your Datadog API key, Neon automatically sends metrics from your project to your selected Datadog site. Some of the key metrics include:

  • Connection counts — Tracks active and idle database connections.
  • Database size — Monitors total size of all databases in bytes.
  • Replication delay — Measures replication lag in bytes and seconds.
  • Compute metrics — Includes CPU and memory usage statistics for your compute.

note

Metrics are sent for all computes in your Neon project. For example, if you have multiple branches, each with an attached compute, metrics will be collected and sent for each compute.

Prerequisites

Before getting started, ensure the following:

Steps to integrate Datadog with Neon

  1. In the Neon Console, navigate to the Integrations page in your Neon project.
  2. Locate the Datadog card and click Add.
  3. Enter your Datadog API key. You can generate or retrieve Datadog API Keys from your Datadog organization.
  4. Select the Datadog site that you used when setting up your Datadog account.
  5. Click Confirm to complete the integration.

Optionally, you can import the Neon-provided JSON configuration file into Datadog, which creates a pre-built dashboard from Neon metrics, similar to the charts available on our Monitoring page. See Import Neon dashboard

Once set up, Neon will start sending metrics to Datadog, and you can use these metrics to create custom dashboards and alerts in Datadog.

note

Neon computes only send metrics when they are active. If the Autosuspend feature is enabled and a compute is suspended due to inactivity, no metrics will be sent during the suspension. This may result in gaps in your Datadog metrics. If you notice missing data in Datadog, check if your compute is suspended. You can verify a compute's status as Idle or Active on the Branches page in the Neon console, and review Suspend compute events on the System operations tab of the Monitoring page.

Additionally, if you are setting up Neon’s Datadog integration for a project with an inactive compute, you'll need to activate the compute before it can send metrics to Datadog. To activate it, simply run a query from the Neon SQL Editor or any connected client on the branch associated with the compute.

Example usage in Datadog

Once integrated, you can create custom dashboards in Datadog by querying the metrics sent from Neon. Use Datadog's Metrics Explorer to search for metrics like neon_connection_counts, neon_db_total_size, and host_cpu_seconds_total. You can also set alerts based on threshold values for critical metrics.

Import the Neon dashboard

As part of the integration, Neon provides a JSON configuration file that you can import into Datadog to start with a pre-built dashboard based on a subset of Neon metrics.

neon dashboard in datadog

Here's how you can import the dashboard:

  1. In the Neon Console, open your Datadog integration from the Integrations page.

  2. Scroll to the bottom of the panel and copy the JSON from there.

    OR

    You can copy the JSON below instead.

  3. Next, create a new dashboard in Datadog.

  4. Open Configure, select Import dashboard JSON, then paste the Neon-provided configuration JSON.

If any of the computes in your project are active, you should start seeing data in the resulting charts right away. By default, the charts show metrics for all active endpoints in your project. You can filter results to one or more selected endpoints using the endpoint_id variable dropdown selector.

select endpoint variable in dashboard

Dashboard JSON

Copy JSON configuration
{
  "title": "Single Neon Compute metrics (with dropdown)",
  "description": "",
  "widgets": [
    {
      "id": 3831219857468963,
      "definition": {
        "title": "RAM",
        "title_size": "16",
        "title_align": "left",
        "show_legend": true,
        "legend_layout": "auto",
        "legend_columns": [
          "avg",
          "min",
          "max",
          "value",
          "sum"
        ],
        "time": {},
        "type": "timeseries",
        "requests": [
          {
            "formulas": [
              {
                "number_format": {
                  "unit": {
                    "type": "canonical_unit",
                    "unit_name": "byte"
                  }
                },
                "alias": "Cached",
                "formula": "query3"
              },
              {
                "alias": "Used",
                "number_format": {
                  "unit": {
                    "type": "canonical_unit",
                    "unit_name": "byte"
                  }
                },
                "formula": "query1 - query2"
              }
            ],
            "queries": [
              {
                "name": "query3",
                "data_source": "metrics",
                "query": "max:host_memory_cached_bytes{$endpoint_id}"
              },
              {
                "name": "query1",
                "data_source": "metrics",
                "query": "max:host_memory_total_bytes{$endpoint_id}"
              },
              {
                "name": "query2",
                "data_source": "metrics",
                "query": "max:host_memory_available_bytes{$endpoint_id}"
              }
            ],
            "response_format": "timeseries",
            "style": {
              "palette": "dog_classic",
              "order_by": "values",
              "line_type": "solid",
              "line_width": "normal"
            },
            "display_type": "line"
          }
        ]
      },
      "layout": {
        "x": 0,
        "y": 0,
        "width": 6,
        "height": 2
      }
    },
    {
      "id": 7296782684811837,
      "definition": {
        "title": "CPU",
        "title_size": "16",
        "title_align": "left",
        "show_legend": true,
        "legend_layout": "auto",
        "legend_columns": [
          "avg",
          "min",
          "max",
          "value",
          "sum"
        ],
        "time": {},
        "type": "timeseries",
        "requests": [
          {
            "formulas": [
              {
                "alias": "Used",
                "formula": "per_minute(query1)"
              }
            ],
            "queries": [
              {
                "name": "query1",
                "data_source": "metrics",
                "query": "max:host_cpu_seconds_total{!mode:idle,$endpoint_id}.as_rate()"
              }
            ],
            "response_format": "timeseries",
            "style": {
              "palette": "dog_classic",
              "order_by": "values",
              "line_type": "solid",
              "line_width": "normal"
            },
            "display_type": "line"
          }
        ]
      },
      "layout": {
        "x": 6,
        "y": 0,
        "width": 6,
        "height": 2
      }
    },
    {
      "id": 7513607855022102,
      "definition": {
        "title": "Connections",
        "title_size": "16",
        "title_align": "left",
        "show_legend": true,
        "legend_layout": "auto",
        "legend_columns": [
          "avg",
          "min",
          "max",
          "value",
          "sum"
        ],
        "type": "timeseries",
        "requests": [
          {
            "formulas": [
              {
                "alias": "Total",
                "formula": "query1"
              },
              {
                "alias": "Active",
                "formula": "query2"
              },
              {
                "alias": "Idle",
                "formula": "query3"
              }
            ],
            "queries": [
              {
                "name": "query1",
                "data_source": "metrics",
                "query": "sum:neon_connection_counts{!datname:postgres,$endpoint_id}"
              },
              {
                "name": "query2",
                "data_source": "metrics",
                "query": "sum:neon_connection_counts{!datname:postgres,state:active ,$endpoint_id}"
              },
              {
                "name": "query3",
                "data_source": "metrics",
                "query": "sum:neon_connection_counts{!datname:postgres,!state:active,$endpoint_id}"
              }
            ],
            "response_format": "timeseries",
            "style": {
              "palette": "dog_classic",
              "order_by": "values",
              "line_type": "solid",
              "line_width": "normal"
            },
            "display_type": "line"
          }
        ]
      },
      "layout": {
        "x": 0,
        "y": 2,
        "width": 6,
        "height": 3
      }
    },
    {
      "id": 5523349536895199,
      "definition": {
        "title": "Database size",
        "title_size": "16",
        "title_align": "left",
        "show_legend": true,
        "legend_layout": "auto",
        "legend_columns": [
          "avg",
          "min",
          "max",
          "value",
          "sum"
        ],
        "type": "timeseries",
        "requests": [
          {
            "formulas": [
              {
                "number_format": {
                  "unit": {
                    "type": "canonical_unit",
                    "unit_name": "byte"
                  }
                },
                "formula": "query2"
              },
              {
                "number_format": {
                  "unit": {
                    "type": "canonical_unit",
                    "unit_name": "byte"
                  }
                },
                "alias": "Size of all databases",
                "formula": "query3"
              },
              {
                "alias": "Max size",
                "number_format": {
                  "unit": {
                    "type": "canonical_unit",
                    "unit_name": "byte"
                  }
                },
                "formula": "query1 * 1024 * 1024"
              }
            ],
            "queries": [
              {
                "name": "query2",
                "data_source": "metrics",
                "query": "max:neon_pg_stats_userdb{kind:db_size,$endpoint_id} by {datname}"
              },
              {
                "name": "query3",
                "data_source": "metrics",
                "query": "max:neon_db_total_size{$endpoint_id}"
              },
              {
                "name": "query1",
                "data_source": "metrics",
                "query": "max:neon_max_cluster_size{$endpoint_id}"
              }
            ],
            "response_format": "timeseries",
            "style": {
              "palette": "dog_classic",
              "order_by": "values",
              "line_type": "solid",
              "line_width": "normal"
            },
            "display_type": "line"
          }
        ],
        "yaxis": {
          "include_zero": false,
          "scale": "log"
        }
      },
      "layout": {
        "x": 6,
        "y": 2,
        "width": 6,
        "height": 3
      }
    },
    {
      "id": 1608572645458648,
      "definition": {
        "title": "Deadlocks",
        "title_size": "16",
        "title_align": "left",
        "show_legend": true,
        "legend_layout": "auto",
        "legend_columns": [
          "avg",
          "min",
          "max",
          "value",
          "sum"
        ],
        "type": "timeseries",
        "requests": [
          {
            "formulas": [
              {
                "alias": "Deadlocks",
                "formula": "query1"
              }
            ],
            "queries": [
              {
                "name": "query1",
                "data_source": "metrics",
                "query": "max:neon_pg_stats_userdb{kind:deadlocks,$endpoint_id} by {datname}"
              }
            ],
            "response_format": "timeseries",
            "style": {
              "palette": "dog_classic",
              "order_by": "values",
              "line_type": "solid",
              "line_width": "normal"
            },
            "display_type": "line"
          }
        ]
      },
      "layout": {
        "x": 0,
        "y": 5,
        "width": 6,
        "height": 2
      }
    },
    {
      "id": 5728659221127513,
      "definition": {
        "title": "Changed rows",
        "title_size": "16",
        "title_align": "left",
        "show_legend": true,
        "legend_layout": "auto",
        "legend_columns": [
          "avg",
          "min",
          "max",
          "value",
          "sum"
        ],
        "type": "timeseries",
        "requests": [
          {
            "formulas": [
              {
                "alias": "Rows inserted",
                "formula": "diff(query1)"
              },
              {
                "alias": "Rows deleted",
                "formula": "diff(query2)"
              },
              {
                "alias": "Rows updated",
                "formula": "diff(query3)"
              }
            ],
            "queries": [
              {
                "name": "query1",
                "data_source": "metrics",
                "query": "max:neon_pg_stats_userdb{kind:inserted,$endpoint_id}"
              },
              {
                "name": "query2",
                "data_source": "metrics",
                "query": "max:neon_pg_stats_userdb{kind:deleted,$endpoint_id}"
              },
              {
                "name": "query3",
                "data_source": "metrics",
                "query": "max:neon_pg_stats_userdb{kind:updated,$endpoint_id}"
              }
            ],
            "response_format": "timeseries",
            "style": {
              "palette": "dog_classic",
              "order_by": "values",
              "line_type": "solid",
              "line_width": "normal"
            },
            "display_type": "line"
          }
        ]
      },
      "layout": {
        "x": 6,
        "y": 5,
        "width": 6,
        "height": 2
      }
    },
    {
      "id": 630770240665422,
      "definition": {
        "title": "Local file cache hit rate",
        "title_size": "16",
        "title_align": "left",
        "show_legend": true,
        "legend_layout": "auto",
        "legend_columns": [
          "avg",
          "min",
          "max",
          "value",
          "sum"
        ],
        "time": {},
        "type": "timeseries",
        "requests": [
          {
            "formulas": [
              {
                "alias": "Cache hit rate",
                "formula": "query1 / (query1 + query2)",
                "number_format": {
                  "unit": {
                    "type": "canonical_unit",
                    "unit_name": "fraction"
                  }
                }
              }
            ],
            "queries": [
              {
                "name": "query1",
                "data_source": "metrics",
                "query": "max:neon_lfc_hits{$endpoint_id}"
              },
              {
                "name": "query2",
                "data_source": "metrics",
                "query": "max:neon_lfc_misses{$endpoint_id}"
              }
            ],
            "response_format": "timeseries",
            "style": {
              "palette": "dog_classic",
              "order_by": "values",
              "line_type": "solid",
              "line_width": "normal"
            },
            "display_type": "line"
          }
        ]
      },
      "layout": {
        "x": 0,
        "y": 7,
        "width": 6,
        "height": 3
      }
    },
    {
      "id": 2040733022455075,
      "definition": {
        "title": "Working set size",
        "title_size": "16",
        "title_align": "left",
        "show_legend": true,
        "legend_layout": "auto",
        "legend_columns": [
          "avg",
          "min",
          "max",
          "value",
          "sum"
        ],
        "time": {},
        "type": "timeseries",
        "requests": [
          {
            "formulas": [
              {
                "alias": "Local file cache size",
                "number_format": {
                  "unit": {
                    "type": "canonical_unit",
                    "unit_name": "byte"
                  }
                },
                "formula": "query2"
              },
              {
                "number_format": {
                  "unit": {
                    "type": "canonical_unit",
                    "unit_name": "byte"
                  }
                },
                "formula": "8192 * query1"
              }
            ],
            "queries": [
              {
                "name": "query2",
                "data_source": "metrics",
                "query": "max:neon_lfc_cache_size_limit{$endpoint_id}"
              },
              {
                "name": "query1",
                "data_source": "metrics",
                "query": "max:neon_lfc_approximate_working_set_size_windows{$endpoint_id} by {duration}"
              }
            ],
            "response_format": "timeseries",
            "style": {
              "palette": "dog_classic",
              "order_by": "values",
              "line_type": "solid",
              "line_width": "normal"
            },
            "display_type": "line"
          }
        ]
      },
      "layout": {
        "x": 6,
        "y": 7,
        "width": 6,
        "height": 3
      }
    }
  ],
  "template_variables": [
    {
      "name": "endpoint_id",
      "prefix": "endpoint_id",
      "available_values": [],
      "default": "*"
    },
    {
      "name": "project_id",
      "prefix": "project_id",
      "available_values": [],
      "default": "*"
    },
    {
      "name": "state",
      "prefix": "state",
      "available_values": [],
      "default": "*"
    }
  ],
  "layout_type": "ordered",
  "notify_list": [],
  "reflow_type": "fixed"
}

Available metrics

Neon makes the following metrics available for export to third parties; for now, availability is limited to the Datadog integration but will soon be expanded to other providers.

All metrics include the following labels:

  • project_id
  • endpoint_id
  • compute_id
  • job

Here's an example of the metric db_total_size with all labels:

neon_db_total_size{project_id="square-daffodil-12345678", endpoint_id="ep-aged-art-260862", compute_id="compute-shrill-blaze-b4hry7fg", job="sql-metrics"} 10485760

note

In Datadog, metric labels are referred to as tags. See Getting Started with Tags in the Datadog Docs.

NameJobDescription
neon_connection_countssql-metricsTotal number of database connections. The state label indicates whether the connection is active (executing queries), idle (awaiting queries), or in a variety of other states derived from the pg_stat_activity Postgres view.
neon_db_total_sizesql-metricsTotal size of all databases in your project, measured in bytes.
neon_lfc_approximate_working_set_size_windowssql-metricsApproximate working set size in pages of 8192 bytes. The metric is tracked over time windows (5m, 15m, 1h) to gauge access patterns. Duration values: duration="5m", duration="15m", duration="1h".
neon_lfc_cache_size_limitsql-metricsThe limit on the size of the Local File Cache (LFC), measured in bytes.
neon_lfc_hitssql-metricsThe number of times requested data was found in the LFC (cache hit). Higher cache hit rates indicate efficient memory use.
neon_lfc_missessql-metricsThe number of times requested data was not found in the LFC (cache miss), forcing a read from slower storage. High miss rates may indicate insufficient compute size.
neon_lfc_usedsql-metricsThe amount of space currently used in the LFC, measured in 1MB chunks. It reflects how much of the cache limit is being utilized.
neon_lfc_writessql-metricsThe number of write operations to the LFC.
neon_max_cluster_sizesql-metricsThe neon.max_cluster_size setting in MB.
neon_pg_stats_userdbsql-metricsAggregated metrics from the pg_stat_database Postgres view.

We collect stats from the oldest non-system databases based on their creation time, but not for all databases. Only the first X databases (sorted by creation time) are included.

datname: The name of the database
kind: The type of value being reported. One of the following:
  • db_size: The size of the database on disk, in bytes (pg_database_size(datname))
  • deadlocks: The number of deadlocks detected
  • inserted: The number of rows inserted (tup_inserted)
  • updated: The number of rows updated (tup_updated)
  • deleted: The number of rows deleted (tup_deleted)
neon_replication_delay_bytessql-metricsThe number of bytes between the last received LSN (Log Sequence Number) and the last replayed one. Large values indiciate replication lag.
neon_replication_delay_secondssql-metricsTime since the last LSN was replayed.
host_cpu_seconds_totalcompute-host-metricsThe number of CPU seconds accumulated in different operating modes (user, system, idle, etc.).
host_load1compute-host-metricsSystem load averaged over the last 1 minute. Example: for 0.25 vCPU, host_load1 of 0.25 means full utilization, >0.25 indicates waiting processes.
host_load5compute-host-metricsSystem load averaged over the last 5 minutes.
host_load15compute-host-metricsSystem load averaged over the last 15 minutes.
host_memory_active_bytescompute-host-metricsThe number of bytes of active main memory.
host_memory_available_bytescompute-host-metricsThe number of bytes of main memory available.
host_memory_buffers_bytescompute-host-metricsThe number of bytes of main memory used by buffers.
host_memory_cached_bytescompute-host-metricsThe number of bytes of main memory used by cached blocks.
host_memory_free_bytescompute-host-metricsThe number of bytes of main memory not used.
host_memory_shared_bytescompute-host-metricsThe number of bytes of main memory shared between processes.
host_memory_swap_free_bytescompute-host-metricsThe number of free bytes of swap space.
host_memory_swap_total_bytescompute-host-metricsThe total number of bytes of swap space.
host_memory_swap_used_bytescompute-host-metricsThe number of used bytes of swap space.
host_memory_swapped_in_bytes_totalcompute-host-metricsThe number of bytes that have been swapped into main memory.
host_memory_swapped_out_bytes_totalcompute-host-metricsThe number of bytes that have been swapped out from main memory.
host_memory_total_bytescompute-host-metricsThe total number of bytes of main memory.
host_memory_used_bytescompute-host-metricsThe number of bytes of main memory used by programs or caches.

Feedback and future improvements

We’re always looking to improve! If you have feature requests or feedback, please let us know via the Feedback form in the Neon Console or on our Discord channel.

Need help?

Join our Discord Server to ask questions or see what others are doing with Neon. Users on paid plans can open a support ticket from the console. For more details, see Getting Support.

Last updated on

Was this page helpful?