{"id":265,"date":"2024-11-15T20:48:59","date_gmt":"2024-11-15T19:48:59","guid":{"rendered":"https:\/\/www.systemdeveloper.nl\/tech\/?p=265"},"modified":"2024-11-23T08:20:47","modified_gmt":"2024-11-23T07:20:47","slug":"building-a-scalable-data-ingestion-api-with-influxdb-and-python","status":"publish","type":"post","link":"https:\/\/www.systemdeveloper.nl\/tech\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\/","title":{"rendered":"Building a Scalable Data Ingestion API with Python"},"content":{"rendered":"<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">In today&#8217;s era of monitoring and predictive analysis, having a flexible and scalable data ingestion system is crucial. In this post, we&rsquo;ll guide you through creating a simple but powerful Scalable API using <a href=\"https:\/\/www.python.org\/\">Python<\/a> and <a href=\"https:\/\/flask.palletsprojects.com\/en\/stable\/\">Flask<\/a> that can ingest custom metrics into an <a href=\"https:\/\/www.influxdata.com\/\">InfluxDB<\/a> instance. This API is designed to accept any metric you define, store it with precise timestamps, and allow easy querying later.<\/p>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">This hands-on project will provide you with the building blocks to expand and customize your own data pipeline.<\/p>\n\n\n<h3 class=\"wp-block-heading\" id=\"step-1-prerequisites\">Step 1: Prerequisites<\/h3>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">Before diving into the code, make sure you have:<\/p>\n\n\n<ol class=\"wp-block-list wp-block-list\">\n<li>\n<strong>InfluxDB<\/strong> (Version 2.x) installed and running.<\/li>\n\n\n\n<li>Python 3.6+ installed on your system.<\/li>\n\n\n\n<li>A bucket created in InfluxDB for storing the metrics (e.g., <code>snmp_monitoring<\/code>).<\/li>\n<\/ol>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">Read our blog about <a href=\"\/tech\/storing-network-monitoring-data-in-influxdb-part-2-of-our-network-monitoring-series\/\">Storing Network Monitoring Data in InfluxDB<\/a> to setup your influx credentials.<\/p>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">Install the required Python libraries:<\/p>\n\n\n<pre class=\"wp-block-code\"><div class=\"copy-to-clipboard\">\n<span>Copied!<\/span><button class=\"click-to-copy-button\" title=\"Copy to clipboard\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" viewbox=\"0 0 32 32\" stroke=\"currentcolor\" stroke-width=\"1.5\" stroke-linecap=\"round\" stroke-linejoin=\"round\" width=\"24\" height=\"24\" fill=\"none\">\n  <path d=\"M12.9975 10.7499L11.7475 10.7499C10.6429 10.7499 9.74747 11.6453 9.74747 12.7499L9.74747 21.2499C9.74747 22.3544 10.6429 23.2499 11.7475 23.2499L20.2475 23.2499C21.352 23.2499 22.2475 22.3544 22.2475 21.2499L22.2475 12.7499C22.2475 11.6453 21.352 10.7499 20.2475 10.7499L18.9975 10.7499Z\"><\/path>\n  <path d=\"M17.9975 12.2499L13.9975 12.2499C13.4452 12.2499 12.9975 11.8022 12.9975 11.2499L12.9975 9.74988C12.9975 9.19759 13.4452 8.74988 13.9975 8.74988L17.9975 8.74988C18.5498 8.74988 18.9975 9.19759 18.9975 9.74988L18.9975 11.2499C18.9975 11.8022 18.5498 12.2499 17.9975 12.2499Z\"><\/path>\n  <path d=\"M13.7475 16.2499L18.2475 16.2499\"><\/path>\n  <path d=\"M13.7475 19.2499L18.2475 19.2499\"><\/path>\n<\/svg><\/button><textarea>pip install flask influxdb-client<\/textarea>\n<\/div><code>pip install flask influxdb-client<\/code><\/pre>\n\n\n<h3 class=\"wp-block-heading\" id=\"step-2-setting-up-influxdb\">Step 2: Setting Up InfluxDB<\/h3>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">You need to create a token in InfluxDB with write permissions for the bucket you&rsquo;ll use. Once you have the token and bucket ready, note the following:<\/p>\n\n\n<ul class=\"wp-block-list wp-block-list\">\n<li>InfluxDB URL (default: <code>http:\/\/localhost:8086<\/code>)<\/li>\n\n\n\n<li>Organization name<\/li>\n\n\n\n<li>Bucket name<\/li>\n<\/ul>\n\n\n<h3 class=\"wp-block-heading\" id=\"step-3-building-the-python-api\">Step 3: Building the Python API<\/h3>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">Here&rsquo;s the full Python code for the API:<\/p>\n\n\n<pre class=\"wp-block-code\"><div class=\"copy-to-clipboard\">\n<span>Copied!<\/span><button class=\"click-to-copy-button\" title=\"Copy to clipboard\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" viewbox=\"0 0 32 32\" stroke=\"currentcolor\" stroke-width=\"1.5\" stroke-linecap=\"round\" stroke-linejoin=\"round\" width=\"24\" height=\"24\" fill=\"none\">\n  <path d=\"M12.9975 10.7499L11.7475 10.7499C10.6429 10.7499 9.74747 11.6453 9.74747 12.7499L9.74747 21.2499C9.74747 22.3544 10.6429 23.2499 11.7475 23.2499L20.2475 23.2499C21.352 23.2499 22.2475 22.3544 22.2475 21.2499L22.2475 12.7499C22.2475 11.6453 21.352 10.7499 20.2475 10.7499L18.9975 10.7499Z\"><\/path>\n  <path d=\"M17.9975 12.2499L13.9975 12.2499C13.4452 12.2499 12.9975 11.8022 12.9975 11.2499L12.9975 9.74988C12.9975 9.19759 13.4452 8.74988 13.9975 8.74988L17.9975 8.74988C18.5498 8.74988 18.9975 9.19759 18.9975 9.74988L18.9975 11.2499C18.9975 11.8022 18.5498 12.2499 17.9975 12.2499Z\"><\/path>\n  <path d=\"M13.7475 16.2499L18.2475 16.2499\"><\/path>\n  <path d=\"M13.7475 19.2499L18.2475 19.2499\"><\/path>\n<\/svg><\/button><textarea>from flask import Flask, request, jsonify\nfrom influxdb_client import InfluxDBClient, Point, WritePrecision\nimport time\nimport logging\n\n# Configure logging\nlogging.basicConfig(level=logging.INFO)\nlogger = logging.getLogger(__name__)\n\n# InfluxDB connection details\nINFLUXDB_URL = 'http:\/\/localhost:8086'\nINFLUXDB_TOKEN = 'your_influxdb_token'\nINFLUXDB_ORG = 'your_organization'\nINFLUXDB_BUCKET = 'snmp_monitoring'\n\n# Initialize Flask app\napp = Flask(__name__)\n\n# Initialize InfluxDB client\nclient = InfluxDBClient(url=INFLUXDB_URL, token=INFLUXDB_TOKEN, org=INFLUXDB_ORG)\nwrite_api = client.write_api()\n\n@app.route('\/api\/submit', methods=['POST'])\ndef submit_data():\n    logger.info(\"Incoming request: %s\", request.json)\n\n    # Haal de parameters op\n    field = request.json.get('field')\n    value = request.json.get('value')\n    timestamp = request.json.get('timestamp', int(time.time() * 1_000_000_000))\n\n    if not field or value is None:\n        logger.warning(\"Missing field or value in request\")\n        return jsonify({\"error\": \"Field and value are required\"}), 400\n\n    try:\n        value = float(value)\n    except ValueError:\n        logger.warning(\"Invalid value format: %s\", value)\n        return jsonify({\"error\": \"Value must be a number\"}), 400\n\n    # Hostname\/IP\n    hostname = request.remote_addr\n\n    # Use current timestamp if none provided\n    try:\n        timestamp = int(timestamp)\n    except ValueError:\n        logger.warning(\"Invalid timestamp format: %s\", timestamp)\n        return jsonify({\"error\": \"Invalid timestamp format\"}), 400\n\n    # Get extra tags (other fields than 'field', 'value' or 'timestamp')\n    extra_tags = {k: v for k, v in request.json.items() if k not in ['field', 'value', 'timestamp']}\n    extra_tags['hostname'] = hostname  # Always add hostname\n\n    # Create an InfluxDB Point\n    point = (\n        Point(field)\n        .field(\"value\", value)\n        .time(timestamp, WritePrecision.NS)\n    )\n\n    # Add extra tags to point\n    for tag_key, tag_value in extra_tags.items():\n        point.tag(tag_key, str(tag_value))\n\n    # Write to InfluxDB\n    try:\n        write_api.write(bucket=INFLUXDB_BUCKET, org=INFLUXDB_ORG, record=point)\n        logger.info(\"Data written to InfluxDB: %s\", point)\n        return jsonify({\"status\": \"success\", \"field\": field, \"value\": value, \"extra_tags\": extra_tags}), 200\n    except Exception as e:\n        logger.error(\"Failed to write to InfluxDB: %s\", str(e))\n        return jsonify({\"error\": \"Failed to write to database\", \"details\": str(e)}), 500\n\n\nif __name__ == '__main__':\n    app.run(host='0.0.0.0', port=5000)<\/textarea>\n<\/div><code>from flask import Flask, request, jsonify\nfrom influxdb_client import InfluxDBClient, Point, WritePrecision\nimport time\nimport logging\n\n# Configure logging\nlogging.basicConfig(level=logging.INFO)\nlogger = logging.getLogger(__name__)\n\n# InfluxDB connection details\nINFLUXDB_URL = 'http:\/\/localhost:8086'\nINFLUXDB_TOKEN = 'your_influxdb_token'\nINFLUXDB_ORG = 'your_organization'\nINFLUXDB_BUCKET = 'snmp_monitoring'\n\n# Initialize Flask app\napp = Flask(__name__)\n\n# Initialize InfluxDB client\nclient = InfluxDBClient(url=INFLUXDB_URL, token=INFLUXDB_TOKEN, org=INFLUXDB_ORG)\nwrite_api = client.write_api()\n\n@app.route('\/api\/submit', methods=['POST'])\ndef submit_data():\n    logger.info(\"Incoming request: %s\", request.json)\n\n    # Haal de parameters op\n    field = request.json.get('field')\n    value = request.json.get('value')\n    timestamp = request.json.get('timestamp', int(time.time() * 1_000_000_000))\n\n    if not field or value is None:\n        logger.warning(\"Missing field or value in request\")\n        return jsonify({\"error\": \"Field and value are required\"}), 400\n\n    try:\n        value = float(value)\n    except ValueError:\n        logger.warning(\"Invalid value format: %s\", value)\n        return jsonify({\"error\": \"Value must be a number\"}), 400\n\n    # Hostname\/IP\n    hostname = request.remote_addr\n\n    # Use current timestamp if none provided\n    try:\n        timestamp = int(timestamp)\n    except ValueError:\n        logger.warning(\"Invalid timestamp format: %s\", timestamp)\n        return jsonify({\"error\": \"Invalid timestamp format\"}), 400\n\n    # Get extra tags (other fields than 'field', 'value' or 'timestamp')\n    extra_tags = {k: v for k, v in request.json.items() if k not in ['field', 'value', 'timestamp']}\n    extra_tags['hostname'] = hostname  # Always add hostname\n\n    # Create an InfluxDB Point\n    point = (\n        Point(field)\n        .field(\"value\", value)\n        .time(timestamp, WritePrecision.NS)\n    )\n\n    # Add extra tags to point\n    for tag_key, tag_value in extra_tags.items():\n        point.tag(tag_key, str(tag_value))\n\n    # Write to InfluxDB\n    try:\n        write_api.write(bucket=INFLUXDB_BUCKET, org=INFLUXDB_ORG, record=point)\n        logger.info(\"Data written to InfluxDB: %s\", point)\n        return jsonify({\"status\": \"success\", \"field\": field, \"value\": value, \"extra_tags\": extra_tags}), 200\n    except Exception as e:\n        logger.error(\"Failed to write to InfluxDB: %s\", str(e))\n        return jsonify({\"error\": \"Failed to write to database\", \"details\": str(e)}), 500\n\n\nif __name__ == '__main__':\n    app.run(host='0.0.0.0', port=5000)<\/code><\/pre>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">Save this code as <code>app.py<\/code>.<\/p>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">This API will accept a JSON array of fields. &#8216;field&#8217; and &#8216;value&#8217; are required because there is little sense in storing no values. All other field are tags you can use to identify the measurement more closely.<\/p>\n\n\n<h3 class=\"wp-block-heading\" id=\"step-4-running-the-api\">Step 4: Running the API<\/h3>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">Start your Flask application:<\/p>\n\n\n<pre class=\"wp-block-code\"><div class=\"copy-to-clipboard\">\n<span>Copied!<\/span><button class=\"click-to-copy-button\" title=\"Copy to clipboard\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" viewbox=\"0 0 32 32\" stroke=\"currentcolor\" stroke-width=\"1.5\" stroke-linecap=\"round\" stroke-linejoin=\"round\" width=\"24\" height=\"24\" fill=\"none\">\n  <path d=\"M12.9975 10.7499L11.7475 10.7499C10.6429 10.7499 9.74747 11.6453 9.74747 12.7499L9.74747 21.2499C9.74747 22.3544 10.6429 23.2499 11.7475 23.2499L20.2475 23.2499C21.352 23.2499 22.2475 22.3544 22.2475 21.2499L22.2475 12.7499C22.2475 11.6453 21.352 10.7499 20.2475 10.7499L18.9975 10.7499Z\"><\/path>\n  <path d=\"M17.9975 12.2499L13.9975 12.2499C13.4452 12.2499 12.9975 11.8022 12.9975 11.2499L12.9975 9.74988C12.9975 9.19759 13.4452 8.74988 13.9975 8.74988L17.9975 8.74988C18.5498 8.74988 18.9975 9.19759 18.9975 9.74988L18.9975 11.2499C18.9975 11.8022 18.5498 12.2499 17.9975 12.2499Z\"><\/path>\n  <path d=\"M13.7475 16.2499L18.2475 16.2499\"><\/path>\n  <path d=\"M13.7475 19.2499L18.2475 19.2499\"><\/path>\n<\/svg><\/button><textarea>python3 app.py<\/textarea>\n<\/div><code>python3 app.py<\/code><\/pre>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">The API will run on <code>http:\/\/0.0.0.0:5000<\/code>. <\/p>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">Running an API like this is fine while you&#8217;re still testing, but it&#8217;s definitely not a production solution. <\/p>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">Read our blog about using <a href=\"\/tech\/deploying-flask-applications-with-gunicorn-as-a-wsgi-server\/\">Gunicorn as a WSGI server<\/a>. And while you&#8217;re browsing, also check out our blog about <a href=\"\/tech\/securing-apis-and-enhancing-flexibility-and-security\/\">Adding A Security Layer to your API<\/a>.<\/p>\n\n\n<h3 class=\"wp-block-heading\" id=\"step-5-submitting-metrics\">Step 5: Submitting Metrics<\/h3>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">Use <code>curl<\/code> to test the API:<\/p>\n\n\n<pre class=\"wp-block-code\"><div class=\"copy-to-clipboard\">\n<span>Copied!<\/span><button class=\"click-to-copy-button\" title=\"Copy to clipboard\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" viewbox=\"0 0 32 32\" stroke=\"currentcolor\" stroke-width=\"1.5\" stroke-linecap=\"round\" stroke-linejoin=\"round\" width=\"24\" height=\"24\" fill=\"none\">\n  <path d=\"M12.9975 10.7499L11.7475 10.7499C10.6429 10.7499 9.74747 11.6453 9.74747 12.7499L9.74747 21.2499C9.74747 22.3544 10.6429 23.2499 11.7475 23.2499L20.2475 23.2499C21.352 23.2499 22.2475 22.3544 22.2475 21.2499L22.2475 12.7499C22.2475 11.6453 21.352 10.7499 20.2475 10.7499L18.9975 10.7499Z\"><\/path>\n  <path d=\"M17.9975 12.2499L13.9975 12.2499C13.4452 12.2499 12.9975 11.8022 12.9975 11.2499L12.9975 9.74988C12.9975 9.19759 13.4452 8.74988 13.9975 8.74988L17.9975 8.74988C18.5498 8.74988 18.9975 9.19759 18.9975 9.74988L18.9975 11.2499C18.9975 11.8022 18.5498 12.2499 17.9975 12.2499Z\"><\/path>\n  <path d=\"M13.7475 16.2499L18.2475 16.2499\"><\/path>\n  <path d=\"M13.7475 19.2499L18.2475 19.2499\"><\/path>\n<\/svg><\/button><textarea>curl -X POST http:\/\/127.0.0.1:5000\/api\/submit -H \"Content-Type: application\/json\" -d '{\n  \"field\": \"os_updates\",\n  \"value\": 73,\n  \"hostname\": \"server01\",\n  \"region\": \"EU-West\",\n  \"environment\": \"production\",\n  \"application\": \"backend\"\n}'<\/textarea>\n<\/div><code>curl -X POST http:\/\/127.0.0.1:5000\/api\/submit -H \"Content-Type: application\/json\" -d '{\n  \"field\": \"os_updates\",\n  \"value\": 73,\n  \"hostname\": \"server01\",\n  \"region\": \"EU-West\",\n  \"environment\": \"production\",\n  \"application\": \"backend\"\n}'<\/code><\/pre>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\"><\/p>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">Expected response:<\/p>\n\n\n<pre class=\"wp-block-code\"><div class=\"copy-to-clipboard\">\n<span>Copied!<\/span><button class=\"click-to-copy-button\" title=\"Copy to clipboard\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" viewbox=\"0 0 32 32\" stroke=\"currentcolor\" stroke-width=\"1.5\" stroke-linecap=\"round\" stroke-linejoin=\"round\" width=\"24\" height=\"24\" fill=\"none\">\n  <path d=\"M12.9975 10.7499L11.7475 10.7499C10.6429 10.7499 9.74747 11.6453 9.74747 12.7499L9.74747 21.2499C9.74747 22.3544 10.6429 23.2499 11.7475 23.2499L20.2475 23.2499C21.352 23.2499 22.2475 22.3544 22.2475 21.2499L22.2475 12.7499C22.2475 11.6453 21.352 10.7499 20.2475 10.7499L18.9975 10.7499Z\"><\/path>\n  <path d=\"M17.9975 12.2499L13.9975 12.2499C13.4452 12.2499 12.9975 11.8022 12.9975 11.2499L12.9975 9.74988C12.9975 9.19759 13.4452 8.74988 13.9975 8.74988L17.9975 8.74988C18.5498 8.74988 18.9975 9.19759 18.9975 9.74988L18.9975 11.2499C18.9975 11.8022 18.5498 12.2499 17.9975 12.2499Z\"><\/path>\n  <path d=\"M13.7475 16.2499L18.2475 16.2499\"><\/path>\n  <path d=\"M13.7475 19.2499L18.2475 19.2499\"><\/path>\n<\/svg><\/button><textarea>{\n  \"extra_tags\": {\n    \"application\": \"backend\",\n    \"environment\": \"production\",\n    \"hostname\": \"127.0.0.1\",\n    \"region\": \"EU-West\"\n  },\n  \"field\": \"os_updates\",\n  \"status\": \"success\",\n  \"value\": 73.0\n}<\/textarea>\n<\/div><code>{\n  \"extra_tags\": {\n    \"application\": \"backend\",\n    \"environment\": \"production\",\n    \"hostname\": \"127.0.0.1\",\n    \"region\": \"EU-West\"\n  },\n  \"field\": \"os_updates\",\n  \"status\": \"success\",\n  \"value\": 73.0\n}<\/code><\/pre>\n\n\n<h3 class=\"wp-block-heading\" id=\"step-6-querying-metrics-in-influxdb\">Step 6: Querying Metrics in InfluxDB<\/h3>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">To query your metrics, use the Influx CLI or the web UI:<\/p>\n\n\n<pre class=\"wp-block-code\"><div class=\"copy-to-clipboard\">\n<span>Copied!<\/span><button class=\"click-to-copy-button\" title=\"Copy to clipboard\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" viewbox=\"0 0 32 32\" stroke=\"currentcolor\" stroke-width=\"1.5\" stroke-linecap=\"round\" stroke-linejoin=\"round\" width=\"24\" height=\"24\" fill=\"none\">\n  <path d=\"M12.9975 10.7499L11.7475 10.7499C10.6429 10.7499 9.74747 11.6453 9.74747 12.7499L9.74747 21.2499C9.74747 22.3544 10.6429 23.2499 11.7475 23.2499L20.2475 23.2499C21.352 23.2499 22.2475 22.3544 22.2475 21.2499L22.2475 12.7499C22.2475 11.6453 21.352 10.7499 20.2475 10.7499L18.9975 10.7499Z\"><\/path>\n  <path d=\"M17.9975 12.2499L13.9975 12.2499C13.4452 12.2499 12.9975 11.8022 12.9975 11.2499L12.9975 9.74988C12.9975 9.19759 13.4452 8.74988 13.9975 8.74988L17.9975 8.74988C18.5498 8.74988 18.9975 9.19759 18.9975 9.74988L18.9975 11.2499C18.9975 11.8022 18.5498 12.2499 17.9975 12.2499Z\"><\/path>\n  <path d=\"M13.7475 16.2499L18.2475 16.2499\"><\/path>\n  <path d=\"M13.7475 19.2499L18.2475 19.2499\"><\/path>\n<\/svg><\/button><textarea>influx query '\nfrom(bucket: \"snmp_monitoring\")\n  |&gt; range(start: -1d)\n  |&gt; filter(fn: (r) =&gt; r._measurement == \"os_updates\")\n  |&gt; limit(n: 10)\n'<\/textarea>\n<\/div><code>influx query '\nfrom(bucket: \"snmp_monitoring\")\n  |&gt; range(start: -1d)\n  |&gt; filter(fn: (r) =&gt; r._measurement == \"os_updates\")\n  |&gt; limit(n: 10)\n'<\/code><\/pre>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">You can also <a href=\"\/tech\/visualizing-snmp-network-data-with-grafana-part-3-of-our-network-monitoring-series\/\">visualize this data<\/a> in tools like <a href=\"https:\/\/grafana.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">Grafana<\/a>.<\/p>\n\n\n<h3 class=\"wp-block-heading\" id=\"example-monitoring-server-updates-with-the-api\">Example: Monitoring Server Updates with the API<\/h3>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">Keeping track of available server updates is essential to maintaining the health and security of your infrastructure. Using our newly developed API, we can efficiently log the number of pending updates into our InfluxDB.<\/p>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">Here&rsquo;s a simple script you can schedule as a cron job to automate this process:<\/p>\n\n\n<pre class=\"wp-block-code\"><div class=\"copy-to-clipboard\">\n<span>Copied!<\/span><button class=\"click-to-copy-button\" title=\"Copy to clipboard\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" viewbox=\"0 0 32 32\" stroke=\"currentcolor\" stroke-width=\"1.5\" stroke-linecap=\"round\" stroke-linejoin=\"round\" width=\"24\" height=\"24\" fill=\"none\">\n  <path d=\"M12.9975 10.7499L11.7475 10.7499C10.6429 10.7499 9.74747 11.6453 9.74747 12.7499L9.74747 21.2499C9.74747 22.3544 10.6429 23.2499 11.7475 23.2499L20.2475 23.2499C21.352 23.2499 22.2475 22.3544 22.2475 21.2499L22.2475 12.7499C22.2475 11.6453 21.352 10.7499 20.2475 10.7499L18.9975 10.7499Z\"><\/path>\n  <path d=\"M17.9975 12.2499L13.9975 12.2499C13.4452 12.2499 12.9975 11.8022 12.9975 11.2499L12.9975 9.74988C12.9975 9.19759 13.4452 8.74988 13.9975 8.74988L17.9975 8.74988C18.5498 8.74988 18.9975 9.19759 18.9975 9.74988L18.9975 11.2499C18.9975 11.8022 18.5498 12.2499 17.9975 12.2499Z\"><\/path>\n  <path d=\"M13.7475 16.2499L18.2475 16.2499\"><\/path>\n  <path d=\"M13.7475 19.2499L18.2475 19.2499\"><\/path>\n<\/svg><\/button><textarea>#!\/bin\/bash\n\n# Count the available updates, filtering out unnecessary output\nUPDATE_COUNT=$(\/usr\/bin\/dnf check-update 2&gt;\/dev\/null | grep -vE \"^(Dependencies resolved|Nothing to do|Complete|Security|Last meta)\" | wc -l)\n\n# Send the update count to the API\ncurl -X POST http:\/\/127.0.0.1:5000\/api\/submit \\\n     -H \"Content-Type: application\/json\" \\\n     -d \"{\\\"field\\\": \\\"os_updates\\\", \\\"value\\\": $UPDATE_COUNT}\"<\/textarea>\n<\/div><code>#!\/bin\/bash\n\n# Count the available updates, filtering out unnecessary output\nUPDATE_COUNT=$(\/usr\/bin\/dnf check-update 2&gt;\/dev\/null | grep -vE \"^(Dependencies resolved|Nothing to do|Complete|Security|Last meta)\" | wc -l)\n\n# Send the update count to the API\ncurl -X POST http:\/\/127.0.0.1:5000\/api\/submit \\\n     -H \"Content-Type: application\/json\" \\\n     -d \"{\\\"field\\\": \\\"os_updates\\\", \\\"value\\\": $UPDATE_COUNT}\"<\/code><\/pre>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">In this script:<\/p>\n\n\n<ol class=\"wp-block-list wp-block-list\">\n<li>\n<strong><code>dnf check-update<\/code><\/strong>: Retrieves a list of available updates. Any unnecessary output (such as &#8220;Dependencies resolved&#8221; or &#8220;Nothing to do&#8221;) is filtered out using <code>grep<\/code>.<\/li>\n\n\n\n<li>\n<strong><code>wc -l<\/code><\/strong>: Counts the number of remaining lines to determine how many updates are available.<\/li>\n\n\n\n<li>\n<strong><code>curl<\/code><\/strong>: Sends this data to our API, storing it in the InfluxDB under the <code>os_updates<\/code> field.<\/li>\n<\/ol>\n\n\n<h3 class=\"wp-block-heading\" id=\"viewing-the-data-in-influxdb\"><strong>Viewing the Data in InfluxDB<\/strong><\/h3>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">Once the script runs and data is stored, you can query InfluxDB to view the collected information. For example, to see the data from the past 24 hours:<\/p>\n\n\n<pre class=\"wp-block-code\"><div class=\"copy-to-clipboard\">\n<span>Copied!<\/span><button class=\"click-to-copy-button\" title=\"Copy to clipboard\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" viewbox=\"0 0 32 32\" stroke=\"currentcolor\" stroke-width=\"1.5\" stroke-linecap=\"round\" stroke-linejoin=\"round\" width=\"24\" height=\"24\" fill=\"none\">\n  <path d=\"M12.9975 10.7499L11.7475 10.7499C10.6429 10.7499 9.74747 11.6453 9.74747 12.7499L9.74747 21.2499C9.74747 22.3544 10.6429 23.2499 11.7475 23.2499L20.2475 23.2499C21.352 23.2499 22.2475 22.3544 22.2475 21.2499L22.2475 12.7499C22.2475 11.6453 21.352 10.7499 20.2475 10.7499L18.9975 10.7499Z\"><\/path>\n  <path d=\"M17.9975 12.2499L13.9975 12.2499C13.4452 12.2499 12.9975 11.8022 12.9975 11.2499L12.9975 9.74988C12.9975 9.19759 13.4452 8.74988 13.9975 8.74988L17.9975 8.74988C18.5498 8.74988 18.9975 9.19759 18.9975 9.74988L18.9975 11.2499C18.9975 11.8022 18.5498 12.2499 17.9975 12.2499Z\"><\/path>\n  <path d=\"M13.7475 16.2499L18.2475 16.2499\"><\/path>\n  <path d=\"M13.7475 19.2499L18.2475 19.2499\"><\/path>\n<\/svg><\/button><textarea>influx query '\nfrom(bucket: \"snmp_monitoring\")\n  |&gt; range(start: -24h)\n  |&gt; filter(fn: (r) =&gt; r._measurement == \"os_updates\")'<\/textarea>\n<\/div><code>influx query '\nfrom(bucket: \"snmp_monitoring\")\n  |&gt; range(start: -24h)\n  |&gt; filter(fn: (r) =&gt; r._measurement == \"os_updates\")'<\/code><\/pre>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">This query will return the timestamps and update counts, giving you insight into when updates were available and their volume.<\/p>\n\n\n<h3 class=\"wp-block-heading\" id=\"conclusion\">Conclusion<\/h3>\n\n\n<p class=\"wp-block-paragraph wp-block-paragraph\" style=\"\">With this simple API, you&rsquo;ve built a scalable system to ingest metrics into InfluxDB. Having a flexible and scalable data ingestion system is crucial.. The design is flexible, allowing you to add custom metrics from any device or service. Whether for SNMP monitoring, application telemetry, or any other use case, this API lays the groundwork for advanced data collection.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In today&#8217;s era of monitoring and predictive analysis, having a flexible and scalable data ingestion system is crucial. In this post, we&rsquo;ll guide you through creating a simple but powerful Scalable API using Python and Flask that can ingest custom metrics into an InfluxDB instance. This API is designed to accept any metric you define, &hellip;<\/p>\n","protected":false},"author":1,"featured_media":266,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[36,46,16],"tags":[15,48,47,38,37,34],"class_list":["post-265","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-future-ideas","category-howto","category-tech-innovation","tag-api","tag-flask","tag-gunicorn","tag-influxdb","tag-monitoring","tag-python"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.9 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Building a Scalable Data Ingestion API with Python - SystemDeveloper.NL<\/title>\n<meta name=\"description\" content=\"This API is designed to accept any metric you define, store it with precise timestamps, and allow easy querying later.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.systemdeveloper.nl\/tech\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Building a Scalable Data Ingestion API with Python - SystemDeveloper.NL\" \/>\n<meta property=\"og:description\" content=\"This API is designed to accept any metric you define, store it with precise timestamps, and allow easy querying later.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.systemdeveloper.nl\/tech\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\/\" \/>\n<meta property=\"og:site_name\" content=\"SystemDeveloper.NL\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/quan.tora.16\" \/>\n<meta property=\"article:published_time\" content=\"2024-11-15T19:48:59+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-11-23T07:20:47+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.systemdeveloper.nl\/tech\/wp-content\/uploads\/2024\/11\/2192e9d4-d4bb-4837-87b9-d569aebf203b-e1731700130815.webp\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"768\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/webp\" \/>\n<meta name=\"author\" content=\"John Timmer\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"John Timmer\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\\\/\"},\"author\":{\"name\":\"John Timmer\",\"@id\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/#\\\/schema\\\/person\\\/5760c2ed5300c56d8ef01dfb00a9763b\"},\"headline\":\"Building a Scalable Data Ingestion API with Python\",\"datePublished\":\"2024-11-15T19:48:59+00:00\",\"dateModified\":\"2024-11-23T07:20:47+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\\\/\"},\"wordCount\":559,\"publisher\":{\"@id\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/wp-content\\\/uploads\\\/2024\\\/11\\\/2192e9d4-d4bb-4837-87b9-d569aebf203b-e1731700130815.webp\",\"keywords\":[\"API\",\"Flask\",\"Gunicorn\",\"InfluxDB\",\"Monitoring\",\"Python\"],\"articleSection\":[\"Future ideas\",\"Howto\",\"Tech Innovation\"],\"inLanguage\":\"en\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\\\/\",\"url\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\\\/\",\"name\":\"Building a Scalable Data Ingestion API with Python - SystemDeveloper.NL\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/wp-content\\\/uploads\\\/2024\\\/11\\\/2192e9d4-d4bb-4837-87b9-d569aebf203b-e1731700130815.webp\",\"datePublished\":\"2024-11-15T19:48:59+00:00\",\"dateModified\":\"2024-11-23T07:20:47+00:00\",\"description\":\"This API is designed to accept any metric you define, store it with precise timestamps, and allow easy querying later.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\\\/#breadcrumb\"},\"inLanguage\":\"en\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en\",\"@id\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/wp-content\\\/uploads\\\/2024\\\/11\\\/2192e9d4-d4bb-4837-87b9-d569aebf203b-e1731700130815.webp\",\"contentUrl\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/wp-content\\\/uploads\\\/2024\\\/11\\\/2192e9d4-d4bb-4837-87b9-d569aebf203b-e1731700130815.webp\",\"width\":1024,\"height\":768,\"caption\":\"Building a Scalable Data Ingestion\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Building a Scalable Data Ingestion API with Python\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/#website\",\"url\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/\",\"name\":\"www.systemdeveloper.nl\",\"description\":\"NextGen IT\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/#organization\",\"name\":\"www.systemdeveloper.nl\",\"url\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en\",\"@id\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/wp-content\\\/uploads\\\/2024\\\/11\\\/qt-logo.png\",\"contentUrl\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/wp-content\\\/uploads\\\/2024\\\/11\\\/qt-logo.png\",\"width\":1346,\"height\":1230,\"caption\":\"www.systemdeveloper.nl\"},\"image\":{\"@id\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/quan.tora.16\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/#\\\/schema\\\/person\\\/5760c2ed5300c56d8ef01dfb00a9763b\",\"name\":\"John Timmer\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en\",\"@id\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/wp-content\\\/uploads\\\/2024\\\/11\\\/cropped-D6E27035-6864-4270-8D72-0D8C0C59F370-96x96.jpeg\",\"url\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/wp-content\\\/uploads\\\/2024\\\/11\\\/cropped-D6E27035-6864-4270-8D72-0D8C0C59F370-96x96.jpeg\",\"contentUrl\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/wp-content\\\/uploads\\\/2024\\\/11\\\/cropped-D6E27035-6864-4270-8D72-0D8C0C59F370-96x96.jpeg\",\"caption\":\"John Timmer\"},\"sameAs\":[\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\"],\"url\":\"https:\\\/\\\/www.systemdeveloper.nl\\\/tech\\\/author\\\/admin\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Building a Scalable Data Ingestion API with Python - SystemDeveloper.NL","description":"This API is designed to accept any metric you define, store it with precise timestamps, and allow easy querying later.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.systemdeveloper.nl\/tech\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\/","og_locale":"en_US","og_type":"article","og_title":"Building a Scalable Data Ingestion API with Python - SystemDeveloper.NL","og_description":"This API is designed to accept any metric you define, store it with precise timestamps, and allow easy querying later.","og_url":"https:\/\/www.systemdeveloper.nl\/tech\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\/","og_site_name":"SystemDeveloper.NL","article_publisher":"https:\/\/www.facebook.com\/quan.tora.16","article_published_time":"2024-11-15T19:48:59+00:00","article_modified_time":"2024-11-23T07:20:47+00:00","og_image":[{"width":1024,"height":768,"url":"https:\/\/www.systemdeveloper.nl\/tech\/wp-content\/uploads\/2024\/11\/2192e9d4-d4bb-4837-87b9-d569aebf203b-e1731700130815.webp","type":"image\/webp"}],"author":"John Timmer","twitter_card":"summary_large_image","twitter_misc":{"Written by":"John Timmer","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.systemdeveloper.nl\/tech\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\/#article","isPartOf":{"@id":"https:\/\/www.systemdeveloper.nl\/tech\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\/"},"author":{"name":"John Timmer","@id":"https:\/\/www.systemdeveloper.nl\/tech\/#\/schema\/person\/5760c2ed5300c56d8ef01dfb00a9763b"},"headline":"Building a Scalable Data Ingestion API with Python","datePublished":"2024-11-15T19:48:59+00:00","dateModified":"2024-11-23T07:20:47+00:00","mainEntityOfPage":{"@id":"https:\/\/www.systemdeveloper.nl\/tech\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\/"},"wordCount":559,"publisher":{"@id":"https:\/\/www.systemdeveloper.nl\/tech\/#organization"},"image":{"@id":"https:\/\/www.systemdeveloper.nl\/tech\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\/#primaryimage"},"thumbnailUrl":"https:\/\/www.systemdeveloper.nl\/tech\/wp-content\/uploads\/2024\/11\/2192e9d4-d4bb-4837-87b9-d569aebf203b-e1731700130815.webp","keywords":["API","Flask","Gunicorn","InfluxDB","Monitoring","Python"],"articleSection":["Future ideas","Howto","Tech Innovation"],"inLanguage":"en"},{"@type":"WebPage","@id":"https:\/\/www.systemdeveloper.nl\/tech\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\/","url":"https:\/\/www.systemdeveloper.nl\/tech\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\/","name":"Building a Scalable Data Ingestion API with Python - SystemDeveloper.NL","isPartOf":{"@id":"https:\/\/www.systemdeveloper.nl\/tech\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.systemdeveloper.nl\/tech\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\/#primaryimage"},"image":{"@id":"https:\/\/www.systemdeveloper.nl\/tech\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\/#primaryimage"},"thumbnailUrl":"https:\/\/www.systemdeveloper.nl\/tech\/wp-content\/uploads\/2024\/11\/2192e9d4-d4bb-4837-87b9-d569aebf203b-e1731700130815.webp","datePublished":"2024-11-15T19:48:59+00:00","dateModified":"2024-11-23T07:20:47+00:00","description":"This API is designed to accept any metric you define, store it with precise timestamps, and allow easy querying later.","breadcrumb":{"@id":"https:\/\/www.systemdeveloper.nl\/tech\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\/#breadcrumb"},"inLanguage":"en","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.systemdeveloper.nl\/tech\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\/"]}]},{"@type":"ImageObject","inLanguage":"en","@id":"https:\/\/www.systemdeveloper.nl\/tech\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\/#primaryimage","url":"https:\/\/www.systemdeveloper.nl\/tech\/wp-content\/uploads\/2024\/11\/2192e9d4-d4bb-4837-87b9-d569aebf203b-e1731700130815.webp","contentUrl":"https:\/\/www.systemdeveloper.nl\/tech\/wp-content\/uploads\/2024\/11\/2192e9d4-d4bb-4837-87b9-d569aebf203b-e1731700130815.webp","width":1024,"height":768,"caption":"Building a Scalable Data Ingestion"},{"@type":"BreadcrumbList","@id":"https:\/\/www.systemdeveloper.nl\/tech\/building-a-scalable-data-ingestion-api-with-influxdb-and-python\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.systemdeveloper.nl\/tech\/"},{"@type":"ListItem","position":2,"name":"Building a Scalable Data Ingestion API with Python"}]},{"@type":"WebSite","@id":"https:\/\/www.systemdeveloper.nl\/tech\/#website","url":"https:\/\/www.systemdeveloper.nl\/tech\/","name":"www.systemdeveloper.nl","description":"NextGen IT","publisher":{"@id":"https:\/\/www.systemdeveloper.nl\/tech\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.systemdeveloper.nl\/tech\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en"},{"@type":"Organization","@id":"https:\/\/www.systemdeveloper.nl\/tech\/#organization","name":"www.systemdeveloper.nl","url":"https:\/\/www.systemdeveloper.nl\/tech\/","logo":{"@type":"ImageObject","inLanguage":"en","@id":"https:\/\/www.systemdeveloper.nl\/tech\/#\/schema\/logo\/image\/","url":"https:\/\/www.systemdeveloper.nl\/tech\/wp-content\/uploads\/2024\/11\/qt-logo.png","contentUrl":"https:\/\/www.systemdeveloper.nl\/tech\/wp-content\/uploads\/2024\/11\/qt-logo.png","width":1346,"height":1230,"caption":"www.systemdeveloper.nl"},"image":{"@id":"https:\/\/www.systemdeveloper.nl\/tech\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/quan.tora.16"]},{"@type":"Person","@id":"https:\/\/www.systemdeveloper.nl\/tech\/#\/schema\/person\/5760c2ed5300c56d8ef01dfb00a9763b","name":"John Timmer","image":{"@type":"ImageObject","inLanguage":"en","@id":"https:\/\/www.systemdeveloper.nl\/tech\/wp-content\/uploads\/2024\/11\/cropped-D6E27035-6864-4270-8D72-0D8C0C59F370-96x96.jpeg","url":"https:\/\/www.systemdeveloper.nl\/tech\/wp-content\/uploads\/2024\/11\/cropped-D6E27035-6864-4270-8D72-0D8C0C59F370-96x96.jpeg","contentUrl":"https:\/\/www.systemdeveloper.nl\/tech\/wp-content\/uploads\/2024\/11\/cropped-D6E27035-6864-4270-8D72-0D8C0C59F370-96x96.jpeg","caption":"John Timmer"},"sameAs":["https:\/\/www.systemdeveloper.nl\/tech"],"url":"https:\/\/www.systemdeveloper.nl\/tech\/author\/admin\/"}]}},"_links":{"self":[{"href":"https:\/\/www.systemdeveloper.nl\/tech\/wp-json\/wp\/v2\/posts\/265","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.systemdeveloper.nl\/tech\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.systemdeveloper.nl\/tech\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.systemdeveloper.nl\/tech\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.systemdeveloper.nl\/tech\/wp-json\/wp\/v2\/comments?post=265"}],"version-history":[{"count":23,"href":"https:\/\/www.systemdeveloper.nl\/tech\/wp-json\/wp\/v2\/posts\/265\/revisions"}],"predecessor-version":[{"id":419,"href":"https:\/\/www.systemdeveloper.nl\/tech\/wp-json\/wp\/v2\/posts\/265\/revisions\/419"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.systemdeveloper.nl\/tech\/wp-json\/wp\/v2\/media\/266"}],"wp:attachment":[{"href":"https:\/\/www.systemdeveloper.nl\/tech\/wp-json\/wp\/v2\/media?parent=265"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.systemdeveloper.nl\/tech\/wp-json\/wp\/v2\/categories?post=265"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.systemdeveloper.nl\/tech\/wp-json\/wp\/v2\/tags?post=265"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}