initial

README.md

@@ -0,0 +1,111 @@
+# FF-Agent Ticker Gateway
+
+Receives **Einsätze** (operations/incidents) from FF-Agent via **webhook** and keeps a local JSON file for a ticker-style microsite. You can also **import** existing data from a CSV export.
+
+## Requirements
+
+- Python 3.8+
+- Dependencies: `flask`, `PyYAML`, `requests` (see `requirements.txt`)
+
+## Installation
+
+```bash
+pip install -r requirements.txt
+```
+
+## Webhook receiver
+
+FF-Agent can push Einsatz details to a URL. Run the webhook server so it accepts POSTs and updates the JSON file.
+
+**Start the server (default: port 5000, data file `data/einsaetze.json`):**
+
+```bash
+python webhook_receiver.py
+```
+
+**Options:**
+
+```bash
+python webhook_receiver.py --host 0.0.0.0 --port 8080 --data data/einsaetze.json
+```
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `--host` | `0.0.0.0` | Bind address. |
+| `--port` | `5000` | Port. |
+| `--data` | `data/einsaetze.json` | Path to the JSON file to read/write. |
+
+**Environment:** Set `EINSATZ_JSON_PATH` to override the data file path without `--data`.
+
+**Endpoints:**
+
+- **POST `/webhook`** or **POST `/`** – Accepts JSON:
+  - A single Einsatz object, or
+  - `{ "einsaetze": [ ... ] }` with an array of Einsätze.
+- **GET `/`** or **GET `/health`** – Health check; returns `data_path` and status.
+
+**Merge behaviour:** Incoming records are merged by `operationId`. If an ID already exists, that record is updated; otherwise it is appended. The file is written after each successful POST.
+
+**Configuring FF-Agent:** In FF-Agent set the webhook URL to your server, e.g. `https://your-server.example.com/webhook`. Ensure the server is reachable (firewall, reverse proxy, HTTPS if needed).
+
+### PHP webhook receiver
+
+A PHP script accepts POST requests with a JSON body and appends the payload to the same data file without assuming a fixed structure.
+
+**Run it:** Point your web server (Apache/Nginx) at `webhook_receiver.php` for the webhook URL, or use the built-in server for local testing:
+
+```bash
+php -S 0.0.0.0:8080
+```
+
+Then send POST requests to `http://localhost:8080/webhook_receiver.php` with `Content-Type: application/json`.
+
+**Behaviour:** The raw POST body is decoded as JSON and stored in `data/einsaetze.json` under the root key **`webhook_events`**: each event is `{ "received": "<ISO8601 UTC>", "data": <your payload> }`. The root key **`updated`** is set to the time of the last webhook write. The **`einsaetze`** array is left unchanged by the webhook. Set the environment variable `EINSATZ_JSON_PATH` to override the data file path (default: `data/einsaetze.json`). Request body is limited to 1 MB.
+
+## CSV import
+
+To seed or replace data from an FF-Agent CSV export (semicolon-delimited):
+
+```bash
+python import_einsaetze_csv.py Einsatzexport_2026-02-03-21-15-53.csv -o data/einsaetze.json
+```
+
+Same output format as the webhook (see below). You can run the CSV import once, then run the webhook receiver so new events update the same file.
+
+## Output format
+
+Both the webhook and the CSV importer use this JSON structure:
+
+- **`einsaetze`** – Array of Einsatz objects (fields depend on FF-Agent / CSV columns).
+- **`updated`** – ISO 8601 timestamp (UTC) of the last write.
+- **`webhook_events`** – (PHP webhook only) Array of `{ "received": "<ISO8601>", "data": <payload> }` for each POST.
+
+Example:
+
+```json
+{
+  "einsaetze": [
+    {
+      "operationId": "T 1.1 260202 828",
+      "keyword": "THL 1 T2714",
+      "message": "klein Fahrzeug öffnen",
+      "location": "Vimystraße 3",
+      "district": "85356 Freising"
+    }
+  ],
+  "updated": "2026-02-03T20:30:00.123456+00:00"
+}
+```
+
+The ticker service reads this file and uses `einsaetze`; `updated` can be used for display or cache invalidation.
+
+## Files
+
+| File | Purpose |
+|------|--------|
+| `webhook_receiver.php` | PHP webhook: POST JSON → append to `webhook_events` in data file. |
+| `webhook_receiver.py` | Webhook server: POST → merge into JSON. |
+| `import_einsaetze_csv.py` | Import CSV export into the same JSON format. |
+| `requirements.txt` | Python dependencies. |
+| `data/einsaetze.json` | Data file (created/updated by webhook or import). |
+| `Dokumentation_WebAPI_ffagent.pdf` | FF-Agent WebAPI documentation. |

config.yml

@@ -0,0 +1,23 @@
+# FF-Agent WebAPI configuration (see Dokumentation_WebAPI_ffagent.pdf)
+# Get webApiToken and webApiKey from Admin > Einstellungen > Sicherheit
+# Get accessToken from Gateways > Soft-Gateway (if required for list endpoint)
+
+baseUrl: https://api.service.ff-agent.com/v1/WebService/
+# baseUrl: https://free.api.service.ff-agent.com/v1/WebService/
+
+webApiToken: YOUR_WEBAPI_TOKEN
+webApiKey: YOUR_WEBAPI_KEY
+# accessToken: YOUR_ACCESS_TOKEN
+
+# List-Einsätze endpoint path (confirm with ff-agent)
+listEinsaetzePath: einsaetze
+
+# Output path for the JSON file (used by ticker backend)
+outputPath: data/einsaetze.json
+
+# Optional: use if server certificate cannot be verified
+# serverCertFile: lets-encrypt-x3-cross-signed.pem
+
+# Optional: required for Live version (paid). Use separate PEM files:
+# clientCertFile: path/to/ffagent-cert.pem
+# clientKeyFile: path/to/ffagent-key.pem

fetch_einsaetze.py

@@ -0,0 +1,155 @@
+#!/usr/bin/env python3
+"""
+FF-Agent WebAPI client: fetch all Einsätze and write them to a local JSON file.
+Config via config.yml (baseUrl, webApiToken, webApiKey, optional certs).
+Endpoint path and HMAC rule are configurable for when ff-agent provides the list-Einsätze spec.
+"""
+
+import argparse
+import hashlib
+import hmac as hmac_lib
+import json
+import logging
+import sys
+from pathlib import Path
+from datetime import datetime, timezone
+
+import requests
+import yaml
+
+logging.basicConfig(
+    level=logging.INFO,
+    format="%(asctime)s %(levelname)s %(message)s",
+    datefmt="%Y-%m-%d %H:%M:%S",
+)
+logger = logging.getLogger(__name__)
+
+DEFAULT_CONFIG_PATH = "config.yml"
+DEFAULT_OUTPUT_PATH = "data/einsaetze.json"
+
+
+def load_config(path: str) -> dict:
+    """Load YAML config. Raises on missing/invalid file."""
+    p = Path(path)
+    if not p.is_file():
+        raise FileNotFoundError(f"Config file not found: {path}")
+    with open(p, "r", encoding="utf-8") as f:
+        data = yaml.safe_load(f)
+    if not data:
+        raise ValueError("Config file is empty")
+    return data
+
+
+def compute_hmac(web_api_key: str, message: str) -> str:
+    """Compute HMAC-SHA256 hex digest over message using webApiKey."""
+    key = web_api_key.encode("utf-8")
+    message_bytes = message.encode("utf-8")
+    sig = hmac_lib.new(key, message_bytes, hashlib.sha256)
+    return sig.hexdigest()
+
+
+def build_headers(cfg: dict) -> dict:
+    """
+    Build request headers including HMAC.
+    For GET list endpoint we use [webApiToken] as sign string until ff-agent specifies otherwise.
+    """
+    token = cfg["webApiToken"]
+    api_key = cfg["webApiKey"]
+    # HMAC rule for GET (no body): sign string = webApiToken (confirm with ff-agent)
+    sign_string = token
+    signature = compute_hmac(api_key, sign_string)
+    return {
+        "Content-Type": "application/json",
+        "Accept": "application/json",
+        "webApiToken": token,
+        "hmac": signature,
+    }
+
+
+def fetch_einsaetze(cfg: dict) -> list:
+    """
+    GET list-Einsätze from FF-Agent WebAPI.
+    Returns list of Einsatz objects. Raises on HTTP or JSON errors.
+    """
+    base_url = cfg["baseUrl"].rstrip("/")
+    path = cfg.get("listEinsaetzePath", "einsaetze").strip("/")
+    url = f"{base_url}/{path}"
+    headers = build_headers(cfg)
+    verify = cfg.get("serverCertFile") or True
+    cert = None
+    if cfg.get("clientCertFile") and cfg.get("clientKeyFile"):
+        cert = (cfg["clientCertFile"], cfg["clientKeyFile"])
+    elif cfg.get("clientCertFile"):
+        # Single file: assume PEM with cert+key or cert only (no p12 support without extra dep)
+        cert = cfg["clientCertFile"]
+
+    logger.info("Requesting %s", url)
+    resp = requests.get(url, headers=headers, cert=cert, verify=verify, timeout=30)
+    resp.raise_for_status()
+    data = resp.json()
+    # Normalize: API might return {"einsaetze": [...]} or direct array
+    if isinstance(data, list):
+        return data
+    if isinstance(data, dict) and "einsaetze" in data:
+        return data["einsaetze"]
+    if isinstance(data, dict):
+        # Some APIs return {"data": [...]} or similar
+        for key in ("data", "items", "operations"):
+            if isinstance(data.get(key), list):
+                return data[key]
+    return data if isinstance(data, list) else [data]
+
+
+def write_output(einsaetze: list, output_path: str) -> None:
+    """Write einsaetze to JSON file with updated timestamp. Creates parent dir if needed."""
+    out = Path(output_path)
+    out.parent.mkdir(parents=True, exist_ok=True)
+    payload = {
+        "einsaetze": einsaetze,
+        "updated": datetime.now(timezone.utc).isoformat(),
+    }
+    with open(out, "w", encoding="utf-8") as f:
+        json.dump(payload, f, ensure_ascii=False, indent=2)
+    logger.info("Wrote %d Einsätze to %s", len(einsaetze), output_path)
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(
+        description="Fetch FF-Agent Einsätze and write to JSON for ticker backend."
+    )
+    parser.add_argument(
+        "--config",
+        default=DEFAULT_CONFIG_PATH,
+        help="Path to YAML config (default: config.yml)",
+    )
+    parser.add_argument(
+        "--output",
+        default=None,
+        help="Output JSON path (overrides config outputPath)",
+    )
+    args = parser.parse_args()
+
+    try:
+        cfg = load_config(args.config)
+    except (FileNotFoundError, ValueError) as e:
+        logger.error("%s", e)
+        return 1
+
+    output_path = args.output or cfg.get("outputPath") or DEFAULT_OUTPUT_PATH
+
+    try:
+        einsaetze = fetch_einsaetze(cfg)
+        write_output(einsaetze, output_path)
+        return 0
+    except requests.RequestException as e:
+        logger.error("Request failed: %s", e)
+        if hasattr(e, "response") and e.response is not None and e.response.text:
+            logger.debug("Response: %s", e.response.text[:500])
+        return 1
+    except json.JSONDecodeError as e:
+        logger.error("Invalid JSON response: %s", e)
+        return 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

import_einsaetze_csv.py

@@ -0,0 +1,120 @@
+#!/usr/bin/env python3
+"""
+Import Einsätze from FF-Agent CSV export and write JSON in the same format
+as fetch_einsaetze.py (for use by the ticker backend).
+CSV format: semicolon-delimited, quoted fields, header row.
+"""
+
+import argparse
+import csv
+import json
+import logging
+import sys
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import List
+
+logging.basicConfig(
+    level=logging.INFO,
+    format="%(asctime)s %(levelname)s %(message)s",
+    datefmt="%Y-%m-%d %H:%M:%S",
+)
+logger = logging.getLogger(__name__)
+
+ENCODINGS = ("utf-8-sig", "utf-8", "cp1252")
+
+
+def detect_encoding(path: Path) -> str:
+    """Try to read first bytes and return encoding that works for the header."""
+    raw = path.read_bytes()
+    for enc in ENCODINGS:
+        try:
+            raw.decode(enc)
+            return enc
+        except UnicodeDecodeError:
+            continue
+    return "utf-8"
+
+
+def parse_csv(path: str) -> List[dict]:
+    """
+    Parse semicolon-delimited CSV with quoted fields.
+    Returns list of row dicts; keys are header names (quotes stripped).
+    """
+    p = Path(path)
+    if not p.is_file():
+        raise FileNotFoundError(f"CSV file not found: {path}")
+
+    encoding = detect_encoding(p)
+    logger.info("Using encoding: %s", encoding)
+
+    with open(p, "r", encoding=encoding, newline="") as f:
+        reader = csv.reader(f, delimiter=";", quotechar='"')
+        rows = list(reader)
+
+    if not rows:
+        raise ValueError("CSV file is empty")
+
+    header = [cell.strip('"') for cell in rows[0]]
+    einsaetze = []
+    for row in rows[1:]:
+        # Pad row to header length so we don't lose columns
+        while len(row) < len(header):
+            row.append("")
+        record = {}
+        for i, key in enumerate(header):
+            value = row[i].strip('"') if i < len(row) else ""
+            record[key] = value
+        # Skip completely empty rows
+        if any(v for v in record.values()):
+            einsaetze.append(record)
+
+    return einsaetze
+
+
+def write_json(einsaetze: List[dict], output_path: str) -> None:
+    """Write einsaetze to JSON with updated timestamp. Creates parent dir if needed."""
+    out = Path(output_path)
+    out.parent.mkdir(parents=True, exist_ok=True)
+    payload = {
+        "einsaetze": einsaetze,
+        "updated": datetime.now(timezone.utc).isoformat(),
+    }
+    with open(out, "w", encoding="utf-8") as f:
+        json.dump(payload, f, ensure_ascii=False, indent=2)
+    logger.info("Wrote %d Einsätze to %s", len(einsaetze), output_path)
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(
+        description="Import FF-Agent CSV export and output JSON for ticker backend."
+    )
+    parser.add_argument(
+        "input",
+        nargs="?",
+        default=None,
+        help="Input CSV file path",
+    )
+    parser.add_argument(
+        "--output",
+        "-o",
+        default="data/einsaetze.json",
+        help="Output JSON path (default: data/einsaetze.json)",
+    )
+    args = parser.parse_args()
+
+    if not args.input:
+        parser.error("Input CSV path required (or pass as positional argument)")
+        return 1
+
+    try:
+        einsaetze = parse_csv(args.input)
+        write_json(einsaetze, args.output)
+        return 0
+    except (FileNotFoundError, ValueError) as e:
+        logger.error("%s", e)
+        return 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

requirements.txt

@@ -0,0 +1,3 @@
+requests>=2.28.0
+PyYAML>=6.0
+flask>=2.0

webhook_receiver.php

@@ -0,0 +1,111 @@
+<?php
+/**
+ * Webhook receiver: accepts POST with JSON body and appends to data/einsaetze.json.
+ * Payload is stored under webhook_events[] with a received timestamp; einsaetze is unchanged.
+ */
+
+declare(strict_types=1);
+
+header('Content-Type: application/json; charset=utf-8');
+
+const DEFAULT_DATA_PATH = 'data/einsaetze.json';
+const MAX_BODY_SIZE = 1024 * 1024 * 10; // 10 MB
+
+$dataPath = getenv('EINSATZ_JSON_PATH') ?: DEFAULT_DATA_PATH;
+
+if ($_SERVER['REQUEST_METHOD'] !== 'POST') {
+    http_response_code(405);
+    echo json_encode(['error' => 'Method Not Allowed']);
+    exit;
+}
+
+$raw = file_get_contents('php://input');
+if ($raw === false || $raw === '') {
+    http_response_code(400);
+    echo json_encode(['error' => 'Invalid or empty JSON']);
+    exit;
+}
+
+if (strlen($raw) > MAX_BODY_SIZE) {
+    http_response_code(400);
+    echo json_encode(['error' => 'Request body too large']);
+    exit;
+}
+
+$payload = json_decode($raw, true);
+if (json_last_error() !== JSON_ERROR_NONE) {
+    http_response_code(400);
+    echo json_encode(['error' => 'Invalid or empty JSON']);
+    exit;
+}
+
+$received = (new DateTimeImmutable('now', new DateTimeZone('UTC')))->format('Y-m-d\TH:i:s.uP');
+
+$dir = dirname($dataPath);
+if (!is_dir($dir)) {
+    if (!@mkdir($dir, 0755, true)) {
+        http_response_code(500);
+        echo json_encode(['error' => 'Could not create data directory']);
+        exit;
+    }
+}
+
+$fp = fopen($dataPath, 'c+');
+if ($fp === false) {
+    http_response_code(500);
+    echo json_encode(['error' => 'Could not open data file']);
+    exit;
+}
+
+if (!flock($fp, LOCK_EX)) {
+    fclose($fp);
+    http_response_code(500);
+    echo json_encode(['error' => 'Could not lock data file']);
+    exit;
+}
+
+$content = stream_get_contents($fp);
+$data = [];
+if ($content !== false && $content !== '') {
+    $data = json_decode($content, true);
+    if (!is_array($data)) {
+        $data = [];
+    }
+}
+
+if (!isset($data['einsaetze'])) {
+    $data['einsaetze'] = [];
+}
+if (!isset($data['webhook_events'])) {
+    $data['webhook_events'] = [];
+}
+
+$data['webhook_events'][] = [
+    'received' => $received,
+    'data' => $payload,
+];
+$data['updated'] = $received;
+
+$json = json_encode($data, JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE);
+if ($json === false) {
+    flock($fp, LOCK_UN);
+    fclose($fp);
+    http_response_code(500);
+    echo json_encode(['error' => 'Could not encode JSON']);
+    exit;
+}
+
+ftruncate($fp, 0);
+rewind($fp);
+$written = fwrite($fp, $json);
+flock($fp, LOCK_UN);
+fclose($fp);
+
+if ($written === false || $written !== strlen($json)) {
+    http_response_code(500);
+    echo json_encode(['error' => 'Could not write data file']);
+    exit;
+}
+
+http_response_code(200);
+echo json_encode(['ok' => true, 'received' => $received]);