你是一個「影片偵探」AI，負責為監控影格做事件描述與風險判讀。
請根據：
1) 影格畫面內容
2) 我提供的偵測結果（物件類別、bbox、信心值）
輸出結構化 JSON，方便後續索引與查詢。

規則：
- 不要編造畫面中不存在的事物。
- 若畫面資訊不足，請在 unsure 字段說明原因。
- suspicious_score 0~100：越高越可疑，但必須用畫面線索與偵測結果解釋。
- 需同時輸出一段適合全文檢索的繁體中文摘要 summary_zh。

輸入：
timestamp: {{timestamp}}
camera_id: {{camera_id}}
detections: {{yolo_detections_json}}
image: <提供影像>

輸出 JSON 欄位：
{
  "timestamp": "...",
  "camera_id": "...",
  "scene_summary": "...",
  "entities": [
    {"type": "...", "count": 1, "notable": "..."}
  ],
  "actions": ["..."],
  "suspicious_score": 0,
  "suspicious_reasons": ["..."],
  "unsure": ["..."],
  "summary_zh": "..."
}

你是一個監控「影片偵探」AI。以下提供同一段時間序列的多張關鍵影格，
每張都有 timestamp 與偵測結果。請完成：

1) 以時間順序描述發生了什麼事（避免臆測）。
2) 是否存在可疑行為？若有，指出關鍵時間點與依據。
3) 產出適合儲存到事件資料庫的 JSON（含 event_type、start/end、關聯track_id）。

輸入格式（重複多次）：
- frame_i.timestamp: ...
- frame_i.detections: ...
- frame_i.image: <圖>

輸出：
- timeline_bullets（條列，含時間）
- event_json：
{
  "event_type": "...",
  "start_time": "...",
  "end_time": "...",
  "actors": [{"track_id":"...","type":"person/vehicle","notes":"..."}],
  "evidence": [{"timestamp":"...","what":"..."}],
  "risk_level": "low/medium/high",
  "search_keywords_zh": ["..."]
}

你是影片偵探助理。你會收到：
- 使用者問題
- 檢索得到的候選事件清單（含摘要、時間、相機、佐證影格描述）

請：
1) 直接回答問題（繁體中文）
2) 附上最相關的事件時間範圍（start~end）
3) 列出 1~3 個佐證點（引用候選事件的 evidence）
4) 若資訊不足，說明缺什麼

輸入：
question: {{user_question}}
candidates: {{retrieved_events_json}}

輸出：
{
  "answer_zh": "...",
  "best_matches": [
    {"camera_id":"...","time_range":"...","why":"...","evidence":["..."]}
  ],
  "missing_info": ["..."]
}

import cv2
import os

def extract_frames(video_path, out_dir, fps_sample=1):
    os.makedirs(out_dir, exist_ok=True)
    cap = cv2.VideoCapture(video_path)
    fps = cap.get(cv2.CAP_PROP_FPS) or 30
    step = max(int(fps / fps_sample), 1)

    idx = 0
    frame_id = 0
    while True:
        ok, frame = cap.read()
        if not ok:
            break
        if frame_id % step == 0:
            out_path = os.path.join(out_dir, f"frame_{idx:06d}.jpg")
            cv2.imwrite(out_path, frame)
            idx += 1
        frame_id += 1

    cap.release()
    return idx

[
  {"type":"person","conf":0.92,"bbox":[x1,y1,x2,y2]},
  {"type":"car","conf":0.88,"bbox":[x1,y1,x2,y2]},
  {"type":"face","conf":0.81,"bbox":[x1,y1,x2,y2]}
]

import json

def build_prompt(ts, cam, detections):
    return f"""
你是一個影片偵探AI。請依據影像與偵測結果輸出JSON（繁體中文）。

timestamp: {ts}
camera_id: {cam}
detections: {json.dumps(detections, ensure_ascii=False)}

輸出JSON欄位：
{{
  "timestamp": "...",
  "camera_id": "...",
  "scene_summary": "...",
  "actions": ["..."],
  "suspicious_score": 0,
  "suspicious_reasons": ["..."],
  "summary_zh": "..."
}}
"""

# 送給 Ollama 的地方，依你使用的套件/HTTP API 填上：
# response = ollama.chat(model="llama3.2-vision:90b", messages=[...], images=[...])

{
  "frame_id": "cam01_20251216_220501_000123",
  "camera_id": "cam01",
  "timestamp": "2025-12-16T22:05:01",
  "image_path": "/frames/cam01/frame_000123.jpg",
  "detections": [],
  "vlm_json": {},
  "summary_zh": "..."
}

{
  "event_id": "evt_20251216_cam01_0007",
  "camera_id": "cam01",
  "start_time": "2025-12-16T22:05:00",
  "end_time": "2025-12-16T22:06:30",
  "event_type": "徘徊/闖入/越線/遺留物/群聚",
  "actors": [{"track_id":"p12","type":"person"}],
  "evidence": [
    {"timestamp":"2025-12-16T22:05:15","what":"人物在後門附近停留，反覆靠近門把"}
  ],
  "risk_level": "medium",
  "search_keywords_zh": ["後門","徘徊","可疑人物"]
}

frame
  -> YOLO detections: [x1,y1,x2,y2, conf, cls]
  -> Tracker.update(detections)
  -> tracks: [track_id, bbox, cls, conf, state]
  -> event_rules.update(tracks, timestamp)

[
  {"track_id":"p12","type":"person","conf":0.92,"bbox":[...],"foot":[...]}
]

[事件類型] 闖入
[地點] cam01 後門 restricted_zone
[摘要] 一名人物在22:05進入禁入區，停留約35秒後離開。
[關鍵詞] 後門, 禁入區, 闖入, 可疑人物

你是影片偵探助理。你會收到：
- 使用者問題 question（繁體中文）
- 候選事件 candidates（由索引檢索得到，含 summary_zh、時間、相機、evidence、frame_link）
你的任務是產生「可行動」的答案：

要求：
1) 先用 3~6 句回答使用者，務必引用 candidates 的內容，不要臆測。
2) 產生時間軸 timeline（依時間排序），每點包含：start~end、camera、event_type、1句描述、frame_link（若有）。
3) 如果 candidates 不足以回答，回 missing_info，並說明需要補哪段時間/哪個相機/哪種事件資料。
4) 全部輸出 JSON，方便前端直接渲染。

輸入：
question: {{question}}
candidates: {{candidates_json}}

輸出 JSON：
{
  "answer_zh": "...",
  "timeline": [
    {
      "camera_id": "...",
      "time_range": "YYYY-MM-DD HH:MM:SS ~ HH:MM:SS",
      "event_type": "...",
      "description": "...",
      "evidence": ["..."],
      "frame_links": ["..."]
    }
  ],
  "missing_info": ["..."]
}

{
  "answer_zh":"在 22:05~22:06 期間，cam01 後門區域出現 1 次闖入事件...",
  "timeline":[
    {
      "camera_id":"cam01",
      "time_range":"2025-12-16 22:05:00 ~ 22:06:30",
      "event_type":"闖入",
      "description":"人物 p12 進入 restricted_zone 停留約35秒後離開。",
      "evidence":["22:05:15 人物靠近門把","22:06:10 人物離開區域"],
      "frame_links":["/frames/cam01_20251216_220515_000130.jpg"]
    }
  ],
  "missing_info":[]
}

<!doctype html>
<html lang="zh-Hant">
<head>
  <meta charset="utf-8" />
  <meta name="viewport" content="width=device-width,initial-scale=1" />
  <title>影片偵探 Demo</title>
  <style>
    body { font-family: system-ui, -apple-system, "Noto Sans TC", sans-serif; margin: 16px; }
    .row { display: flex; gap: 8px; flex-wrap: wrap; margin-bottom: 12px; }
    input, select, button { padding: 8px; }
    .card { border: 1px solid #ddd; border-radius: 12px; padding: 12px; margin: 10px 0; }
    .timeline-item { cursor: pointer; padding: 10px; border-radius: 10px; }
    .timeline-item:hover { background: #f5f5f5; }
    .grid { display: grid; grid-template-columns: 1fr 1fr; gap: 12px; }
    img { max-width: 100%; border-radius: 12px; border: 1px solid #eee; }
    pre { background:#fafafa; padding:10px; border-radius:10px; overflow:auto; }
  </style>
</head>
<body>
  <h1>影片偵探 Demo</h1>

  <div class="row">
    <input id="q" size="40" placeholder="例如：後門有沒有人徘徊？白色車出現在哪？" />
    <input id="from" type="datetime-local" />
    <input id="to" type="datetime-local" />
    <select id="camera">
      <option value="">全部相機</option>
      <option value="cam01">cam01</option>
      <option value="cam02">cam02</option>
    </select>
    <button id="btn">查詢</button>
  </div>

  <div class="grid">
    <div>
      <div class="card">
        <h3>答案</h3>
        <div id="answer">尚未查詢</div>
      </div>

      <div class="card">
        <h3>時間軸</h3>
        <div id="timeline"></div>
      </div>

      <div class="card">
        <h3>除錯（回傳 JSON）</h3>
        <pre id="raw">{}</pre>
      </div>
    </div>

    <div>
      <div class="card">
        <h3>影格預覽</h3>
        <div id="frameInfo">點選左側時間軸項目查看影格</div>
        <img id="frame" src="" alt="" style="display:none;" />
      </div>
    </div>
  </div>

<script>
  const $ = (id) => document.getElementById(id);

  function toISOParam(dtLocal) {
    // 直接傳給後端即可（你可在後端再轉時區）
    return dtLocal ? encodeURIComponent(dtLocal) : "";
  }

  function renderTimeline(items) {
    const root = $("timeline");
    root.innerHTML = "";
    if (!items || items.length === 0) {
      root.innerHTML = "<div>沒有找到事件</div>";
      return;
    }
    items.forEach((it, idx) => {
      const div = document.createElement("div");
      div.className = "timeline-item";
      div.innerHTML = `
        <div><b>${it.time_range}</b>｜${it.camera_id}｜${it.event_type}</div>
        <div>${it.description}</div>
      `;
      div.onclick = () => {
        const link = (it.frame_links && it.frame_links[0]) ? it.frame_links[0] : "";
        $("frameInfo").innerText = link ? `影格：${link}` : "此事件沒有影格連結";
        if (link) {
          $("frame").src = link;
          $("frame").style.display = "block";
        } else {
          $("frame").style.display = "none";
        }
      };
      root.appendChild(div);
    });
  }

  $("btn").onclick = async () => {
    const q = $("q").value.trim();
    const from = toISOParam($("from").value);
    const to = toISOParam($("to").value);
    const camera = encodeURIComponent($("camera").value);

    const url = `/api/search?q=${encodeURIComponent(q)}&from=${from}&to=${to}&camera=${camera}`;
    const res = await fetch(url);
    const data = await res.json();

    $("answer").innerText = data.answer_zh || "";
    renderTimeline(data.timeline || []);
    $("raw").innerText = JSON.stringify(data, null, 2);
  };
</script>
</body>
</html>

video-detective-mvp/
  app.py
  requirements.txt
  data/
    events.json
  frames/
    cam01_20251216_220515_000130.jpg   (放你自己的影格檔)
  static/
    index.html

Flask==3.0.3

[
  {
    "event_id": "evt_20251216_cam01_0007",
    "camera_id": "cam01",
    "start_time": "2025-12-16T22:05:00",
    "end_time": "2025-12-16T22:06:30",
    "event_type": "闖入",
    "actors": [{"track_id":"p12","type":"person"}],
    "summary_zh": "一名人物進入後門禁入區，停留約35秒後離開。",
    "keywords_zh": ["後門","禁入區","闖入","可疑人物"],
    "evidence": [
      {"timestamp":"2025-12-16T22:05:15","what":"人物靠近門把並進入禁入區"},
      {"timestamp":"2025-12-16T22:06:10","what":"人物離開禁入區"}
    ],
    "frame_links": ["/frames/cam01_20251216_220515_000130.jpg"]
  },
  {
    "event_id": "evt_20251216_cam02_0011",
    "camera_id": "cam02",
    "start_time": "2025-12-16T21:40:10",
    "end_time": "2025-12-16T21:41:05",
    "event_type": "越線",
    "actors": [{"track_id":"v03","type":"vehicle"}],
    "summary_zh": "一輛車跨越門檻線由外往內進入區域。",
    "keywords_zh": ["越線","車輛","進入","門檻線"],
    "evidence": [
      {"timestamp":"2025-12-16T21:40:22","what":"車輛前輪跨越門檻線"}
    ],
    "frame_links": []
  },
  {
    "event_id": "evt_20251216_cam01_0009",
    "camera_id": "cam01",
    "start_time": "2025-12-16T23:10:00",
    "end_time": "2025-12-16T23:11:00",
    "event_type": "停留",
    "actors": [{"track_id":"p33","type":"person"}],
    "summary_zh": "一名人物在後門區域停留約60秒，移動緩慢，疑似徘徊。",
    "keywords_zh": ["後門","停留","徘徊","可疑"],
    "evidence": [
      {"timestamp":"2025-12-16T23:10:20","what":"人物在後門附近停留且來回踱步"}
    ],
    "frame_links": []
  }
]

<!doctype html>
<html lang="zh-Hant">
<head>
  <meta charset="utf-8" />
  <meta name="viewport" content="width=device-width,initial-scale=1" />
  <title>影片偵探 Demo</title>
  <style>
    body { font-family: system-ui, -apple-system, "Noto Sans TC", sans-serif; margin: 16px; }
    .row { display: flex; gap: 8px; flex-wrap: wrap; margin-bottom: 12px; }
    input, select, button { padding: 8px; }
    .card { border: 1px solid #ddd; border-radius: 12px; padding: 12px; margin: 10px 0; }
    .timeline-item { cursor: pointer; padding: 10px; border-radius: 10px; }
    .timeline-item:hover { background: #f5f5f5; }
    .grid { display: grid; grid-template-columns: 1fr 1fr; gap: 12px; }
    img { max-width: 100%; border-radius: 12px; border: 1px solid #eee; }
    pre { background:#fafafa; padding:10px; border-radius:10px; overflow:auto; }
  </style>
</head>
<body>
  <h1>影片偵探 Demo</h1>

  <div class="row">
    <input id="q" size="42" placeholder="例如：後門有沒有人徘徊？禁入區闖入？越線？" />
    <input id="from" type="datetime-local" />
    <input id="to" type="datetime-local" />
    <select id="camera">
      <option value="">全部相機</option>
      <option value="cam01">cam01</option>
      <option value="cam02">cam02</option>
    </select>
    <button id="btn">查詢</button>
  </div>

  <div class="grid">
    <div>
      <div class="card">
        <h3>答案</h3>
        <div id="answer">尚未查詢</div>
      </div>

      <div class="card">
        <h3>時間軸</h3>
        <div id="timeline"></div>
      </div>

      <div class="card">
        <h3>除錯（回傳 JSON）</h3>
        <pre id="raw">{}</pre>
      </div>
    </div>

    <div>
      <div class="card">
        <h3>影格預覽</h3>
        <div id="frameInfo">點選左側時間軸項目查看影格</div>
        <img id="frame" src="" alt="" style="display:none;" />
      </div>
    </div>
  </div>

<script>
  const $ = (id) => document.getElementById(id);

  function renderTimeline(items) {
    const root = $("timeline");
    root.innerHTML = "";
    if (!items || items.length === 0) {
      root.innerHTML = "<div>沒有找到事件</div>";
      return;
    }
    items.forEach((it) => {
      const div = document.createElement("div");
      div.className = "timeline-item";
      div.innerHTML = `
        <div><b>${it.time_range}</b>｜${it.camera_id}｜${it.event_type}</div>
        <div>${it.description}</div>
      `;
      div.onclick = () => {
        const link = (it.frame_links && it.frame_links[0]) ? it.frame_links[0] : "";
        $("frameInfo").innerText = link ? `影格：${link}` : "此事件沒有影格連結";
        if (link) {
          $("frame").src = link;
          $("frame").style.display = "block";
        } else {
          $("frame").style.display = "none";
        }
      };
      root.appendChild(div);
    });
  }

  $("btn").onclick = async () => {
    const q = $("q").value.trim();
    const from = $("from").value;
    const to = $("to").value;
    const camera = $("camera").value;

    const url = `/api/search?q=${encodeURIComponent(q)}&from=${encodeURIComponent(from)}&to=${encodeURIComponent(to)}&camera=${encodeURIComponent(camera)}`;
    const res = await fetch(url);
    const data = await res.json();

    $("answer").innerText = data.answer_zh || "";
    renderTimeline(data.timeline || []);
    $("raw").innerText = JSON.stringify(data, null, 2);
  };
</script>
</body>
</html>

from __future__ import annotations

import json
import os
from dataclasses import dataclass
from datetime import datetime
from typing import Any, Dict, List, Optional, Tuple

from flask import Flask, jsonify, request, send_from_directory


BASE_DIR = os.path.dirname(os.path.abspath(__file__))
DATA_PATH = os.path.join(BASE_DIR, "data", "events.json")
FRAMES_DIR = os.path.join(BASE_DIR, "frames")


def parse_dt(s: str) -> Optional[datetime]:
    """
    Accepts:
      - "" -> None
      - HTML datetime-local: "2025-12-16T22:05"
      - ISO: "2025-12-16T22:05:00"
    """
    if not s:
        return None
    s = s.strip()
    for fmt in ("%Y-%m-%dT%H:%M", "%Y-%m-%dT%H:%M:%S"):
        try:
            return datetime.strptime(s, fmt)
        except ValueError:
            pass
    # Fallback: try fromisoformat (may accept seconds/microseconds)
    try:
        return datetime.fromisoformat(s)
    except ValueError:
        return None


def load_events() -> List[Dict[str, Any]]:
    if not os.path.exists(DATA_PATH):
        return []
    with open(DATA_PATH, "r", encoding="utf-8") as f:
        return json.load(f)


def dt_in_range(start: datetime, end: datetime, q_from: Optional[datetime], q_to: Optional[datetime]) -> bool:
    if q_from and end < q_from:
        return False
    if q_to and start > q_to:
        return False
    return True


def text_score(query_tokens: List[str], event: Dict[str, Any]) -> int:
    """
    Minimal keyword scoring:
      - match in event_type, summary_zh, keywords_zh, evidence.what
    """
    blob_parts = []
    blob_parts.append(str(event.get("event_type", "")))
    blob_parts.append(str(event.get("summary_zh", "")))
    blob_parts.extend(event.get("keywords_zh", []) or [])
    for ev in event.get("evidence", []) or []:
        blob_parts.append(str(ev.get("what", "")))

    blob = " ".join(blob_parts).lower()
    score = 0
    for t in query_tokens:
        if not t:
            continue
        if t in blob:
            score += 10
    return score


def tokenize_zh(query: str) -> List[str]:
    """
    MVP 簡化版 tokenization：
    - 以空白切
    - 再把常見關鍵詞直接保留（你可改成 jieba/ckip/hfl 分詞）
    """
    q = (query or "").strip().lower()
    if not q:
        return []
    tokens = [x for x in q.replace("，", " ").replace(",", " ").split() if x]
    # 也把一些常見事件詞補入（避免使用者只打「徘徊」但資料用「停留」）
    synonyms = {
        "徘徊": ["停留"],
        "停留": ["徘徊"],
        "闖入": ["入侵", "禁入"],
        "越線": ["跨線", "穿越"],
        "後門": ["門口"]
    }
    expanded = []
    for t in tokens:
        expanded.append(t)
        for s in synonyms.get(t, []):
            expanded.append(s)
    return expanded


def build_timeline_item(e: Dict[str, Any]) -> Dict[str, Any]:
    def fmt_range(s: str, t: str) -> str:
        try:
            sdt = datetime.fromisoformat(s)
            tdt = datetime.fromisoformat(t)
            return f"{sdt:%Y-%m-%d %H:%M:%S} ~ {tdt:%Y-%m-%d %H:%M:%S}"
        except Exception:
            return f"{s} ~ {t}"

    evidence = []
    for ev in e.get("evidence", []) or []:
        ts = ev.get("timestamp", "")
        what = ev.get("what", "")
        if ts and what:
            evidence.append(f"{ts} {what}")
        elif what:
            evidence.append(what)

    return {
        "camera_id": e.get("camera_id", ""),
        "time_range": fmt_range(e.get("start_time", ""), e.get("end_time", "")),
        "event_type": e.get("event_type", ""),
        "description": e.get("summary_zh", ""),
        "evidence": evidence[:3],
        "frame_links": e.get("frame_links", []) or [],
    }


def build_answer(query: str, matched: List[Dict[str, Any]]) -> str:
    if not query.strip():
        return "請輸入查詢內容（例如：後門 徘徊、禁入區 闖入、越線）。"
    if not matched:
        return "沒有找到符合條件的事件。你可以嘗試放寬時間範圍、改用關鍵詞（例如：後門、闖入、停留、越線）。"

    # MVP：用最前面幾筆拼一段可讀答案（之後你可接 LLM/RAG）
    top = matched[:3]
    parts = []
    parts.append(f"找到 {len(matched)} 筆候選事件，最相關的如下：")
    for e in top:
        parts.append(f"- {e.get('camera_id','')}｜{e.get('event_type','')}｜{e.get('summary_zh','')}")
    return "\n".join(parts)


app = Flask(__name__, static_folder="static", static_url_path="/")


@app.get("/")
def index():
    # 直接用 static/index.html
    return app.send_static_file("index.html")


@app.get("/frames/<path:filename>")
def frames(filename: str):
    # 提供影格圖片
    return send_from_directory(FRAMES_DIR, filename)


@app.get("/api/search")
def api_search():
    q = request.args.get("q", "")
    camera = request.args.get("camera", "").strip()
    q_from = parse_dt(request.args.get("from", ""))
    q_to = parse_dt(request.args.get("to", ""))

    events = load_events()
    tokens = tokenize_zh(q)

    # 先做基本篩選（camera + time range）
    filtered = []
    for e in events:
        if camera and e.get("camera_id") != camera:
            continue
        s = parse_dt(e.get("start_time", "")) or datetime.min
        t = parse_dt(e.get("end_time", "")) or datetime.min
        if not dt_in_range(s, t, q_from, q_to):
            continue
        filtered.append(e)

    # 關鍵字打分排序（MVP）
    scored: List[Tuple[int, Dict[str, Any]]] = []
    for e in filtered:
        score = text_score(tokens, e)
        if tokens and score <= 0:
            continue
        scored.append((score, e))

    scored.sort(key=lambda x: x[0], reverse=True)
    matched = [e for _, e in scored][:20]

    resp = {
        "answer_zh": build_answer(q, matched),
        "timeline": [build_timeline_item(e) for e in matched],
        "missing_info": [] if matched else ["建議擴大時間範圍或指定相機，並使用關鍵字：後門、闖入、停留、越線。"],
    }
    return jsonify(resp)


if __name__ == "__main__":
    os.makedirs(FRAMES_DIR, exist_ok=True)
    app.run(host="0.0.0.0", port=5000, debug=True)

cd video-detective-mvp
python -m venv .venv
# Windows: .venv\Scripts\activate
source .venv/bin/activate

pip install -r requirements.txt
python app.py