# Project Heloha

A detailed, actionable task list for building a unified, multi-radar severe weather analysis and alerting platform for Oklahoma.

---

## Phase 0: Foundation & Setup (The First Week)

*Goal: Establish a clean, professional Go project environment.*

-   [x] **Initialize Git Repository:**
    -   [x] `git init`
    -   [x] Create a `.gitignore` file for Go and common OS files.
    -   [x] Create a `README.md` with the project's mission statement.
-   [x] **Establish Go Project Structure:**
    -   [x] `go mod init github.com/blakeridgway/heloha` 
    -   [x] Create a standard Go project layout:
        -   `/cmd/heloha-server`: Main application entry point.
        -   `/internal/radar`: Code for fetching, parsing, and processing radar data.
        -   `/internal/server`: HTTP handlers and server logic.
        -   `/internal/config`: Configuration management.
        -   `/web/templates`: HTML templates for the frontend.
        -   `/web/static`: CSS and JavaScript assets.
-   [x] **Create a Basic Web Server:**
    -   [x] In `/cmd/heloha-server/main.go`, set up an `http.Server`.
    -   [x] Choose and implement a router (e.g., `go-chi/chi` is a good choice for middleware and flexibility).
    -   [x] Create a simple `/healthz` endpoint that returns `200 OK`.
-   [x] **Set Up Build Automation:**
    -   [x] Create a `Makefile` with targets for:
        -   `build`: `go build -o bin/heloha-server ./cmd/heloha-server`
        -   `run`: `go run ./cmd/heloha-server`
        -   `test`: `go test ./...`
        -   `tidy`: `go mod tidy`

---

## Phase 1: Single Radar Ingestion & Display (The MVP)

*Goal: Get raw data from one radar onto a map on a webpage. This proves the core data pipeline is viable.*

-   [ ] **Data Acquisition (`/internal/radar/fetch.go`):**
    -   [ ] Add the AWS SDK for Go V2 (`aws-sdk-go-v2`) as a dependency.
    -   [ ] **Use NEXRAD Level 3 data** from the `unidata-nexrad-level3` S3 bucket. Level 3 products are pre-processed by the RPG into a simple binary format — no polar-to-Cartesian projection needed, and no per-record bzip2 complexity.
    -   [ ] Implement logic to find the *latest* file for a given radar (`KTLX`) and product code (`N0B` = Base Reflectivity 0.5° tilt) using the S3 key naming convention.
    -   [ ] Write a function to download a specific Level 3 file from S3 into memory.
    -   [ ] *Note: Level 2 data (raw I/Q + full velocity fields) will be introduced in Phase 3 when TVS detection requires it. Level 3 is sufficient for reflectivity display and mosaic.*
-   [ ] **Data Parsing (`/internal/radar/parse.go`):**
    -   [ ] Implement a Level 3 product parser targeting the Graphic Product Message format (ICD 2620001). The header is fixed-width; the symbology block contains pre-gridded radial data.
    -   [ ] Define Go `structs` to hold the parsed data: a `RadarProduct` struct with base reflectivity (a 2D array of floats), timestamp, elevation angle, and radar site coordinates.
-   [ ] **Visualization Backend (`/internal/server/handlers.go`):**
    -   [ ] Create a new HTTP handler for map tiles: `/api/v1/tile/ktlx/{z}/{x}/{y}.png`.
    -   [ ] Inside the handler, implement logic to:
        -   Load the latest parsed `KTLX` data.
        -   Use the Z/X/Y tile coordinates to determine the required geographic bounds.
        -   Map the pre-gridded radar data onto the requested tile canvas (no polar projection needed at this stage).
        -   Use a color lookup table (LUT) to convert dBZ values to colors (e.g., green for light rain, red for heavy).
        -   Render the final tile as a PNG image using Go's standard `image` package.
-   [ ] **Tile Caching (`/internal/server/tilecache.go`):**
    -   [ ] Implement an in-memory tile cache as a `map[string][]byte` (key: `"z/x/y"`) protected by a `sync.RWMutex`.
    -   [ ] Wrap the tile handler: check the cache first; on a miss, render the PNG, store it in the cache, and return it.
    -   [ ] Tie cache invalidation to the data update cycle — when a new `RadarProduct` is ingested, clear the cache for that radar. This ensures tiles are never stale beyond one update interval.
-   [ ] **Frontend Display (`/web/templates/index.html`):**
    -   [ ] Set up a basic HTML page that includes Leaflet.js from a CDN.
    -   [ ] Initialize a Leaflet map centered on Oklahoma.
    -   [ ] Add a `L.tileLayer` pointing to your new `/api/v1/tile/ktlx/{z}/{x}/{y}.png` endpoint.
    -   [ ] Add the HTMX script tag.
    -   [ ] Use `hx-get` and `hx-trigger="every 60s"` on a container element to periodically refresh the map layer or associated metadata.

---

## Phase 2: Multi-Radar Fusion (The Core Challenge)

*Goal: Transition from a single, siloed view to a single, authoritative statewide weather picture.*

-   [ ] **Concurrent Ingestion (`/internal/radar/manager.go`):**
    -   [ ] Design a concurrent manager that spawns one "fetcher" goroutine per radar (KTLX, KINX, KFDR, KVNX, etc.).
    -   [ ] Each fetcher goroutine periodically checks for new data for its assigned radar.
    -   [ ] Use a buffered Go `channel` to pass newly downloaded and parsed `RadarProduct` objects to a central processing component.
-   [ ] **Data Mosaicing (`/internal/radar/mosaic.go`):**
    -   [ ] **Grid Definition:** Define a constant statewide grid in your code. This includes its geographic boundaries (min/max lat/lon), resolution (e.g., 1km x 1km), and dimensions (width/height in pixels).
    -   [ ] **Projection Logic:** Implement robust math functions to convert from (radar, azimuth, range) -> (lat, lon) -> (grid X, grid Y). This is crucial.
    -   [ ] **Compositing Engine:**
        -   Create a function that runs in a loop, triggered by new data or a timer.
        -   This function initializes a new, empty statewide grid.
        -   It iterates through every cell of the grid. For each cell, it checks all available radars to see which one provides the best data (lowest beam altitude above that grid point).
        -   It then "paints" the data from the best radar onto that grid cell.
    -   [ ] **State Management:** Store the latest completed mosaic grid in memory, protected by a mutex, so it can be safely accessed by the tile-serving handler.
-   [ ] **Update Visualization Backend:**
    -   [ ] Modify the `/api/v1/tile/...` handler to render tiles from the statewide mosaic grid instead of a single radar product.
    -   [ ] The tile handler no longer needs the radar ID in the URL. New URL: `/api/v1/tile/reflectivity/{z}/{x}/{y}.png`.

---

## Phase 3: Advanced Analysis & Alerting

*Goal: Find the "so what?" in the data. Identify and track dangerous storms.*

-   [ ] **Expand Data Parsing (introduce Level 2):**
    -   [ ] Add Level 2 ingestion from the `noaa-nexrad-level2` S3 bucket alongside the existing Level 3 pipeline. Level 2 files are compressed with bzip2 on a per-record basis — implement record-level decompression.
    -   [ ] Parse Level 2 Velocity, Differential Reflectivity (ZDR), and Correlation Coefficient (CC) data fields. The official format spec is ICD RDA/RPG 2620002U.
    -   [ ] Update the mosaicing engine to create composite grids for these new data types.
-   [ ] **Storm Cell Identification (`/internal/analysis/segmentation.go`):**
    -   [ ] Implement a "blob detection" algorithm on the composite reflectivity grid. This can be a simple algorithm like flood-fill or a more advanced one like DBSCAN.
    -   [ ] The output should be a list of distinct `StormCell` objects, each with a polygon defining its boundary.
-   [ ] **Signature Detection (`/internal/analysis/signatures.go`):**
    -   [ ] For each identified `StormCell`, analyze the underlying data within its polygon.
    -   [ ] Implement a TVS (Tornadic Vortex Signature) detector: Search for strong, compact, inbound/outbound velocity couplets.
    -   [ ] Implement a Hail Core detector: Search for areas of high reflectivity (>55 dBZ) co-located with low CC (< 0.95) and near-zero ZDR.
-   [ ] **Storm Tracking (`/internal/analysis/tracking.go`):**
    -   [ ] Implement a centroid tracking algorithm.
    -   [ ] For each frame, calculate the center-of-mass for each `StormCell`.
    -   [ ] Correlate cells between frames by finding the closest centroid from the previous frame.
    -   [ ] Assign a persistent `TrackID` to each storm. Store the storm's history (location, time, intensity) in memory.
-   [ ] **Threat Generation (`/internal/analysis/threats.go`):**
    -   [ ] Define a `Threat` struct: `TrackID`, `ThreatType` (Tornado, Hail), `Severity`, `PredictedPath` (a GeoJSON line or polygon).
    -   [ ] Create a rules engine that ingests `StormCell` objects and generates `Threat` objects.
    -   [ ] Example Rule: IF `StormCell.HasTVS == true` AND `StormCell.Intensity > Threshold`, THEN generate a "Tornado Warning" threat object.
    -   [ ] Extrapolate a simple predicted path based on the storm's recent movement vector.

---

## Phase 4: Frontend Refinement with HTMX

*Goal: Build a dynamic, interactive, and useful user interface without a heavy JS framework.*

-   [ ] **Create API Endpoints for UI Components:**
    -   [ ] `/ui/storm-list`: Returns an HTML fragment (`<ul>...</ul>`) of currently active tracked storms.
    -   [ ] `/ui/storm-detail/{track_id}`: Returns an HTML fragment (`<div>...</div>`) with detailed stats for a specific storm.
-   [ ] **Dynamic UI with HTMX:**
    -   [ ] Create a sidebar on the main page. Use `hx-get="/ui/storm-list"` and `hx-trigger="every 15s"` to keep the list of active storms up-to-date.
    -   [ ] Make each item in the storm list clickable. Use `hx-get="/ui/storm-detail/{track_id}"` and `hx-target="#detail-pane"` to load storm details into another panel without a page reload.
-   [ ] **Map Interactivity:**
    -   [ ] When a storm detail is loaded, add a GeoJSON layer to the Leaflet map showing its current position and predicted track.
    -   [ ] Use a small vanilla JavaScript `htmx:afterSwap` event listener to command the Leaflet map to pan and zoom to the selected storm's coordinates (passed as `data-` attributes on the swapped HTML fragment).
    -   [ ] Create a search bar that allows a user to input an address. Use an external geocoding API to get coordinates and place a marker on the map.

---

## Phase 5: SRE & Productionization

*Goal: Ensure the system is reliable, scalable, and observable, ready for real-world use.*

-   [ ] **Containerization:**
    -   [ ] Write a multi-stage `Dockerfile`. The first stage builds the Go binary, the second copies the binary into a minimal `distroless` or `alpine` base image for a small, secure result.
-   [ ] **Configuration Management:**
    -   [ ] Move all hardcoded settings (radar lists, thresholds) into a configuration file (e.g., `config.yaml`) or environment variables. Use a library like Viper to manage this.
-   [ ] **Deployment (Docker Compose):**
    -   [ ] Write a `docker-compose.yml` that starts the `heloha-server` container alongside a Prometheus container scraping its `/metrics` endpoint.
    -   [ ] Implement proper liveness (`/healthz`) and readiness probes in your Go server. The readiness check should fail if no radar data has been successfully processed recently.
    -   [ ] *Future path: when ready to scale, migrate to Kubernetes manifests (Deployment, Service, Ingress) packaged with Helm. The Dockerfile and `/healthz`/readiness endpoints you build now translate directly.*
-   [ ] **Observability:**
    -   [ ] **Metrics:** Add the `prometheus/client_golang` library. Instrument your code to export key metrics:
        -   `heloha_radar_files_processed_total{radar="KTLX", status="success"}`
        -   `heloha_data_latency_seconds` (time from file timestamp to processing completion)
        -   `heloha_active_threats_gauge`
    -   [ ] **Logging:** Use Go's stdlib `slog` with JSON output. Log key events (new threat detected, radar feed lost, tile cache stats).
    -   [ ] **Alerting:** Configure Prometheus and Alertmanager (included in the Compose stack). Create critical alerts:
        -   `ALERT RadarFeedDown IF (time() - last_success_timestamp{radar="KTLX"}) > 600`
        -   `ALERT HighProcessingLatency`
        -   `ALERT AppDown (based on container health check)`

---

## Backlog / Future Ideas

-   [ ] **Mesonet Integration:** Add another data ingestion pipeline for the Oklahoma Mesonet. Correlate ground-level wind speed/pressure drops with radar signatures.
-   [ ] **Database Persistence:** Store historical storm tracks and threat polygons in a time-series or geospatial database (e.g., TimescaleDB, PostGIS).
-   [ ] **Push Notifications:** Implement a system (e.g., WebSockets or a mobile app) to send real-time alerts to users based on their registered location.
-   [ ] **Machine Learning:** Train a model on historical radar data and confirmed tornado reports to create a more accurate probabilistic threat model.