riprap / flood and heat briefings NYC
methodology gallery

About

About Riprap: method, AI disclosure and limits

Riprap answers flood and heat questions about New York City places from public records. Rules or an open Granite model read your question and choose the evidence. Every sentence of evidence is written by code from a public record and cites its source and date.

How an answer is made

For each place, Riprap reads the public sources that cover it. Code turns each record into one sentence that carries the record's source and the date of its data. A question is answered by rules over its own words: they pick which of those sentences answer it, and whether the figures support a yes, a no or a count. The sentences are shown unchanged.

The sentences are written by Riprap's code from the records. They are not quotations of a publisher's own prose, and some state a figure Riprap computed from a record, such as the share of a district inside a mapped extent. The citation beside each sentence names the record and its date so the figure can be checked. The method in full.

Where a language model can run

By default, none runs. The public site and every saved briefing in the gallery are built with no language model.

A person who runs their own copy can connect one. It was tested with IBM's Granite 4.1 8B, an open-weight model that runs on a laptop. With a model connected:

  • The rules still answer first. The model is called only when no rule handles the query.
  • In planning, the model may set the intent among six (code overrides it when the parser found an address or a district), the target text, the focus, and which sources run beyond a fixed floor. It also writes a rationale of at most 300 characters, which appears only in the JSON plan and the stream's plan event, never in the briefing.
  • In answering, the model may propose a lead from a fixed list (yes, no, partly, a count, or cannot answer) and pick up to four existing sentences and their order.
  • By default the model cannot write a sentence or a number in a briefing: the sentences it picks are shown as code wrote them.
  • No yes or no stands on the model's word. Code overrides the lead for questions about a past event, about now and about a forecast, requires a yes-or-no question shape, and checks the lead against the cited figures. A failure is sent back to the model once; after that the lead and the picks are dropped.
  • Each answer says which path produced it (answer_path in the JSON, rules or llm), for example: "Answered by rules over the question's words; no language model was used."

One developer setting, off by default (RIPRAP_LLM_BARE=1), is the one mode in which a model rewrites evidence as sentences, checked for citation ids and numbers. It is not used on the public site or in the gallery. How answers are checked.

Experimental models

One other open model can add sentences: an estimate of paved and green land from recent satellite images. It is a fine-tune made by Riprap's maintainer and is not an official product. Every sentence from it opens with "Experimental", states the model's limits and its tested accuracy, and follows the sentence from the city's own land cover map.

Two more were tested and taken out. A forecast of surge at the Battery is out of default briefings since 5 October 2026: on 639 held-out four-day windows its mean error was 0.115 m, and damped persistence, a one-line rule, scored 0.108 m and beats it. It foresaw 1 of the 23 windows that reached flood stage. A person running their own copy can switch it back on. The model card hosted on Hugging Face was corrected to say so; its source is in the repository. A satellite water layer was tested twice and retired. The heat briefing uses no model: its forecast is the National Weather Service's. What each model was tested against, and the results.

How Riprap was built

AI coding agents wrote most of Riprap's code, tests and documents, working from prompts written by the maintainer, and agent sessions also reviewed that work. The maintainer, Adam Munawar Rahman, directed it, decides what Riprap claims, and is accountable for all of it.

The output was checked in three ways:

  • automated tests of the code and of the wording of answers;
  • sets of test questions with answer keys, some written without sight of the code (docs/GROUNDING.md);
  • a sanity check on 5 October 2026, run by AI agents separate from those that wrote the code, which worked from the code, the public sources and the app's output before reading any project document. It re-derived 487 briefing sentences from the public sources with separate code: 475 were confirmed, 10 were wrong and 2 sat on the edge of a map cell. The 10 wrong sentences came from three defects (an address elevation read from a neighbouring map cell, hospitals counted twice, and a construction permits count). The first two have since been corrected in the code, and the permits sentence was taken out of plain briefings.

The agents that ran that check worked read-only, under the maintainer's direction, and were not the agents that wrote the code. Its method, its figures, and what was done about each problem it found. The repository records no audit by a person outside the project. Riprap's checks are patterns and rules: they do not read meaning, and they can miss a wrong inference.

These checks confirm that a sentence matches its source. Nothing yet tests the joined evidence against flooding that was observed, so a briefing can agree with every record it reads and still miss what happened on a block.

Riprap is built by one person. No community group, agency or resident has yet shaped or tested it. To send a correction, or an account of flooding or heat that the records miss, see Corrections.

What the data cannot say

  • 311 counts are complaints filed, not floods measured. Published research finds that some neighbourhoods report less often for the same conditions, so a low count is not evidence of a dry block.
  • The city's four stormwater flood maps are modelled scenarios, not forecasts or observations. The city says its map "does not provide the exact depth of flooding at any location".
  • Outside a mapped area is not safe. NYC Emergency Management reports that during Hurricane Ida the most heavily impacted areas, "representing over half of all damaged buildings, were also outside of any flood risk scenario".
  • A FloodNet depth is a measurement at one point under one sensor. It is not the depth across a street or at a building. Riprap counts only the events FloodNet has verified, and it quotes a sensor's status as FloodNet lists it today without interpreting it.
  • A neighbourhood name can be answered for a larger official area, or for part of one. The line under the briefing's title says which area was used.
  • Surface temperature is not air temperature, and the Heat Vulnerability Index is a rank among neighbourhoods. The Health Department says "a neighborhood with low vulnerability does not mean no risk". For air temperature across the city, see BetaNYC's NYC Urban Heat Portal.
  • No one has compared briefings with an independent record of where water stood in a storm.
  • A briefing describes public records about a place. It is not a judgement on a property or on the people who live there, and it is not advice.
  • Riprap is in English only.

The sources for each of these statements, and the distances and time windows Riprap uses, are in docs/METHODOLOGY.md.

Whose project this is

Riprap is an independent, open-source project. It is not a product of the City of New York and is not endorsed by or affiliated with FloodNet, New York University, the City University of New York, FEMA, NOAA, USGS or the City. The City publishes its open data for information only and does not warrant its completeness, accuracy, content or fitness for any use (Local Law 11 of 2012). FloodNet's data, and the sentences and figures made from it, are licensed CC BY-NC-SA 4.0 and are not under Riprap's Apache-2.0 licence. FloodNet's Data Access License Agreement binds anyone who uses its data; FloodNet has not been asked about Riprap's use and has not approved it (what Riprap does about each clause). Other tools for the same records are credited in Related work.

NYC Open Data used

Each dataset by the name NYC Open Data gives it, with its ID. What Riprap does to each before a sentence is written, the copy dates and the sources published elsewhere (federal, state, the Health Department, FloodNet) are in docs/DATA-SOURCES.md.

Corrections

If a sentence is wrong, or reads as saying more than its record does, fill in the correction form with the place and the sentence. The form asks nothing technical; it is public and needs a free GitHub account. Corrections are made in the code and recorded in the changelog. If the error is in a source dataset, the issue will say so and point to the publisher. An account of flooding or heat that the records miss is welcome on the same form; Riprap has no way yet to put one into a briefing, so it is kept there.