Antero Helismaa, chair of the Civic Signal Forum, submitted his resignation on 2 October, shortly after the study appeared in Public Systems Quarterly. Researchers tested 180,000 Japanese-language alerts and summaries across 12 AI models; 1,106 outputs changed an instruction’s practical meaning, including whether people should leave an area or stay clear of it.
Helismaa had led the forum’s work on voluntary standards for private-sector software used to prepare municipal alerts. His departure was not announced in advance. The forum said its board would meet next week to choose an interim chair, while Helismaa told The Plausible Post he had decided to step aside as members review the findings.
How the errors appeared
The study’s authors found that the software most often lost short but crucial qualifiers, such as “not”, or turned a warning about one street into a broader instruction. In one test, a message telling residents not to use a low-lying road became a summary advising them to use it. Researchers said 4,812 outputs changed the practical instruction, but only the 1,106 reversals directly contradicted it.
“A notice can be grammatically clean and operationally wrong,” said Mohit Chaudry, the study’s lead author at the fictional Tokyo Centre for Reliable Software. The team said the errors appeared across products and were not tied to one model maker. Its tests used archived public notices and controlled variations, rather than live emergency broadcasts.
The authors stressed that the results do not show that any resident received a false alert. The test set deliberately included difficult phrasing, and the researchers did not measure how often local staff would catch a faulty summary before sending it. Still, they argued that even a low error rate matters when an alert gives directions during a flood, earthquake or severe-weather warning.
Several Japanese municipalities have been trialling automated tools to shorten notices for mobile phones and translate them for visitors. Kate Boye, a municipal technology manager who reviewed the paper, said staff often work under pressure to issue clear messages quickly. “A human check has to be more than a final glance at fluent text,” she said, calling for systems to show the original alert beside every generated summary.
Standards face a reset
The Civic Signal Forum is an industry group that brings software suppliers and local-government technology staff together to propose common practices; its guidance is voluntary. Helismaa said he had backed a requirement for human approval but now wanted the board to assess whether that safeguard was specific enough. He declined to say whether the study alone prompted his resignation.
The forum said it would pause work on its next alert-software standard until its board reviews the paper. Chaudry’s team has offered to share its test set with municipalities and suppliers, while the researchers plan a follow-up using staff-reviewed drafts. For local authorities, the immediate question is whether they can verify what automated systems change before a short message becomes an official instruction.
