Using LLMs to Explain Historical Code: FLOW-MATIC Investigation
The new generation of code assistance tools powered by Large Language Models (LLM)s may be useful in efficiently categorizing and translating historical software corpora. In this qualitative study, we examine LLM performance in explaining and translating FLOW-MATIC programs. We also test their reliability by injecting OCR-like errors into the text. Released in 1958, FLOW-MATIC targeted business applications and used a heavily English-inspired syntax that later influenced COBOL. We find that the models perform inconsistently for describing the key aspects of a program but fare better when translating business logic into SQL. LLMs appear robust against OCR-like errors in the source code, but we caution against relying on the current technology.