Glossary

This glossary is a work in progress! Please reach out if there are terms you would like to see defined here.

A list of instructions that a computer program can follow, often written by a developer in a programming language. If the algorithm is written in or close to plain language (like Missouri’s Nursing Facility Level of Care algorithm), a non-technical person could also read and follow the same instructions.

When an entity, usually a third party like an Inspector General, compares the system’s real-world performance with its expected performance. Audits are important because otherwise it can be hard to see a clear picture of the effects of a system. Also, the auditor likely has access to sensitive information that others may not have, meaning they can do a more thorough inspection of the system.

Benefits tech that makes measurement decisions is often referred to as an automated decision system (ADS). ADSs are used to support decision-making processes, usually in conjunction with some human decision-making, like determining whether someone is qualified for benefits, how much support they get, or whether someone meets income or asset rules.

The part of a system that most people do not see, which receives, processes, and stores information. Technical staff probably have access to logs that show anything the system is designed to keep track of, like when certain processes (e.g. the process to create a new account) are set off in the program — but otherwise, case workers may not have access to this part of the system. Many failures can occur in the back-end, especially if the system doesn’t consider how to handle certain real-world scenarios: For example, if someone’s application makes it past the front-end without certain pieces of information, it may not proceed past certain checks within the program and seem to get “lost,” especially if the system isn’t designed to handle errors like that and report them back up to the front-end to be dealt with.

A type of design document that shows how the system is supposed to operate at a high-level. This includes how someone’s application gets processed by the system, and where/how people like caseworkers or applicants can interact with the system. These kinds of documents are important because they are legible to someone who can’t read source code, and can reveal potentially problematic points in a system. Sometimes these types of documents have other names including “business logic,” “domain logic,” or “user flow” documents.

The process of checking whether information from one system exists in another system. This is done in many contexts, often for determining eligibility or for identity verification and fraud detection. Data-matching may seem simple but can be hard to implement robustly: systems can reject a valid match if one of their profiles in a system has out of date information or has slightly different formatting (capitalization, hyphens, middle initials). Systems can also pick the wrong profiles as matches, especially with common names or congregate living situations where people have the same address.

A structure for storing information. There are different types of databases, but they generally are made up of category labels and corresponding entries that can be searched, processed, and retrieved. Some are simple, like a spreadsheet, but others can be more complex, like a model to represent an applicant’s profile.

The part of a system that a person interacts with. This includes case workers and applicants, who may interact with a case management system or an online application form. Usually, different types of users in a system will be authorized to see different things, which means they see different options on the front-end of the same system. Sometimes, there can be problems with the front-end of a system, like if applicants cannot find the pages they need, or if an online form does not let them submit certain information (e.g. names with special characters).

The physical parts of the computer that actually run the software’s instructions and store information.

The stage before a system goes online for everyone, when only certain groups of people use it. For example, sometimes a state decides to roll out the system only to certain counties for a certain amount of time. Theoretically, this is when the state should be doing a lot of work to make sure the system is operating as it’s supposed to, as well as investigating whether the system is having a negative impact on any individuals or groups.

A description of a project to be implemented and a call for vendors to submit their plans and bids for the project. The RFP can contain lots of useful information about a project’s goals, requirements, and timeline, as well as terminology that can be used to narrow public records requests.

A broad term for any computer program.

The written program, which includes the front- and back-end. This is the part that developers write in a programming language (e.g. Java or C) for the computer to run. Edits to the source code will change how the software operates.

The process of making sure a system is functioning as intended. System testing happens on many levels — in a software program, developers need to test each individual feature in a program to make sure it handles all possible inputs in a way that is correct and predictable. They also need to test the system for real-world performance, like making sure that it can handle a high volume of users. Additionally, the system needs to be tested for how people actually use the interface of the system, to see if unexpected problems come up. In assessments, this can include validation testing, as well as testing for discrimination against groups on the basis of qualities like race or disability.

The end-user’s interaction with a process or system. With benefits tech, end users include people applying for or enrolled in benefits programs as well as the caseworkers at government agencies. A core tenet of user experience is that a system should work well for end users: this means it should be as easy to use as possible, accessible to all end users, and allow end users to accomplish all the tasks they need to do.

The process of confirming that a standardized assessment is effectively measuring the quality it claims to. If an assessment is not validated, it might be making completely arbitrary or unreliable decisions about people. However, validation studies are limited and may not consider all the impacts of the system — so even if a tool is validated according to its developers, it can still be discriminatory or fail to adequately account for people’s needs and circumstances.