Legal Services Offshore research · Evidence Quality

Measuring intake completeness in a distributed legal team

A bounded protocol for measuring whether matter intake records carry the request, source evidence, conflict status, and decision needed for attorney review.

Published · 5 sources

Measuring intake completeness in a distributed legal team research illustration

Research context

This research brief is designed for legal operations planning. Use the findings to ask better workflow questions. Test any proposed process against the firm’s jurisdiction, professional duties, systems, and client commitments. Treat every local result as a starting point for review rather than a universal benchmark.

Question and decision use

This protocol asks which required elements are present in a matter intake record at the moment it reaches attorney review. The elements are the client and matter identity, the requested scope, the source for each key fact, the conflict search status, the open questions, and the decision requested. The result is intended to help a legal operations team improve forms, routing, and reviewer coverage rather than to rank individual preparers. A useful measurement question names the decision it will inform.

If the team is deciding whether to change the intake form, the study should report which fields are missing most often and where the missing value originates. If the team is deciding whether to add a reviewer, the study should show whether incomplete packets cluster around particular matter types, offices, or times of day. The question should be written before the sample is drawn. This is not an audit opinion and it is not a legal conclusion. The study observes documentation in a defined window and cannot prove that a complete record produced a better client outcome. It can show whether the team's own evidence standard is being met and where the process breaks down.

Population and unit of analysis

The population should be defined from a frozen export of intake records, with the query time, systems, date boundaries, and status filters recorded. Include new matters, reopened matters, and changes to existing matters only if the protocol treats them as separate units. If they are combined, report the mix so the reader can see what the completeness rate represents. The unit is one intake record followed from its opening event to the attorney decision or the end of the observation window.

A record that is returned for correction is still one unit, with a linked correction event. A record that is reopened later creates a new event but may remain the same matter. The protocol should state which choice it uses and apply it consistently. Excluded records need a reason. Common exclusions include duplicates, test records, matters handled outside the intake queue, and records missing the identifiers needed to link them to a source. Report the excluded count by reason so the denominator can be challenged. A large unexplained exclusion is a limitation, not a cleaning detail.

Evidence dictionary

Define each field before reviewing any record. The client name, matter identifier, request source, scope statement, conflict search date, search result status, open question, owner, and decision requested should each have an allowed value and a source rule. Distinguish missing, unknown, not applicable, and confirmed negative, because those states have different operational meanings. A source rule should say where the value must come from.

The conflict status should come from the approved conflict system or a dated search record, not from a note that says checked. The scope should come from the client request or the attorney's instruction, not from a summary written after a call. The requested decision should be explicit enough that a reviewer can answer it. Where a field is derived, record the formula and the source fields. If the team calculates an age in business days, define the calendar, the start event, and the stop event. If a field is free text, provide examples of acceptable entries and a rule for what makes an entry insufficient. The dictionary is part of the study and should be archived with the sample.

Sampling and reviewer calibration

Draw the sample from the frozen frame using a documented method. A simple random sample may be sufficient for a small queue, while a stratified sample can ensure coverage of matter types, offices, and exception states. Record the random seed and any replacement. Do not let reviewers choose the records that look most informative after seeing preliminary results.

Two reviewers should classify a small calibration set before the main review. Compare their answers field by field, discuss the ambiguous cases, and revise the dictionary. Then double-review a prespecified subset of the main sample. Report agreement counts and preserve the initial ratings so the reader can see whether the final result depended on extensive harmonization. If only one reviewer is available, say so and treat the result as a single-reviewer observation. That does not make the study useless, but it limits claims about consistency. A second reviewer on a subsample is often enough to reveal whether the definitions are clear.

Measures and reporting

Report completeness as a count and a rate for each required element, with the denominator and the missing count visible. A single overall percentage can hide the difference between a missing client name and a missing conflict date. The reader should be able to see which element drives the result. Include the age of incomplete records at the observation date, not only the share that are incomplete.

A queue with many fresh records may look worse than a queue with a few old ones, even though the older items carry more risk. Report the distribution and the open count by owner group, with a caution that routing and case difficulty may explain differences. Separate first-pass completeness from corrected completeness. A record that was returned and fixed is not the same as a record that arrived complete. Report both, and keep the reason for each return. This distinction helps the team see whether the form is weak, the reviewer is catching problems, or the preparer is learning.

Analysis and interpretation

Compare completeness across the variables the protocol prespecified, such as matter type, intake channel, office, and time of day. Do not add segments after the fact simply because they produce an interesting difference. With a small sample, many apparent differences will be noise. Examine the narratives behind the largest gaps before assigning a cause.

A missing conflict search may reflect a system outage, an unfamiliar entity, or a preparer who did not know the search was required. Each cause suggests a different remedy. A training note will not fix a broken integration, and a new field will not fix a reviewer who never asks for it. Report uncertainty in plain language. If the sample contains nine incomplete records, say nine rather than implying a precise population rate. If the design cannot support a comparison between offices, say that the comparison is descriptive. A modest claim that the team can defend is more useful than a strong claim that collapses under review.

Limitations and sources of error

The study cannot observe intake work that never reached the queue. Matters opened by a partner through a direct conversation, records kept in a local drive, and requests handled by another office may be missing from the frame. The result describes the sampled records, not every matter the firm accepts. Reviewer judgment is another limitation.

Two people may disagree about whether a scope statement is sufficiently specific or whether a conflict result is confirmed. Calibration reduces but does not eliminate that variation. The study should preserve the disagreements and the adjudication rule rather than reporting only the final consensus. System changes can also affect the result. A form update, a new matter management release, or a change in the conflict database may alter what the team can record. Note the relevant change dates and avoid comparing periods as if the instrument stayed constant. Public guidance can frame the questions but cannot validate a local record.

Privacy and role boundaries

Use the minimum client and matter information needed for the study. Replace names with study keys, restrict access to the crosswalk, and follow the firm's retention and deletion rules. Do not copy privileged or sensitive material into the analysis file when a reference to the approved source is enough. The offshore team may prepare the sample, classify administrative fields, and maintain the study log.

The supervising attorney or authorized firm reviewer decides whether a record is sufficient for acceptance and whether a legal question is resolved. The study should not be used to make a client-facing statement about the firm's compliance or the strength of a matter. If a record contains information from another client or an unrelated matter, stop and report it through the firm's incident path. Do not include it in the sample or discuss it in a group review. The privacy rule applies to the study just as it applies to the underlying work.

From finding to controlled change

Choose one change based on the most common or most consequential gap. It might be a required field, a different source system, a routing rule, or a second-review step. Name the owner, the due date, and the measure that will show whether the change helped. Recheck the same measure after two cycles rather than declaring success immediately.

A short-term improvement may reflect reviewer attention rather than a durable process change. If the measure does not move, examine whether the change addressed the cause or only the symptom. Keep a decision log with the finding, the change, the owner, and the result. This log is the institutional memory of the intake process and helps a new manager understand why the form looks the way it does. The goal is not a perfect record; it is a record that makes the next decision easier and the next correction visible.

Sources and method

Open each source to confirm its scope, publication date, and applicability before relying on it for a legal workflow decision.

  1. American Bar Association — Law Practice Resources
  2. U.S. Courts — Rules and Policies
  3. NIST Cybersecurity Framework 2.0
  4. U.S. Government Accountability Office
  5. Legal Services Corporation

Related research

Bring the messy task list

We will help turn it into one clear legal support role.

Share the work, tools, hours, and review needs. The next step is a practical staffing scope, not a public rate card.