Auditing access changes around privilege log

Question and scope

A useful scope is narrow enough to reproduce and specific enough to show whether incomplete document index is being detected. If one request produces several matter intake updates, define whether the unit is the request or each update before sampling. The definitions should be short enough to use at the queue and precise enough to defend in a review with vendor. If the study is exploratory, label it exploratory and do not present the result as a stable baseline. Use the same note in the case file and the study extract; do not create a second version in a personal document. Write the primary question and one supporting question, then stop; a long list of questions usually hides an undefined unit. If one request produces several conflict packet updates, define whether the unit is the request or each update before sampling. The definitions should be short enough to use at the queue and precise enough to defend in a review with supervising attorney. State whether the study covers one team, one client, or all cases in the defined service. When a case needs an exception, record the reason, the owner, and the date before continuing the review.

Population and unit

Record how duplicate requests were matched, merged, or kept separate, because that choice changes the denominator. Exclude cases opened outside the window only when the exclusion rule was written before collection. The start event should be the first recorded request, not the first time the team opened the case. If a request was reopened after the window, keep it in the original population and report the later event separately. Keep the measurement note short enough to read with the case and specific enough to support the reported matter reopen rate. Record the opening count by owner and channel, because a single total can hide a queue concentrated in one person. A case that moves between owners stays one case unless the process defines a new requirement at the handoff. If the definition of deadline completion changes, keep the old and new values side by side for the transition period. If a request was reopened after the window, keep it in the original population and report the later event separately. If two reviewers disagree, preserve both classifications until the definition or the evidence is resolved.

Definitions and measures

If the definition of completion changes, keep the old and new rules side by side for the transition period. Define allowed values for status, reason, owner, and evidence state before collection. Before collection, test the definitions on a missing document index, a reopened case, and a handoff. Define whether the measure counts requests, cases, or destination updates, and use that unit consistently. Link the observation to the exact field in the billing portal and to the document index that supports it. Define how a correction after close is linked to the original case and whether it creates a new case. Use a stable case key and a separate identity crosswalk stored in an approved system. A duplicate request is a separate intake event even when it points to the same underlying need; record the link. Define whether the measure counts requests, cases, or destination updates, and use that unit consistently. Do not let a summary written later replace the evidence that was available at the first decision.

Data dictionary and evidence

Record the version or date of the privilege log and the matter management system value so a later reviewer can see what was known at the first action. Use a controlled list for exception reasons; do not let free text become the analysis category. A missing value should have a reason code, not a blank cell or a zero. Do not use a zero or an empty string to represent missing evidence. If the case moved between owners, include both owner periods in the review note and name the handoff time. Keep free-text notes separate from coded fields so the analysis does not depend on inconsistent wording. Keep the identity crosswalk in the approved system and restrict it to the review owner. Store sensitive evidence in the approved repository and keep only the reference in the analysis file. Pilot the dictionary with two reviewers and compare their classifications before the main sample. Keep the review note with the case so the next sampler does not have to reconstruct the decision from memory. Each case needs the source, the observation time, the field supported, the owner, and the reviewer. Version the dictionary and report which version was used for each observation period. Run a five-case pilot before the full sample and revise any field that two reviewers interpret differently. Archive the dictionary with the study so a later reviewer can reproduce the categories. When a value is missing, record the missing state and the reason instead of leaving an empty field in the extract.

Sampling and collection

Use consecutive privilege log cases where possible, then stratify by owner, channel, and risk class. Do not clean the data by filling missing values; preserve the missing state and report it as a result. Do not expand the sample only because the early result looks weak or strong. If a selected case is out of scope, replace it only with the next case in the same stratum and record the substitution. At Legal Services Offshore, the privilege log review should end with a dated note and a named owner for the next check. Include adverse cases: a missing outside counsel invoice, a conflicting source, an unavailable supervising attorney, and a tight deadline. If a source cannot be accessed, mark it unavailable and record the access problem rather than asking the operator to reconstruct it from memory. Use a random or systematic start rule and then take consecutive cases within each stratum. Do not let the reviewer choose the sample after seeing the results. If the evidence is sensitive, record the approved reference and access path rather than copying the content. A convenience sample of recently closed cases will overrepresent easy work and underrepresent the tail that needs attention. A second reviewer should re-extract a small subsample and record disagreements before the full analysis. Include reopened, unresolved, and exception cases in the sample frame. Keep a table of planned, eligible, selected, and reviewed counts for each stratum. The note should distinguish what was observed, what was inferred, and what still needs an owner decision.

Analysis and reporting

Compare conflict check cycle across owners only after checking the case mix, the channel, and the evidence standard. Keep descriptive findings separate from causal claims. When two groups differ, check whether the difference survives after adjusting for case mix and evidence standard. Report the numerator, denominator, observation window, and exclusion count for each measure. Record this decision in the study log with the case key, the review date, and the person who approved the classification. Separate observation from interpretation; a high completion rate on a small easy sample is not evidence of a strong control. The analysis should end with the decision the evidence can support and the decision it cannot support. Use a simple before-and-after comparison only when the definitions and observation window are the same. Report the numerator, denominator, observation window, and exclusion count for each measure. Keep the evidence reference beside the coded value so a reviewer can test the same case without rebuilding the queue.

Limitations

Changes in definitions, staffing, or the deadline calendar during the window can create a trend that is not a process effect. Report the unknown population and the evidence that would be needed to describe it. If the queue changed during the window, explain how the change affects the denominator and the interpretation. A missing record is evidence about the record, not proof that the work was never done. If the source changes after the observation, preserve the earlier value and add a dated note instead of replacing it. The record may be incomplete, delayed, or maintained outside the team, so the method cannot infer missing work from a clean summary. The study cannot determine whether a professional judgment was correct, only whether the record supports it. State that the result describes the observed sample and not the entire industry or every team in the service. If the reviewer performed the work, use a second reviewer for a subsample and report the disagreement. At Legal Services Offshore, the reviewer should be able to explain this step to supervising attorney from the case record alone.

Review and next action

Record the decision and the next check so the study does not end as an archive. Name the owner, the review date, and the decision the study is expected to inform. If the action requires a policy or access change, route it to the owner and record the decision. Record the owner, due date, and exact change that will be tested at the next review. Use the same note in the case file and the study extract; do not create a second version in a personal document. If the action affects stale matter record, route it to client contact and record the approval before implementation. Name the owner, the review date, and the decision the study is expected to inform. Set the next sample date before closing the review, so the action does not drift. Keep the prior procedure available so the next reviewer can compare the old and new states. When a case needs an exception, record the reason, the owner, and the date before continuing the review.

Related research

• Testing whether deadline calendar monitoring catches silent failures
• Testing evidence completeness in conflict packet without a convenience sample
• Measuring cycle time for matter intake from request to verified close