HINT
Queries containing the words “does”, “is”, “must”, or “should” are perfect candidates for test queries.
The Test function is designed for closed yes/no questions, where you want:
A quick pass/fail flag.
A summary explaining the reasoning behind the determination.
Highlighted sections in the document that support the conclusion.
Test works best when:
You want to check whether a requirement is met or unmet.
You want a clear, concise answer without generating long summaries.
Follow the steps outlined here
The name field is a human-friendly label. Add a label which makes sense to you (and anyone else using this Playbook), e.g. “Limitation of Liability”
The term description describes the query you are trying to perform. Find tips for writing solid queries here.
Phrase it as a statement describing the criteria or threshold
Use “and” if multiple conditions must all be met
If the term does not meet the criteria…
Flag it - Select this when you only need a quick pass/fail result and don’t need to suggest amendments. You’ll get the clear flag (pass or fail) quickly.
Attempt to redraft - Select this when you need to propose amendments or departures. Playbooks will generate a suggested clause that meets the stated requirement. For more information, check out the Redrafting guide.
No external knowledge
Test only uses the information in your document. For example, it can’t answer “What provisions of the Civil Liability Act are relevant to this document?” or “Which clauses in this document are off-market?” These queries require external knowledge.
This means that you can trust that every part of an answer from a Test query is grounded in the actual document that you are reviewing. There is no risk of hallucination.
Test queries are ideal for quickly validating whether a document meets specific conditions. We always recommend validating a Test query’s conclusions by jumping to the relevant page in the document.
Avoid open-ended questions
Test interprets any request as a requirement to be met. Open-ended questions will be treated as a requirement the document should meet and may not extract information in a useful format.
For summaries or exploratory questions, use the Extract query type instead.
Use easily-verifiable closed questions
Use a close-ended operative word such as “does”, “is”, “must”, or “should”
“Does this document contain a signature block?”
“Is a privacy disclaimer clearly defined?”
“The limitation of liability clause must not exceed $1 million”
“Confidentiality provisions should be included in the agreement”
Consider the positive/negative wording carefully
Think carefully about whether you’re testing for something to be present or absent
“This document must contain a limitation of liability”
If a limitation of liability is found, the test will pass.
If a limitation of liability is not found, the test will fail.
“This document must not include a signature block”
If a signature block is not found, the test will pass.
If a signature block is found, the test will fail.
Consider whether your query would be better split into multiple queries
If your query contains the word “and”, the conditions on either side of the word “and” need to be met to achieve a pass.
“Does the document contain a limitation of liability and confidentiality provision?”
If either statement is missing, the whole test will fail.