Before You Test: A Decision Framework for Reverse Engineering and Evidence Mining

Why product testing and reverse engineering should be the outcome of an evidence strategy—not the starting point

You have identified a product that appears to practice the claimed technology. The product is commercially available. Its specifications look promising. Product demonstrations appear to show the claimed functionality. Perhaps a technical document even describes a feature that closely resembles one of the claim limitations.

The natural next question is: “Can we test it?”

But that may be the wrong first question.

In complex patent investigations, testing and reverse engineering can produce valuable technical evidence. They can reveal how a product actually operates, expose hidden software or hardware functionality, capture network behavior, and provide evidence that public documentation does not disclose.

They can also become expensive exercises when the underlying evidence gap has not been clearly defined. Testing the wrong product version, investigating an unobservable claim limitation, or attempting to prove something that is already established through stronger public evidence can consume significant time and resources without materially improving the analysis.

The better question is:

What claim limitation remains unresolved, what evidence already exists, and can testing or reverse engineering realistically close that gap?

This distinction changes the investigation strategy. Testing should be the outcome of an evidence strategy—not the starting point.

This article presents a practical framework for making that determination before committing resources to product testing or reverse engineering.

---

Testing is Not the Same as Evidence

In a patent investigation, the objective is to establish whether a product or system satisfies particular claim limitations—not simply to understand how the product works.

A product may clearly advertise that it supports a particular feature. That may establish useful evidence for some claim limitations. But a claim may require something much more specific, such as:

• a particular sequence of operations;
• a particular relationship between components;
• processing performed by a particular module;
• a specific protocol message;
• a condition that triggers an operation; or
• functionality occurring internally rather than at the user interface.

Public documentation may therefore identify what a product does without revealing how the relevant functionality is implemented. That distinction matters.

Testing observes behavior. Reverse engineering investigates implementation. Neither automatically proves every limitation of a claim.

Testing can demonstrate what a product does under defined conditions.

Reverse engineering can go further by examining internal components, software behavior, architecture, communications, or other technical characteristics that may not be externally visible.

Evidence mining provides the bridge between the two: it determines which facts matter to the claim, what the available evidence already establishes, and what additional investigation—if any—is required.

The strongest investigations connect all three.

---

The Evidence-Mining Decision Framework

A robust investigation can be viewed as a progression:

This structure makes the investigation easier to review, reproduce, challenge, and ultimately use in a claim chart.

---

1. Start with the Claim—Not the Product

The first step is to understand exactly what must be established.

A claim should be decomposed into its individual technical limitations rather than treated as a single block of functionality.

For example, consider a hypothetical software-related claim requiring:

1. receiving information from a remote system;
2. processing the information according to a particular rule;
3. generating a modified data set;
4. storing the modified data set; and
5. displaying the result according to a specified relationship between two user inputs.

A product may clearly perform the overall functionality. But that does not necessarily establish every limitation. The first question should therefore be: Which limitations are already supported, and which ones remain uncertain?

A useful classification: Each limitation can initially be classified as:

This simple classification prevents the investigation from treating every limitation as if it requires the same level of technical effort.

---

2. Identify the Exact Product and Version

A common mistake is to establish that a product generally has a feature and then assume that the relevant version had the same implementation. That assumption can be dangerous. Software changes. Firmware changes. Hardware revisions are introduced. Features move between product generations. Cloud functionality can change without a corresponding change to the physical product.

For evidence-mining purposes, the relevant question is therefore not simply: Does Product X have Feature Y?

It is:

“Did the relevant version of Product X implement Feature Y at the relevant point in time?”

Before testing begins, investigators should establish, where possible:

• Product name and model;
• hardware revision;
• software or firmware version;
• release date;
• relevant configuration;
• operating environment;
• geographic or service-specific variation;
• cloud/backend dependency; and
• whether the product under investigation corresponds to the version discussed in the available evidence.

A technically perfect test performed on the wrong version may have limited evidentiary value.

---

3. Exhaust the Public Evidence Before Opening the Product

Before opening a device, intercepting packets, disassembling firmware, or designing a test environment, investigators should first determine whether the relevant evidence is already publicly available.

A product may leave an extensive evidence trail:

• Product manuals and technical specifications
• Developer/API documentation
• Support articles and release notes
• Regulatory filings
• Standards documentation
• Product demonstrations
• Archived webpages
• Source-code repositories
• Teardown/technical analyses
• Academic and technical publications

This evidence is often faster to obtain, easier to cite, and less expensive to validate than evidence generated through physical testing. More importantly, publicly available evidence can reveal what should be tested.

A technical manual might identify a particular communication protocol. A developer document might identify an API. A release note might establish when a feature was introduced. A standards document might identify the expected protocol behavior. A product teardown might reveal the relevant hardware architecture. Together, these sources can transform an open-ended testing exercise into a narrowly defined investigation.

The important question is not simply whether a source mentions the product. It is whether the source establishes the specific claim limitation for which it is being relied upon.

---

4. Distinguish Observation From Inference

Once the claim has been decomposed and available evidence has been collected, the next question is not simply whether evidence exists.

It is:

What does that evidence actually establish?

Consider a hypothetical product that displays a result on a screen. A demonstration video may clearly establish the displayed result. But suppose the claim requires that the result be generated through the following sequence:

Receiving data → Applying a specific processing operation → Storing an intermediate data structure → Retrieving that structure → Displaying the result

The video may establish the beginning and end of the sequence without revealing the intermediate operations. Inferring the backend implementation solely from visible behavior can therefore create an evidentiary gap. Instead of asking: “Does the product appear to do this?”

the investigation should ask:

“What observable evidence establishes each required step?”

This distinction is particularly important when the claim depends on internal processing, data structures, communications, or component relationships that may not be visible through ordinary product use.

---

5. Conduct an Evidence Gap Analysis

Once the available evidence has been mapped to the claim, the unresolved limitations become much easier to identify.

A useful evidence-gap matrix might look like this:

This is the point at which testing becomes a decision, rather than an assumption. Once the evidence gap has been identified, the next question is whether testing can actually close it.

A Hypothetical Example: When Testing Is—and Isn't—the Right Next Step

Consider a hypothetical software product that appears to satisfy a claim requiring a system to:

1. receive user-generated information;
2. process that information;
3. store a resulting data object; and
4. retrieve and display the data object in response to a particular user interaction.

A product demonstration clearly shows the user entering information and subsequently viewing a result.

The product documentation also describes the relevant feature. At first glance, the product appears to be an excellent testing candidate. But the evidence analysis reveals something important. The first, second, and fourth limitations are reasonably well supported. The unresolved issue is whether the intermediate result is actually stored as the claimed data object before being retrieved for display.

Testing the user interface again is unlikely to resolve that question. The investigation therefore shifts from: “Can we demonstrate the feature?” to: “Can we observe the relevant storage operation or data flow?”

The next step may involve examining application files, network traffic, APIs, logs, or other technical artifacts. If those sources can establish the storage operation, extensive reverse engineering may be unnecessary. If they cannot, deeper technical investigation may then be justified.

The important point is that the investigation method was selected after identifying the evidence gap.

---

6. Determine Whether the Gap Is Actually Testable

Not every evidence gap can be solved through behavioral testing.

Suppose a claim requires a particular internal algorithm. Running the product may demonstrate the output, but multiple algorithms could potentially produce the same output. In that situation, behavioral testing alone may not establish the claimed implementation. Conversely, a claim requiring a particular observable interaction may be highly suitable for controlled testing.

The question at this stage is:

Can the unresolved limitation be observed, measured, captured, or otherwise established through a technically sound investigation?

Potentially testable evidence may include:

• Inputs and outputs;
• system responses;
• network traffic;
• timing relationships;
• displayed behavior;
• device state changes;
• API requests and responses;
• file creation or modification;
• configuration changes;
• communication between components;
• hardware behavior; and
• interactions between software modules.

Where the relevant implementation is not externally observable, other forms of investigation may be necessary, including firmware extraction, binary analysis, source-code review, file-system examination, protocol analysis, component inspection, or other forms of reverse engineering.

---

7. Define the Testing Objective Before Designing the Test

A weak testing objective sounds like: “Investigate whether the product infringes the patent.” That objective is too broad.

A stronger objective identifies the exact technical proposition that needs to be established. For example:

Determine whether the product transmits parameter X to the backend when condition Y occurs, and whether the backend uses that parameter to generate response Z.

The investigation can now be designed around a specific proposition.

A well-defined test objective should identify:

• the claim limitation being investigated;
• the technical behavior to be observed;
• the conditions under which it should occur;
• the expected result;
• the evidence that would support the proposition; and
• the alternative explanations that must be ruled out.

This transforms testing from exploratory experimentation into targeted evidence collection.

---

8. Select the Appropriate Investigation Method

Once the evidence gap and objective are clear, the next question is:

What is the least invasive method capable of answering the question reliably?

The appropriate method often depends on the visibility of the claim limitation.

Where the limitation is externally observable

Controlled testing may be sufficient. For example:

• activating a feature produces a specific output;
• a device automatically connects under a defined condition;
• a particular command generates a particular response; or
• a product changes state when a specific input is received.

Where the limitation concerns communications

Network or protocol analysis may provide stronger evidence. For example:

• a particular message is transmitted;
• a protocol field contains specified information;
• a device initiates communication after a defined event; or
• information is exchanged between components in a particular sequence.

Where the limitation concerns internal implementation

Deeper investigation may be necessary. For example:

• a particular algorithm is executed;
• a parameter is stored in a specific location;
• a particular data structure is generated; or
• encryption or authentication is performed in a particular manner.

The goal is not to perform the most sophisticated investigation. It is to perform the minimum investigation necessary to establish the technical proposition reliably.

---

9. Evaluate Testing Feasibility and Evidentiary Value

Even where testing appears technically appropriate, the investigation should determine whether it is practical and worthwhile.

Technical Feasibility

Important questions include:

• Product availability: Can the relevant product and version be obtained?
• Environment: Can the required hardware, software, network, account, or service environment be reproduced?
• Observability: Can the claimed behavior actually be observed?
• Instrumentation: What tools are required to capture the relevant evidence?
• Repeatability: Can the observed behavior be reproduced consistently?
• Configuration: Are there dependencies that could change the result?
• Access: Does the investigation require credentials, specialized equipment, cloud access, or other restricted resources?

Evidentiary value

Technical feasibility alone is not enough. A test may successfully generate a result while still failing to establish the relevant claim limitation. Before committing resources, ask:

“If the test succeeds, what will we be able to prove that we cannot already prove today?”

This question helps distinguish a technically interesting experiment from an investigation that materially strengthens the evidence.

The decision should weigh: Expected evidentiary value × confidence in the investigation methodology

against: Cost + time + technical risk + remaining uncertainty

Testing becomes more attractive when the expected evidence can materially strengthen the analysis and there is a reliable path to obtaining it.

---

10. Build Independent Validation into the Investigation

A common risk in technical investigations is designing a test around an expected conclusion. An investigator may observe a product behavior and interpret it as evidence of a particular backend operation without adequately considering alternative explanations.

Independent validation helps prevent this.

The validation process should ask:

• Does the test actually isolate the claimed feature?
• Could another implementation produce the same result?
• Are there alternative explanations for the observed behavior?
• Was the correct product version tested?
• Was the environment adequately controlled?
• Can the result be reproduced?
• Are the logs, captures, or other artifacts complete?
• Does the evidence directly support the claim limitation?
• Are assumptions being presented as facts?

The purpose of validation is not to make the evidence appear stronger. It is to determine whether the evidence actually supports the proposition being made.

---

A Practical Pre-Testing Checklist

Before approving product testing or reverse engineering, an investigation team should be able to answer:

• Have the relevant claim limitations been clearly identified?
• Has the target product and relevant version been established?
• Has available public evidence been comprehensively reviewed?
• Has each available source been mapped to specific claim limitations?
• Have explicit disclosures been separated from inference?
• Have the unresolved evidence gaps been clearly documented?
• Is the unresolved limitation technically observable or investigable?
• Can testing realistically close the identified gap?
• Has a specific test objective been defined?
• Is the proposed testing method proportionate to the evidentiary need?
• Have alternative explanations been considered?
• Can the test environment be reproduced and controlled?
• Can the resulting evidence be independently validated?
• Has the expected evidentiary value been weighed against cost and effort?

If several of these questions cannot be answered, the investigation may not yet be ready for testing.

---

Common Mistakes Before Testing

1. Testing because the product “looks promising”

A promising product is not necessarily a good testing candidate. The claim limitations—not the overall similarity of the product—should drive the decision.

2. Testing the latest version without establishing historical relevance

A modern product may contain functionality that was absent from the relevant earlier version. Version history should therefore be established before relying on current behavior.

3. Treating user-visible behavior as proof of backend implementation

The same visible result can sometimes be produced by multiple underlying implementations. Where the claim requires an internal operation, behavioral similarity alone may not be enough.

4. Starting with reverse engineering before conducting basic technical research

Deep technical investigation can sometimes answer questions that publicly available documentation already answers. The investigation should first determine whether the required evidence already exists.

5. Designing a test without defining what success looks like

A test without a defined evidentiary objective can produce large amounts of data without resolving the actual claim issue.

6. Failing to document the test environment

Hardware version, software version, configuration, network conditions, inputs, and other variables can affect results. Without adequate documentation, reproducing or interpreting the test may become difficult.

7. Treating a successful experiment as the end of the investigation

The final question should not be: “Did the product do it?” It should be: “What does this result establish, and how directly does it support the claim limitation?”

---

From Testing to Litigation-Ready Evidence

A successful test is not the final deliverable. The result must still be connected to the claim limitation it is intended to establish. A strong evidence chain preserves the relationship between:

Claim Limitation → Technical Proposition → Evidence Source → Investigation Methodology → Observation → Interpretation → Validation

Each link matters.

If the claim limitation is misunderstood, the investigation may target the wrong behavior. If the wrong product version is examined, the resulting evidence may not be relevant. If the methodology is inadequate, the observation may not be reliable. If the observation does not directly establish the claimed feature, the interpretation may remain vulnerable to challenge.

The strongest investigations therefore treat evidence collection as a connected chain rather than a collection of isolated findings. The ultimate objective is to develop evidence that can be clearly traced from the technical observation back to the claim limitation it is intended to establish.

---

The Decision Tree: Should We Test?

A practical decision process can be summarized as follows:

This framework helps ensure that testing is a deliberate evidence decision rather than an exploratory exercise.

---

Key Takeaways

• Start with the claim—not the product.
• Establish what public evidence already proves before testing.
• Tie every testing decision to a specific evidence gap.
• Choose the least invasive method capable of resolving that gap.
• Define the investigation objective before conducting the test.
• Technical success does not automatically equal evidentiary success.
• Validate the result and map it back to the relevant claim limitation.

---

Where iCuerious Comes In

Effective evidence mining begins well before a product is placed on a test bench. Knowing whether to test is as important as knowing how to test. For over a decade, iCuerious has helped clients move from claim-level analysis and product research to focused evidence strategies. Our role is to determine what the available evidence establishes, isolate the technical gaps that remain, assess whether further technical investigation could meaningfully close those gaps, and help identify the appropriate resources when specialized testing or reverse engineering is required.

Talk to the iCuerious Evidence Mining & Product Research Team → www.icuerious.com