Understanding Status Checks vs Clayton PR Reviews

It's crucial to first establish its connection with Protect What's New and the Legacy code filter. Essentially, status checks are functionalities offered by Git providers (like GitHub or Bitbucket) to integrate third-party tools and implement a quality gate into Pull Requests (PRs). You can read more about status checks in GitHub's documentation: Status Checks
​

​

Status checks aim to maintain code quality by evaluating the entire codebase at a specific revision. While this approach ensures comprehensive code health, it often highlights issues unrelated to the changes made in the current PR.
​
To help implement a quality gate in codebases with significant technical debt, Clayton offers a tailored approach:

​Legacy Code Filter: Excludes legacy code from the analysis based on a specified reference date.
​

​Protect What's New: Using this protection scheme and setting up quality gates protects PRs by using status checks while specifically targeting new code, leveraging the Legacy Code Filter.
​

​

When creating a PR in Git repositories like GitHub or Bitbucket, you might notice a discrepancy where:

This apparent contradiction is rooted in how these tools analyze code changes differently.
​

A status check in Gearset, when integrated with Clayton, provides a thorough evaluation of the entire source commit linked to the PR. Unlike basic code reviews that only assess the new changes introduced in the PR, status checks analyze all files involved in the specific revision, regardless of whether the changes were made in the current or previous PRs.
​

This method ensures that the entire codebase remains healthy rather than just focusing on the changes being introduced. Even if a PR modifies just a few lines, the status check will flag any pre-existing issues within the affected commit, treating the entire commit as a single unit. Therefore, the report is considered "all-or-nothing" because any issues, newly introduced or inherited will result in a failed status check.
​

This comprehensive approach can impact the workflow since a PR might fail, not because of new issues, but due to existing problems from previous commits that are now part of the revision. As a result, developers might need to address these existing issues within the same PR, even if they did not originally introduce them. This practice ensures that the project’s overall quality does not degrade over time.
Thus, the status check functions as a robust quality gate, preserving consistency and integrity throughout the entire codebase.
​

The feedback from Clayton[bots] takes a more targeted approach. It analyzes only the files that have been changed in the current PR, highlighting issues specifically introduced by those changes while ignoring pre-existing problems. This method keeps feedback relevant to what was actually modified, avoiding legacy issues.
​

Consider this: you open a PR that modifies just three files. Clayton scans them and finds no new issues. Yet, the build status still fails. This happens because the status check examines the entire project, including untouched files that might contain legacy issues.

If you've enabled the Legacy Code Filter, Clayton's targeted review will correctly exclude those older problems. For example, if you set the reference date to "yesterday," only issues that have existed in the entire project since yesterday would be flagged in the status section. This allows your build to pass without being held up by pre-existing issues. Crucially, any items within your PR itself would remain unaffected by the reference date, no matter what it's set to.