Spec quality & scoring
Elva evaluates an OpenAPI specification with a deterministic check catalog. The report includes category scores, individual findings, and coverage so you can see both the result and how much of the specification could be evaluated.
The four categories that make your score
The overall score balances four categories: Design 30%, Developer Experience 30%, AI Readiness 20%, and Security 20%. These are category weights; individual checks have their own weights within each category.
Category | Weight | What it evaluates |
Design | 30% | Resource naming, HTTP semantics, responses, pagination, versioning, and lifecycle guidance. |
Developer Experience | 30% | Descriptions, examples, error models, navigation, authentication guidance, and structural integrity. |
Category | Weight | What it evaluates |
AI Readiness | 20% | Operation identifiers, semantic descriptions, context, schema discoverability, constraints, enums, and formats. |
Security | 20% | HTTPS, authentication declarations, credentials in URLs, authorization errors, rate limits, and secret hygiene. |
A spec score measures the API description. It does not prove the running API is secure, compliant, correct, or suitable for every agent. Verify behavior with realistic requests and the permissions your consumers will use.
How a score is computed
Each check returns a status and may also return a fractional score. When score is present, its value, clamped to 0–1, determines credit. Otherwise Pass earns 1, Partial earns 0.5, and Fail earns 0. NotApplicable is excluded from the evaluated weight.
Status | Fallback credit | Meaning |
Pass | 1 | The evaluated requirement is satisfied. |
Partial | 0.5 | Some evaluated cases need improvement. |
Fail | 0 | The evaluated requirement is not satisfied. |
NotApplicable | Excluded | The check could not meaningfully evaluate this specification. |
A category percentage is 100 × the sum of check credit × check weight, divided by evaluated check weight. Its coverage is evaluated weight divided by total category weight. Below 70% coverage the category reports N/A and is excluded from the overall average. The overall score uses unrounded percentages from scorable categories, with their category weights renormalized.
Grade | Percentage before rounding |
A | 84 or higher |
B | 74 to less than 84 |
C | 64 to less than 74 |
D | 52 to less than 64 |
F | Below 52 |
N/A | Insufficient evidence to grade |
Read coverage before comparing scores
A report is unscorable when there are no operations, references cannot be evaluated sufficiently, or total rubric coverage is below 70%. Inspect scorable, unscorableReason, operationCount, and rubricCoverage before using a score in a release gate. A zero percentage with grade N/A is not a measured zero-quality result.
Compare reports with similar coverage. For example, a category with 80% coverage and 75% credit is a meaningful B; a category with only 50% coverage is N/A even if every evaluated check passes. Displayed percentages are rounded, so a displayed boundary value can differ from the grade calculated before rounding.
On this page
- Spec quality & scoring