The Open Catalog of Test Smells

Welcome!

Search for a test smell using the search or the sidebar — works with test smells descriptions too — if you already know what to look for. If not, you may browse by category in the left panel. Download options (PDF and EPUB) are available in the Read the Docs menu (bottom of left panel). There is a lot to see!

The catalog provides a visualization of the data set compiled in a Multivocal Literature Review published here. Initially, the catalog unifies information from 127 formal and informal sources. As the catalog is open-source, any community member can submit a contribution as a pull request and help improve it.

Every listed test smell contains a name, AKA (when available), definition, code example (likely extracted from the listed references), and the listed references, which of course provide further information. We expect the catalog to be a helpful resource for software testing community members to better understand test smells. Furthermore, it may help avoid proposing new names (AKA) to existing test smells in an overlap.

Tip

Please help us keep this Catalog updated and contribute with your own test smell findings.

Send in your helpful comments or ideas to easy@ic.ufal.br or contribute directly by clicking Edit on GitHub in the top right corner of this page.

If you are not familiar with directly editing, give us a new issue! Click here and create a new GitHub issue

Contents

Test Smell Categories

• Code related
  • Code duplication
    • Army Of Clones
    • Assertion Chorus
    • Copy-Paste
    • Cut-And-Paste Code Reuse
    • Data-Ja Vu
    • Duplicate Alt Branches
    • Duplicate Assert
    • Duplicate Code
    • Duplicate Component Definition
    • Duplicate In-Line Templates
    • Duplicate Local Variable/Constant/Timer
    • Duplicate Statements
    • Duplicate Template Fields
    • Duplicate Test Code
    • Duplicate Test Runs
    • Duplicated Actions
    • Duplicated Code In Conditional
    • Duplicated Code
    • Half A Helper Method
    • Missing Parameterised Test
    • Missing Test Data Factory
    • Multiple Tests Testing The Same Or Similar Things
    • Reinventing The Wheel
    • Second Class Citizens
    • Test Clones
    • Test Code Duplication
    • Test Redundancy
    • The First And Last Rites
    • Two For The Price Of One
    • Very Similar Test Cases
  • Complex - Hard to understand
    • Absence Of Why
    • Bad Comment Rate
    • Boilerplate Hell
    • Comments
    • Complicated Logic In Tests
    • Hard-Coded Test Data
    • Hard-Coded Values
    • Hard-To-Test Code
    • Hard-To-Write Test
    • Hardcoded Environment Configuration
    • Hardcoded Environment
    • Hardcoded Literals
    • Hardcoded Test Data
    • Herp Derp
    • Hidden Complexity
    • It Looks Right To Me
    • Keyword Driven Testing
    • Lack Comments
    • Large Macro Component
    • Large Module
    • Large Test File
    • Long Class
    • Long Function
    • Long Macro Event
    • Long Method
    • Long Statement Block
    • Long Test Steps
    • Long Test
    • Long/Complex/Verbose/Obscure Test
    • Magic Number Test
    • Magic Values
    • Obscure Test
    • Optimizing Dry
    • Over Refactoring Of Tests
    • Overcommented Test
    • Overly Complex Tests
    • Overly Dry Tests
    • Overuse Of Abstractions
    • Self Important Test Data
    • Tests Are Difficult To Write
    • Using Complicated Data Store
    • Using Complicated X-Path Or Css Selectors
    • Using Test Case Inheritance To Test Source Code Polymorphism
    • Verbose Test
    • What Are We Testing?
  • In association with production code
    • Behavior Sensitivity
    • Changing Implementation To Make Tests Possible
    • Code Pollution
    • Code Run Only By Tests
    • Context Logic In Production Code
    • Equality Pollution
    • Fire And Forget
    • For Testers Only
    • Hooks Everywhere
    • Indecent Exposure
    • Interface Sensitivity
    • Mixing Production And Test Code
    • Multiple Points Of Failure
    • Overly Elaborate Test Code
    • Overspecification
    • Overspecified Software
    • Overspecified Tests
    • Plate Spinning
    • Production Logic In Test
    • Test Dependency In Production
    • Test Hook
    • Test Logic In Production Code
    • Test Tautology
    • Tests Cluttered With Business Logic
    • Tests Require Too Much Intimate Knowledge Of The Code To Run
    • The Telltale Heart
    • The Ugly Mirror
    • Trying To Test The Untestable
    • Ui Tests Should Not Expose Interactions With Web Elements
    • Well, My Setup Works
  • Mock and stub related
    • Excessive Mocking
    • Is Mockito Working Fine?
    • Making A Mockery Of Design
    • Mismatched Stubs
    • Mock Everything
    • Mock Happy
    • Mock Madness
    • Mock’Em All
    • Mockers Without Borders
    • Mocking A Mocking Framework
    • Mocking What You Don’T Own
    • Mockito Any() Vs. Isa()
    • Overmocking
    • Overuse Mocking
    • Remote Control Mocking
    • Subclass To Test
    • Surreal
    • Testing The Framework
    • The Dead Tree
    • The Mockery
    • Unnecessary Stubs
  • Violating coding best practices
    • Accidental Test Framework
    • Ambiguous Tests
    • Anonymous Test Case
    • Anonymous Test
    • Bad Documentation Comment
    • Bad Naming
    • Badly Structured Test Suite
    • Blethery Prefixes
    • Comments Only Test
    • Constant Actual Parameter Value
    • Dead Field
    • Default Test
    • Directly Executing Javascript
    • Disabled Test
    • Empty Method Category
    • Empty Test-Method Category
    • Empty Test
    • Erratic Test
    • Erratic Tests
    • Everything Is A Property
    • Flaky Locator
    • Flaky Test
    • Fully-Parameterized Template
    • Goto Statement
    • Having Flaky Or Slow Tests
    • Hidden Meaning
    • Hidden Test Call
    • Ignored Test
    • Improper Test Method Location
    • Inconsistent Hierarchy
    • Integration Test, Masquerading As Unit Test
    • Intermittent Test Failures
    • Is There Anybody There?
    • Lack Of Encapsulation
    • Lack Of Macro Events
    • Literal Pollution
    • Long Parameter List
    • Lost Test
    • Meaningless Variable Names
    • Messy Test
    • Missed Skip Rotten Green Test
    • Missing Variable Definition
    • Missing Verdict
    • Mistaken Identity
    • Mixed Selectors
    • Name-Clashing Import
    • Non-Functional Statement
    • Nondeterministic Test
    • Opaque Output
    • Over-Eager Helper
    • Over-Specific Runs On
    • Overreferencing
    • Proper Organization
    • Sensitive Locators
    • Short Template
    • Singular Component Variable/Constant/Timer Reference
    • Singular Template Reference
    • Stop In Function
    • Test Body Is Somewhere Else
    • Test-Class Name
    • Test-Method Category Name
    • Testing For A Specific Bug
    • Testy Testy Test Test
    • The Conjoined Twins
    • The Enumerator
    • The Flickering Test
    • The Forty Foot Pole Test
    • The Sleeper
    • The Stepford Fields
    • The Test With No Name
    • Time Bomb Data
    • Time Bombs
    • Time Sensitive Test
    • Treating Test Code As A Second Class Citizen
    • Unclassified Method Category
    • Unreachable Default
    • Unrestricted Imports
    • Unsuitable Naming
    • Unused Imports
    • Unused Inputs
    • Unused Parameter
    • Unused Shared-Fixture Variables
    • Unused Variable Definition
    • Unworldly Test Data
    • Using Faker
    • Verbless And Noun-Full
    • Wasted Variable Definition
• Dependencies
  • Dependencies among tests
    • Chain Gang
    • Constrained Test Order
    • Coupled Tests
    • Coupling Between Test Methods
    • Dependent Test
    • Identity Dodgems
    • Interacting Test Suites
    • Interacting Tests
    • Lack Of Cohesion Of Test Cases
    • Lack Of Cohesion Of Test Methods
    • Litter Bugs
    • Lonely Test
    • Order Dependent Tests
    • Test Pollution
    • Test Run War
    • The Environmental Vandal
    • The Inhuman Centipede
    • The Leaky Cauldron
    • The Parasite
    • The Peeping Tom
    • Unusual Test Order
  • External dependencies
    • Context Sensitivity
    • Counting On Spies
    • Data Sensitivity
    • External Data
    • External Dependencies
    • Factories Depending On Database Records
    • Hidden Dependency
    • Hidden Integration Test
    • Local Only Testing
    • Middle Man
    • Mystery Guest
    • Programming Paradigms Blend
    • Remote Mystery Guest
    • Require External Resources
    • Resource Leakage
    • Resource Optimism
    • Stinky Synchronization Syndrome
    • Stinky Synchronization
    • Tests Depend On Something Outside Of The Test Suite
    • The Local Hero
    • The Operating System Evangelist
    • Web-Browsing Test
• Design related
  • Not using test patterns
    • Autogeneration
    • Constructor Initialization
    • Contortionist Testing
    • Disorder
    • I’Ll Believe It When I See Some Flashing Guis
    • Missing Log
    • Narcissistic
    • No Clear Structure Within The Test
    • No Structure When Creating Test Cases
    • Not Using Page-Objects
    • So The Automation Tool Wrote This Crap
    • Test::Class Hierarchy
    • Testing Business Rules Through Ui
    • The Turing Test
    • Unclassified Method Category
    • Undefined Test
    • Unknown Test
• Issues in test steps
  • Issues in assertions
    • 7 Layer Testing
    • Asserting Pre-Condition And Invariants
    • Assertion Diversion
    • Assertion Mismatch Scenario
    • Assertion-Free
    • Assertionless Test
    • Assertions Should Be Merciless
    • Blinkered Assertions
    • Brittle Assertion
    • Brittle Test
    • Brittle Ui Tests
    • Broad Assertion
    • Bumbling Assertions
    • Calculating Expected Results On The Fly
    • Celery Data
    • Circumstantial Evidence
    • Commented Code In The Test
    • Commented Test
    • Complex Assertions
    • Conspiracy Of Silence
    • Early Returning Test
    • Equality Sledgehammer Assertion
    • Fantasy Tests
    • Fragile Test
    • Fuzzy Assertions
    • Happenstance Testing
    • Inadequate Assertion
    • Inappropriate Assertions
    • Incidental Coverage
    • Invisible Assertions
    • Likely Ineffective Object-Comparison
    • Line Hitter
    • Martini Assertion
    • Missed Fail Rotten Green Test
    • Missing Assertions
    • Missing Test
    • No Assertions
    • No Traces Left
    • On The Fly
    • Over Exertion Assertion
    • Over-Checking
    • Premature Assertions
    • Primitive Assertion
    • Redundant Assertion
    • Returning Assertion
    • Second Guess The Calculation
    • Self-Test
    • Sensitive Equality
    • Shotgun Surgery
    • Testing The Internal Monologue
    • The Butterfly
    • The Nitpicker
    • Under-The-Carpet Assertion
    • Under-The-Carpet Failing Assertion
    • Using Assertions As A Substitute For All Class-Based Exceptions
    • Using Assertions As A Substitute For All Defensive Programming Techniques
    • Using The Wrong Assert
  • Issues in exception handling
    • Catching Unexpected Exceptions
    • Exception Catch/Throw
    • Exception Catching Throwing
    • Exception Handling
    • Expected Exceptions And Verification
    • Expecting Exceptions Anywhere
    • Issues In Exception Handling
    • No Error Handling In Tests
    • The Greedy Catcher
    • The Secret Catcher
    • The Silent Catcher
  • Issues in setup
    • Badly Used Fixture
    • Bury The Lede
    • Complicated Set Up Scenarios Within The Tests Themselves
    • Curdled Test Fixtures
    • Empty Shared-Fixture
    • Excessive Inline Setup
    • Excessive Setup
    • Factories That Contain Unnecessary Data
    • Fragile Fixture
    • Fragile Locators
    • General Fixture
    • Hidden Test Data
    • Idle Ptc
    • Inappropriately Shared Fixture
    • Irrelevant Information
    • Isolated Ptc
    • Large Fixture
    • Max Instance Variables
    • Noisy Logging
    • Noisy Setup
    • Obscure In-Line Setup
    • Oversharing On Setup
    • Refused Bequest
    • Scattered Test Fixtures
    • Separate Fixture Files
    • Share The World
    • Superfluous Setup Data
    • Taking Environment State For Granted
    • Test Maverick
    • Test Objects Initialized In A Describe Block
    • Test Objects Initialized In Each Test
    • Test Setup Is Somewhere Else
    • The Cuckoo
    • The Mother Hen
    • There Is Too Much Setup To Run The Test Cases
    • Unused Definition
    • Using Fixtures
    • Vague Header Setup
    • Verifying In Setup Method
    • Where Does This One Go?
  • Issues in teardown
    • Activation Asymmetry
    • Complex Teardown
    • External Shared-State Corruption
    • Generous Leftovers
    • Improper Clean Up After Tests Have Been Run
    • It Was Like That When I Got Here
    • Leaving Temporary Files Behind
    • Not Idempotent
    • Shared-State Corruption
    • Sloppy Worker
    • Teardown Only Test
    • The Painful Clean-Up
    • The Soloist
    • Unrepeatable Test
    • Wet Floor
• Test execution - behavior
  • Other test execution - behavior
    • Abnormal Utf-Use
    • Chatty Logging
    • Frequent Debugging
    • Interactive Test
    • Manual Assertions
    • Manual Event Injection
    • Manual Fixture Setup
    • Manual Intervention
    • Manual Result Verification
    • Print Statement
    • Redundant Print
    • Requires Supervision
    • The Loudmouth
    • Transcripting Test
    • Trying To Use Ui Automation To Replace Manual Testing
    • Unnecessary Navigation
  • Performance
    • Asynchronous Test
    • Factories Pulling Too Many Dependencies
    • Hardcoded Sleep
    • Inefficient Waits
    • Long Running Tests
    • Sleeping For Arbitrary Amount Of Time
    • Sleepy Test
    • Slow Component Usage
    • Slow Running Tests
    • Slow Test
    • The Bandwidth Demander
    • The Slow Poke
    • The Temporal Tip Toe
    • Too Many Tests
    • Wait And See
• Test semantic - logic
  • Other test logic related
    • Anal Probe
    • Assert 1 = 2
    • Asynchronous Code
    • Branch To Assumption Anti-Pattern
    • Chafing
    • Complex Conditional
    • Conditional Assertions
    • Conditional Logic Test
    • Conditional Logic
    • Conditional Test Logic
    • Conditional Tests
    • Conditional Verification Logic
    • Conditionals In Tests
    • Contaminated Test Subject
    • Context-Dependent Rotten Green Assertion Test
    • Control Logic
    • Easy Tests
    • Embedding Implementation Detail In Your Features/Scenarios
    • Evolve Or
    • Factories With Random Data Instead Of Sequences
    • Flexible Test
    • Fully Rotten Green Test
    • Generative
    • Get Really Clever And Use Random Numbers In Your Tests
    • Ground Zero
    • Guarded Test
    • Happy Path
    • I Wrote It Like This
    • Incidental Details
    • Inconsistent Wording
    • Indecisive
    • Indented Test
    • Insufficient Grouping
    • Invalid Test Data
    • Invasion Of Privacy
    • Lazy Test
    • London School Orthodoxy
    • Multiple Test Conditions
    • Nested Conditional
    • Neverfail Test
    • Only Easy Tests
    • Only Happy Path Tests
    • Paranoid
    • Parsed Data
    • Quixotic
    • Rewriting Private Methods As Public
    • Rotten Green Test
    • Skip Rotten Green Test
    • Skip-Epidemic
    • Sneaky Checking
    • Success Against All Odds
    • Tangential
    • Test By Number
    • Test-Per-Method
    • Testing Internal Implementation
    • Testing The Authentication Framework
    • Tests That Can’T Fail
    • The Dodger
    • The Inspector
    • The Liar
    • The Sequencer
    • Underspecification
    • Untestable Test Code
    • Use Smart Values
    • Wheel Of Fortune
    • X-Ray Specs
    • You Do Weird Things To Get At The Code Under Test
  • Testing many things
    • Assert The World
    • Assertion Roulette
    • Eager Test
    • Field-Level Assertion
    • Many Assertions
    • Method-Based Testing
    • Missing Assertion Message
    • Multiple Assertions
    • Piggybacking On Existing Tests
    • Split Logic
    • Split Personality
    • Test Cases Are Concerned With More Than One Unit Of Code
    • The Free Ride
    • The Giant
    • The One
    • The Test It All
  • Testing many units
    • Feature Envy
    • Indirect Testing
    • Test Envy
    • The Stranger

Miscellanous

• Easy Lab
• How to contribute