At Consumer Reports, we’ve tested hundreds of thermostat models over the years. And while once upon a time they weren’t very accurate, most now are able to keep rooms close to the set temperature. It’s for this reason that we no longer test manual thermostats—all they do is hold a set temperature.

With temperature accuracy a given, CR focuses our testing on ease of use—taking note of how simple each thermostat is to set up, program, read, and operate.

For programmable thermostats, the ease of use testing is broken down into four aspects of the product experience: setup, ease of reading the display, programming, and manual operation.

Our setup test evaluates how easy the thermostat is to configure for your HVAC system after physical installation, and our display test judges how well information is presented and how easy it is to read. The programming test looks at the number of steps it takes to create a heating and cooling schedule and the cues provided for understanding the schedule-building process. The manual operation test assesses the ease of using a thermostat’s primary functions, including controlling the setpoint, building a schedule, setting home/away status, and creating a vacation period.

For smart thermostats, our testing has a different focus. We factor in smart features like alerts, app control, and geofencing (where the thermostat tracks your phone’s location to know whether you’re home). We also account for machine learning and automation features, which allow smart thermostats to learn your habits and routines to adjust temperatures for you. And with the growing popularity of digital assistants, we consider voice control with Amazon Alexa, Apple HomeKit/Siri, and Google Assistant, if those integrations are available. These criteria factor into our unique Smart IQ and automation scores for smart thermostats. We also consider ease of wireless setup and, as we do with programmable thermostats, evaluate ease of manual operation.

Our test engineers then take all of this data and incorporate it into our Overall Scores for both smart and programmable thermostats.