Building the backbone of a smarter smart home

The right algorithms can make smart homes smarter, more efficient, and more communicative

The state of artificial intelligence (AI) in smart homes nowadays might be likened to a smart, but moody teenager: It’s starting to hit its stride and discover its talents. But it doesn’t really feel like answering any questions about what it’s up to and would really rather be left alone, OK?

William Yeoh, assistant professor of computer science & engineering in the School of Engineering & Applied Science at Washington University in St. Louis, is working to help smart home AI to grow up.

The National Science Foundation awarded Yeoh a $300,000 grant to assist in developing smart home AI algorithms that can determine what a user wants by both asking questions and making smart guesses, and then plan and schedule accordingly. Beyond being smart, the system needs to be able to communicate; to explain why it is proposing the schedule it proposed to the user.

 These challenges rely heavily on communication. And like that moody teen, communication is not currently one of AI’s strengths.

“So far, a lot of AI is successful in isolation,” Yeoh said.

Feats such as winning a game of Go or recognizing faces do not require significant user interaction; a computer can do these tasks mostly on its own. “But if AI is going to help people,” he said, “interaction with people is pretty important.”

The first step in this project is figuring out what the user actually wants: The temperature to be 70 degrees when she gets home, the car fully charged by 8 a.m., and so on. And it must decipher what the user doesn’t want: The air conditioner set at full-blast all day long, or the car to charge during peak hours.

An AI system could meticulously and continuously ask a user for every one of her preferences. But because that is not possible (or would be, at the least, extremely annoying), the research sets out to determine, “Without bothering users too much, how many questions should we ask them and what should those questions be?” Yeoh noted in his grant application. The reason for asking: to make the smartest possible decisions when the user has not supplied specific information.

The algorithm that determines which questions to ask will be restricted by how many questions it can ask, and so will have to decide which questions have answers that will yield the most useful information.

Once the parameters and constraints are set, and the system has all of the available information it’s going to have, it devises a schedule that honors the user’s preferences, and maximizes the user’s comfort, while minimizing energy usage.

The system’s job won’t be done, however, once the parameters are set and the scheduling is done; it still needs to work on its communication skills.

Communication is a two-way street. Not only does Yeoh want to develop smart home systems that a user can tell what to do, he also thinks that the system should be able to explain itself to the user.

For several reasons, “the system needs to be able to explain to the user ‘why,’” Yeoh said. “The field has been working on getting good answers from AI. Now it’s time to get good explanations.”

Say a user rarely goes in the basement, but she heads down one night to get something out of storage. All of a sudden, the lights go out. Without being able to ask the system why, she may be wondering, “Is there a power outage? Is someone here? Is my house out to get me?”

If she could simply ask the system what happened, however, she would realize that in an attempt to save energy, the system has been turning off the lights in the basement every day after a certain time, because (via sensors, perhaps) it knows that she rarely goes downstairs in the evening.

Yeoh’s project is developing the tools to give users the ability to ask those questions, vocally or by turning to a visual interface designed to give users access to a host of information, from details about specific devices to a broader view of energy consumption.

Having access to that information will allow the user to more narrowly tailor her settings, working with the system instead of simply having to accept its settings. After all, there are lots of variables for a smart home AI system to take into account if it’s monitoring the temperature, locking the doors, charging a vehicle, preheating the oven, and monitoring a security system — all while trying to conserve energy usage.

“Energy prices, weather, the status of the devices,” Yeoh said of all the changing conditions that AI will need to monitor. “And humans, the most fickle variable of all.”

This research is supported by the National Science Foundation, grant #1838364

Click on the topics below for more stories in those areas

• Research
• Computer Science & Engineering