Caffeine is unquestionably the world’s favorite stimulant, and among Americans a good cup of joe reigns as the caffeine-delivery vehicle of choice. That’s a good thing. While coffee used to have a bad reputation, the last decade of research suggests black coffee is actually pretty great for you.

But like other stimulants, caffeine can both build tolerance and trigger withdrawal symptoms. Especially if you’re the type who drinks a few cups of the stuff at the same time each day (i.e., a normal human being), skipping your morning coffee is going to mess with your brain and body in a number of ways.

Expect a headache, for starters. Half of habitual coffee drinkers felt one after going without their joe for a day, found a study from American University. The more coffee you drink every day, the more likely you are to experience a headache when you go without, the research suggests.

According to resources from Johns Hopkins University—a school that has spearheaded the “coffee is a drug!” campaign—irritability, difficulty concentrating, and feeling fatigued are some of the other common symptoms of caffeine withdrawal. Some people have also reported nausea, depression, anxiety, or “flu-like” symptoms. “Caffeine withdrawal syndrome” is even listed in the DSM—the American Psychiatric Association’s compendium of mental disorders.

How could coffee do all that? Like alcohol, but unlike most of the stuff you eat or drink, caffeine can cross the “blood-brain barrier.” And once it gets into your head, caffeine attaches to brain receptors that both reduce fatigue and trigger the flow of feel-good, fell-sharp hormones like dopamine and adrenaline, says David Elmenhorst, Ph.D., of Germany’s Institute of Neuroscience and Medicine. (That adrenaline surge explains why, if you drink a lot more coffee than you’re used to, your heart starts pounding and you feel anxious or over amped.)

A handful of experts have even argued that, once you build up tolerance to caffeine, drinking the stuff doesn’t perk you up. It just keeps you from crashing into withdrawal. A recent study from the UK found coffee drinkers felt no more alert or awake then non-drinkers when they had their morning joe. But if they were deprived their habitual coffee, their focus tanked and they reported feeling fatigued.

“We don’t gain an advantage from consuming caffeine,” said the author of that study, Peter Rogers, Ph.D., in a press release that accompanied his study’s publication. “Although we feel alerted by it, this is caffeine just bringing us back to normal.”

No one’s arguing that caffeine doesn’t perk you up if you drink it sparingly—and so don’t build up tolerance to it. But if you drink the stuff every day, the evidence suggests you come to depend on it to function.

This kind of language—words like tolerance and dependence and withdrawal—is usually associated with illicit drugs or other stuff that’s bad for you. But, again, when it comes to coffee, there doesn’t seem to be much risk unless you’re loading it up with sugar (which is unquestionable awful for you) or drinking it so much that you can’t sleep at night.

Not cool with the idea of being dependent on a stimulant, health benefits be damned? Research shows it usually takes between two days and a week to get over any withdrawal symptoms.