Dopamine and norepinephrine signaling differentially mediate the exploration-exploitation tradeoff

In an uncertain world, we balance two goals: exploiting rewarding options when they are available, and exploring potentially better alternatives. One neuromodulatory system that has been implicated in mediating the transition between exploration and exploitation is the catecholamine system, in particular, norepinephrine (NE) and dopamine (DA). Although both molecules have been implicated in decision making, their contributions have not been directly compared. When each neuromodulatory system is examined in isolation, they have been assigned similar roles in the latent cognitive processes that mediate exploration and exploitation. To understand the differences and overlaps of the role of these two catecholamine systems in regulating exploration and exploitation, a direct comparison using the same dynamic decision making task is needed. Here, we ran mice in a restless two-armed bandit task, which encourages both exploration and exploitation. We systemically administered a NE beta-receptor antagonist (propranolol), NE beta-receptor agonist (isoproterenol), a nonselective DA receptor antagonist (flupenthixol), and a nonselective DA receptor agonist (apomorphine) within subjects across sessions and examined changes in exploration. We found that modulating NE and DA receptor function had opposing effects on exploration - decreasing NE beta receptor activity or increasing DA receptor activity decreased exploration and resulted in stickier behaviors. Fitting a reinforcement learning model revealed that changes in exploration through manipulating NE and DA were due to changes in different latent processes - decreasing NE receptor activity decreased decision noise and increasing DA receptor activity decreased both learning rate and decision noise. Together, these findings suggested that the mechanisms that govern the transition between exploration and exploitation are sensitive to changes in both catecholamine functions and revealed differential roles for NE and DA in regulating exploration.