A theta sum is an exponential sum of the form $S_N^f(t, r, s) := \sum_{n \in \mathbb Z + s}f(n/N)e(1/2 (t n^2 + r n))$, where $t, r$ and $s$ are real numbers and $f$ is a sufficiently regular cut-off function. Upper bounds for theta sums have been well studied, and in this generality, estimates go back to work of J. Marklof, L. Flaminio and G. Forni, among others. Through their work one has, for instance, that for Lebesgue almost every $t$ the estimate $|S_N^f (t,r,s)| \ll_{f,t} \sqrt N \log N$ holds, for any pair $(r,s)$. We contrast this result by showing that there exist rational pairs $(r,s)$ such that for any Schwartz cut-off $f$, there exists a constant $C$ independent of $t$ for which $|S_N^f| \leq C \sqrt N$. A key feature of the proof is to realise that $S_N^f$, when normalised appropriately, agrees with a theta function $\Theta_f$ along a special curve known as a horocycle lift, which depends on the pair $(r,s)$. The result follows from showing that for certain rational pairs $(r,s)$, the horocycle lift avoids all regions where the modulus of $\Theta_f$ can be large. Time permitting, we will also discuss extensions of this result to theta sums in more than one variable. This talk is based on joint work with Francesco Cellarosi as well as a separate project with Michael Lu.