Let $U = \{1, 2, 3, \dots, 12\}$. Let $A$ be the set of divisors of 12 in $U$, and $B$ be the set of divisors of 18 in $U$. We define a set of ordered pairs $S = \{ (x, y) \in A \times B : \text{gcd}(x, y) \in A \cap B \}$. What is the cardinality of $S$?