A rectangular prism has a base area of $20$ cm$^2$ and a surface area of $108$ cm$^2$. If the height is $4$, what are the dimensions of the base?