Well I guessed right, it was the upper strut mount. It began to oval.
I decided to do the sway bar bushings as well, so I decided to paint it with rustoleum to see how that paint holds up. Got an alignment setup for 10AM tomorrow.
I really hope this fixes all my issues for now.
This photo is installing the strut after I did the work, proof for the haterz.
Here is the old top strut mount...