OK so I just put it all back in with fresh everything.
Spigot, thrust bearings, rear main, the clutch got completely degreased and a mad scrub.
Took it for a small run up the driveway and the massive wobble that was there seems to have gone so I'm guessing the wobble was oil contamination from either the rear main or thrust bearing leaking. It has this weird creaking noise coming from it but I'm hoping shit will settle and that will go.