Well, if these mirror attachments haven't changed, it's an easy fix. Went in there with the intention of trying to roughen up the surface of the ball part, to find that the problem is (of course, with hindsight) that the socket part had split down one side.
Thought about trying to glue it, decided it wouldn't hold; thought about trying to melt the edges of the crack together with a soldering iron tip. Decided I'd need to hold the crack shut with something while I did this, hit upon a small tyrap as the solution. Tried it back on the car with just the tyrap, and it's a winner! No more floppiness for me!