so I was wondering if when and if you get the inevitable leave me alone pervert and they leave and don;t seem to ever come back do you have to Restart from the menu to get them back?

I am afraid so.

A tip would be to first save the game before trying something, like abusive language or showing erotic objects (dildo, penis).

If it does not work out, load the last saved state and try something else.