Hey Mr T.
Yes is the straight answer. You should remove 85-95% of the paint off the substrate.
I personally find some elbow grease and a thin bladed painters scraper & a stanley blade does the job.
If you
tile onto PVA you are asking the Adhesive to adhere to a thin layer of glue (Other major PVA flaws obvious will occur but that is another story)
So bearing the above in mind, if you try to adhere adhesive to paint it has to adhere to a thin layer of paint which wont hold anything but also PAINT is a hard material to stick to (full stop)
Get a piece of selotape and stick onto paint... it wont stick.
Hope this helps buddy