@"they called me because of some website I was on"

Nobody can learn your phone number by visiting a website.

Unless you are naive enough to think that websites will ask you for your number because they need it for some reason, and you fell for it and gave it to them.