There can be several reasons why ants are in your succulent soil, but often it is because they are “farming” other insects as a food source. Since you seem to have so many coming out of the soil when you water, it sounds as though they’ve made a home for themselves in the soil as well.
Pests such as mealy bugs, aphids, and scale love succulents, and ants love the honeydew these pests excrete.
Check your plants for signs of mealybugs and other pests. If you see web-like substances, little white or black bugs, or small brown spots, your plant is probably infected already, and the ants can be drawn to that.
If you can eliminate the pest infestation, you should be able to then eradicate the ants. Isopropyl alcohol is the cheapest, easiest, and often most effective treatment for mealy bugs and scale. It will usually kill ants as well. Follow the directions in this post on how to use it.
You can also try Safer Soap, but personally, I only use isopropyl alcohol because it’s a lot cheaper and I know exactly what’s in it. I also know it’s not a problem to get on my skin.
If you believe the ants have made a colony in the soil of your succulent, it’s best to repot your plant into fresh soil. Do this after eliminating the bug problem as best you can, so that they’re not transferred to the new soil.