In a medium mixing bowl, use a hand mixer or whisk to combine the peanut butter and sugar until smooth.
Add the milk and mix on low speed until the sugar is dissolved, about 1 to 2 minutes.
Stir in the heavy cream and vanilla.
Pour the mixture into freezer bowl.
Turn on your machine and let mix until thickened, about 25 to 35 minutes. The ice cream will have a soft, creamy texture.
If a firmer consistency is desired, transfer the ice cream to an airtight container and place in freezer for about 2 hours. Remove from freezer about 15 minutes before serving.